How to create Periodic Tasks with Django Celery?

Celery provides asynchronous job queues, which allows you to run Python functions in the background. Celery is on the Python Package Index (PyPi), and can be easily installed with pip or easy_install and its dependencies.

Installation:
$ pip install django-celery && pip install redis

Celery uses a broker to pass messages between your application and Celery worker processes. We are having like rabbitmq, redis but we are using redis as a broker in this article. To initiate a task, an application will adds a message to the queue, which the broker then delivers to a worker.

So we can install redis using the following command:;

sudo apt-get install redis-server

Setting up celery in your project:

1. We need to add 'djcelery' into your project Installed Apps.

    INSTALLED_APPS += (
        'djcelery',
    )

2. To configure celery, we are using redis broker which is easy to install, lightweight, fast:

    import djcelery
    djcelery.setup_loader()
    BROKER_URL = 'redis://localhost:6379/1'

3. We should specify imports i.e from where tasks are being loaded using CELERY_IMPORTS command in settings.py file

    CELERY_IMPORTS = ("testapp.tasks")


4. Celery Periodic Task means which runs at a regular intervals of time. These periodic tasks are scheduled by a celery beat which will executed by a worker.

  • You can set the interval of time using crontab, timedelta. By Using time delta function, we can run a task at particular point of time, day like 
     timedelta(seconds=30)

     timedelta(minutes=30),

     timedelta(minutes=*/5) --> which will execute at every 5 minutes
  • Like with cron, the tasks may overlap if the first task does not complete before the next.
     crontab(0, 0, day_of_month='23') --> which will execute on the 23rd day of the every month.

     crontab(minute='*/5') --> which will execute at every 5 minutes

     crontab(hour='*', minute='5', day_of_week='mon') --> which will execute for every 5 minutes on mon

5. We can setup celery periodic tasks settings using celery beat schedular.

    from celery.schedules import crontab

    CELERY_TIMEZONE = "Asia/Calcutta"

    CELERYBEAT_SCHEDULER = "djcelery.schedulers.DatabaseScheduler"

    CELERYBEAT_SCHEDULE = {
        # Executes for every 5 minutes on mon,tue,wed,thu,fri,sat
        'add-every-day-evening': {
            'task': 'testapp.tasks.load_messages_into_database',
            'schedule': crontab(hour='*', minute='5', day_of_week='mon,tue,wed,thu,fri,sat'),
        },
    }

    You need to ensure that scheduling tasks run with a single schedular only, otherwise it leads to duplicate tasks which runs by multiple schedulers at the same time.

Posted On 23 February 2014 By MicroPyramid


Need any Help in your Project?Let's Talk

Latest Comments
Related Articles
Understanding django serializers with examples

Serializers are used for “translating” Django models into other formats like xmi,json,yaml(YAML Ain’t a Markup Language)

from django.core import serializers
data = serializers.serialize("xml", SomeModel.objects.all())

Continue Reading...
Working with Django Plugins

This blog describes about how to work with django-plugins. Django Plugin is a Simple Plugin Framework for Django. By using django-plugins, you can make your ...

Continue Reading...
Django Conditional Expressions in Queries

Reduce database queries in django with Conditional Expressions. By using Conditional Expressions we can use "If...Elif...Else" expressions while querying the database so that we can ...

Continue Reading...
open source packages

Subscribe To our news letter

Subscribe and Stay Updated about our Webinars, news and articles on Django, Python, Machine Learning, Amazon Web Services, DevOps, Salesforce, ReactJS, AngularJS, React Native.
* We don't provide your email contact details to any third parties