How to create Periodic Tasks in Celery?

Celery is a powerful, production-ready asynchronous job queue, which allows you to run time-consuming Python functions in the background. Celery is on the Python Package Index (PyPi), and can be easily installed with pip or easy_install and its dependencies.

$ pip install django-celery && pip install redis

Celery uses a broker to pass messages between your application and Celery worker processes. We are having like rabbitmq, redis but we are using redis as a broker in this article. To initiate a task, an application will adds a message to the queue, which the broker then delivers to a worker.

So we can install redis using the following command:;

sudo apt-get install redis-server

Setting up celery in your project:

1. We need to add 'djcelery' into your project Installed Apps.

    INSTALLED_APPS += (
        'djcelery',
    )

2. To configure celery, we are using redis broker which is easy to install, lightweight, fast:

    import djcelery
    djcelery.setup_loader()
    BROKER_URL = 'redis://localhost:6379/1'

3. We need to tell celery from where tasks are being loaded using CELERY_IMPORTS command

    CELERY_IMPORTS = ("testapp.tasks")

4. Celery Periodic Task means which runs at a regular intervals of time. These periodic tasks are scheduled by a celery beat which will executed by a worker.

• You can set the interval of time using crontab, timedelta. Using the time delta, we can execute a task at particular point of time like 

     timedelta(seconds=30)

     timedelta(minutes=30),

     timedelta(minutes=*/5) --> which will execute at every 5 minutes

• Like with cron, the tasks may overlap if the first task does not complete before the next.

     crontab(0, 0, day_of_month='23') --> which will execute on the 23rd day of the every month.

     crontab(minute='*/5') --> which will execute at every 5 minutes

     crontab(hour='*', minute='5', day_of_week='mon') --> which will execute for every 5 minutes on mon

5. We can setup celery periodic tasks settings using celery beat schedular.

    from celery.schedules import crontab

    CELERY_TIMEZONE = "Asia/Calcutta"

    CELERYBEAT_SCHEDULER = "djcelery.schedulers.DatabaseScheduler"

    CELERYBEAT_SCHEDULE = {
        # Executes for every 5 minutes on mon,tue,wed,thu,fri,sat
        'add-every-day-evening': {
            'task': 'testapp.tasks.load_messages_into_database',
            'schedule': crontab(hour='*', minute='5', day_of_week='mon,tue,wed,thu,fri,sat'),
        },
    }

    You have to ensure only a single scheduler is running for a schedule at a time, otherwise you would end up with duplicate tasks.