How to create Periodic Tasks in Celery?

Reading Time : ~ .

Celery is a powerful, production-ready asynchronous job queue, which allows you to run time-consuming Python functions in the background. Celery is on the Python Package Index (PyPi), and can be easily installed with pip or easy_install and its dependencies.

Installation:
$ pip install django-celery && pip install redis

Celery uses a broker to pass messages between your application and Celery worker processes. We are having like rabbitmq, redis but we are using redis as a broker in this article. To initiate a task, an application will adds a message to the queue, which the broker then delivers to a worker.

So we can install redis using the following command:;

sudo apt-get install redis-server

Setting up celery in your project:

1. We need to add 'djcelery' into your project Installed Apps.

    INSTALLED_APPS += (
        'djcelery',
    )

2. To configure celery, we are using redis broker which is easy to install, lightweight, fast:

    import djcelery
    djcelery.setup_loader()
    BROKER_URL = 'redis://localhost:6379/1'

3. We need to tell celery from where tasks are being loaded using CELERY_IMPORTS command

    CELERY_IMPORTS = ("testapp.tasks")


4. Celery Periodic Task means which runs at a regular intervals of time. These periodic tasks are scheduled by a celery beat which will executed by a worker.

  • You can set the interval of time using crontab, timedelta. Using the time delta, we can execute a task at particular point of time like 
     timedelta(seconds=30)

     timedelta(minutes=30),

     timedelta(minutes=*/5) --> which will execute at every 5 minutes
  • Like with cron, the tasks may overlap if the first task does not complete before the next.
     crontab(0, 0, day_of_month='23') --> which will execute on the 23rd day of the every month.

     crontab(minute='*/5') --> which will execute at every 5 minutes

     crontab(hour='*', minute='5', day_of_week='mon') --> which will execute for every 5 minutes on mon

5. We can setup celery periodic tasks settings using celery beat schedular.

    from celery.schedules import crontab

    CELERY_TIMEZONE = "Asia/Calcutta"

    CELERYBEAT_SCHEDULER = "djcelery.schedulers.DatabaseScheduler"

    CELERYBEAT_SCHEDULE = {
        # Executes for every 5 minutes on mon,tue,wed,thu,fri,sat
        'add-every-day-evening': {
            'task': 'testapp.tasks.load_messages_into_database',
            'schedule': crontab(hour='*', minute='5', day_of_week='mon,tue,wed,thu,fri,sat'),
        },
    }

    You have to ensure only a single scheduler is running for a schedule at a time, otherwise you would end up with duplicate tasks.

    By Posted On
SENIOR DEVELOPER at MICROPYRAMID

Need any Help in your Project?Let's Talk

Latest Comments
Related Articles
Sendgrid Inbound Email Parsing with django Nikhila Mergu

Using the Inbound parse webhook, we can parse the contents, attachments of an incoming email.

Inbound Parse API follows 3 steps:
1. sending an ...

Continue Reading...
Integration Of Linkedin API in python django Nikhila Mergu

Using Linkedin integration, we can get the user verified email id, general information, work history in a less span of time, and a user can ...

Continue Reading...
Django search with haystack and whoosh Ashwin Kumar

Haystack is a Django plugin to allow text search, while Whoosh is a pure Python search backend and it provides a library of classes and ...

Continue Reading...

Subscribe To our news letter

Subscribe to our news letter to receive latest blog posts into your inbox. Please fill your email address in the below form.
*We don't provide your email contact details to any third parties