How to create Periodic Tasks with Django Celery?

Celery provides asynchronous job queues, which allows you to run Python functions in the background. Celery is on the Python Package Index (PyPi), and can be easily installed with pip or easy_install and its dependencies.

Installation:

$ pip install django-celery && pip install redis

Celery uses a broker to pass messages between your application and Celery worker processes. We are having like rabbitmq, redis but we are using redis as a broker in this article. To initiate a task, an application will adds a message to the queue, which the broker then delivers to a worker.

So we can install redis using the following command:;

sudo apt-get install redis-server

Setting up celery in your project:

1. We need to add 'djcelery' into your project Installed Apps.

    INSTALLED_APPS += (
        'djcelery',
    )

2. To configure celery, we are using redis broker which is easy to install, lightweight, fast:

    import djcelery
    djcelery.setup_loader()
    BROKER_URL = 'redis://localhost:6379/1'

3. We should specify imports i.e from where tasks are being loaded using CELERY_IMPORTS command in settings.py file

    CELERY_IMPORTS = ("testapp.tasks")

4. Celery Periodic Task means which runs at a regular intervals of time. These periodic tasks are scheduled by a celery beat which will executed by a worker.

You can set the interval of time using crontab, timedelta. By Using time delta function, we can run a task at particular point of time, day like

     timedelta(seconds=30)

     timedelta(minutes=30),

     timedelta(minutes=*/5) --> which will execute at every 5 minutes

Like with cron, the tasks may overlap if the first task does not complete before the next.

     crontab(0, 0, day_of_month='23') --> which will execute on the 23rd day of the every month.

     crontab(minute='*/5') --> which will execute at every 5 minutes

     crontab(hour='*', minute='5', day_of_week='mon') --> which will execute for every 5 minutes on mon

5. We can setup celery periodic tasks settings using celery beat schedular.

    from celery.schedules import crontab

    CELERY_TIMEZONE = "Asia/Calcutta"

    CELERYBEAT_SCHEDULER = "djcelery.schedulers.DatabaseScheduler"

    CELERYBEAT_SCHEDULE = {
        # Executes for every 5 minutes on mon,tue,wed,thu,fri,sat
        'add-every-day-evening': {
            'task': 'testapp.tasks.load_messages_into_database',
            'schedule': crontab(hour='*', minute='5', day_of_week='mon,tue,wed,thu,fri,sat'),
        },
    }

You need to ensure that scheduling tasks run with a single schedular only, otherwise it leads to duplicate tasks which runs by multiple schedulers at the same time.

Posted On 23 February 2014 By MicroPyramid

Previous post Next post

Need any Help in your Project?Let's Talk

Latest Comments

OUR SERVICES

OPEN SOURCE PACKAGES

Django-CRM :Customer relationship management based on Django

Django-blog-it : django blog with complete customization and ready to use with one click installer Edit

Django-webpacker : A django compressor tool

Django-MFA : Multi Factor Authentication

Docker-box : Web Interface to manage full blown docker containers and images

More...

How to create Periodic Tasks with Django Celery?

Need any Help in your Project?Let's Talk

How to maintain user session across sub domains in Django

What is Django CRM and it’s advantages

Implementation of single sign on using auth0 in django application

Customizing Django REST API Serializers

How to implement TokenBasedAuthentication in DjangoRestFramework?

Understanding middleware functionality in django2.0

Django acceptance testing automation with robot framework

How to filter a Django Queryset using Extra

Overriding Django Model behaviour with Proxy Model

how to pass extra context data to serializers in django-rest-framework ?

Understanding Routers in Django-Rest-Framework

Django Single Sign On(SSO) to multiple applications

How to Document API Requests using Django Rest Swagger

Django webpacker - A compression tool to bundles css, js files

What's great about Django girls to inspire women into programming

How to use nested formsets in django

How to convert xml content into json using xmltodict

Custom validations for serializer fields Django Rest Framework

How To Add A Custom Managers In Django

Introduction to API development using Django REST framework with Example

Django-REST Framework Object Level Permissions and User Level Permissions

Using Python weasyprint generate HTML to PDF in django

How to create thumbnail image in Django templates using sorl-thumbnail?

Deploying Your Django app on Heroku

Django Template language Intro

Using Gitlab API, integrating gitlab in django project for authentication and access

Django Conditional Expressions in Queries

Sorl-thumbnail to generate thumbnails in django

How to Create initial django migrations for existing DB schema.

Django - Database access optimization

Querying with Django Q objects

Integration Of GitHub API with python django

Django - migrating from function based views to class based views

How To Write Custom migrations in Django

Dynamically Adding Google Maps with Marker In Django

Django Unit Test cases with Forms and Views

Understanding Checkout flow in Django Oscar.

How to monitor django application live events with sentry

Integration Of Linkedin API in python django

Django efficient implementation of Amazon s3 and Cloudfront CDN for faster loading.

How to create Custom User Model or Extend User Model in Django?

Working with Django Plugins

Export html web page to pdf using jspdf

Preserve file names with sorl for better SEO

Custom Password Less Authentication in Django

Improving page speed score in Google Page Score test- PART2

Extract text with OCR for all image types in python using pytesseract

Factory Boy - An alternative for Fixtures

Improving page speed score in Google Page Score test- PART1

Integration of 2checkout with django

Get count, average, min, max values from model field using Django Aggregate

Set Up Travis CI For Django project

Sending emails using sendgrid on heroku for a Django App

Django Custom Template Tags And Filters

Python Memcached Implementation for Django project

How to customize Django Oscar Models, Views and URL's

Integrate Django-Oscar-Accounts with Django-Oscar

Google plus API integration with django

Extract data from PDF and all Microsoft Office files in python

django Payu Payment gateway Integration

Running Django with PyPy to boost performance

How to Create your own e-commerce shop using Django-Oscar.

Setting Up Coveralls for Django Project

Sendgrid Inbound Email Parsing with django

How to customize the admin actions in list pages of Django admin?

Handling Custom Error Pages(404, 500) In Django

Autocomplete with Django-Haystack and Elasticsearch with single letter querying.

How to create Periodic Tasks with Django Celery?

Best Practices of Using Django Celery in Django Social Auth

Custom Decorators To Check User Roles And Permissions In Django

Implement search with Django-haystack and Elasticsearch Part-1

How to index binary files in django haystack

Understanding django serializers with examples

Introduction to Django's Class based views - Understanding how a class based view works

Understanding Django Permissions And Groups

Django Inclusion Tags

Dynamic Models in Django (adding new fields to models from admin)

E-commerce (paypal) integration with Django