Periodic Tasks in Django with Celery Beat (5.x Guide)

To run scheduled or recurring tasks in Django, use Celery Beat — Celery's built-in scheduler. You define a schedule (a crontab expression or a fixed interval), run the beat process to send due tasks onto the broker queue, and run one or more worker processes to execute them. For schedules you need to add or change at runtime without redeploying, store them in the database with django-celery-beat.

This guide uses the current Celery 5.x application API. If you have seen older tutorials using djcelery, import djcelery / djcelery.setup_loader(), or from celery.task import periodic_task, those are removed in modern Celery — don't use them. Everything below works with Celery 5.x on Python 3.

Install Celery and a broker

Celery needs a message broker to pass task messages between your app and the workers. Redis and RabbitMQ are the common choices; this guide uses Redis because it is light to install and run.

pip install celery redis django-celery-beat

Install and start a Redis server (on Debian/Ubuntu):

sudo apt-get install redis-server
sudo systemctl enable --now redis-server

Set up the Celery app in a Django project

The canonical layout for a project named proj (the package that holds settings.py) is a proj/celery.py module that creates the app and reads its configuration from Django settings. This is the structure the official Celery docs recommend for Celery 5.x.

# proj/celery.py
import os

from celery import Celery

# Point Celery at your Django settings module before creating the app.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'proj.settings')

app = Celery('proj')

# Read config from Django settings, using keys prefixed with CELERY_.
app.config_from_object('django.conf:settings', namespace='CELERY')

# Auto-discover tasks.py modules in every app listed in INSTALLED_APPS.
app.autodiscover_tasks()


@app.task(bind=True, ignore_result=True)
def debug_task(self):
    print(f'Request: {self.request!r}')

Make sure the app is loaded when Django starts, so @shared_task and the worker can find it. Add this to the project package's __init__.py:

# proj/__init__.py
from .celery import app as celery_app

__all__ = ('celery_app',)

Then configure the broker and a few sensible defaults in settings.py. Because of the namespace='CELERY' above, every Celery setting is written with a CELERY_ prefix:

# proj/settings.py
CELERY_BROKER_URL = 'redis://localhost:6379/0'
CELERY_RESULT_BACKEND = 'redis://localhost:6379/1'

CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_ACCEPT_CONTENT = ['json']

# Keep this in sync with TIME_ZONE; see the timezone note below.
CELERY_TIMEZONE = 'Asia/Kolkata'
CELERY_TASK_TRACK_STARTED = True
CELERY_TASK_TIME_LIMIT = 30 * 60  # 30 minutes

Define tasks with @shared_task

Inside any Django app, put your task functions in a tasks.py module so autodiscover_tasks() finds them. Use @shared_task (not @app.task) so the task is not tied to one specific app instance — this is the right choice for reusable Django apps.

# myapp/tasks.py
from celery import shared_task


@shared_task
def load_messages_into_database():
    # ... your slow / recurring work here ...
    return 'done'


@shared_task
def add(x, y):
    return x + y

Schedule tasks with app.conf.beat_schedule

For schedules that ship with your code, define them in app.conf.beat_schedule. Each entry maps a name to a task path, a schedule, and optional args/kwargs. The schedule value can be a crontab(...) object, a datetime.timedelta, or a plain number of seconds (a float).

# proj/celery.py (added below the app setup)
from datetime import timedelta

from celery.schedules import crontab

app.conf.beat_schedule = {
    # Run every 30 seconds (float = seconds).
    'heartbeat-every-30s': {
        'task': 'myapp.tasks.add',
        'schedule': 30.0,
        'args': (16, 16),
    },
    # Run every 5 minutes (timedelta).
    'sync-every-5-min': {
        'task': 'myapp.tasks.load_messages_into_database',
        'schedule': timedelta(minutes=5),
    },
    # Run on weekdays at 7:30 a.m. in the configured timezone (crontab).
    'weekday-morning-report': {
        'task': 'myapp.tasks.load_messages_into_database',
        'schedule': crontab(hour=7, minute=30, day_of_week='mon-fri'),
    },
}

crontab field reference

crontab() accepts minute, hour, day_of_week, day_of_month, and month_of_year. Any field left out defaults to * (every value). Here are common patterns:

Schedule	crontab expression	Runs
Every minute	`crontab()`	Once per minute
Every 5 minutes	`crontab(minute='*/5')`	At :00, :05, :10, ...
Daily at 7:30 a.m.	`crontab(hour=7, minute=30)`	Once a day at 07:30
Every Monday at 7:30 a.m.	`crontab(hour=7, minute=30, day_of_week=1)`	Mondays at 07:30
First of every month at midnight	`crontab(minute=0, hour=0, day_of_month=1)`	1st of each month
Every 3 hours	`crontab(minute=0, hour='*/3')`	00:00, 03:00, 06:00, ...

day_of_week accepts numbers (0 = Sunday) or names (mon, tue, ...) and ranges/lists like mon-fri or thu,fri.

The timezone gotcha

By default Celery schedules in UTC. If a task fires at the wrong wall-clock time, the cause is almost always timezone. Set CELERY_TIMEZONE (or app.conf.timezone) to your local zone so crontab(hour=7, minute=30) means 7:30 a.m. local time. Keep it consistent with Django's own TIME_ZONE to avoid confusion:

# Either in settings.py (preferred, with the CELERY_ prefix)...
CELERY_TIMEZONE = 'Asia/Kolkata'

# ...or directly on the app object:
app.conf.timezone = 'Asia/Kolkata'

Edit schedules at runtime with django-celery-beat

The beat_schedule dict above is static — changing it means a code change and redeploy. To add, edit, enable, or disable schedules at runtime (for example, from the Django admin), use the django-celery-beat package. It stores schedules in the database and exposes them as the PeriodicTask, CrontabSchedule, IntervalSchedule, SolarSchedule, and ClockedSchedule models in the admin.

Add it to INSTALLED_APPS and run migrations:

# proj/settings.py
INSTALLED_APPS = [
    # ... your other apps ...
    'django_celery_beat',
]

python manage.py migrate

Now run beat with the database scheduler. It reads the schedule from the DB, so editing a PeriodicTask in the admin takes effect without restarting anything:

celery -A proj beat -l info --scheduler django_celery_beat.schedulers:DatabaseScheduler

In the admin you create an IntervalSchedule (e.g. every 5 minutes) or a CrontabSchedule (e.g. 7:30 on weekdays), then create a PeriodicTask that points at your task path (myapp.tasks.load_messages_into_database) and references that schedule. Toggle its Enabled checkbox to pause or resume it instantly.

Run it: worker and beat are separate processes

The worker executes tasks; beat only decides when a periodic task should be queued. In production they are two separate processes. Start a worker:

celery -A proj worker -l info

And start beat (use the DatabaseScheduler form shown above if you use django-celery-beat):

celery -A proj beat -l info

For local development only, you can embed beat inside the worker with -B to avoid running two terminals:

# DEV ONLY -- do not use -B in production
celery -A proj worker -B -l info

Production notes

Run exactly one beat instance. If two beat schedulers run against the same schedule, every periodic task is queued twice. Workers can and should scale horizontally; beat must not.
Use a process manager. Run worker and beat under systemd or supervisor so they restart on failure and start on boot.
Make tasks idempotent. A task can run more than once (retries, redeploys, broker redelivery). Design tasks so a duplicate run is harmless — guard with a lock or a unique key.
Set the broker visibility timeout (Redis) higher than your longest task so messages aren't redelivered while still being processed.
Monitor with Flower. Run pip install flower then celery -A proj flower to watch workers, queues, and task history in a web UI.
Pick a durable broker. RabbitMQ or a properly persisted Redis is what you want in production rather than an ephemeral instance.

Celery Beat vs cron vs APScheduler vs cloud schedulers

Option	Best when	Trade-offs
Celery Beat	You already use Celery for background tasks and want scheduling in the same system	Needs a broker and a separate beat process; one beat instance only
OS cron	Simple, host-local jobs; running a management command on a box	No retries, no result tracking, no distribution; tied to one machine
APScheduler	Lightweight in-process scheduling without a broker	In-process means it stops with the app; no built-in distributed workers
Cloud schedulers (AWS EventBridge / Cloud Scheduler)	Serverless or multi-service triggers, infra-managed reliability	Vendor-specific; triggers an endpoint/queue rather than running your task code directly

If your app already runs Celery workers, Celery Beat is usually the natural fit because schedules, retries, and monitoring all live in one place.

Frequently Asked Questions

What is the difference between the Celery worker and Celery Beat?

The worker is the process that actually executes your tasks. Beat is a lightweight scheduler that watches your schedule and, when a periodic task is due, places a message on the broker queue. Beat never runs task code itself — a worker picks the message up and runs it. In production you run one beat process and one or more worker processes.

How do I change a schedule without redeploying?

Use django-celery-beat with its DatabaseScheduler. It stores schedules in the database as PeriodicTask, CrontabSchedule, and IntervalSchedule records you can edit in the Django admin. Run beat with --scheduler django_celery_beat.schedulers:DatabaseScheduler and any change you save takes effect on the next beat tick, with no code change or restart needed.

Why is my periodic task running at the wrong time?

This is almost always a timezone issue. Celery schedules in UTC unless you tell it otherwise, so a crontab(hour=7, minute=30) fires at 07:30 UTC by default. Set CELERY_TIMEZONE (or app.conf.timezone) to your local zone, keep it consistent with Django's TIME_ZONE, and the crontab times will match your local wall clock.

Can I run multiple Celery Beat schedulers?

No — run exactly one beat instance per schedule. Two beat processes reading the same schedule will each queue every periodic task, so tasks run twice. Scale workers horizontally as much as you need, but keep beat as a single process, ideally supervised by systemd or supervisor so it restarts cleanly if it dies.

Celery Beat or OS cron — which should I use?

Use OS cron for simple, host-local jobs where you just need to invoke a script or management command and don't care about retries or distribution. Use Celery Beat when you already run Celery, want tasks distributed across workers, and need retries, result tracking, and monitoring. Celery Beat keeps scheduling, execution, and observability in one system.

How do I stop a periodic task from running twice?

First, make sure only one beat instance is running — duplicate beat processes are the most common cause. Then make the task idempotent so an occasional duplicate (from a retry or broker redelivery) is harmless: guard the work with a distributed lock (for example via Redis) or a unique database constraint, and set the broker visibility timeout longer than your longest-running task.

Need help with Django background jobs?

MicroPyramid has built and operated Celery-based task pipelines across 50+ projects over 12+ years. If you want help designing reliable scheduled and async workloads, explore our Django development services and broader Python development services.

How to Create Periodic Tasks in Django with Celery Beat