Django Celery + Social Auth: Async Pipeline Best Practices

Combining Celery with social login in Django comes down to one rule: the social-auth pipeline runs inside the login HTTP request, so any slow step in it — downloading the user's avatar, calling a third-party profile API, syncing CRM data, or sending a welcome email — directly delays the redirect back into your app. The best practice is to keep the pipeline lean and hand every slow, non-essential step to a Celery task with .delay() the moment the user and social-account records exist. Login then finishes in milliseconds while the heavy work runs in the background and retries on its own if a provider is flaky.

This 2026 guide uses the maintained social-auth-app-django package (the successor to the long-dead django-social-auth) together with Celery 5.x. If you are starting fresh, django-allauth is the other mainstream option — we compare both below.

Key takeaways

• The pipeline is synchronous. Every function in SOCIAL_AUTH_PIPELINE runs during the login request, so a slow step blocks the user's redirect.
• Offload, don't inline. Move avatar fetches, profile/CRM syncs, and welcome emails into Celery tasks called with .delay() from a custom pipeline step.
• Never couple login success to task success. A failed welcome email must not fail the login — fire-and-forget after the user exists.
• Pass IDs, not ORM objects. Send user.id to the task and re-query inside it; pickled model instances go stale and bloat the broker.
• Make tasks idempotent and retried. Use autoretry_for + retry_backoff so transient provider outages self-heal without creating duplicates.
• Use the maintained stack. social-auth-app-django (import social_django) replaces the dead django-social-auth; Celery 5.x replaces the obsolete django-celery/djcelery.
• Some steps must pause (e.g. asking for a phone number) — use the partial pipeline pattern, not a blocking call.

Why does social login feel slow?

When a user clicks "Sign in with Google," python-social-auth walks an ordered list of pipeline functions — social_details, social_uid, social_user, create_user, load_extra_data, user_details, and any custom steps you append. All of that happens inside the request/response cycle before the user is redirected back into your app.

Third-party providers make this fragile:

• Latency — a remote profile or avatar call is far slower than local DB access, yet users expect an instant login.
• Rate limits — every provider has different (and sometimes undocumented) limits; a burst of logins can throttle you.
• Instability — provider outages and random failures will, eventually, happen mid-login.

If you do that work synchronously in the pipeline, a slow or down provider becomes a slow or failed login. Celery lets login succeed immediately and pushes the risky work to a worker.

What is the social-auth pipeline (and which library should I use)?

The pipeline is python-social-auth's extension point: a tuple of dotted paths in settings.py that every authentication flows through. You customize behavior by inserting your own functions into that tuple — which is exactly where we'll schedule background work.

Modernization note (this matters): the original django-social-auth library is dead and deprecated — do not install it. Its maintained successor is social-auth-app-django, part of the python-social-auth project; the Django app you add to INSTALLED_APPS is social_django. Likewise, the old django-celery/djcelery packages are obsolete — Celery 5.x has native Django support and needs no Django-specific shim.

If you have no legacy code to keep, django-allauth is the other mainstream choice. The two differ in philosophy — see the comparison table below.

Sync step vs async Celery task: what changes?

Aspect	Slow step inline in the pipeline	Same step as a Celery task
Login latency	Blocks the redirect until it finishes	Login returns immediately
Provider outage	Login fails or hangs	Login succeeds; task retries later
Rate-limit spikes	Throttling stalls real users	Absorbed by the worker queue
Retries	None (you would hand-roll them)	Built in via autoretry_for / retry_backoff
Scaling	Tied to web worker count	Scale workers independently
Best for	Cheap, essential, local work (create user, set username)	Avatar fetch, profile/CRM sync, welcome email, analytics

social-auth-app-django vs django-allauth

Question	social-auth-app-django	django-allauth
Project	python-social-auth	independent, very active
Core idea	Customizable pipeline of functions	Signals + adapter classes
Best hook for Celery	A custom pipeline step that calls .delay()	pre_social_login / user_signed_up signal receivers
Local email/password + social	Add-on	Built in, batteries-included
Account-management UI	Bring your own	Ships login/signup/connect views
Choose when	You want fine-grained pipeline control or are migrating from django-social-auth	You want a full auth UI out of the box

Both keep login fast the same way: do the minimum synchronously, then enqueue Celery tasks. The rest of this guide uses social-auth-app-django; with allauth, put the identical .delay() calls inside a user_signed_up signal receiver instead of a pipeline function.

# settings.py

INSTALLED_APPS = [
    # ...
    "social_django",            # python-social-auth's Django app (NOT the dead django-social-auth)
]

AUTHENTICATION_BACKENDS = [
    "social_core.backends.google.GoogleOAuth2",
    "django.contrib.auth.backends.ModelBackend",
]

# The pipeline runs DURING the login request. Keep it lean and append one
# custom step that schedules the slow work on Celery.
SOCIAL_AUTH_PIPELINE = (
    "social_core.pipeline.social_auth.social_details",
    "social_core.pipeline.social_auth.social_uid",
    "social_core.pipeline.social_auth.auth_allowed",
    "social_core.pipeline.social_auth.social_user",
    "social_core.pipeline.user.get_username",
    "social_core.pipeline.social_auth.associate_by_email",
    "social_core.pipeline.user.create_user",
    "social_core.pipeline.social_auth.associate_user",
    "social_core.pipeline.social_auth.load_extra_data",
    "social_core.pipeline.user.user_details",
    "accounts.pipeline.schedule_post_login_tasks",   # <-- our async hand-off
)

# Celery 5.x settings (CELERY_ namespace). Redis broker; separate result DB.
CELERY_BROKER_URL = "redis://localhost:6379/0"
CELERY_RESULT_BACKEND = "redis://localhost:6379/1"
CELERY_TASK_ACKS_LATE = True

How do I wire up Celery 5.x in Django?

Celery 5 talks to your app through a broker (Redis or RabbitMQ) and, optionally, a result backend. There is no djcelery anymore — you create one Celery app and let it read settings straight from Django.

Create proj/celery.py, register it in proj/__init__.py, then start a worker with celery -A proj worker -l info.

# proj/celery.py
import os
from celery import Celery

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "proj.settings")

app = Celery("proj")
# Read every CELERY_* value from Django settings.py
app.config_from_object("django.conf:settings", namespace="CELERY")
# Auto-discover tasks.py in each installed app
app.autodiscover_tasks()


# proj/__init__.py
from .celery import app as celery_app

__all__ = ("celery_app",)

What does a good post-login task look like?

Four rules keep these tasks safe:

1. Pass IDs, not objects. Send user.id, not the User instance, then re-query inside the task so you never act on a stale, pickled row.
2. Be idempotent. A retried task must not send two welcome emails or save two avatars — guard with a check or a flag.
3. Retry transient failures. Use autoretry_for for network errors plus retry_backoff (exponential) and retry_jitter so a flaky provider self-heals instead of hammering.
4. Stay decoupled from login. The task runs after the user exists; if it ultimately fails, login is already done and the user is unaffected.

Define each task with @shared_task so it isn't bound to one specific Celery app instance.

# accounts/tasks.py
import requests
from celery import shared_task
from django.contrib.auth import get_user_model
from django.core.files.base import ContentFile
from django.core.mail import send_mail

User = get_user_model()


@shared_task(
    bind=True,
    autoretry_for=(requests.RequestException,),
    retry_backoff=True,        # 1s, 2s, 4s, 8s ... exponential
    retry_backoff_max=600,
    retry_jitter=True,
    max_retries=5,
)
def fetch_avatar(self, user_id, avatar_url):
    """Download the provider avatar without blocking login."""
    user = User.objects.filter(pk=user_id).first()
    if not user or not avatar_url:
        return
    profile = user.profile
    if profile.avatar:           # idempotent: already done on a prior run
        return
    resp = requests.get(avatar_url, timeout=10)
    resp.raise_for_status()
    profile.avatar.save(f"{user_id}.jpg", ContentFile(resp.content), save=True)


@shared_task
def send_welcome_email(user_id):
    user = User.objects.filter(pk=user_id).first()
    if not user or not user.email:
        return
    if getattr(user.profile, "welcomed", False):   # idempotent: send once
        return
    send_mail(
        "Welcome aboard",
        "Thanks for signing in.",
        "noreply@example.com",
        [user.email],
    )
    user.profile.welcomed = True
    user.profile.save(update_fields=["welcomed"])

# accounts/pipeline.py
from .tasks import fetch_avatar, send_welcome_email


def schedule_post_login_tasks(strategy, details, backend, user=None, is_new=False, *args, **kwargs):
    """Custom pipeline step: hand slow work to Celery, then return instantly.

    Runs after create_user / associate_user, so `user` already exists in the DB.
    We pass only IDs and plain strings to the tasks -- never the ORM object.
    """
    if user is None:
        return

    # kwargs["response"] is the provider's raw OAuth payload.
    response = kwargs.get("response", {}) or {}
    avatar_url = response.get("picture")   # Google key; differs per provider

    if avatar_url:
        fetch_avatar.delay(user.id, avatar_url)

    if is_new:                             # only greet brand-new accounts
        send_welcome_email.delay(user.id)

    # Returning None keeps the pipeline going; login is NOT blocked by the tasks.
    return

What about a step that must pause for user input?

Sometimes a step can't be fire-and-forget — for example, you must ask a first-time user for a phone number or to accept terms before the account is complete. You can't background that, but you also shouldn't block; instead use the partial pipeline pattern. Decorate the step with @partial and return strategy.redirect(...). python-social-auth saves the pipeline state and resumes from the same step once the user returns to the social:complete URL.

Reserve partials for genuine input gates only — everything that doesn't need the user should still go to Celery.

# accounts/pipeline.py
from social_core.pipeline.partial import partial


@partial
def require_phone_number(strategy, details, backend, user=None, is_new=False, *args, **kwargs):
    """Pause the pipeline to collect extra data, then resume at THIS step."""
    if is_new and not strategy.request_data().get("phone"):
        # Pipeline state is persisted; the user is sent to a form and returns
        # to /complete/<backend>/?partial_token=... which re-enters here.
        return strategy.redirect("/onboarding/phone/")
    # Phone present (or returning user) -> continue the pipeline normally.
    return

Putting it together: the production checklist

• Run workers under a process supervisor so a crash restarts automatically — see our guide to running Celery workers with Supervisor.
• Schedule recurring cleanups (e.g. re-syncing stale profiles) with Celery Beat — see how to create periodic tasks in Celery.
• Set a timeout on every outbound call inside a task, paired with autoretry_for.
• Use acks_late (set earlier) so a task isn't lost if a worker dies mid-run.
• Make every task idempotent before you turn on retries.
• Test the pipeline step in isolation — it's a plain function; assert it calls .delay() with the right ID.

If you're also choosing an identity strategy, our walkthrough of single sign-on with Auth0 in Django covers the hosted-IdP alternative to self-managed social backends.

Frequently Asked Questions

Why not just run the avatar fetch inside the pipeline?

Because the pipeline runs inside the login HTTP request. A remote avatar download adds its full latency to the redirect, and if the provider is slow or down, the login is too. Calling fetch_avatar.delay(user.id, url) from a custom pipeline step returns instantly and lets the work retry on a worker.

Is django-social-auth still usable in 2026?

No. django-social-auth is deprecated and unmaintained. Use social-auth-app-django (the python-social-auth project) and add social_django to INSTALLED_APPS. The pipeline concept is the same — only the package name and import paths changed.

Should I pass the User object to a Celery task?

No — pass user.id and re-query inside the task. Serializing an ORM instance onto the broker risks acting on stale data and bloats the message. Passing the primary key is small, safe, and always current.

How do I stop a retried task from sending two welcome emails?

Make the task idempotent. Check a flag (e.g. profile.welcomed) or the existing state before acting, and set the flag only after success. Combined with autoretry_for, a retried run becomes a no-op once the work is already done.

What if a pipeline step needs extra input from the user?

Use the partial pipeline: decorate the step with @partial and return strategy.redirect(...). python-social-auth persists the state and resumes at the same step when the user returns to the social:complete URL. Reserve this for genuine input gates; everything else should be a Celery task.

Do I still need django-celery or djcelery?

No. Those shims are obsolete. Celery 5.x has native Django support: create a Celery app, call config_from_object('django.conf:settings', namespace='CELERY') and autodiscover_tasks(), then run celery -A proj worker -l info.

Build a login that stays fast under load

Social login is where first impressions are made — it should never feel slow because a provider is having a bad day. Keeping the pipeline lean and pushing every heavy step to idempotent, retrying Celery tasks gives you logins that are fast, resilient, and observable.

MicroPyramid has built Django systems since 2014 — 12+ years and 50+ delivered projects spanning authentication, async pipelines, and AWS infrastructure. If you want help designing or hardening your social-auth and Celery setup, explore our Django development services.