Browse Source

explain when to use a task queue

remove deprecated abstract attr from celery
add explanation of example task
[ci skip]
pull/2055/head
David Lord 8 years ago
parent
commit
d5a88bf0d3
No known key found for this signature in database
GPG Key ID: 7A1C87E3F5BC42A8
  1. 78
      docs/patterns/celery.rst

78
docs/patterns/celery.rst

@ -1,24 +1,27 @@
Celery Based Background Tasks Celery Background Tasks
============================= =======================
Celery is a task queue for Python with batteries included. It used to If your application has a long running task, such as processing some uploaded
have a Flask integration but it became unnecessary after some data or sending email, you don't want to wait for it to finish during a
restructuring of the internals of Celery with Version 3. This guide fills request. Instead, use a task queue to send the necessary data to another
in the blanks in how to properly use Celery with Flask but assumes that process that will run the task in the background while the request returns
you generally already read the `First Steps with Celery immediately.
<http://docs.celeryproject.org/en/latest/getting-started/first-steps-with-celery.html>`_
guide in the official Celery documentation.
Installing Celery Celery is a powerful task queue that can be used for simple background tasks
----------------- as well as complex multi-stage programs and schedules. This guide will show you
how to configure Celery using Flask, but assumes you've already read the
`First Steps with Celery <http://docs.celeryproject.org/en/latest/getting-started/first-steps-with-celery.html>`_
guide in the Celery documentation.
Celery is on the Python Package Index (PyPI), so it can be installed with Install
standard Python tools like :command:`pip` or :command:`easy_install`:: -------
Celery is a separate Python package. Install it from PyPI using pip::
$ pip install celery $ pip install celery
Configuring Celery Configure
------------------ ---------
The first thing you need is a Celery instance, this is called the celery The first thing you need is a Celery instance, this is called the celery
application. It serves the same purpose as the :class:`~flask.Flask` application. It serves the same purpose as the :class:`~flask.Flask`
@ -36,15 +39,18 @@ This is all that is necessary to properly integrate Celery with Flask::
from celery import Celery from celery import Celery
def make_celery(app): def make_celery(app):
celery = Celery(app.import_name, backend=app.config['CELERY_RESULT_BACKEND'], celery = Celery(
broker=app.config['CELERY_BROKER_URL']) app.import_name,
backend=app.config['CELERY_RESULT_BACKEND'],
broker=app.config['CELERY_BROKER_URL']
)
celery.conf.update(app.config) celery.conf.update(app.config)
TaskBase = celery.Task
class ContextTask(TaskBase): class ContextTask(celery.Task):
abstract = True
def __call__(self, *args, **kwargs): def __call__(self, *args, **kwargs):
with app.app_context(): with app.app_context():
return self.run(*args, **kwargs) return self.run(*args, **kwargs)
celery.Task = ContextTask celery.Task = ContextTask
return celery return celery
@ -53,11 +59,12 @@ from the application config, updates the rest of the Celery config from
the Flask config and then creates a subclass of the task that wraps the the Flask config and then creates a subclass of the task that wraps the
task execution in an application context. task execution in an application context.
Minimal Example An example task
--------------- ---------------
With what we have above this is the minimal example of using Celery with Let's write a task that adds two numbers together and returns the result. We
Flask:: configure Celery's broker and backend to use Redis, create a ``celery``
application using the factor from above, and then use it to define the task. ::
from flask import Flask from flask import Flask
@ -68,26 +75,27 @@ Flask::
) )
celery = make_celery(flask_app) celery = make_celery(flask_app)
@celery.task() @celery.task()
def add_together(a, b): def add_together(a, b):
return a + b return a + b
This task can now be called in the background: This task can now be called in the background::
>>> result = add_together.delay(23, 42) result = add_together.delay(23, 42)
>>> result.wait() result.wait() # 65
65
Running the Celery Worker Run a worker
------------------------- ------------
Now if you jumped in and already executed the above code you will be If you jumped in and already executed the above code you will be
disappointed to learn that your ``.wait()`` will never actually return. disappointed to learn that ``.wait()`` will never actually return.
That's because you also need to run celery. You can do that by running That's because you also need to run a Celery worker to receive and execute the
celery as a worker:: task. ::
$ celery -A your_application.celery worker $ celery -A your_application.celery worker
The ``your_application`` string has to point to your application's package The ``your_application`` string has to point to your application's package
or module that creates the `celery` object. or module that creates the ``celery`` object.
Now that the worker is running, ``wait`` will return the result once the task
is finished.

Loading…
Cancel
Save