导航

你正在阅读 Celery 3.1 的文档。开发版本文档见： 此处.

Running the worker as a daemon

Celery does not daemonize itself, please use one of the following daemonization tools.

Generic init scripts

See the extra/generic-init.d/ directory Celery distribution.

This directory contains generic bash init scripts for the celery worker program, these should run on Linux, FreeBSD, OpenBSD, and other Unix-like platforms.

Init script: celeryd

Usage:/etc/init.d/celeryd {start|stop|restart|status}
Configuration file:
 /etc/default/celeryd

To configure this script to run the worker properly you probably need to at least tell it where to change directory to when it starts (to find the module containing your app, or your configuration module).

The daemonization script is configured by the file /etc/default/celeryd, which is a shell (sh) script. You can add environment variables and the configuration options below to this file. To add environment variables you must also export them (e.g. export DISPLAY=":0")

Superuser privileges required

The init scripts can only be used by root, and the shell configuration file must also be owned by root.

Unprivileged users do not need to use the init script, instead they can use the celery multi utility (or celery worker –detach):

$ celery multi start worker1 \
    --pidfile="$HOME/run/celery/%n.pid" \
    --logfile=""$HOME/log/celery/%n.log"

$ celery multi restart worker1 --pidfile="$HOME/run/celery/%n.pid"

$ celery multi stopwait worker1 --pidfile="$HOME/run/celery/%n.pid"

Example configuration

This is an example configuration for a Python project.

/etc/default/celeryd:

# Names of nodes to start
#   most will only start one node:
CELERYD_NODES="worker1"
#   but you can also start multiple and configure settings
#   for each in CELERYD_OPTS (see `celery multi --help` for examples).
CELERYD_NODES="worker1 worker2 worker3"

# Absolute or relative path to the 'celery' command:
CELERY_BIN="/usr/local/bin/celery"
#CELERY_BIN="/virtualenvs/def/bin/celery"

# App instance to use
# comment out this line if you don't use an app
CELERY_APP="proj"
# or fully qualified:
#CELERY_APP="proj.tasks:app"

# Where to chdir at start.
CELERYD_CHDIR="/opt/Myproject/"

# Extra command-line arguments to the worker
CELERYD_OPTS="--time-limit=300 --concurrency=8"

# %N will be replaced with the first part of the nodename.
CELERYD_LOG_FILE="/var/log/celery/%N.log"
CELERYD_PID_FILE="/var/run/celery/%N.pid"

# Workers should run as an unprivileged user.
#   You need to create this user manually (or you can choose
#   a user/group combination that already exists, e.g. nobody).
CELERYD_USER="celery"
CELERYD_GROUP="celery"

# If enabled pid and log directories will be created if missing,
# and owned by the userid/group configured.
CELERY_CREATE_DIRS=1

Example Django configuration

Django users now uses the exact same template as above, but make sure that the module that defines your Celery app instance also sets a default value for DJANGO_SETTINGS_MODULE as shown in the example Django project in First steps with Django.

Available options

  • CELERY_APP

    App instance to use (value for --app argument). If you’re still using the old API, or django-celery, then you can omit this setting.

  • CELERY_BIN

    Absolute or relative path to the celery program. Examples:

    • celery
    • /usr/local/bin/celery
    • /virtualenvs/proj/bin/celery
    • /virtualenvs/proj/bin/python -m celery
  • CELERYD_NODES

    List of node names to start (separated by space).

  • CELERYD_OPTS

    Additional command-line arguments for the worker, see celery worker –help for a list. This also supports the extended syntax used by multi to configure settings for individual nodes. See celery multi –help for some multi-node configuration examples.

  • CELERYD_CHDIR

    Path to change directory to at start. Default is to stay in the current directory.

  • CELERYD_PID_FILE

    Full path to the PID file. Default is /var/run/celery/%N.pid

  • CELERYD_LOG_FILE

    Full path to the worker log file. Default is /var/log/celery/%N.log

  • CELERYD_LOG_LEVEL

    Worker log level. Default is INFO.

  • CELERYD_USER

    User to run the worker as. Default is current user.

  • CELERYD_GROUP

    Group to run worker as. Default is current user.

  • CELERY_CREATE_DIRS

    Always create directories (log directory and pid file directory). Default is to only create directories when no custom logfile/pidfile set.

  • CELERY_CREATE_RUNDIR

    Always create pidfile directory. By default only enabled when no custom pidfile location set.

  • CELERY_CREATE_LOGDIR

    Always create logfile directory. By default only enable when no custom logfile location set.

Init script: celerybeat

Usage:/etc/init.d/celerybeat {start|stop|restart}
Configuration file:
 /etc/default/celerybeat or /etc/default/celeryd

Example configuration

This is an example configuration for a Python project:

/etc/default/celerybeat:

# Absolute or relative path to the 'celery' command:
CELERY_BIN="/usr/local/bin/celery"
#CELERY_BIN="/virtualenvs/def/bin/celery"

# App instance to use
# comment out this line if you don't use an app
CELERY_APP="proj"
# or fully qualified:
#CELERY_APP="proj.tasks:app"

# Where to chdir at start.
CELERYBEAT_CHDIR="/opt/Myproject/"

# Extra arguments to celerybeat
CELERYBEAT_OPTS="--schedule=/var/run/celerybeat-schedule"

Example Django configuration

You should use the same template as above, but make sure the DJANGO_SETTINGS_MODULE variable is set (and exported), and that CELERYD_CHDIR is set to the projects directory:

export DJANGO_SETTINGS_MODULE="settings"

CELERYD_CHDIR="/opt/MyProject"

Available options

  • CELERY_APP

    App instance to use (value for --app argument).

  • CELERYBEAT_OPTS

    Additional arguments to celerybeat, see celerybeat –help for a list.

  • CELERYBEAT_PID_FILE

    Full path to the PID file. Default is /var/run/celeryd.pid.

  • CELERYBEAT_LOG_FILE

    Full path to the celeryd log file. Default is /var/log/celeryd.log

  • CELERYBEAT_LOG_LEVEL

    Log level to use for celeryd. Default is INFO.

  • CELERYBEAT_USER

    User to run beat as. Default is current user.

  • CELERYBEAT_GROUP

    Group to run beat as. Default is current user.

  • CELERY_CREATE_DIRS

    Always create directories (log directory and pid file directory). Default is to only create directories when no custom logfile/pidfile set.

  • CELERY_CREATE_RUNDIR

    Always create pidfile directory. By default only enabled when no custom pidfile location set.

  • CELERY_CREATE_LOGDIR

    Always create logfile directory. By default only enable when no custom logfile location set.

Usage systemd

Service file: celery.service

Usage:systemctl {start|stop|restart|status} celery.service
Configuration file:
 /etc/conf.d/celery

To create a temporary folders for the log and pid files change user and group in /usr/lib/tmpfiles.d/celery.conf. To configure user, group, chdir change settings User, Group and WorkingDirectory defines in /usr/lib/systemd/system/celery.service.

Example configuration

This is an example configuration for a Python project:

/etc/conf.d/celery:

# Name of nodes to start
# here we have a single node
CELERYD_NODES="w1"
# or we could have three nodes:
#CELERYD_NODES="w1 w2 w3"

# Absolute or relative path to the 'celery' command:
CELERY_BIN="/usr/local/bin/celery"
#CELERY_BIN="/virtualenvs/def/bin/celery"

# How to call manage.py
CELERYD_MULTI="multi"

# Extra command-line arguments to the worker
CELERYD_OPTS="--time-limit=300 --concurrency=8"

# %N will be replaced with the first part of the nodename.
CELERYD_LOG_FILE="/var/log/celery/%N.log"
CELERYD_PID_FILE="/var/run/celery/%N.pid"

Example Django configuration

This is an example configuration for those using django-celery:

# Name of nodes to start
# here we have a single node
CELERYD_NODES="w1"
# or we could have three nodes:
#CELERYD_NODES="w1 w2 w3"

# Absolute path to "manage.py"
CELERY_BIN="/opt/Myproject/manage.py"

# How to call manage.py
CELERYD_MULTI="celery multi"

# Extra command-line arguments to the worker
CELERYD_OPTS="--time-limit=300 --concurrency=8"

# %N will be replaced with the first part of the nodename.
CELERYD_LOG_FILE="/var/log/celery/%N.log"
CELERYD_PID_FILE="/var/run/celery/%N.pid"

To add an environment variable such as DJANGO_SETTINGS_MODULE use the Environment in celery.service.

Troubleshooting

If you can’t get the init scripts to work, you should try running them in verbose mode:

# sh -x /etc/init.d/celeryd start

This can reveal hints as to why the service won’t start.

If the worker starts with “OK” but exits almost immediately afterwards and there is nothing in the log file, then there is probably an error but as the daemons standard outputs are already closed you’ll not be able to see them anywhere. For this situation you can use the C_FAKEFORK environment variable to skip the daemonization step:

C_FAKEFORK=1 sh -x /etc/init.d/celeryd start

and now you should be able to see the errors.

Commonly such errors are caused by insufficient permissions to read from, or write to a file, and also by syntax errors in configuration modules, user modules, 3rd party libraries, or even from Celery itself (if you’ve found a bug, in which case you should report it).

CentOS

In CentOS we can take advantage of built-in service helpers, such as the pid-based status checker function in /etc/init.d/functions. See the sample script in http://github.com/celery/celery/tree/3.1/extra/centos/.

上一个主题

Tutorials

下一个主题

Debugging Tasks Remotely (using pdb)

本页

导航

© 版权所有 2009-2013, Ask Solem & Contributors.