셀러리 / bunnymq에서 보류중인 모든 작업 삭제

셀러리 / bunnymq에서 보류중인 모든 작업 삭제

---

task_id각 작업에 대한 정보 를 모르고 보류중인 모든 작업을 어떻게 삭제 합니까?

---

로부터 문서 :

$ celery -A proj purge

또는

from proj.celery import app
app.control.purge()

(편집 : 현재 방법으로 업데이트되었습니다.)

---

셀러리 3.0 이상 :

$ celery purge

특정 대기열을 제거하려면

$ celery -Q queue_name purge

---

Celery 2.x 및 3.x의 경우 :

예를 들어 -Q 매개 변수와 함께 worker를 사용하여 큐를 정의하는 경우

celery worker -Q queue1,queue2,queue3

다음, celery purge당신이 그것에 큐 PARAMS을 통과 할 수 없기 때문에, 작동하지 않습니다. 기본 대기열 만 삭제합니다. 해결책은 다음과 --purge같은 매개 변수로 작업자를 시작하는 것입니다 .

celery worker -Q queue1,queue2,queue3 --purge

그러나 이것은 노동자를 운영 할 것입니다.

다른 옵션은 셀러리의 amqp 부속 명령을 사용하는 것입니다

celery amqp queue.delete queue1
celery amqp queue.delete queue2
celery amqp queue.delete queue3

---

셀러리 3 이상에서 :

CLI :

$ celery -A proj purge

프로그래밍 방식으로 :

>>> from proj.celery import app
>>> app.control.purge()

http://docs.celeryproject.org/en/latest/faq.html#how-do-i-purge-all-waiting-tasks

---

그 발견 celery purge내 더 복잡한 셀러리 설정 작동하지 않습니다. 다른 목적으로 여러 개의 명명 된 대기열을 사용합니다.

$ sudo rabbitmqctl list_queues -p celery name messages consumers
Listing queues ...  # Output sorted, whitespaced for readability
celery                                          0   2
celery@web01.celery.pidbox                      0   1
celery@web02.celery.pidbox                      0   1
apns                                            0   1
apns@web01.celery.pidbox                        0   1
analytics                                       1   1
analytics@web01.celery.pidbox                   0   1
bcast.361093f1-de68-46c5-adff-d49ea8f164c0      0   1
bcast.a53632b0-c8b8-46d9-bd59-364afe9998c1      0   1
celeryev.c27b070d-b07e-4e37-9dca-dbb45d03fd54   0   1
celeryev.c66a9bed-84bd-40b0-8fe7-4e4d0c002866   0   1
celeryev.b490f71a-be1a-4cd8-ae17-06a713cc2a99   0   1
celeryev.9d023165-ab4a-42cb-86f8-90294b80bd1e   0   1

첫 번째 열은 큐 이름이고 두 번째 열은 큐에서 대기중인 메시지 수이고 세 번째 열은 해당 큐의 리스너 수입니다. 대기열은 다음과 같습니다.

• 셀러리-표준 dem 등성 셀러리 작업 대기열
• apns-Push 등원이 아닌 Apple 푸시 알림 서비스 작업 대기열
• 분석-야간 야간 분석을위한 대기열
• * .pidbox-종료 및 재설정과 같은 작업자 명령에 대한 대기열, 작업 자당 하나씩 (셀러리 작업자 2 명, apns 작업자 1 명, 분석 작업자 1 명)
• bcast. *-브로드 캐스트 큐 (큐를 잡는 것이 아니라 큐를 수신하는 모든 작업자에게 메시지 보내기)
• celeryev. *-작업 분석보고를위한 Celery 이벤트 큐

The analytics task is a brute force tasks that worked great on small data sets, but now takes more than 24 hours to process. Occasionally, something will go wrong and it will get stuck waiting on the database. It needs to be re-written, but until then, when it gets stuck I kill the task, empty the queue, and try again. I detect "stuckness" by looking at the message count for the analytics queue, which should be 0 (finished analytics) or 1 (waiting for last night's analytics to finish). 2 or higher is bad, and I get an email.

celery purge offers to erase tasks from one of the broadcast queues, and I don't see an option to pick a different named queue.

Here's my process:

$ sudo /etc/init.d/celeryd stop  # Wait for analytics task to be last one, Ctrl-C
$ ps -ef | grep analytics  # Get the PID of the worker, not the root PID reported by celery
$ sudo kill <PID>
$ sudo /etc/init.d/celeryd stop  # Confim dead
$ python manage.py celery amqp queue.purge analytics
$ sudo rabbitmqctl list_queues -p celery name messages consumers  # Confirm messages is 0
$ sudo /etc/init.d/celeryd start

---

In Celery 3+

http://docs.celeryproject.org/en/3.1/faq.html#how-do-i-purge-all-waiting-tasks

CLI

Purge named queue:

 celery -A proj amqp queue.purge <queue name>

Purge configured queue

celery -A proj purge

I’ve purged messages, but there are still messages left in the queue? Answer: Tasks are acknowledged (removed from the queue) as soon as they are actually executed. After the worker has received a task, it will take some time until it is actually executed, especially if there are a lot of tasks already waiting for execution. Messages that are not acknowledged are held on to by the worker until it closes the connection to the broker (AMQP server). When that connection is closed (e.g. because the worker was stopped) the tasks will be re-sent by the broker to the next available worker (or the same worker when it has been restarted), so to properly purge the queue of waiting tasks you have to stop all the workers, and then purge the tasks using celery.control.purge().

So to purge the entire queue workers must be stopped.

---

1. To properly purge the queue of waiting tasks you have to stop all the workers (http://celery.readthedocs.io/en/latest/faq.html#i-ve-purged-messages-but-there-are-still-messages-left-in-the-queue):

$ sudo rabbitmqctl stop

or (in case RabbitMQ/message broker is managed by Supervisor):

$ sudo supervisorctl stop all

2. ...and then purge the tasks from a specific queue:

$ cd <source_dir>
$ celery amqp queue.purge <queue name>

3. Start RabbitMQ:

$ sudo rabbitmqctl start

or (in case RabbitMQ is managed by Supervisor):

$ sudo supervisorctl start all

---

celery 4+ celery purge command to purge all configured task queues

celery -A *APPNAME* purge

programmatically:

from proj.celery import app
app.control.purge()

all pending task will be purged. Reference: celerydoc

---

If you want to remove all pending tasks and also the active and reserved ones to completely stop Celery, this is what worked for me:

from proj.celery import app
from celery.task.control import inspect, revoke

# remove pending tasks
app.control.purge()

# remove active tasks
i = inspect()
jobs = i.active()
for hostname in jobs:
    tasks = jobs[hostname]
    for task in tasks:
        revoke(task['id'], terminate=True)

# remove reserved tasks
jobs = i.reserved()
for hostname in jobs:
    tasks = jobs[hostname]
    for task in tasks:
        revoke(task['id'], terminate=True)

참고URL : https://stackoverflow.com/questions/7149074/deleting-all-pending-tasks-in-celery-rabbitmq