Celery

Celery is an open source software that is used to implement an asynchronous task queue. Although it is written in Python, it can be used in any programming language. Task queue, based on distributed messaging, although it is mainly focused on real-time operation, it also allows pre-scheduled operations. It is a kind of mechanism that deals with the organization and distribution of work between threads or machines. Its data inputs in Celery form a single unit of work called a task. The software constantly monitors the queue and checks for more tasks to run on one or more servers simultaneously using multiprocessing. It also allows delegation of long-running tasks while responding quickly to a request. So, it can handle tasks such as sending e-mails to a specific user, or it can update data internally in its system after a task is completed.

Celery - a task queue based on distributed messages

Celery uses messages in the process of communicating the classic producer/consumer model. Both producers (the system involved in placing a task in the task queue) and consumers (workers/users) check the beginning of the task queue for pending tasks, and then select and execute the first one. The task queue that mediates the communication between the producer and the consumer is the so-called broker. Also, workers themselves can add new tasks to the queue, so they can act as a producer.
Celery takes care of all the project control logic, and it is software that is easy and quick to set up. It includes a number of tools to automate and facilitate monitoring of the task queue. Moreover, it is easy to integrate with Django, Flask or Pylons. It also allows changing its behavior or extending its functionality through developers' use of appropriate webhooks, which here means customized return executions. What's more, webhooks elements use the HTTP protocol, which allows them to integrate with web services without creating new structures in Celery. Due to the fact that implementation-related task queues are not among the simplest parts of Python, understanding this tool can be difficult for novice programmers.