Move Celery check task to the high priority queue

Description

Currently, LMS uses 3 Celery workers: `lms_default_1`, `lms_high_1`, and `lms_high_mem_1`. Each Celery worker sends messages to a single queue: `edx.core.default`, `edx.core.high` and `edx.core.high_mem`, respectively.
The number of child processes per Celery worker is set to 1. Due to this configuration, any task in a queue blocks all other tasks.

Currently, the Celery check task submitted by the /heartbeat?extended LMS HTTP API endpoint runs in the default queue. When some slow task (eg course grades creation) is sent to the default queue, it will block the Celery check task, which will expire and the heartbeat endpointwill fail. This patch moves the task to another queue which has only shorter tasks and in which this problem will not occur.

Use a Django setting for the Celery check task routing key so that it can be overriden by individual OpenEDX instances via JSON env files.

*JIRA tickets*: Implements

*Sandbox URL*: TBD - sandbox is being provisioned.

*Testing instructions*:

1. Load the heartbeat endpoint.
2. See that it runs in the high priority queue.

*Reviewers*

  • [ ] (@giovannicimolin)

  • [ ] edX reviewer[s] TBD

Assignee

Fred Smith

Reporter

Open Source Pull Request Bot

Labels

Contributor Name

Josh McLaughlin

Repo

edx/edx-platform

Customer

Epic Link

None

OSCM Assignee

None

Platform Map Area (Levels 1 & 2)

Developer Experiences

Platform Map Area (Levels 3 & 4)

None

Blended Hour Utilization Percentage

None

Priority

Unset
Configure