Tags: AppsDevTeam/background-queue
Tags
Allow dynamic AMQP arguments based on queue name Introduces an optional `queueArguments` parameter to the `Manager` constructor. This enables applying specific AMQP arguments, such as `x-single-active-consumer`, to queues whose names contain a defined string. This provides more granular and flexible control over queue behavior without requiring separate queue parameter definitions for each unique configuration.
Handle deadlocks in job coalescing The `markCoalescedJobsRedundant` operation can encounter transient database deadlocks (1213) during concurrent writes. This could leave jobs stuck in a PROCESSING state. This change adds a retry mechanism with a back-off delay for the coalesce UPDATE to overcome these transient deadlocks. Additionally, a new index `(serial_group, coalesce_threshold)` is added to optimize the WHERE clause, reducing lock contention and preventing deadlocks by allowing targeted index scans.
Lower background queue job fetch limit Reduces the maximum number of background jobs fetched and processed per iteration from 10,000 to 1,000. While the previous increase to 10,000 aimed to reduce database round-trips, processing such a large batch concurrently could still lead to excessive memory consumption for specific job types, resulting in out-of-memory errors. This adjustment provides a better balance between database efficiency and peak memory usage during job execution.
Implement job coalescing for background queue Allows jobs to mark overlapping, unstarted jobs within the same serial group as REDUNDANT. This prevents redundant processing, for example, when a "recalculate from X" job covers the work of later "recalculate from Y (where Y >= X)" jobs.
Improve memory handling for background queue processing Addresses potential out-of-memory errors when processing a large number of background jobs. * Limits the number of background jobs fetched per iteration to 10,000. * Increases the PHP memory limit for the console `process` command to 1GB.
PreviousNext