/
Queue Devices

Queue Devices

The Queue Device is a core component of Connexion and Gateway platforms, and is typically used at least once in every channel. It defines the server-side behavior of each channel as well as exposes messages (and metadata) via the user interface.

There are different queue devices depending on the platform and database used, and you must use the correct queue device for your deployment. When creating or importing a channel, the correct type is automatically selected.

  • Queue Device → Sql Server message repository

  • PostgreSql Queue Device → Postgres message repository

  • Mongo Queue Device → MongoDb message repository (Gateway platform)

image-20250908-184913.png

Behavior

The default behavior of the queue is to accept a message from the upstream device and store it in the associated message database. In this state, the message is Queued (queue type Q). A separate thread reads this message from the database (or from a memory buffer) and pushes the message to each downstream device. Processing metadata (called Processing History) and logs are stored when the message state changes.

By default, queues are ordered (FIFO) - messages are processed in the order they are received. Certain scenarios support out-of-order processing, such as the Delay queue type or batch processing.

Message processing typically results in one of the following:

  • The message passes successfully to each downstream device, and the message is moved to the Processed queue.

  • A downstream device cancels the processing (context.cancel = true) and the message remains in the Queued state. Processing will be retried.

  • A downstream device throws an exception (without a retry policy) or logs an error. This results in the message being moved to the Error queue.

  • A downstream device filters the message (context.FilterMessage) and it is moved to either the Filtered or Errored queue. Filtering removes a message from the processing pipeline along with a user-provided reason (for example, message type not supported).

  • A downstream device Delays message processing. In this scenario, the queue skips the delayed message until a specified amount of time has elapsed, and processes the next Queued message. The Delay queue type typically results in out-of-order message processing.

This page covers queue Configuration / Settings (everything under the Settings tab). To learn about interacting with messages, view the Working With Messages page.

Configuration

The queue device is configured via the Settings top tab and then the category left tabs.

image-20250908-201204.png

The Queue Operation tab has the following configuration:

Empty Sleep Time: When a queue has no Queued messages, the amount of time (in ms) to wait before checking for newly arrived messages. The larger this number, the longer a queued message may wait before being processed. The lower the number, the higher the load on the database. Minimum value is 5000ms (5s).

Maximum Batch Count: The maximum number of queued messages to return from the database. The lower the number, the lower the memory footprint. A higher number means fewer round trips to the database. For queues containing small messages, a higher number is acceptable (for example, 100). For queues with large messages (>1 MB), smaller numbers are better.

Limit Total Batch Size: When set, the payload returned from the database will be capped at the given size. This will limit the amount of memory required for queues with very large messages.

Disable Message Caching: Each queue has a memory-based message cache which can be used under certain conditions. When available, this cache can bypass the need to read messages from the database, lowering the database and application server load. This cache typically uses no more than 5 MB of memory.

Queue Operation: This determines the behavior of the queue.

  • Regular: Messages are processed first-in, first-out, one message at-a-time (ProcessMessageAsync). Messages are stored in the queue as soon as they are received (at the queue device), and dequeue / processing is decoupled from ingestion. A paused queue can accept messages, but will not process them.

  • Synchronous: Messages are not stored in the queue until processing completes. The message is pushed down the channel until it is either successful (in which case it is stored in the queue) or fails (in which case the error is pushed back to the message source).

  • Batch: The same behavior as Regular, except a batch of messages is passed instead of a single message (ProcessMessagesAsync). Device developers can choose to support batching, which can be significantly faster for certain workloads. Read more.

  • Multi-threaded: This is a legacy behavior no longer recommended. Use the Batch mode instead.

Store Message Change History: By default, Connexion stores the received message as well as the message at the end of channel (or until the next queue device). If you have no use for the updated / transformed message, you can disable this feature and reduce storage and horsepower requirements.

Store...Errors Only: Same as above, except message change history will be stored for messages which error.

Maximum Change History Size: Truncate any message larger than this size which is being stored for history / viewing purposes only.

Throttling

Throttling enables you to cap the throughput of a queue. This is done by injecting a minimum delay between messages being processed. This delay can be scheduled using our standard scheduling UI.

Another option is temporarily disable a queue. A disabled queue will not process any messages (but can receive messages).

image-20250909-162828.png

Logging

Each queue has the ability to generate events for specific conditions. These events are processed by the eventing mechanism and can be hooked or acted upon (notify support, etc.). Each event can be scheduled, as message volumes and states may only matter during specific times.

image-20250909-163620.png

Message Retention

By default, all queues will retain messages until the repository-level retention period elapses. After this period, messages are purged as part of the maintenance routines (Queued and Delayed messages are never purged).

image-20250909-164328.png

Queues can override this setting and provide their own retention period. To revert back to using the repository-level setting, clear out the queue-level Message Retention value and save the setting.

image-20250909-164850.png

Two additional options available are Delete Message…On Processed and Delete Message…On Filtered. These are useful when the message content isn’t needed (ie, it’s available in an upstream or downstream system) and you want to minimize the size of the Connexion message databases. All the message metadata is retained, but the content is no longer visible (all standard functionality like message tracing still works).

Message retention values are reported in dashboards and metrics, in both Connexion and Gateway platforms.

External Access

Connexion supports sending and receiving messages to/from other Connexion instances via a proprietary secured protocol. This feature is intended for message transport within an internal network - remote agents are typically used for transporting messages across the WAN.

Read more about the External Access feature.

Display Settings

Choose which columns are visible in the messages view, as well as the number of messages to display (Queue Page Size).

image-20250909-170950.png

Diagnostics

Depending on the database stack in use (Sql Server, Postgres, Mongo), a diagnostics tab may be available. This can be useful for digging into the raw database-provided diagnostic information for a specific queue.

image-20250909-171642.png
The diagnostics tab for the MongoDb queue. Click the Run Diagnostic link to view state information. An interactive console is also available for diagnostics and debugging (super-users only).