File Writer Device
The File Writer device is used to output messages to the file system, FTP server, S3 bucket, SQS queue, or Azure Blob Storage.
This device includes configuration for the file target, conditions under which files should be written, support for attachments, and built-in purging.
The File Writer offers two distinct modes of operation:
Synchronous: Message(s) are written to a file, and then the file is copied to the target system. No further message processing can occur until the file is successfully copied. For example, if you are copying to an FTP target, and it is currently unavailable, message processing halts until the target is available.Asynchronous: Message(s) are written to local files and an independent thread is used to copy those local files to the target. This mode is usually more performant as it can run the file creation and copying concurrently.
You can also choose if files will be written as text or binary, where base64-encoded string or byte arrays should use the binary output option.
Select the desired Target Type, and input the configuration to connect to the given target. The UI automatically changes depending on which Target Type is selected.
For the File target, you can choose to Use Impersonation if the Connexion service account does not have the required file permissions. There is also a Write Breadcrumbs option which will write a tiny file to the target directory showing the Group, Tab, and Channel which is writing files.
Other target types have a specialized UI for that specific target. For example, the S3 target has the following UI.
The Set Filename option lets you specify how files should be named, and supports a number of different variables (described in detail below). If a date/time variable is used, you have the option to use the date/time the first message was received, or the date/time of the last message written.
You can also enable ZIP compression, with an optional password.
The Temp File Path field lets you override the default path used to work with temporary files. This can be useful if you have limited drive space, or, are running a Connexion cluster and are using a shared path. It may also be useful if you have access to high-performance I/O.
The Obfuscate File Names option is designed to hide any PHI which might accidentally become part of a filename. This feature is automatically enabled if the filename uses the UserContext or UserStorage variables.
Variables
A number of variables are supported in the Filename and Destination fields (where Destination represents some target folder, bucket, or similar concept). These are listed below:
Variable | Description |
|---|---|
| Inserts the day of the year, padded to 3 characters. For example, January 30th is '030'. |
| Inserts the number of seconds elapsed since midnight, padded to 5 characters. For example, 1 AM is '03600'. |
| Inserts an incrementing integer. The counter is reset when the device is started. |
| Inserts the filename from the message context Source Uri property (shown in the SourceUri column of the queue device). |
| Inserts the host computer name. |
| To insert a date/time stamp, use the {d:x} format where x is the string specifying the format. String formats are based on the .NET 4.0 standards which are available at http://msdn.microsoft.com/en-us/library/8kb3ddd4.aspx. yyyy = year For example, {d:yyyyMMddHHmmss} for January 1st, 2012, 1 PM would output 20120101130000. |
| Inserts the value from the MessageContext.UserContext object with the key X. By using a custom code device upstream of the file writer, information can be inserted into the MessageContext.UserContext dictionary which is accessible in the ProcessMessage method. The key used when adding items to the dictionary must match the value specified for X. In the destination folder field, the UserContext only works when the file writer is in 'synchronous' mode and has a single 'file count' condition with a value of 1. For example, in the ProcessMessage method of your custom code device:
and in your file writer:
|
| Inserts the value from the MessageContext.UserStorage object with the key X. UserStorage has the same restrictions as UserContext variables. |
| Inserts the group, tab, or channel name. |
Conditions
Conditions determine how and when a file is created. Files can be created based on:
Maximum file size: Once the reaches a specific size, route messages to a new file.Maximum message count: Once a specific number of messages have been written into a file, start a new file.File age: Once the file reaches a specific age, route messages to a new file.Time of day: Once the time of day reaches a specific value, route messages to a new file.
Multiple conditions can be combined - for example, you can have a maximum message count and a maximum file size.
Attachments
Attachments are additional payloads associated with a message. When your File Writer has the exact condition of Number of messages reaches 1 (ie, one message per file), then the Attachments tab is enabled.
When enabled, you can choose to write one or more attachments, or, write only the attachments. By default, all attachments will be written when the Write Attachments option is selected. If you only wish to write specific attachment(s), use the Attachment Extensions field to choose which will be written.
You can also choose to rename the attachments to match the name of the primary file, with the attachment extension unchanged.
Framing
Choose the optional framing and encoding to use when writing messages. Framing is only used when writing multiple messages to the same file.
Purging
If your target is a file system, you can choose to purge files out of the Destination folder based on age or folder size.
Diagnostics
Use the diagnostics tab to view device state as well as file and directory information. This can be useful when debugging file permission issues.