About Store-and-Forward

Application Server communicates with Historian Server through the Historian Client Access Layer (HCAL). HCAL can establish and maintain a connection to one or more historians either synchronously or asynchronously. If a disconnection from the historian occurs, HCAL attempts to restore the connection.

If HCAL cannot communicate with the historian, all data currently being processed can be stored locally on the computer running HCAL. This hard drive location is called the store-and-forward path. Historical data is stored until the threshold capacity of the path is reached or communication to the historian is restored. In the event that all store-forward disk capacity is used to store historical data, no more data is stored. An error message is logged. Remote store-and-forward paths are not supported.

HCAL can go into store-and-forward mode even if it has never been connected to the historian. After the tags are configured successfully on the historian, current data will start to be sent, along with the locally-stored data.

If an AppEngine is deployed before Historian Server is started, history data can be stored locally by HCAL until the objects successfully register with the Historian Server.

In a Galaxy with several engines configured for historization, when all engines come out of store- forward at the same time, all of them will try to send store-forward data to historian simultaneously. If not controlled, it could result in network congestion or historian overload. All engines could go back into store-forward again.

To prevent this problem, before the engine can start sending store-forward data, it will send a request to historian which will give permission based on its workload. The engine will start sending store-forward data once permission is granted.

To support AppEngines configured as a redundant pair, HCAL contacts a component named HCAP to create a second copy of the store-forward repository on the partner computer. This way, when a fail-over or recovery occurs, the store-forward data is already duplicated on the partner node. There are TWO store-forward repositories, one for each Historian Server.

Store-and-Forward Configuration

You can specify how disconnects between Application Server and the Historian Server should be reflected in the data until the disconnect period can be backfilled with store-and-forward data.

If you select the Enable late data option for an AppEngine, NULL values are injected into the data stream for the disconnect period. For a trend, this means that a line gap appears during the period of NULL values. The tag remains in store-and-forward mode until the timestamps become greater than the startup time of the server or the time that the connection was restored. If you do not enable this option, NULL values are not injected and no gap is shown in client-side trends.

Storing Data through HCAL

HCAL uses a local instance of the storage subsystem for store-and-forward data. HCAL can go into store-and-forward mode even if it is not connected to the historian. Store-and-forward is only supported for streamed original data if the server is not available. The following parameters are used to configure store-and-forward for HCAL.

On WinPlatform

Store-and-Forward Directory

This is the folder on disk where the local data is stored. The path for this folder is configured in WinPlatform General settings. The same store-forward path is used for all Engines on the Platform. If the configured directory path is invalid or unreachable, then the Platform will use the default location on the local drive.

Each Engine's store-forward repository is placed inside a sub-directory inside the main store- forward directory. The name of the subdirectory is the same as the unique tagname of the Engine or Platform.

The following is the store-and-forward directory structure created for a Galaxy where the WinPlatform (GRPlatform) and the engine (AppEngine1) both are enabled for storage to Historian (X01STD). The custom store-and-orward directory was configured as C:\S&F.

 On the Engine

If an HCAL client will be operating in a shared network environment, you can limit the bandwidth usage for HCAL communications to minimize the impact on other applications. This bandwidth limit applies to all types of data sent from HCAL. If the limit is set below what is required for the current load, HCAL goes into store-and-forward mode. There are two parameters defined here.

Enable compression – If enabled, history data is compressed before it is sent over the network to the Historian Server. This is used to reduce network utilization, but enabling this setting will increase the CPU load.

Throttling network bandwidth – Specifies limit of bandwidth usage, in kbps, for network communication used by HCAL when communicating with Historian Server. A value of 0 disables this feature (default). For more information on estimating your bandwidth needs, see the performance and sizing recommendations for the Historian Server in the Wonderware System Platform Installation Guide.

Store-Forward Threshold

When the free disk space on the disk configured for store-and-forward reaches the minimum free disk space configured, the storage subsystem engine switches to read-only mode to prevent total disk consumption and periodically logs error messages in the ArchestrA Logger.

Store-Forward Minimum Duration

When the network connection is unstable, HCAL will go into store-and-forward mode and stay in store-and-forward mode for the duration specified. This avoids the situation where the client goes into and comes out of store-and-forward too often due to an intermittent network connection. This value is configured on the Engine.

Engine Parameters to Monitor Store-and-Forward

• Engine.Historian.InStoreForward – If set to TRUE, HCAL is in store-and-forward mode. In a redundant historian setup, the InStoreForward attribute is TRUE if either one of the historians is in store-and-forward mode.
• Engine.Historian.StoreForwardProblem – If set to TRUE, the HCAL on the active engine is unable to store data while in store-and-forward mode.
• Engine.Historian.StoreForwardSpaceAvailable – The available storage space, in MB, for the HCAL of the active engine.
• Engine.Historian.StoreFwdDataLost – Set to TRUE when HCAL is in store-and-forward mode, the store-and-forward deletion threshold has been exceeded, and history data has been lost. This value remains TRUE until the connection to the historian has been re- established and store-and-forward is no longer in effect.
• Engine.Historian.StoreForwardDeletionThreshold – The size, in MB, of free space to reserve on the HCAL store-and-forward disk. The space designated will not be used during store-and-forward. This value cannot be a negative number
• Engine.Historian.StoreForwardMinDuration – The minimum duration, in seconds, for HCAL to function in store-and-forward mode. HCAL will function in store-and-forward mode for this length of time even if the condition that caused HCAL to function in store- and-forward mode no longer exists.
• Engine.Historian.BandwidthLimit – Specifies limit of bandwidth usage, in kbps, for network communication used by HCAL when communicating with Historian Server. A value of 0 disables this feature (default). For more information on estimating your bandwidth needs, see the performance and sizing recommendations for the Historian Server in the Wonderware System Platform Installation Guide.
• Engine.Historian.CompressionEnable – If enabled, history data is compressed before it is sent over the network to Historian Server. This is used to reduce network utilization, but enabling this setting will increase the CPU load.