Historical mode of gateway
Today I'd like to write about a recently implemented enhancement to the D2000 Gateway process and the transfer of values from remote system archives.
In the blog Communication in test environments, I already presented the functionality of the transparent gateway in 2017. Using it, it is possible to transfer the values of I/O tags (e.g. from the production environment to the test or from the SCADA system to the MES) and then archive them. Even recently, the automatic transfer of the so-called OLDVALs (old values from communication - e.g. saved data from data loggers or profile data from electricity meters) via D2000 Gateway has been implemented.
But one thing still remained problematic: what about script-filled archives? These require some ESL or Java script to process the source data (CSV, XML and other files, or data sources downloaded from the Internet or data inserted via the D2000 REST API). If we need to get the same data to another system, it is necessary to duplicate data processing in the target D2000 system and for some data sources to "double" the source (e.g. CSV files copied from the server and subsequently deleted on the server). Additionally, various shutdowns or outages can cause different readings in each D2000 system. And it may be necessary to import historical data or use another way of filling the archive in the past.
To make it easier for users to solve this problem, we taught the D2000 Gateway Client process to pretend to be a D2000 Archive - and called this functionality Historical mode of gateway.
The new start parameter /ARC is used to activate the historical mode. When started with this parameter, the D2000 Gateway Client will connect to the D2000 Server as an ARC process:
If we create historical values whose parent is this process (NEW.ARC in the picture above) with the same names as those on the source D2000 system (easiest by using XML export from the source system and subsequent XML import to the target system), the D2000 Gateway Client will execute as follow:
1. It transfers the values of the simple historical values published by the D2000 Archiv in the source system, so that if they are connected to schemes or graphs, they will work:
2. Requests for reading data (in graphs or from a script) are forwarded to the source Scada system (the same functionality is used that is used for the needs of the GETOLDVAL command). In addition, archive values are marked as "read-only", so they cannot be edited.
D2000 Gateway Client in historical mode (similar to transparent gateway mode) works as a so-called data diode – it does not allow modification of data in the source system.
Most actions for reading historical values from a script are supported:
Actions for writing to the archive (DELETEARCHDATA, INSERTARCHARR, UPDATEARCHVAL) are not supported due to the functionality of the data diode and the calculation of any archive expression (CALCONDEMANDSTAT) is not supported due to excessive complexity.
In historical mode, the /MON parameter is also supported, which allows you to modify object names. In our example, all configured historical values have names starting with „H.Scada.“ on the target system (so that we can distinguish them from „local“ historical values). With the parameter „/MONScada\.,“ we achieve that the prefix „Scada.“ is deleted (the parameter „/MONScada\.,something“ would transform the name H.Scada.Ainto H.somethingA - the backslash is to suppress the special meaning of the dot in regular expressions).
It is therefore possible, for example, to configure several fictitious ARC processes for several SCADA systems on the MES system and connect the MES system to these SCADA systems using the D2000 Gateway Client processes. Subsequently, we can import historical values, schemes and graphs and display the same data to MES users as seen by SCADA systems - in addition, without the risk of changes (the MES user cannot control or change archive data) and without demands on disk space in the MES system (all readings are performed against SCADA systems). It is possible to set different prefixes in object names for individual SCADA systems, so that it is clear at first glance whether it is a "local" object or an object transferred via the gateway in historical mode.
If someone had a request to arbitrarily modify the names of archive objects (so even the /MON parameter would not be enough), the support for /UID parameter as been also added in the historical mode. It causes the objects in the source and target systems to be matched according to the unique identifier of the object - UID. Attention - for this feature to work, it is also necessary to start the D2000 Gateway Server with the /UID parameter on the source system.
The support for historical mode of gateway was added for D2000 versions 21.1.72 and above.
Conclusion
The historical mode of the gateway is already deployed at one of our customers, where it serves MES users to read script-filled archives from the SCADA system. In the near future, it is planned to be used on a test environment at another customer, where it will be deployed as an instance archive ([2]_SELF.ARC). In addition to the gateway, there will also be a standard instance archive ([1]_SELF.ARC), which will perform archiving with a limited time depth - e.g. 2 months. By changing the active instance, it will be possible to switch reading between the short-term local archive and the archive of the production system.
As can be seen, the historical gateway mode has several interesting uses that enable even better interconnection of multiple systems built on the Ipesoft D2000 real-time application server.