Continuous integration in an archive or how to do a sawtooth diagram

Image Description

Peter Humaj

May 30 2019, 8 min read

A few days ago, a colleague came to me and asked for advice. “I need to sum, more precisely I need to integrate the energy consumption but in a way that at the beginning of every work shift, the number will be zeroed. So, the diagram would be resembling teeth of a saw – it will rise, and rise, and then on the edge of the work shift, it will be zero again. Could this be done in an elegant way – preferably right in the archive without needing to calculate somewhere in the ESL script?”

If the assignment was that he wants an integral for an 8-hour work shift, there would be nothing to think about. A statistical archive that would calculate integral form the primary archive would be simply configured:

Figure 1: When configuring a statistical archive, it is necessary to first enter the archive purpose (statistical) and to choose the archive object of which statistics will be calculated.
Figure 2: We set a period and an offset of the archiving in the tab with time parameters (in the example, there is 1 minute used but in reality, it was 8 hours).
Figure 3: On the last tab, we choose the type of the statistical function – integral and we choose time units (if the I/O tag is in kilowatts, we will have the option of integral in kilowatt-hours, kilowatt-minutes or kilowatt-seconds).
Figure 4: This shows a display of values of the statistical archive in the table – values are with a minute period.

As we can see on the figure above, such archive would be automatically recalculated every minute. However, the colleague’s requirements were different – the integration should run continuously and the user should see also intermediate results and not just one number on the end of the period (8-hour work shift).

So, the statistical archive is not the right choice. The solution might be to use a calculated archive and to give it required properties.

For me to debug the example, I created a line, station with active simulation and a two-second polling period, and the M.MySimulPower I/O tag. This I/O tag will change the value according to time parameters of the station, meaning around every two seconds (and the values will have a sinusoidal course with an amplitude respecting the setting of LL and HL limits on the I/O tag). Afterward, I created the H.MySimulPower primary archive that archives this I/O tag:

Figure 5: This is the primary archive archiving the M.MySimulPower I/O tag value.

Then, I started to create a calculated archive. The integration principle is simple. The integral (or else a sum) is composed of sums of Value * DeltaT where DeltaT is the size of the time interval during which the Value was valid. The first version of the calculated archive looked like this:

Figure 6: This is a calculated archive implementing a simple integral.

The line
_prevVal := %ARC_GetValue(H.MySimulPowerIntegral, @EvalTime - 0.001, @FALSE)
will find out the previous value of the integral.

The time is specified as a time of calculation (represented by the @EvalTime constant) reduced by 1 ms, which is a basic time unit supported by the D2000 system. The @FALSE parameter states that if there is no value available for the required time, the closest older value will be used.

The line
_prevTime:= _prevVal\TIM
will assign into the _prevTime variable the timestamp of the previous value of the integral.

At last, the line
_prevVal + (@EvalTime - _prevTime) * H.MySimulPower
in the FINALLY section will perform the integration – it will add to the previous value of the integral the current value of the integrated quantity H.MySimulPowermultiplied by the size of the time interval (the current time minus the time of the latest integrated value).

Such integral, however, does not meet the requirements of the task because it is permanently increasing. So, how to implement the required zeroing at the beginning of another shift?

For speeding up the debugging, we will not zero after eight hours but our work shifts will be 15-seconds-long. Some would appreciate such work shifts in real life  :-)

How to achieve this? First, we will create an auxiliary archive object H.MyPeriodicTrigger which will have the required period (15 seconds). We will use this to trigger the calculation on the edge of the work shift. This 15-second-long archive can archive anything – more precisely, we have used the system object Day which changes once a day and gives the current day in a month.

Some readers may object that we will unnecessarily fill the archive database – archiving the same value every 15 seconds? This objection can be justified in other systems but not in D2000. Thanks to on-change mode of storing periodic data in the D2000 archive database, only one value will be stored every day.

Figure 7: This is a configuration of a primary periodic archive which archives Day.

The setting of time parameters is also important – the length of the work shift (15 seconds) and storing the end time of period.

Figure 8: These are the time parameters of the H.MyPeriodicTrigger archive object.

How will the calculated archive change? The answer is in the next figure:

Figure 9: This is the calculated archive implementing integral zeroed at the beginning of the interval.

For one thing, we need to include the trigger H.MyPeriodicTriggerinto the calculation (even though we won’t use the value of this object at all but it will trigger the calculation). That is done by the line
_prevVal := H.MyPeriodicTrigger

The next line (which stayed without any change) will rewrite this value with the previous value of the integral.
_prevVal := %ARC_GetValue(H.MySimulPowerIntegral, @EvalTime - 0.001, @FALSE)

Further, lines implementing resetting were added. If the time of the previous integral value is a multiple of a period, the result will be zeroed (because we are calculating the first value in the new work shift). Otherwise nothing else will happen; the previous value will be used (we continue with integrating).
IF %ModTime(_prevTime, 15) = 0 THEN
_prevVal := 0

The last line implementing integration remained without a change.

Figure 10: The diagram displaying values of an adjusted archive shows that maxima are in multiples of 15 seconds and subsequently the next calculation integrates from zero again.

Is this formula final or is there something missing? Two possible improvements come to mind:

·        Marking the choice “Replace Invalid values with 0”, which will cause that coming of an invalid value from communication will not make the integral calculation invalid but zero will be integrated.

·        In a case that value from communication would not change for a long time, users would look at a constant value of the integral. For such case, it is possible to “enliven” the calculation so that the period of the H.MyPeriodicTrigger trigger will be substantially reduced (in this case from 15 seconds to e.g. 5 seconds, in production from 8 hours to 5 seconds). This change will ensure integral calculation once in a chosen period even though the value of the I/O tag itself does not change. So as not to break zeroing at the beginning of the work shift, the work shift (8 hours) must be a multiple of the trigger period. So, 1, 2, 3, 4, 5, 6 or 10 seconds is all right, 7 seconds would not work.

Figure 11: After changing parameters of a simulation to 1 value in 20 seconds, there will be regular 5-second long calculations caused by the H.MyPeriodicTrigger trigger. In the time 14:37, we can see zeroing of the integral and writing “almost zero” caused by the change of I/O tag value closely after 14:37.


The described calculated archive demonstrates the strength of an archive subsystem of the D2000 application server to simply perform users’ calculations – using only the archive and without any external means.

On the Amper Trade Fair 2019, when I was talking with technicians of a renowned Austrian PLCs and control systems supplier and asked them how they would implement the functionality of “calculated archives”, they spoke about an external script written in Python which would periodically read data of primary archive objects, calculate and send the results to an archive.

Other, this time an American producer of SCADA systems, uses periodically run external programs in the C language with functionality similar to aforementioned Python scripts.

Some disadvantages of these approaches are obvious – instead of a simple configuration in a native environment of the SCADA system (using the same syntax as calculated tags or ESL scripts), an application programmer has to use different languages (Python, C) and make efforts to not only write the calculation itself but also to acquire data and write results.

Other disadvantages are more subtle. When upgrading, changing versions or else migrating on another platform, external calculations need to be preserved or recompiled (because of the changes in API or because of the linking with new API library). It is necessary to keep and manage source codes somewhere. In redundant systems, it is necessary to distribute current versions of external scripts or binary files to two or more servers and keep in mind where the newest source codes are.

Moreover, moving calculations outside the SCADA system will cause a loss of linkage between source and calculated archive objects. One of the key features of the D2000 system called referential integrity is based on keeping information about the linkage among all system objects. The information about every object saying  which objects it uses and which objects use this object, is available at all times. Deleting an object in use thus cannot happen. At the same time, the referential integrity is a priceless aid when debugging and looking for errors (it gives answers to questions such as “How and from where did this value come?”) and when building and especially maintainig solutions with a range of tens of thousands to hundreds of thousand objects.

With this practical example, I wanted to demonstrate how you can relatively easily solve non-trivial problems in D2000 – with the use of built-in functionality of calculated archives and without the need to resort to a crutch such as external programs.

Subscription was successful

Thank you for submitting form.

Image Description

Your message was successfully sent.

Thank you for submitting the form.

Image Description

Your message was successfully sent.

Thank you for submitting the form.

Image Description

Your message was successfully sent.

Thank you for submitting the form.

Image Description