NoSQL databases and the D2000
In the beginning, let us look at a simple question: is the D2000 a NoSQL database?
Before we categorically answer "Of course not!", let's try to think about what a NoSQL database is.
The first sentence in the Wikipedia article on NoSQL databases talks about what NoSQL is by comparing it to SQL databases - the NoSQL database provides a different mechanism for storing and retrieving data than the table relations used in relational databases:
"A NoSQL (originally referring to “non-SQL“ or “non-relational“) database provides a mechanism for storage and retrieval of data that is modelled in means other than the tabular relations used in relational databases."
Wikipedia lists several differences between NoSQL databases and relational SQL databases. It uses terms like "many" and "multiple" NoSQL databases, or "most" NoSQL databases.
The abbreviation NoSQL itself first meant "non-SQL", later it was redefined as "Not Only SQL".
In his article „NosqlDefinition“, Martin Fowler lists several characteristics common to NoSQL databases but also notes that none of them are definitive.
According to the list available at http://nosql-database.org, the NoSQL category also includes Lotus Notes or Yserial, which is a Python wrapper over sqlite3.
This quick overview suggests that the concept of NoSQL databases encompasses a relatively broad set of technologies and is "defined" rather by differentiating it from the SQL definition.
Let's take a look at the D2000 core and the DODM (Dynamic Object Data Model) technology. According to the description from the documentation:
"The requirement of a fast application response to events in the monitored technological process led to the creation of the technology of data saving and data processing - DODM (Dynamic Object Data Model). This technology combines an object-oriented approach with the high-performance data model of an application operating in real-time. ”
So the DODM is a technology for storing and processing data. The DODM is implemented in the D2000 Server process, which serves as the basis or core of the D2000 system. It does not depend on other D2000 processes, so the D2000 Server itself can be perceived as a "database" that stores data in memory (so it is in-memory) and possibly persist them (object values in the configuration database, event records in the log database). If the D2000 Archive process is running, depending on the configuration of individual objects, it stores them in the archive database as optimized time series - production applications manage the archiving of more than a hundred thousand objects.
The D2000 Server includes strong support for fast distribution of information about data changes, which is available for the D2000 system processes as well as for external processes (using VBApi, OBJApi and Java API interfaces). There is also an ODBC driver for access to the current values of the D2000 system as well as data from the archive of historical values.
By the way, I recently had the opportunity to observe a deployed SCADA system, which was built on an in-memory database but without the support of the distribution of data changes. Instead, the client processes cyclically read data (polling) from in-memory tables. The (im)practical consequence was that, for example, the user interface was able to display data changes at a maximum frequency of once every two seconds. So, when the Boolean value came from the communication changing every second (True/False), the changes were not visible in the user interface!
In contrast, the D2000 client receives push notifications via the API about each change in the value of the object of interest. The D2000 Server is capable of processing and distributing tens to hundreds of thousands of value changes per second. Where do value changes actually come from? The values of some objects (system variables and predefined structured variables) are generated directly by the D2000 Server. The values of other objects are supplied to the system by clients - native or external processes via the above-mentioned API.
In redundant D2000 systems, one or more standby D2000 Servers are connected to the active ("master") D2000 server. The active D2000 server replicates real-time data to standby servers so that they all have an identical image of the data in memory, which can be understood as asynchronous master-slave replication. This is conforming to the description on Wikipedia:
„Most NoSQL stores lack true ACID transactions, although a few databases have made them central to their designs. Instead, most NoSQL databases offer a concept of "eventual consistency" in which database changes are propagated to all nodes "eventually" (typically within milliseconds) so queries for data might not return updated data immediately or might result in reading data that is not accurate, a problem known as stale reads.“
The dynamic object data model is a technology developed by IPESOFT. It defines various relations between objects, but they are of a different nature than relations in SQL databases. An example is an "object is used by another object" relation or a "parent-child" relation. DODM was created in response to specific problems that SCADA or MES class systems must solve. All these facts are in consonance with Wikipedia's description:
„The particular suitability of a given NoSQL database depends on the problem it must solve. Sometimes the data structures used by NoSQL databases are also viewed as "more flexible" than relational database tables“.
Speaking of, Wikipedia lists IBM Domino (formerly Lotus Notes) among NoSQL databases.
A while back, I read a sci-fi story in which an American inventor invented a one-way teleport. The problem was that he didn't know where the things put into the teleport would teleport to. So he started selling it as ... a garbage disposal. Unlike standard waste shredders, it did not discharge mixed waste into the sewer, but teleported them somewhere in space :-)
The inventor was locked up relatively quickly and interrogated. He defended himself by the fact that he was only selling garbage disposals. What the investigator responded, was "Declaring your invention a waste shredder is like opening a beer bottle on the edge of a Polaris nuclear submarine and then declaring Polaris a bottle opener."
So - using this analogy - we can answer the question from the introduction and declare Lotus Notes or the D2000 as specialized NoSQL databases, although their use is much wider. Lotus Notes is also an application, mail and web server and a system for supporting team collaboration. The D2000 is a real-time application server and technology with which it is possible to create large SCADA and MES solutions.
By the way, on the website http://nosql-database.org you can find not only the Lotus Notes in the list of NoSQL databases but also the D2000 ...
And one last note - there is a very interesting blog where the author writes about the return of SQL databases after a decade in which NoSQL databases were promoted. In contrast to the advantages of NoSQL databases (performance, scaling), it poses problems with non-standard (absence of a common query language, denormalization of data causing an increase in their volume, differences of individual databases). As an example, he originally mentions the NoSQL Spanner database developed by Google for their internal needs, which developed into a relational database system with the SQL query language during the years 2010-2017.
Ing. Peter Humaj, www.ipesoft.com