OPC UA is a 100% open source protocol found in many industrial networks. Much like the Apache Foundation stewards the Hadoop project the OPC Foundation https://opcfoundation.org/ is the steward of the OPC UA standard. You can access the standard at https://opcfoundation.org/developer-tools/specifications-unified-architecture for the standard.
In addition to the standard itself, the OPC foundation has released implementations of clients, data servers and discovery servers for OPC UA in Java, .NET, and C. The Java repo can be found here OPC UA servers. https://github.com/OPCFoundation/UA-Java
The following Nifi Service & Processor bundle can be used to retrieve data from OPC UA severs. https://github.com/wadesalazar/NIFI-OPCUA Information on building the bundle will be provided in a separate article
To use the service, start by dropping the GetNodeIds onto the pallet
Most OPC servers provide a properties dialog that provides the information necessary to configure the GetNodeIds processor and its companion OPC UA Service. The following example properties are from an instance of KepServerEx's OPC UA Server
In this dialog there are 3 endpoints associated with the example server. One for each NIC, 10.0.2.6 and 192.168.189.10, and one for the localhost or 127.0.0.1. Each is using port 49320 for incoming communications. The security column shows the security handshaking mechanisms supported by the endpoint.
Once you have this information for your OPC UA server return to Nifi, right click the GetNodeIDs processor and select configure. On the properties tab select value box for the OPC UA Sevrvice property field and select Create new service on the following dialog and to create a new instance of the StandardOPCUAService
Access the Process Group Configuration by selecting the black arrow to the right of the value box of the OPC UA Service property.
Edit the newly created StandardOPCUAService and enter the information corresponding to the endpoint of your OPC UA server. Take care to ensure the security setting here matches one of one of the available security modes on the server. If you have been give a certificate for authenticating your client with the OPC UA server, you must place the certificate in a folder that nifi can access. Provide the complete path and filename to your certificate in the Certificate field. If you do not have a certificate leave this property field blank and Nifi will create a certificate for you, as they are required for OPC UA authentication. The application name is arbitrary but is used to identify your client in various dialogs on the server. Make this something you will recognize in a list of other clients.
Once the configuration of the controller service is complete then save and start the controller.
Returning to the GetNodeId configuration dialog. The StandardOPCUAService you just configured should be selected in the OPC UA Service property field. Recursive Depth defines how may branches "deep" the processor will browse into the OPC server's namespace. WARNING!!! The namespace can be both very large and may have circular references start this processors off with the default setting of 0 and increase it only after successful reads. The Starting Node property defines the starting node when browsing the server's namespace. If you do not know where to start browsing, leave the field blank and the browse will start from the designated root node. On the scheduling tab set the Run Schedule to something appropriate for your use case greater than 250ms. On the Settings tab select both the Failure and Success checkbox under Automatically Terminate Relationships, we will change this in following steps.
Select accept and transition the processor to "Run". You should the metrics shown on the face of the processor change indicating that a flowfile has been generated and written out that is greater that 1B. This is a successful read of the name space
Right clicking the processor and selecting Data provenance will allow you to inspect the contents of the newly generated flowfile. You should see something like the following
Most of the nodes starting with NSU are internall nodes that do not contain data we are interested in. Somewhere in your name space you will find the tags that contain the data you would like to retrieve. For this example we are looking for the Nodes under the "Simulation Examples Functions".
Now we need to refine the GetNodeIDs processor so that only these items are returned each time the processor is triggered. To do this we change the starting node to ns=2;s=Simulation Examples.Functions and the Recursive Depth to 0.
Now we have the information needed to query the OPC server for time stamped values for each NodeID in our list. The list can be saved to disk or keep only in Nifi's memory. Either way it is a best practice to periodically check the OPC Server for changes in the name space as in some applications this happens frequently.
Next add a split text processor and configure it as follows to split each line of this flowfile into separate flow files.
Now add the GetValue OPC processor to the pallet and connect it to the "Splits" output from the split text processor. There is only one configuration parameter available on the Properties tab of the GetValue processor. For this field select the OPC UA Service created earlier. On the settings tab select both Failure and Success for initial testing.
Select apply and transitioning the whole data flow to run.
The output data from the GetValue processor is formatted as lines of CSV that can be merged into a single document and treated as a CSV. Future iterations of the processor will support various serialization techniques to avoid processing text in later flows.
