Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Labels (1)
Contributor

OPC UA is a 100% open source protocol found in many industrial networks. Much like the Apache Foundation stewards the Hadoop project the OPC Foundation https://opcfoundation.org/ is the steward of the OPC UA standard. You can access the standard at https://opcfoundation.org/developer-tools/specifications-unified-architecture for the standard.

In addition to the standard itself, the OPC foundation has released implementations of clients, data servers and discovery servers for OPC UA in Java, .NET, and C. The Java repo can be found here OPC UA servers. https://github.com/OPCFoundation/UA-Java

The following Nifi Service & Processor bundle can be used to retrieve data from OPC UA severs. https://github.com/wadesalazar/NIFI-OPCUA Information on building the bundle will be provided in a separate article

To use the service, start by dropping the GetNodeIds onto the pallet

13965-1.png

Most OPC servers provide a properties dialog that provides the information necessary to configure the GetNodeIds processor and its companion OPC UA Service. The following example properties are from an instance of KepServerEx's OPC UA Server

13967-opc-settings.png

In this dialog there are 3 endpoints associated with the example server. One for each NIC, 10.0.2.6 and 192.168.189.10, and one for the localhost or 127.0.0.1. Each is using port 49320 for incoming communications. The security column shows the security handshaking mechanisms supported by the endpoint.

Once you have this information for your OPC UA server return to Nifi, right click the GetNodeIDs processor and select configure. On the properties tab select value box for the OPC UA Sevrvice property field and select Create new service on the following dialog and to create a new instance of the StandardOPCUAService

13969-addcontrollerservice.png

Access the Process Group Configuration by selecting the black arrow to the right of the value box of the OPC UA Service property.

Edit the newly created StandardOPCUAService and enter the information corresponding to the endpoint of your OPC UA server. Take care to ensure the security setting here matches one of one of the available security modes on the server. If you have been give a certificate for authenticating your client with the OPC UA server, you must place the certificate in a folder that nifi can access. Provide the complete path and filename to your certificate in the Certificate field. If you do not have a certificate leave this property field blank and Nifi will create a certificate for you, as they are required for OPC UA authentication. The application name is arbitrary but is used to identify your client in various dialogs on the server. Make this something you will recognize in a list of other clients.

13979-opcserversettings.png

Once the configuration of the controller service is complete then save and start the controller.

Returning to the GetNodeId configuration dialog. The StandardOPCUAService you just configured should be selected in the OPC UA Service property field. Recursive Depth defines how may branches "deep" the processor will browse into the OPC server's namespace. WARNING!!! The namespace can be both very large and may have circular references start this processors off with the default setting of 0 and increase it only after successful reads. The Starting Node property defines the starting node when browsing the server's namespace. If you do not know where to start browsing, leave the field blank and the browse will start from the designated root node. On the scheduling tab set the Run Schedule to something appropriate for your use case greater than 250ms. On the Settings tab select both the Failure and Success checkbox under Automatically Terminate Relationships, we will change this in following steps.

13977-initialgetnodeidproperties.png

Select accept and transition the processor to "Run". You should the metrics shown on the face of the processor change indicating that a flowfile has been generated and written out that is greater that 1B. This is a successful read of the name space

13973-successfullflow.png

Right clicking the processor and selecting Data provenance will allow you to inspect the contents of the newly generated flowfile. You should see something like the following

13975-topofnamespace.png

Most of the nodes starting with NSU are internall nodes that do not contain data we are interested in. Somewhere in your name space you will find the tags that contain the data you would like to retrieve. For this example we are looking for the Nodes under the "Simulation Examples Functions".

13976-simulationexamplesfunctions.png

Now we need to refine the GetNodeIDs processor so that only these items are returned each time the processor is triggered. To do this we change the starting node to ns=2;s=Simulation Examples.Functions and the Recursive Depth to 0.

13978-getnodeidsproperites.png

Now we have the information needed to query the OPC server for time stamped values for each NodeID in our list. The list can be saved to disk or keep only in Nifi's memory. Either way it is a best practice to periodically check the OPC Server for changes in the name space as in some applications this happens frequently.

Next add a split text processor and configure it as follows to split each line of this flowfile into separate flow files.

13980-splittext.png

Now add the GetValue OPC processor to the pallet and connect it to the "Splits" output from the split text processor. There is only one configuration parameter available on the Properties tab of the GetValue processor. For this field select the OPC UA Service created earlier. On the settings tab select both Failure and Success for initial testing.

13981-getvalueproperties.png

Select apply and transitioning the whole data flow to run.

13985-wholeflow.png

The output data from the GetValue processor is formatted as lines of CSV that can be merged into a single document and treated as a CSV. Future iterations of the processor will support various serialization techniques to avoid processing text in later flows.

13986-output.png


2.pngopcserversettings.pngwholeflow.pngselect-the-black-arrow.pnggetnodeidsproperites.pngcreate-new-service.png
11,642 Views
Comments
New Contributor

Hi, where can I find the article how to build the NIFI-OPCUA bundle?

Thanks in advance.

Explorer

Hi Friedel, I believed you're looking for this https://github.com/hashmapinc/nifi-opcua-bundle . Can you share your result after complete this steps?

Thanks.

Not applicable

"Information on building the bundle will be provided in a separate article" - Can't seem to find this article, would you mind sharing a link?

Not applicable

Nevermind, I see someone has already put the link in the comments!

Not applicable

Hi we are running a data flow from PI servers to Phoenix/Hbase using Nifi workflows. As part of this Minifi is deployed at the source system to connect with OPC UA and it is extracting data using Getvalue processor. However, we are having an issue where the minifi is throwing the following error : org.opcfoundation.ua.common.ServiceResultException: Bad_RequestTypeInvalid (code=0x80530000, description="The security token request type is not valid.") We are using the code wrapped around the opcua foundation code and using it to connect from kepware client.

Any idea what the issue might be ? I can provide any further details as required.

Thanks

The whole OPC UA package is now in .net standard - can some one share any articles on how to build a .nar file from the open source GIT repo ?

Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
2 of 2
Last update:
‎08-17-2019 01:41 PM
Updated by:
 
Contributors
Top Kudoed Authors