question Re: Can NiFi connect to OPC DA and HDA..?? in Support Questions

Can NiFi connect to OPC DA and HDA..??

saikrishna_tara — Thu, 01 Jun 2017 03:12:27 GMT

Hi ,

Is there a way where we can connect NiFi to OPC Servers using OPC DA and\or OPC HDA specifications and retrieve both real-time and archived data .?

Regards,Sai

Re: Can NiFi connect to OPC DA and HDA..??

mriggs1 — Thu, 01 Jun 2017 03:38:00 GMT

@Saikrishna Tarapareddy Yes, you should be able to. Take a look at this HCC article and see if it helps: https://community.hortonworks.com/articles/98394/accessing-data-from-osi-softs-pi-system.html.

Re: Can NiFi connect to OPC DA and HDA..??

wsalazar — Fri, 07 Jul 2017 03:46:04 GMT

One option is to use a proxy server like KepServerEx www.kepware.com to access OPC DA data via OPC UA and use https://github.com/wadesalazar/NIFI-OPCUA

If you need more direct access to OPC DA you have 2 choices. 1) build native Java client and wrap it with the service and/or processor Nifi APIs. or 2) build / obtain a Windows executable program that can be called or interfaced with Java and Nifi

The reason why 1) is so difficult and 2) is necessary is COM. COM was, is, a binary-interface standard for introduced by Microsoft in 1993. It is used to enable inter-process communication and dynamic object creation Ironically I believe it was an attempt at an "open" standard for RPC.

There is no public Java implementation for COM and developing for OPC DA / HDA / A&E in Java is a real pain. The only tried and true path forward for a native Java OPC DA client is is JInterop http://www.j-interop.org/ . See http://openscada.org/projects/utgard/ for a reference implementation. J-Interop is old and sparsely supported There may be a compiler that can create the necessary Java class files ( similar to what JInterop has provided ) https://msdn.microsoft.com/en-us/library/ms691398(v=vs.85).aspx but it is in Visual J++ 6.0. ( if you happen to have this please email me

Going with option 2 is better supported but still not easy. You may use a commercial OPC DA stack/client/library, such as https://www.prosysopc.com/products/ to build an executable that you can call with nifi execute process or Integrate into a processor using JNI. This does require your Nifi hosts to be Windows machines.

If you are looking to stay on Linux but use this approach. A relatively recent development is Rad Hat's .Net frame work for RHEL, it isnt clear to me if it supports COM but I believe it may. This might allow one to use a commercial OPC stack to build an executable that could be used.

I love to help you or anyone else build an OPC DA connector for Apache Nifi. Let me know how I can help.

Re: Can NiFi connect to OPC DA and HDA..??

yimsiokek — Thu, 23 Nov 2017 17:40:34 GMT

@wsalazar

I already have the KepServer and I need to add OPCUA as additional NiFi processor to get the data from that server. I downloaded your git (https://github.com/wadesalazar/NIFI-OPCUA) and run 'mvn package' then copy 'opc-ua-stack-1.03.342.1-SNAPSHOT.jar' to Nifi's lib but seem like not working. Any step I missed or went wrong?

Re: Can NiFi connect to OPC DA and HDA..??

alexmc — Thu, 23 Jan 2020 13:25:08 GMT

Does anyone have any update to this? Does anyone use NiFi to read data from OPC DA, or OPC UA?

There was talk about Hortonworks having a connector for those data sources three years ago. Does anyone know if that is supported by Cloudera now?

I am thinking that the fact that OPC DA has to be on Windows and uses COM has been the sticking point for us. It may be that moving to OPC UA is possible.

Re: Can NiFi connect to OPC DA and HDA..??

cjervis — Thu, 23 Jan 2020 14:23:49 GMT

@alexmc As this thread is a couple of years old, you would be better served by creating a new thread.