Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Can NiFi connect to OPC DA and HDA..??

Solved Go to solution
Highlighted

Can NiFi connect to OPC DA and HDA..??

Super Collaborator

Hi ,

Is there a way where we can connect NiFi to OPC Servers using OPC DA and\or OPC HDA specifications and retrieve both real-time and archived data .?

Regards,Sai

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Can NiFi connect to OPC DA and HDA..??

Contributor

One option is to use a proxy server like KepServerEx www.kepware.com to access OPC DA data via OPC UA and use https://github.com/wadesalazar/NIFI-OPCUA

If you need more direct access to OPC DA you have 2 choices. 1) build native Java client and wrap it with the service and/or processor Nifi APIs. or 2) build / obtain a Windows executable program that can be called or interfaced with Java and Nifi

The reason why 1) is so difficult and 2) is necessary is COM. COM was, is, a binary-interface standard for introduced by Microsoft in 1993. It is used to enable inter-process communication and dynamic object creation Ironically I believe it was an attempt at an "open" standard for RPC.

There is no public Java implementation for COM and developing for OPC DA / HDA / A&E in Java is a real pain. The only tried and true path forward for a native Java OPC DA client is is JInterop http://www.j-interop.org/ . See http://openscada.org/projects/utgard/ for a reference implementation. J-Interop is old and sparsely supported There may be a compiler that can create the necessary Java class files ( similar to what JInterop has provided ) https://msdn.microsoft.com/en-us/library/ms691398(v=vs.85).aspx but it is in Visual J++ 6.0. ( if you happen to have this please email me

Going with option 2 is better supported but still not easy. You may use a commercial OPC DA stack/client/library, such as https://www.prosysopc.com/products/ to build an executable that you can call with nifi execute process or Integrate into a processor using JNI. This does require your Nifi hosts to be Windows machines.

If you are looking to stay on Linux but use this approach. A relatively recent development is Rad Hat's .Net frame work for RHEL, it isnt clear to me if it supports COM but I believe it may. This might allow one to use a commercial OPC stack to build an executable that could be used.

I love to help you or anyone else build an OPC DA connector for Apache Nifi. Let me know how I can help.

View solution in original post

5 REPLIES 5
Highlighted

Re: Can NiFi connect to OPC DA and HDA..??

Rising Star

@Saikrishna Tarapareddy Yes, you should be able to. Take a look at this HCC article and see if it helps: https://community.hortonworks.com/articles/98394/accessing-data-from-osi-softs-pi-system.html.

Highlighted

Re: Can NiFi connect to OPC DA and HDA..??

Contributor

One option is to use a proxy server like KepServerEx www.kepware.com to access OPC DA data via OPC UA and use https://github.com/wadesalazar/NIFI-OPCUA

If you need more direct access to OPC DA you have 2 choices. 1) build native Java client and wrap it with the service and/or processor Nifi APIs. or 2) build / obtain a Windows executable program that can be called or interfaced with Java and Nifi

The reason why 1) is so difficult and 2) is necessary is COM. COM was, is, a binary-interface standard for introduced by Microsoft in 1993. It is used to enable inter-process communication and dynamic object creation Ironically I believe it was an attempt at an "open" standard for RPC.

There is no public Java implementation for COM and developing for OPC DA / HDA / A&E in Java is a real pain. The only tried and true path forward for a native Java OPC DA client is is JInterop http://www.j-interop.org/ . See http://openscada.org/projects/utgard/ for a reference implementation. J-Interop is old and sparsely supported There may be a compiler that can create the necessary Java class files ( similar to what JInterop has provided ) https://msdn.microsoft.com/en-us/library/ms691398(v=vs.85).aspx but it is in Visual J++ 6.0. ( if you happen to have this please email me

Going with option 2 is better supported but still not easy. You may use a commercial OPC DA stack/client/library, such as https://www.prosysopc.com/products/ to build an executable that you can call with nifi execute process or Integrate into a processor using JNI. This does require your Nifi hosts to be Windows machines.

If you are looking to stay on Linux but use this approach. A relatively recent development is Rad Hat's .Net frame work for RHEL, it isnt clear to me if it supports COM but I believe it may. This might allow one to use a commercial OPC stack to build an executable that could be used.

I love to help you or anyone else build an OPC DA connector for Apache Nifi. Let me know how I can help.

View solution in original post

Highlighted

Re: Can NiFi connect to OPC DA and HDA..??

Explorer

@wsalazar

I already have the KepServer and I need to add OPCUA as additional NiFi processor to get the data from that server. I downloaded your git (https://github.com/wadesalazar/NIFI-OPCUA) and run 'mvn package' then copy 'opc-ua-stack-1.03.342.1-SNAPSHOT.jar' to Nifi's lib but seem like not working. Any step I missed or went wrong?

Highlighted

Re: Can NiFi connect to OPC DA and HDA..??

Contributor

Does anyone have any update to this? Does anyone use NiFi to read data from OPC DA, or OPC UA?

 

There was talk about Hortonworks having a connector for those data sources three years ago. Does anyone know if that is supported by Cloudera now? 

 

I am thinking that the fact that OPC DA has to be on Windows and uses COM has been the sticking point for us. It may be that moving to OPC UA is possible. 

Re: Can NiFi connect to OPC DA and HDA..??

Community Manager

@alexmc As this thread is a couple of years old, you would be better served by creating a new thread. 



Cy Jervis, Community Manager

Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community:
Community Guidelines
How to use the forum
Don't have an account?
Coming from Hortonworks? Activate your account here