How to read HL7 message in NiFi Processor. Is it mandatory to use Kafka?

[#1]

In my scenario I want to receive the HL7 message from a HL7 simulator or through some asset which send the HL7 message to the configured IP:Port. I tried two ways for receiving HL7 message

1. Through ListenTCP and GetTCP processors. But with these Processor I am unable to receive the HL7 message in NiFi.

2. Tried the steps mentioned in Ingesting HL7 Data in NiFi. To follow this I need the Kafka Set up. For that I followed the steps mentioned on the kafka site for installing Kafka. Also verified the installation by running the Kafka Producer and Consumer bat file. As a next step instead of Kafka Consumer console, I used NiFi ConsumeKafka Processor as a Consumer. The data I sent from Kafka Producer console was received in ConsumeKafka Processor which I tested by sending the data through PutFile Processor.

But here the problem is I don't have a configuration section where I can mention topic. I can only set the IP, Port and Message in the HL7 simulator app. Is there any Processor which can receive the HL7 data by configuring Ip and Port. Or any suggestions/help on receiving HL7 message into NiFi will be really helpful.

Thanks.

[#2]

@Dhamotharan P

Can you share why ListenTCP and GetTCP didn't work? I did you get errors?

[#3]

I didn't get any errors rather didn't receive the data packet.

[#4]

No need for kafka

Follow my articles

https://community.hortonworks.com/articles/149910/handling-hl7-records-part-1-hl7-ingest.html

https://community.hortonworks.com/articles/149891/handling-hl7-records-and-storing-in-apache-hive-fo...

You can read it from files.

which simulator is it?

[#5]

Hi @Timothy Spann Thanks for the Links. I followed the links mentioned. But I want the HL7 message directly to NiFi so that I can do further processing on NiFi Processors and send to my back end. I don't want the HL7 message to write into some file and send to NiFi. Because I want to avoid the time it takes to write into file and send to NiFi. Is it possible? Or any other alternative ?

And I am using iNTERFACEHL7 Simulator. I tested the Simulator. I am able to receive the HL7 message in my console which I send through Simulator. Problem is, NiFi Processor are unable to receive HL7 messages.

[#6]

Hi @Michael Young @Timothy Spann @Matt Clarke @Matt Burgess@anarasimham

I am trying to receive the Hl7 message from Simulator to Listen TCP but I am not able to receive it. Any directions from here will be helpful. So far what I have tried are

1. Sent the same message HL7 from console application to ListenTCP Processor I am able to receive message in Processor

2. Also from the simulator to Console application I am able to pass the HL7 message through the port configured.

Attached the screenshot.

[#7]

@Dhamotharan P

ListenTCP doesn't really have any concept of a "message". It is based on buffer size. If you are sending multiple messages per request, then the Batching Message Delimiter can pull out individual messages from a single request, but I believe that only happens once the buffer size has been reached. It's a nuance of working with something like ListenTCP.

Try setting the Receive Buffer Size to a much smaller value than the 65507 B that you currently have it set to

[#8]

@Michael Young I have created the custom processor to solve the issue. But currently I am stuck with HL7 message processing and I have created a new question on that here https://community.hortonworks.com/questions/226511/extracthl7attribute-processor-throws-error-cauhnh.... Can you please let me know your suggestion on that? Thank you

[#9]

I am unable to receive even after adjusting the buffer size. Now I am in the process of creating a custom processor on top of ListenTCP which will receive the HL7 message and also send back the acknowledgement. If anyone have an idea or implemented the same please share me the steps which will reduce my development time in creating the custom processor.

Thanks

[#10]

We are stuck with the same issue. We are unable to receive the HL7 messages. Could you guide us with the custom processor creation? Any help in this regard will be appreciated. Thank you.