Hi All, I have the following requirement: Split a single NiFi flowfile into multiple flowfiles, eventually to insert the contents (after extracting the contents from the flowfile) of each of the flowfiles as a separate row in a Hive table. Sample input flowfile: MESSAGE_HEADER | A | B | C LINE|1 | ABCD | 1234 LINE|2 | DEFG | 5678 LINE|3 | HIJK | 9012 . . . Desired output files: Flowfile 1: MESSAGE_HEADER | A | B | C LINE|1 | ABCD | 1234 Flowfile 2: MESSAGE_HEADER | A | B | C LINE|2 | DEFG | 5678 Flowfile 3: MESSAGE_HEADER | A | B | C LINE|3 | HIJK | 9012 . . . The number of lines in the flowfile is not known ahead of time. I would like to know what's the best way to accomplish this with the different NiFi processors that are available; The splitting can be done at the flowfile level or after the contents of the flowfile are extracted out of the flowfile, but before Hive insert statements are created. Thanks. ... View more

Hi All, May be this is a networking question and not specific to NiFi. I've a locally installed NiFi instance on my laptop, we don't have NiFi installed on our Hadoop cluster yet. On a remote server a TCP/IP port is setup, to receive real-time streaming data. Using NiFi's ListenTCP processor on my local machine, is it possible to listen to the port on the remote server and receive the real-time streaming data into my local NiFi instance. Thanks. ... View more

Hi All, Here is some background on what I want to do: I'm reading HL7 message files and want to create an HBase table with that data. The ExtractHL7Attributes processor parses HL7 messages and creates attributes in this fashion - <Segment Name> <dot> <Field Index>. If the segment is repeating, the naming will be <Segment Name> <underscore> <Segment Index> <dot> <Field Index>. Examples of attributes it creates are "MHS.12" with a value of "2.1" and "OBX_11.3" with a value of "93000^CPT4". I want to write these attributes and their values to an HBase table, where the attributes (MHS.12) become column names and the values become the cells in the HBase table. This works fine, but the column names MHS.12 and OBX_11.3 are not very meaningful for an HBase table. So, using an UpdateAttributes processor, I'm able to rename the columns as follows: OBX_1.1 --> OBX_1_Set ID OBX_1.2 --> OBX_1_ValueType . . . OBX_8.1 --> OBX_8_Set ID OBX_8.2 --> OBX_8_ValueType But for this to work, you need to know how many attributes (or OBX segments) will be in the flow file at design time; Since OBX segment can repeat itself many times in the message and since we can’t anticipate the max number of OBX segments a message could have at design time, I am unable to rename all attributes at design time; Any suggestions on how to make this work ? Is creating a custom processor the route to take ? Thanks in advance. ... View more