Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

NiFi convert string to key/value pair

Solved Go to solution

NiFi convert string to key/value pair

Contributor

i'm still learning the ropes of NiFi, I have some XML (See Below)

<measInfo measInfoId="1542455297">
	<measTypes>1542455297 1542455298 1542455299 1542455300 1542455301 1542455302 1542455303 1542455304 1542455305 1542455306 1542455307 1542460296 1542460297 </measTypes>
	<measValue measObjLdn="LTHAB0113422/ETHPORT:Cabinet No.=0, Subrack No.=1, Slot No.=7, Port No.=0, Subboard Type=BASE_BOARD">
		<measResults>116967973 585560 496041572 682500 0 12583680 72080 520454 46670568 73432 2205837 1000000 1000000 </measResults>
	</measValue>
	<measValue measObjLdn="LTHAB0113422/ETHPORT:Cabinet No.=0, Subrack No.=1, Slot No.=7, Port No.=1, Subboard Type=BASE_BOARD">
		<measResults>0 0 0 0 0 0 0 0 0 0 0 0 0 </measResults>
	</measValue>
</measInfo>


I successfully parsed the XML, but need further processing. I'm looking for suggestions as to the best way to handle this. Ultimately I would like a key/value pair between MeasTypes and MeasResults to insert into Hbase or Pivot data into a Hive table (MeasType would get translated into a Column Name). The exact number of MeasTypes and MeasResults pairs is variable.

measTypes	measResults
1542455297	116967973
1542455298	585560
1542455299	496041572
1542455300	682500
1542455301	0
1542455302	12583680
1542455303	72080
1542455304	520454
1542455305	46670568
1542455306	73432
1542455307	2205837
1542460296	1000000
1542460297	1000000


I have 10GB of XML arriving every 30 minutes, so efficiency is also a concern. I'm able to take the measTypes and put it into a file and parse it with a ExtractText and ReplaceText (using a Regex), so the next challenge is to replace the GetFIle with a feed from my existing XML template, how do I do it? Another Option is to InvokeScriptedProcessor (Maybe Python)? Sample code would be greatly appreciated. :)

1 ACCEPTED SOLUTION

Accepted Solutions

Re: NiFi convert string to key/value pair

One option I can think of is using xquery or xslt.

5 REPLIES 5

Re: NiFi convert string to key/value pair

One option I can think of is using xquery or xslt.

Re: NiFi convert string to key/value pair

Super Guru

Re: NiFi convert string to key/value pair

Contributor

Wow, didn't know the xQuery could do this. Based on quick research it appears that fn:tokenize might be just what I'm looking for. I'll have to play with this to see if I can this to work.

Re: NiFi convert string to key/value pair

New Contributor

Hi Hans

Did you manage to accomplish this using xquery? Your solution would be greatly appreciated. Thanks

Re: NiFi convert string to key/value pair

Contributor

Sorry I did not get this to work in xQuery. I ended up using XSLT to do the transformation.