Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Text processing of FlowFiles content in Apache NiFi

Highlighted

Text processing of FlowFiles content in Apache NiFi

New Contributor

Hi All,

    I have single JSON object in a flow file. I just want to append and prepend text to it, and pass it over as flowfiles for subsequent processors. Here is what I want to achieve:

 

Input FlowFile Content:

{ "first_name": "alpha", "age": "30" }

Expected output in FlowFile Content:

{ $set: { "first_name": "alpha", "age": "30" } }

 

So basically i want to prepend "{ $set " and append " } "

I found crude way of updating it, in multiple steps. But just wanted to do it in proper and concise manner.

5 REPLIES 5
Highlighted

Re: Text processing of FlowFiles content in Apache NiFi

Master Collaborator

@arunnalpet This sounds like a perfect job for ReplaceText:

 

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache...

 

I did notice that you have set as $set.  In order to maintain a real json object, I have used just "set".  The set object is available as $.set if you use EvalateJsonPath.   If you still need $set, please explain... 

 

 

Below is the configuration you need for ReplaceText.  It regex matches the entire flow file then adds the prepend before $1 and adds the append after.

 

{ "set" : $1 }

(this is the only change in the processors from default)

 

Screen Shot 2020-04-28 at 7.46.49 AM.png

 

 

And the output:

 

{ "set" : { "first_name": "alpha", "age": "30" } }

 

 


 


If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.  


 


Thanks,



Steven

Highlighted

Re: Text processing of FlowFiles content in Apache NiFi

New Contributor

Thank you! This gives me some directions.

One thing though is that, i really want it to be "$set" and I am not expecting it to comply to JSON standard after doing this.

 

So I guess i can just use $set in the replacement value and get the job done. (i think we need to escape $ value). Will try it out and get back.

Highlighted

Re: Text processing of FlowFiles content in Apache NiFi

Master Collaborator

@arunnalpet   yes that is correct.  I wanted to avoid $set in basic testing and also advocating for valid code practice to avoid issues further with the data source.    

 

 

If the answer above resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.  

 

Thanks,


Steven @ DFHZ

 


 


If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.  


 


Thanks,



Steven

Highlighted

Re: Text processing of FlowFiles content in Apache NiFi

New Contributor

Hi,

      Actually I am working on MongoDB queries, which supports 'Extended JSON' as opposed to 'Standard JSON'. For which I need to enforce $set. Is there a way we can implement this?

Re: Text processing of FlowFiles content in Apache NiFi

Master Collaborator

@arunnalpet You should be able to just add the $. If it doesn't work, maybe have to escape it:  \$

 


 


If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.  


 


Thanks,



Steven

Don't have an account?
Coming from Hortonworks? Activate your account here