Support Questions

Find answers, ask questions, and share your expertise
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

how to use evaluatejsonpath processor with two files feeded simultaneously

New Contributor

Master Guru
@Tinkle Mahendru

Each thread used to run the EvaluateJsonPath processor will pull one FlowFile from an incoming queue and evaluate against one or more configured JsonPath expressions.

By default every processor you add to your canvas is only configured to use 1 concurrent task. You can add additional concurrent tasks to a processor. By doing so each concurrent task can pull a different file form an incoming queue and work on those files in parallel. Each Concurrent task works on its own FlowFile. Keep in mind that each running concurrent task is another cpu thread. So if you configure 10 concurrent tasks on a processor and their are enough incoming queued FlowFiles, this one processor could end up consuming 10 threads from your NiFi thread resource pool.

The "Max Timer Driven Thread Count" from which processor get their threads (concurrent tasks) is configured via the "controller settings" UI found in the upper right corner menu of the UI. This values is defaulted to 10. This controls how many system threads the NiFi controller will consume from your hardware to service thread requests form your various components added to the NiFi canvas. This value should be set between 2-4 times the number of cores you have available. If you are a cluster, this values is set per node and not per cluster. So if you have 4 nodes each with 16 cores, this values should still only be set to 32 to 64. Be sure to monitor CPU utilization on your server(s) to see what inpact these changes have on your hardware.

Thanks, Matt

New Contributor

thanks matt but actually i am looking to extract json property name from the evaluatejsonpath processor and change its property to some other property but i want to do this for two files in flow

for eg:- if i have in evaluatejsonpath "name":"filename1" for file1 and "name":"filename2" for other file but i want to change name for both the files as name1 and name2 respectively . how should i go about it . even if i use extract text and replace text it , will change name for 1 file, how will it change the same property for other file . The full flow is i am changing file content with package info using replace text which is created in some open data source .Now with that package info i am calling a package creating api which is creating a package in the open data source. now i am using the payload of the package creation api to create a resource in that open source. After that i am evaluating the payload using evaluatejsonpath . in that processor i am setting filename1 as ${.result.name1} and filename2 as ${.result.name2} . but the problem is in json payload which i received from resource creation api has property "name" and if i give two files at a time both have property "name" in their json payload. so i want to change property"name" of one file1 to name1 and file2 to name2 in the jayson payload so that i can use evaluatejsonpath to evalaute name1 for file1 and name2 for file 2 in payload instead of name in the same dataflow

Please let me know if this is not clear enough i will try to make the question more clear.

Thanks in advance!!!!

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.