Hi @sally sally, You can do that by using invokeHTTP processor, once you make first service call then keep Response relationship to trigger next service. This way we can only triggers next service once we get response from previous service. Example:- In my below flow service 1 is triggered by GenerateFlowFile processor then i connected response relationship to trigger service2 InvokeHTTP processor. This service2 processor only triggers when it got response from service1 processor and keep in mind the response from service1 will be overwritten by response of service2. ... View more

@swathi thukkaraju, You can use the below serde properties to read your data correctly CREATE TABLE test(a string, b string,..) ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.OpenCSVSerde' WITH SERDEPROPERTIES ( "separatorChar" = ",", "quoteChar" = "\"" ) STORED AS TEXTFILE location 'location of csv file'; ... View more

Hi @Rohit Ravishankar, yeah you will have impact on the flow. You are going to have 3 node cluster and thinking to use RPG after ListFile processor. let's consider you are having M01,M02,M03 are 3 NiFi nodes in the cluster and M01 is the Primary Node of the cluster. 1.So when ListFile processor runs then gives output to RPG, it is not guaranteed the file will goes to Primary node(M01). 2. RPG will take care of load balancing of nifi cluster and distributes the flowfiles accordingly. 3.If you are running ExecuteStreamCommand on Primary Node only, then it will triggers the command only if the flowfile will be on primary node at the time.in our assumption above processor will triggers the shell script only when flowfile will be on M01 node. 5.If RPG distributes the flowfile to M02(or)M03 nodes but the ExecuteStreamCommand processor is running on Primary Node only, in this cases those flow files won't triggers off the shell script. ... View more

Hi @sally sally, List Hdfs processor are developed as store the last state.. i.e when you configure ListHDFS processor you are going to specify directory name in properties. once the processor lists all the files existed in that directory at the time it will stores the state as maximum file time when it got stored into HDFS. you can view the state info by clicking on view state button. if you want to clear the state then you need to get into view state and click on clear the state. 2. so once it saves the state in listhdfs processor, if you are running the processor by scheduling as cron(or)timer driven it will only checks for the new files after the state timestamp. Note:- as we are running ListHDFS on primary node only, but this state value will be stored across all the nodes of NiFi cluster as primary node got changed, there won't be any issues regarding duplicates. Example:- hadoop fs -ls /user/yashu/test/ Found 1 items -rw-r--r-- 3 yash hdfs 3 2017-09-15 16:16 /user/yashu/test/part1.txt when i configure ListHDFS processor to list all the files in the above directory if you see the state of ListHDFS processor that should be same as when part1.txt got stored in HDFS in our case that should be 2017-09-15 16:16 it would be unix time in milliseconds when we convert the state time to date time format that should be Unixtime in milliseconds:- 1505506613479 Timestamp :- 2017-09-15 16:16:53 so the processor has stored the state, when it will run again it will lists only the new files that got stored after the state timestamp in to the directory and updates the state with new state time (i.e maximum file created in hadoop directory). ... View more

Hi @sally sally, as you said you are getting rid of [] in the flow file content but in my case i haven't get rid of [] you can use your existing functionality. After that use another replace text processor with search value property as (?<=]})(,) if you are having any spaces or new line characters before , then make use of below regex (?<=]})(\s*,) and replacement value property as shift+enter it will replaces your , after } with new line Input:- { "squadName": "Super hero squad", "homeTown": "Metro City", "formed": 2016, "secretBase": "Super tower", "active": true, "Data": { "row": [{ "name": "Molecule Man", "age": 29, "secretIdentity": "Dan Jukes", "powers": ["Radiation resistance", "Turning tiny", "Radiation blast"] }, { "name": "Madame Uppercut", "age": 39, "secretIdentity": "Jane Wilson", "powers": ["Million tonne punch", "Damage resistance", "Superhuman reflexes"] }, { "name": "Eternal Flame", "age": 1000000, "secretIdentity": "Unknown", "powers": ["Immortality", "Heat Immunity", "Inferno", "Teleportation", "Interdimensional travel"] }] }} Output:- [{"name":"Molecule Man","age":29,"secretIdentity":"Dan Jukes","powers":["Radiation resistance","Turning tiny","Radiation blast"]} {"name":"Madame Uppercut","age":39,"secretIdentity":"Jane Wilson","powers":["Million tonne punch","Damage resistance","Superhuman reflexes"]} {"name":"Eternal Flame","age":1000000,"secretIdentity":"Unknown","powers":["Immortality","Heat Immunity","Inferno","Teleportation","Interdimensional travel"]}] Replacetext configs:- In addition in future if you are expecting different flow files for each record make a use splitJson processor with JsonPath Expression property as $.Data.row connect the split property to another processor, it will results as in our array of data.row having 3 records so it will results 3 different flow files as a result from split json processor. Results after splitjson configs:- Output:- as we are having 3 flow files above and the contents of those flowfiles as follows. flowfile1:- {"name":"Eternal Flame","age":1000000,"secretIdentity":"Unknown","powers":["Immortality","Heat Immunity","Inferno","Teleportation","Interdimensional travel"]} flowfile2:- {"name":"Molecule Man","age":29,"secretIdentity":"Dan Jukes","powers":["Radiation resistance","Turning tiny","Radiation blast"]}flowfile3:- flowfile3:- {"name":"Madame Uppercut","age":39,"secretIdentity":"Jane Wilson","powers":["Million tonne punch","Damage resistance","Superhuman reflexes"]} ... View more