I have a use case where JSON files are read from an API, transformed to CSV and imported to Hive tables, however my flow fails at the replace text processor. Can you give some advice on the configuration of the processor or on where my approach fails? InvokeHTTP --> EvaluateJsonPath --> ReplaceText --> MergeContent --> UpdateAttribute --> PutHDFS My flow does several HTTP calls with InvokeHTTP (Each call with different ID), extracts attributes from each JSON that is returned (each JSON is unique) and then creates the csv's in the ReplaceText processor as following: ${attribute1},${attribute2},${attribute3},${attribute4},${attribute5},${attribute6},${attribute7} However after the MergeContent processor inthe merged CSV there is really a lot of duplicate data while all incoming JSONs contain unique data. ... View more

I need to read an API where my first call will return a JSON with objectIDs in this form: { objectIdFieldName: "ID", objectIds: [ 64916, 67266, 67237, 64511, .... ..] } I need to use the objectIds above to send requests for each of these IDs to the API that will return the data that i need. I was thinking a flow: GetHTTP (get response JSON) --> EvaluateJSONpath (parse only objectIds field: $.objectIds) --> ? --> InvokeHTTP (new query per ID) My problem comes after that as what i get is a sort of objectIds array in the form of: [64916, 67266, 67237, 64511,...,] How to i manage to split/parse each ID from this array in a flowfile attribute so that i can send it along with other data/headers to the InvokeHTTP processor? I thought to use SplitJson processor but i am having difficulties to understand its usage in this case. Any help much appreciated! ... View more

Trying to get files from an FTP server with ListFTP/FetchFTP, but these speicifc processors are so confusing to me. I have scheduled ListFTP to fire also with Cron and time-driven every 10 secs or so, but even though it shows a task is executed in the ListFTP processor, no flowfiles come out of it! GetFTP works fine, but i wanna implement it with List/Fetch to get only the new files in the dir. Initially i scheduled the flow with cron at 03:00 last night when i knew a new file would become available in the FTP server around 1. However this morning i saw that nothing was transfered to HDFS. So i changed scheduling to every 60 secs to test it right away and guess what, i got the new file! So i thought Cron is the problem, deleted the test-file from HDFS and scheduled the flow to run at 10:00 this morning, as expected a task was created but no flowfiles were passed to FetchFTP. Switching back to timer-driven scheduling to get the file i deleted back to HDFS from the FTP server, but this time there are no flowfiles created even for this scheduling option. What is going on here guys? ... View more