Hi, i need to ingest only the JSON files following a valid schema. i am trying to achieve this by validate-record processor. i am supplying the same schema for both JSONTreeReader and JSONRecsetWriter. I am not using AVRO because my input contains _ in the names. (but i came up with this schema by modifying the input file without _ and using inferAvroSchema and then changed both to use _ to match the input file) my schema and files are matching but its sending it to invalid relation. Anything wrong that i am doing..?? Schema : { "type": "record", "name": "iHist", "fields": [ { "name": "file_name", "type": "string" }, { "name": "plant", "type": "string" }, { "name": "collector", "type": "string" }, { "name": "name", "type": "string" }, { "name": "unique_id", "type": "string" }, { "name": "description", "type": "string" }, { "name": "general_1", "type": "string" }, { "name": "general_2", "type": "string" }, { "name": "general_3", "type": "string" }, { "name": "general_4", "type": "string" }, { "name": "general_5", "type": "string" }, { "name": "data_points", "type": { "type": "array", "items": { "type": "record", "name": "data_points", "fields": [ { "name": "timestamp", "type": "string" }, { "name": "value", "type": "string" }, { "name": "quality", "type": "string" } ] } } } ] } Data file.. { "file-name": "tp-tcollec.tag.json", "plant": "P11A3", "collector": "test_Collector", "name": "tag_SAFETY_MARGN.F_CV", "unique-id": "1532358720761", "description": "test", "general-1": "", "general-2": "", "general-3": "", "general-4": "", "general-5": "", "datapoints": [ { "timestamp": "2016-07-19T10:25:43.000Z", "value": "177", "quality": "100" }, { "timestamp": "2016-07-19T10:25:42.000Z", "value": "177", "quality": "100" }, { "timestamp": "2016-07-19T10:25:41.000Z", "value": "177", "quality": "100" } ] } I just need to validate if the input file is following the schema. any better ways to do this.?? ... View more

Hi, Our OU in the LDAP connection has changed and we are getting invalid ID,PWD errors when trying to log in to NiFi, I think this is expected. But if i change the old ou=Users and Groups,ou=EU,ou=same,ou=Organizations,dc=comp,dc=com to new ou=Users and Groups,ou=USA,ou=same,ou=Organizations,dc=comp,dc=com do i need to restart nifi.?? is there a way that this could be avoided .?? that is instead of giving the whole string can i just give the top level ou like ou=Users and Groups Regards, Sai ... View more

Hi, Is there anyway that we can run Remote Processor Group with multi threading (concurrent tasks) in NiFi Site-2-Site.? Regards, Sai ... View more

Hi, i need to transform the input JSON below { "TransactionId": "-1", "Source": "ihrs", "Name": "tag-data", "Id": "126", "Records": [ { "Name": "tag-master", "PayLoad": { "file-name": "testfile.json", "plant": "1086", "collector": "testcollector", "tag-name": "testtag", "tag-description": "test desc" } }, { "Name": "tag-detail", "PayLoad": { "tag-value": "98998.55", "tag-timestamp": "2018-01-02T16:09:39.000000+00:00", "tag-quality": "100" } }, { "Name": "tag-detail", "PayLoad": { "tag-value": "91009.47", "tag-timestamp": "2018-01-02T16:09:340000000+00:00", "tag-quality": "100" } }, { "Name": "tag-detail", "PayLoad": { "tag-value": "91021.80", "tag-timestamp": "2018-01-02T16:09:41.000000+00:00", "tag-quality": "100" } } ] } to output like below. { "TransactionId": "-1", "Source": "ihrs", "Name": "tag-data", "Id": "100", "Records": [ { "Name": "tag-master", "PayLoad": { "file-name": "testfile.json", "plant": "1000", "collector": "testcollector", "tag-name": "testtag", "tag-description": "test desc", "Children": [ { "Name": "tag-detail", "PayLoad": { "tag-value": "98998.55", "tag-timestamp": "2018-01-02T16:09:39.000000+00:00", "tag-quality": "100" } }, { "Name": "tag-detail", "PayLoad": { "tag-value": "98998.55", "tag-timestamp": "2018-01-02T16:09:39.000000+00:00", "tag-quality": "100" } } ] } } ] } i need to move the details records under master as array.. basically first element of the record array (tag master)is parent to next 3 elements(tag details). i tried to use the convert record processor with JSON converters giving correct input and output schemas for files. it didnt convert. it looks like i may have to write JOLT which i need some help with. Thanks in advance!!! attached files for reference.. input.jsonoutput.json Regards, Sai ... View more

Hi, I am trying to update header of my CSV file with a regular expression to remove special chars from header line only. how to do that.? i tried to do that by reading the file and on one route to RouteText,ReplaceText,ExtractText to get the firstline and storing it in headerline attribute. and another route to move the file without the header and tried to Merge it by using headerline from route1. But It only shows header in output when i first file arrives from Route 1 into MergeContent processor as it has the headerline property where as if it gets it from Route 2 , the output file doesnt have the headerline as it doent have that property. any idea how to solve this..? ... View more