@JohnYaya  if you can show a sample, it would be very helpful.   If your header and footer is static, and always predictable for a specific file you can replaceText and get "in between" lines using those static header/footer matches with a very creative regex...     ... View more

Try:   $.outputs.['EM_CLASSIFICATION']   that's $.mainobject.['array name'] to access the value for that outputs array item   Fun with Json... ... View more

As mentioned above, make sure you are sending full JSON Object.  Then in your EvaluateJsonPath processor click a + for each one you want to map to an attribute.  Then enter the key → values as below:   test1 → $.outputs.test1 test2 → $.outputs.test2 test3 → $.outputs.test3 test4 → $.outputs.test4 test5 → $.outputs.test5 test6 → $.outputs.test6 If this solution works please accept it as the solution to close the topic.     ... View more

@AndyTech Can you share some details about the hive table (schema, number rows, data size, etc)?  Can you describe your hive setup (configuration, # of nodes, tez/yarn container size, queue setup, etc)?   Can you speak to the current speed benchmark, and your expected speed?   Depending on the data itself, partition can have some performance.  There are a bunch of things you can do to hive itself, that will have a huge impact on performance.  With working table, and working query, especially parquet, I would want to investigate Hive Performance Tuning before making any changes to the data structure.   Additionally, there would be some discussion about parquet vs orc which is known to be faster.   Share some details and myself or others will comment further. ... View more

@wengelbrecht thank you that is exactly what i needed to see.  I am having an issue with the parquet-hadoop-1.10 and need to get a 1.12 version working in NiFi and Hive....      ... View more

@MattWho do you know what version of Parquet is supported by the new readers? ... View more

@wengelbrecht  do you know version of parquet this reader is supposed to support? ... View more