Thanks for the info, was very helpful! ... View more

Hi,   Exit code 143 is related to Memory/GC issues. Your default Mapper/reducer memory setting may not be sufficient to run the large data set. Thus, try setting up higher AM, MAP and REDUCER memory when a large yarn job is invoked.  For more please refer to this link. https://stackoverflow.com/questions/42972908/container-killed-by-the-applicationmaster-exit-code-is-143   Thanks AKR ... View more

Hey can you please help me with extracting regex attribute from a json file. I am trying to take attribute from a json file, this attributes have regex values. For ex: json file is - { "key1" : "(.{2})(.{4})", "key2" : "$1,$2"} From this json file I would need to take key1 and key2 in replacetext processor to search(key1) and replace by (key2). Please suggest a way to do this. Thanks!! ... View more

@Shu_ashu  This is my dataflow -  https://drive.google.com/file/d/1SWtSAPKxRcgAWT7ca0dytePjfwMZIpgR/view?usp=sharing   The ListSFTP processor works fine. It lists all the files in the server. But FetchSFTP doesn't work as expected. I get comms.failure. My ListSFTP  and FetchSFTP configurations are the same.    Also I had one more doubt - How does FetchSFTP gets its state from LISTSFTP.? Is there any additional configuration that has to be done.? I am just joining ListSFTP and FetchSFTP with relationship Success. Not doing anything else.   Please help.      ... View more

@Satish Karuturi This is an expected behaviour from ExecuteStreamCommand processor and best practice is to place shell script file on all nodes of NiFi cluster. As you are having 2 node nifi cluster and we are not able to control which will be primary node and ExecuteStreamCommand processor will run only on primary node. In case of primary node changes nifi will pick the shell script from the active primary node and continue to execute the script without any issues..! In addition you can also use ExecuteProcess processor to execute Shell script in NiFi. ... View more

This is the error if I process the queue with the above content in PuthiiveQL PutHiveQL[id=89b230de-8dc5-18e0-9cac-668cfe8085b0] Failed to update Hive for StandardFlowFileRecord[uuid=1bb63fbe-29af-4422-b8a3-a282063514e8,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1564127716134-1, container=default, section=1], offset=186312, length=69],offset=0,name=9d11a5b0-78e8-43f1-9514-9e4c4509855c,size=69] due to Error while compiling statement: FAILED: ParseException line 1:58 cannot recognize input near '?' ',' '?' in value row constructor; routing to failure: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: ParseException line 1:58 cannot recognize input near '?' ',' '?' in value row constructor ... View more

Splits a JSON File into multiple, separate FlowFiles for an array element specified by a JsonPath expression. Each generated FlowFile is comprised of an element of the specified array and transferred to relationship 'split,' with the original file transferred to the 'original' relationship. If the specified JsonPath is not found or does not evaluate to an array element, the original file is routed to 'failure' and no files are generated. regards, titaniumtv download ... View more

HI @Shu. Could you please explain what sysdate, current_date, etc would do for me with the spark job? I dont fully understand how to use them and the benefits that this technique would offer. ... View more

The above question and the entire response thread below was originally posted in the Community Help track. On Sun Jun 30 21:03 UTC 2019, a member of the HCC moderation staff moved it to the Data Ingestion & Streaming track. The Community Help Track is intended for questions about using the HCC site itself, not technical questions about Nifi processors. ... View more