Found the solution!  Just needed to apply the regex  (?:"b":.\w+?)(\d+) thanks to @Shu_ashu   ... View more

Thanks a lot!  ... View more

Thanks for the info, was very helpful! ... View more

Hi,   Exit code 143 is related to Memory/GC issues. Your default Mapper/reducer memory setting may not be sufficient to run the large data set. Thus, try setting up higher AM, MAP and REDUCER memory when a large yarn job is invoked.  For more please refer to this link. https://stackoverflow.com/questions/42972908/container-killed-by-the-applicationmaster-exit-code-is-143   Thanks AKR ... View more

@Shu_ashu  This is my dataflow -  https://drive.google.com/file/d/1SWtSAPKxRcgAWT7ca0dytePjfwMZIpgR/view?usp=sharing   The ListSFTP processor works fine. It lists all the files in the server. But FetchSFTP doesn't work as expected. I get comms.failure. My ListSFTP  and FetchSFTP configurations are the same.    Also I had one more doubt - How does FetchSFTP gets its state from LISTSFTP.? Is there any additional configuration that has to be done.? I am just joining ListSFTP and FetchSFTP with relationship Success. Not doing anything else.   Please help.      ... View more

@Haijin Li Try with this and this approaches and also to test out the regex serde functionality, Create new file using vi editor in shell and move it to HDFS directory and create table on top of this directory. ... View more

The above querstion and the entire reply thread below was originally posted in the Community Help Track. On Tue Jun 4 03:55 UTC 2019, a member of the HCC moderation staff moved it to the Data Processing track. The Community Help Track is intended for questions about using the HCC site itself. ... View more