Hi @PJ, Could you please share what you ended up using at the end? Thanks ... View more

Thank you. I solved it by using notify and wait processors. But I think the LookupAttribute solution sounds very interesting way for enrichment...Thanks Matt, I 've learned a lot from your responses. ... View more

@Matt Burgess, just to clarify my above question, I need to get some values from 1.ctl and use these values as attributes for 2.txt. GetFile>RouteOnAttribute(filename)>(if filename ends with ctl then: extractText>UpdateAttribute) How can I add these attributes to the attributes of 2.txt? I really appreciate your help. ... View more

Hi @Matt Burgess, I have a question with regards to combining attributes from two different flows. I would like to merge the attributes of two different flows without merging the contents. For example, looking at the following flow: I have different types of files and I would like to add some attributes based on the content of one type of files and then merge all attributes. I know using MergeContent I will be able to keep all unique attributes if the output of the MergeContent is set to merged. But I do not want to merge the contents of my files and would like to keep them as their original format. So, is there any way to merge the attributes of two different flows? Thanks, Sam ... View more

Hi @dts were you able to find a solution for automatic tagging of hive tables and columns? could you please share an example? ... View more

Hi @allen huang. I faced the same issue. so you solved the problem by removing some of the properties of hive-site.xml? Do you know which properties were making the problem? @Kuldeep Kulkarni, @Artem Ervits do you guys have any idea in this regards? ... View more

hi, @yang jifei, were you able to solve your issue? I am having similar problem as well. Could you please help me? ... View more

Hi, Thank you for providing these examples. I went through this one but could not make it work. the Job is being Killed for some reason. Looking at the LogError of the workflow, the following is the message I got: USER[admin] GROUP[-] TOKEN[] APP[pigwf] JOB[0000006-180125094008677-oozie-oozi-W] ACTION[0000006-180125094008677-oozie-oozi-W@pig_1] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.PigMain], exit code [2] Also, I looked at /var/log/oozie/oozie-error.log and I got the following message: 2018-01-25 11:20:07,800 WARN ParameterVerifier:523 - SERVER[my.server.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] The application does not define formal parameters in its XML definition 2018-01-25 11:20:07,808 WARN LiteWorkflowAppService:523 - SERVER[my.server.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] libpath [hdfs://my.server.com:8020/tmp/data/lib] does not exist 2018-01-25 11:20:30,527 WARN PigActionExecutor:523 - SERVER[PD-Hortonworks-DATANODE2.network.com] USER[admin] GROUP[-] TOKEN[] APP[pigwf] JOB[0000004-180125094008677-oozie-oozi-W] ACTION[0000004-180125094008677-oozie-oozi-W@pig_1] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.PigMain], exit code [2] I ran the same script as you suggested and also tested it on shell which got me the result I was looking for. Also, I tested the oozie workflow with your part1 tutorial "making a shell command" and it worked. Also, I checked the workflow.xml files and everything looks like yours. Could you please help me find what my problem is? Thanks, Sam ... View more

Hi Matt, I am trying to convert a series of XML files to csv files and save them on hdfs. I wrote a script that does the conversion and now I would like to use ExecutingStreamCommand processor. I tried to simulate your explanation of using ListFile->ExecutingStreamCommand->FetchFile->PutHDFS. It works well however, the name of the FlowFile is not updating after using the FetchFile processor, which I know that FetchFile was never design to change the FlowFile. So, the converted files that are saved on hdfs are now .xml, instead of .csv. Now my question is how can I update the name of the FlowFile? Should I add an other ExecutingStreamCommand for renaming? Thanks ... View more

Hi, Have you been able to make it work? Could you please share your script here? Thanks ... View more