Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Cannot parse morphline file error

Cannot parse morphline file error

Explorer

I am trying to use a Morphline conf for simple transform commands on my input data. I am using the same mapreduce code as specified at:-

https://github.com/sequenceiq/sequenceiq-samples/blob/master/mapreduce-morphline/src/main/java/com/s...

 

Mapreduce job is getting submitted but is failing with the following error:-

Error: org.kitesdk.morphline.api.MorphlineCompilationException: Cannot parse morphline file: conv2avro_morphline.conf

....................

Caused by: java.io.FileNotFoundException: File not found: convert2avro_morphline.conf

 

The conf file exists in the location where I am running the JAR from.

 

Can you please guide me here?

 

Thanks

 

6 REPLIES 6

Re: Cannot parse morphline file error

Explorer
Please help, thanks
Highlighted

Re: Cannot parse morphline file error

Expert Contributor

The morphline conf file must be present on the local file system of each Mapper task. For example, you can submit it there via the --files CLI option of MR job, in which case the file will end up in the CWD of the mapper task process, where you can point to it via a relative file path. 

Re: Cannot parse morphline file error

Explorer
thanks for the response. Is there a sample code for this?

Re: Cannot parse morphline file error

Explorer

Ok, I modified the code to process the add it to distributed cache, but I am getting the following error:-

Error: java.io.FileNotFoundException: File /home/myuser/convert2avro_morphline.conf does not exist. at org.apache.hadoop.util.GenericOptionsParser.validateFiles(GenericOptionsParser.java:403) at org.apache.hadoop.util.GenericOptionsParser.processGeneralOptions(GenericOptionsParser.java:301) at org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:485) at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170) at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153) at com.bbt.Hadoop.HandleMorphline.runMorphline.addDistributedCacheFile(runMorphline.java:74) at com.bbt.Hadoop.HandleMorphline.runMorphline.access$0(runMorphline.java:68) at com.bbt.Hadoop.HandleMorphline.runMorphline$Cleaner.setup(runMorphline.java:53) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142) at


Please note that I tried copying the conf file to /home/myuser folder manually on all datanodes with no luck.

Please help out as soon as you can.

Thanks

Re: Cannot parse morphline file error

Explorer

I was able to find resolution for the below, now my morphline conf is being executed by the mapper without any issue, but my output file doesn't contain valid AVRO data, its not even binary as shown below:-

 

{_attachment_body=[[B@2c0f7678], affectedproduct=[Client Service], agencycloseddate=[], agencyduedate=[], agencyrefnumber=[], agencyvendorinvolved=[], assignedforreview=[], bankname=[FUNNY RESTAURANT OF NORTH DAKOTA], casesubmittedby=[], centerid=[7171539], centername=[FUNNY RESTAURANT PHONE 24], CLERKuserbnumber=[], CLERKuserfullname=[], CLERKuserid=[], channelid=[PRT], cifkey=[NonProduct], clienthomephoneno=[0000000000], clientworkphoneno=[8506401131]}

 

My question is should the outputKeyClass/ValueClass etc. in my map-only job be specific to Avro or Text, is that causing this issue?

 

Would really appreicate if someone atleast answer this Q.

Re: Cannot parse morphline file error

Explorer

Can someone please help out.

 

Thanks

Don't have an account?
Coming from Hortonworks? Activate your account here