Reply
Highlighted
PNS
Explorer
Posts: 38
Registered: ‎05-20-2014
Accepted Solution

Ignoring a morphline error

[ Edited ]

Hi...

 

Is there any way of ignoring an error in a morphline, which consists of a series of commands, without modiying the offending command to be more tolerant?

 

For example, if one of the commands is Grok and a large file is processed and somewhere in the file a record does not match the regular expression, the execution of the morphline stops. Is there any configuration option to allow it to continue?

 

Thanks,

 

PNS

 

Cloudera Employee
Posts: 146
Registered: ‎08-21-2013

Re: Ingnoring a morphline error

PNS
Explorer
Posts: 38
Registered: ‎05-20-2014

Re: Ignoring a morphline error

[ Edited ]

Thanks for the prompt answer!

 

Adding to the Morphlines Reference Guide, the tryRules block has to be placed inside the commands block, e.g.

 

morphlines : [
    {
        id : morphline1
        importCommands : ["org.kitesdk.**", "org.apache.solr.**"]

        commands : [
            {
                readLine {
                    charset : UTF-8
                }
            }

            {
                tryRules {
                    catchExceptions : false
                    throwExceptionIfAllRulesFailed : true

                    rules : [

                    # next rule of tryRules cmd:
                    {
                        commands : [
                          { ... }
                          { ... }
                        ]
                    }

                    # next rule of tryRules cmd:
                    {
                        commands : [
                          { ... }
                          { ... }
                        ]
                    }
                }
            }
         ]
     }

 

 

Announcements
The Kite SDK is a collection of docs, sample code, APIs, and tools to make Hadoop application development faster. Learn more at http://kitesdk.org.