question Ignoring a morphline error in Archives of Support Questions (Read Only)

Ignoring a morphline error

PNS — Thu, 29 May 2014 11:39:01 GMT

Hi...

Is there any way of ignoring an error in a morphline, which consists of a series of commands, without modiying the offending command to be more tolerant?

For example, if one of the commands is Grok and a large file is processed and somewhere in the file a record does not match the regular expression, the execution of the morphline stops. Is there any configuration option to allow it to continue?

Thanks,

PNS

Re: Ingnoring a morphline error

whosch — Thu, 29 May 2014 12:31:30 GMT

See the answer here: https://groups.google.com/a/cloudera.org/forum/#!topic/cdk-dev/4PkFFmG59vk

Wolfgang.

Re: Ignoring a morphline error

PNS — Thu, 29 May 2014 18:39:10 GMT

Thanks for the prompt answer!

Adding to the Morphlines Reference Guide, the tryRules block has to be placed inside the commands block, e.g.

morphlines : [
    {
        id : morphline1
        importCommands : ["org.kitesdk.**", "org.apache.solr.**"]

        commands : [
            {
                readLine {
                    charset : UTF-8
                }
            }

            {
                tryRules {
                    catchExceptions : false
                    throwExceptionIfAllRulesFailed : true

                    rules : [

                    # next rule of tryRules cmd:
                    {
                        commands : [
                          { ... }
                          { ... }
                        ]
                    }

                    # next rule of tryRules cmd:
                    {
                        commands : [
                          { ... }
                          { ... }
                        ]
                    }
                }
            }
         ]
     }