Support Questions
Find answers, ask questions, and share your expertise

Flume Morphline Error with Solr

Hi all,

I am trying to ingest data to Solr 6.3 using flume 1.5 on Hortonworks 2.5 platform Facing below issue while sinking the data

19 Apr 2017 19:54:26,943 ERROR [lifecycleSupervisor-1-3] (org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run:253) - Unable to start SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@130344d7 counterGroup:{ name:null counters:{} } } - Exception follows.

org.kitesdk.morphline.api.MorphlineCompilationException: No command builder registered for name: detectMimeType near: {

# /etc/flume/conf/morphline.conf: 48

"detectMimeType" : {

# /etc/flume/conf/morphline.conf: 50

"includeDefaultMimeTypes" : true

}

}

The morphline config file is as below

id : morphline1

importCommands : ["org.kitesdk.**", "org.apache.solr.**"]

# importCommands : ["com.cloudera.**", "org.kitesdk.**"]

commands :

[

{ detectMimeType { includeDefaultMimeTypes : true } }

{

solrCell {

solrLocator : ${solrLocator}

captureAttr : true

lowernames : true

capture : [_attachment_body, _attachment_mimetype, basename, content, content_encoding, content_type, file, meta,text]

parsers : [ # { parser : org.apache.tika.parser.txt.TXTParser }

# { parser : org.apache.tika.parser.AutoDetectParser }

#{ parser : org.apache.tika.parser.asm.ClassParser }

#{ parser : org.gagravarr.tika.FlacParser }

#{ parser : org.apache.tika.parser.executable.ExecutableParser }

#{ parser : org.apache.tika.parser.font.TrueTypeParser }

#{ parser : org.apache.tika.parser.xml.XMLParser }

#{ parser : org.apache.tika.parser.html.HtmlParser }

#{ parser : org.apache.tika.parser.image.TiffParser }

# { parser : org.apache.tika.parser.mail.RFC822Parser }

#{ parser : org.apache.tika.parser.mbox.MboxParser, additionalSupportedMimeTypes : [message/x-emlx] }

#{ parser : org.apache.tika.parser.microsoft.OfficeParser }

#{ parser : org.apache.tika.parser.hdf.HDFParser }

#{ parser : org.apache.tika.parser.odf.OpenDocumentParser }

#{ parser : org.apache.tika.parser.pdf.PDFParser }

#{ parser : org.apache.tika.parser.rtf.RTFParser }

{ parser : org.apache.tika.parser.txt.TXTParser }

#{ parser : org.apache.tika.parser.chm.ChmParser }

]

fmap : { content : text }

}

}

{ generateUUID { field : id } }

{ sanitizeUnknownSolrFields { solrLocator : ${solrLocator} } }

{ logDebug { format : "output record: {}", args : ["@{}"] } }

{ loadSolr: { solrLocator : ${solrLocator} } }

]

}

]

I have copied all required jars files to Flume Classpath Kindly let me know the solution for this issue

Regards,

~Sri