Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Flume Morphline Error with Solr

Flume Morphline Error with Solr

Hi all,

I am trying to ingest data to Solr 6.3 using flume 1.5 on Hortonworks 2.5 platform Facing below issue while sinking the data

19 Apr 2017 19:54:26,943 ERROR [lifecycleSupervisor-1-3] (org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run:253) - Unable to start SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@130344d7 counterGroup:{ name:null counters:{} } } - Exception follows.

org.kitesdk.morphline.api.MorphlineCompilationException: No command builder registered for name: detectMimeType near: {

# /etc/flume/conf/morphline.conf: 48

"detectMimeType" : {

# /etc/flume/conf/morphline.conf: 50

"includeDefaultMimeTypes" : true

}

}

The morphline config file is as below

id : morphline1

importCommands : ["org.kitesdk.**", "org.apache.solr.**"]

# importCommands : ["com.cloudera.**", "org.kitesdk.**"]

commands :

[

{ detectMimeType { includeDefaultMimeTypes : true } }

{

solrCell {

solrLocator : ${solrLocator}

captureAttr : true

lowernames : true

capture : [_attachment_body, _attachment_mimetype, basename, content, content_encoding, content_type, file, meta,text]

parsers : [ # { parser : org.apache.tika.parser.txt.TXTParser }

# { parser : org.apache.tika.parser.AutoDetectParser }

#{ parser : org.apache.tika.parser.asm.ClassParser }

#{ parser : org.gagravarr.tika.FlacParser }

#{ parser : org.apache.tika.parser.executable.ExecutableParser }

#{ parser : org.apache.tika.parser.font.TrueTypeParser }

#{ parser : org.apache.tika.parser.xml.XMLParser }

#{ parser : org.apache.tika.parser.html.HtmlParser }

#{ parser : org.apache.tika.parser.image.TiffParser }

# { parser : org.apache.tika.parser.mail.RFC822Parser }

#{ parser : org.apache.tika.parser.mbox.MboxParser, additionalSupportedMimeTypes : [message/x-emlx] }

#{ parser : org.apache.tika.parser.microsoft.OfficeParser }

#{ parser : org.apache.tika.parser.hdf.HDFParser }

#{ parser : org.apache.tika.parser.odf.OpenDocumentParser }

#{ parser : org.apache.tika.parser.pdf.PDFParser }

#{ parser : org.apache.tika.parser.rtf.RTFParser }

{ parser : org.apache.tika.parser.txt.TXTParser }

#{ parser : org.apache.tika.parser.chm.ChmParser }

]

fmap : { content : text }

}

}

{ generateUUID { field : id } }

{ sanitizeUnknownSolrFields { solrLocator : ${solrLocator} } }

{ logDebug { format : "output record: {}", args : ["@{}"] } }

{ loadSolr: { solrLocator : ${solrLocator} } }

]

}

]

I have copied all required jars files to Flume Classpath Kindly let me know the solution for this issue

Regards,

~Sri

Don't have an account?
Coming from Hortonworks? Activate your account here