Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here. Want to know more about what has changed? Check out the Community News blog.

How to test morphlines

How to test morphlines

Expert Contributor

I'm looking into constructing a Flume morphlines interceptor and came across this sample code to test the morphlines.conf against sample records:

 

 https://github.com/cloudera/cdk/blob/master/cdk-morphlines/cdk-morphlines-core/src/test/java/com/clo...

 

But when I clone the cdk git repo and run 'mvn clean build' I get this error:

 

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:1.3.1:enforce (clean) on project cdk-parent: Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed. -> [Help 1]

 

I'm new to the mvn world and hopefully someone can point me to the right instructions to build the cdk project. Thanks!

5 REPLIES 5

Re: How to test morphlines

Expert Contributor

The CDK has been renamed to Kite and instructions how to build it are here:

https://github.com/kite-sdk/kite/

 

Re: How to test morphlines

Expert Contributor

Re: How to test morphlines

Expert Contributor

Thanks, I was able to build kite-examples-morphlines after updating the kite version to 0.17.1-SNAPSHOT.

 

How can we look at the output record when I run this test?

 

mvn test -DskipTests exec:java -Dexec.mainClass="org.kitesdk.morphline.api.MorphlineDemo" -Dexec.args="src/test/resources/test-morphlines/addValues.conf src/test/resources/test-documents/email.txt" -Dexec.classpathScope=test

 

Since I only notice this with Trace enabled but not the actual output record:

 

[org.kitesdk.morphline.api.MorphlineDemo.main()] DEBUG org.kitesdk.morphline.stdlib.LogDebugBuilder$LogDebug  - output record: [{_attachment_body=[java.io.BufferedInputStream@29b41961], names=[@{first_name}], pids=[456, hello], source_host=[123], source_type=[text/log, text/log2]}]

 

 

Re: How to test morphlines

Expert Contributor

To automatically print diagnostic information such as the content of records as they pass through the morphline commands, consider enabling TRACE log level, for example by adding the following line to your log4j.properties file:

 

log4j.logger.org.kitesdk.morphline=TRACE

 

Also see http://kitesdk.org/docs/current/kite-morphlines/morphlinesReferenceGuide.html#/logTrace

Highlighted

Re: How to test morphlines

Expert Contributor

As mentioned in the previous comment, I did already enbled TRACE but did not find anything related to the output records in the stdout except for this:

 

1588 [org.kitesdk.morphline.api.MorphlineDemo.main()] DEBUG org.kitesdk.morphline.stdlib.LogDebugBuilder$LogDebug  - output record: [{_attachment_body=[java.io.BufferedInputStream@5cde2dcb], names=[@{first_name}], pids=[456, hello], source_host=[123], source_type=[text/log, text/log2]}]

 

Please let me know how to look at the resulting transformation on the sample input records? Thanks!