Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HowTo test morphline.conf with saxon using java?

Solved Go to solution

HowTo test morphline.conf with saxon using java?

Expert Contributor

Hi, I've seen this:

https://github.com/cloudera/cdk/blob/master/cdk-morphlines/cdk-morphlines-core/src/test/java/com/clo...

 

And I have no idea how to get access to parsed records.

 

I've seen this:

https://github.com/cloudera/cdk/blob/master/cdk-morphlines/cdk-morphlines-saxon/src/test/java/com/cl...

 

and I can't use it, because

1. it uses junit

2. because can't get access to com.cloudera.cdk.morphline.api.Collector, I don't see where artifact with classifier "test" is published.

 

What are the right approaches?

 

1 ACCEPTED SOLUTION

Accepted Solutions

Re: HowTo test morphline.conf with saxon using java?

Expert Contributor
That?s pretty much what it already does. Read the first paragraph at https://github.com/kite-sdk/kite-examples/tree/master/kite-examples-morphlines again and look into the dirs mentioned there.


6 REPLIES 6

Re: HowTo test morphline.conf with saxon using java?

Expert Contributor
Highlighted

Re: HowTo test morphline.conf with saxon using java?

Expert Contributor

Hi, thanks for the reply. It really looks like more "debug", than "test".

I do expect something like:

 

//groovy-like pseudocode using hamcrest

@Test

void testParseSmthUsingMorphline(){

    def aResult = doSomeTrickyStuff('a_path_to_morphline_config', 'a_path_to_input_dataset')

    assertThat(result, hasSize(3))

    assertThat(result.get(0).get('myProperty'), equalTo('some cool value'))

}

 

P.S.

Please add code highlighting!

 

 

 

Re: HowTo test morphline.conf with saxon using java?

Expert Contributor
That?s pretty much what it already does. Read the first paragraph at https://github.com/kite-sdk/kite-examples/tree/master/kite-examples-morphlines again and look into the dirs mentioned there.


Re: HowTo test morphline.conf with saxon using java?

Expert Contributor

Oh, I've used wrong artifact, here is the right with <type>test-jar</type>:

Thanks!

 

 

<dependency>
<groupId>org.kitesdk</groupId>
<artifactId>kite-morphlines-core</artifactId>
<type>test-jar</type>
<scope>test</scope>
<version>${kite-version}</version>
</dependency>

Re: HowTo test morphline.conf with saxon using java?

Expert Contributor

Is there any possibility to contribute to project?

It would be great to decouple "test basement".

 

I see these major problems:

1. tightly coupled with junit

2. i have to download dozens of deps to make it run

3.   protected static final java.lang.String RESOURCES_DIR = "target/test-classes"; 

forces me to put configs under test resource. What is the reason to hardcode it.

 

I do get 

java.io.FileNotFoundException: File not found: target/test-classes/dummy-xml.conf while trying to run my test :(

 

I did put config to desired place then it just throws NPE

 

java.lang.NullPointerException: null
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:187)
at org.kitesdk.morphline.base.AbstractCommand.<init>(AbstractCommand.java:71)
at org.kitesdk.morphline.stdlib.Pipe.<init>(Pipe.java:38)
at org.kitesdk.morphline.stdlib.PipeBuilder.build(PipeBuilder.java:40)

 

checkNotNull what? Not to much info to make it work :(

Re: HowTo test morphline.conf with saxon using java?

Expert Contributor

NPE reason is in wrong test initialization order. I'ev found the problem