Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: The Cloudera Community will undergo maintenance on Saturday, August 17 at 12:00am PDT. See more info here.

HowTo test morphline.conf with saxon using java?

SOLVED Go to solution

HowTo test morphline.conf with saxon using java?

Expert Contributor

Hi, I've seen this:

https://github.com/cloudera/cdk/blob/master/cdk-morphlines/cdk-morphlines-core/src/test/java/com/clo...

 

And I have no idea how to get access to parsed records.

 

I've seen this:

https://github.com/cloudera/cdk/blob/master/cdk-morphlines/cdk-morphlines-saxon/src/test/java/com/cl...

 

and I can't use it, because

1. it uses junit

2. because can't get access to com.cloudera.cdk.morphline.api.Collector, I don't see where artifact with classifier "test" is published.

 

What are the right approaches?

 

1 ACCEPTED SOLUTION

Accepted Solutions

Re: HowTo test morphline.conf with saxon using java?

Cloudera Employee
That?s pretty much what it already does. Read the first paragraph at https://github.com/kite-sdk/kite-examples/tree/master/kite-examples-morphlines again and look into the dirs mentioned there.


6 REPLIES 6

Re: HowTo test morphline.conf with saxon using java?

Cloudera Employee

Re: HowTo test morphline.conf with saxon using java?

Expert Contributor

Hi, thanks for the reply. It really looks like more "debug", than "test".

I do expect something like:

 

//groovy-like pseudocode using hamcrest

@Test

void testParseSmthUsingMorphline(){

    def aResult = doSomeTrickyStuff('a_path_to_morphline_config', 'a_path_to_input_dataset')

    assertThat(result, hasSize(3))

    assertThat(result.get(0).get('myProperty'), equalTo('some cool value'))

}

 

P.S.

Please add code highlighting!

 

 

 

Re: HowTo test morphline.conf with saxon using java?

Cloudera Employee
That?s pretty much what it already does. Read the first paragraph at https://github.com/kite-sdk/kite-examples/tree/master/kite-examples-morphlines again and look into the dirs mentioned there.


Re: HowTo test morphline.conf with saxon using java?

Expert Contributor

Oh, I've used wrong artifact, here is the right with <type>test-jar</type>:

Thanks!

 

 

<dependency>
<groupId>org.kitesdk</groupId>
<artifactId>kite-morphlines-core</artifactId>
<type>test-jar</type>
<scope>test</scope>
<version>${kite-version}</version>
</dependency>

Re: HowTo test morphline.conf with saxon using java?

Expert Contributor

Is there any possibility to contribute to project?

It would be great to decouple "test basement".

 

I see these major problems:

1. tightly coupled with junit

2. i have to download dozens of deps to make it run

3.   protected static final java.lang.String RESOURCES_DIR = "target/test-classes"; 

forces me to put configs under test resource. What is the reason to hardcode it.

 

I do get 

java.io.FileNotFoundException: File not found: target/test-classes/dummy-xml.conf while trying to run my test :(

 

I did put config to desired place then it just throws NPE

 

java.lang.NullPointerException: null
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:187)
at org.kitesdk.morphline.base.AbstractCommand.<init>(AbstractCommand.java:71)
at org.kitesdk.morphline.stdlib.Pipe.<init>(Pipe.java:38)
at org.kitesdk.morphline.stdlib.PipeBuilder.build(PipeBuilder.java:40)

 

checkNotNull what? Not to much info to make it work :(

Re: HowTo test morphline.conf with saxon using java?

Expert Contributor

NPE reason is in wrong test initialization order. I'ev found the problem