Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

HowTo test morphline.conf with saxon using java?

avatar
Expert Contributor

Hi, I've seen this:

https://github.com/cloudera/cdk/blob/master/cdk-morphlines/cdk-morphlines-core/src/test/java/com/clo...

 

And I have no idea how to get access to parsed records.

 

I've seen this:

https://github.com/cloudera/cdk/blob/master/cdk-morphlines/cdk-morphlines-saxon/src/test/java/com/cl...

 

and I can't use it, because

1. it uses junit

2. because can't get access to com.cloudera.cdk.morphline.api.Collector, I don't see where artifact with classifier "test" is published.

 

What are the right approaches?

 

1 ACCEPTED SOLUTION

avatar
Super Collaborator
That?s pretty much what it already does. Read the first paragraph at https://github.com/kite-sdk/kite-examples/tree/master/kite-examples-morphlines again and look into the dirs mentioned there.


View solution in original post

6 REPLIES 6

avatar
Super Collaborator

avatar
Expert Contributor

Hi, thanks for the reply. It really looks like more "debug", than "test".

I do expect something like:

 

//groovy-like pseudocode using hamcrest

@Test

void testParseSmthUsingMorphline(){

    def aResult = doSomeTrickyStuff('a_path_to_morphline_config', 'a_path_to_input_dataset')

    assertThat(result, hasSize(3))

    assertThat(result.get(0).get('myProperty'), equalTo('some cool value'))

}

 

P.S.

Please add code highlighting!

 

 

 

avatar
Super Collaborator
That?s pretty much what it already does. Read the first paragraph at https://github.com/kite-sdk/kite-examples/tree/master/kite-examples-morphlines again and look into the dirs mentioned there.


avatar
Expert Contributor

Oh, I've used wrong artifact, here is the right with <type>test-jar</type>:

Thanks!

 

 

<dependency>
<groupId>org.kitesdk</groupId>
<artifactId>kite-morphlines-core</artifactId>
<type>test-jar</type>
<scope>test</scope>
<version>${kite-version}</version>
</dependency>

avatar
Expert Contributor

Is there any possibility to contribute to project?

It would be great to decouple "test basement".

 

I see these major problems:

1. tightly coupled with junit

2. i have to download dozens of deps to make it run

3.   protected static final java.lang.String RESOURCES_DIR = "target/test-classes"; 

forces me to put configs under test resource. What is the reason to hardcode it.

 

I do get 

java.io.FileNotFoundException: File not found: target/test-classes/dummy-xml.conf while trying to run my test 😞

 

I did put config to desired place then it just throws NPE

 

java.lang.NullPointerException: null
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:187)
at org.kitesdk.morphline.base.AbstractCommand.<init>(AbstractCommand.java:71)
at org.kitesdk.morphline.stdlib.Pipe.<init>(Pipe.java:38)
at org.kitesdk.morphline.stdlib.PipeBuilder.build(PipeBuilder.java:40)

 

checkNotNull what? Not to much info to make it work 😞

avatar
Expert Contributor

NPE reason is in wrong test initialization order. I'ev found the problem