Reply
Highlighted
Expert Contributor
Posts: 162
Registered: ‎07-29-2013
Accepted Solution

HowTo test morphline.conf with saxon using java?

Hi, I've seen this:

https://github.com/cloudera/cdk/blob/master/cdk-morphlines/cdk-morphlines-core/src/test/java/com/clo...

 

And I have no idea how to get access to parsed records.

 

I've seen this:

https://github.com/cloudera/cdk/blob/master/cdk-morphlines/cdk-morphlines-saxon/src/test/java/com/cl...

 

and I can't use it, because

1. it uses junit

2. because can't get access to com.cloudera.cdk.morphline.api.Collector, I don't see where artifact with classifier "test" is published.

 

What are the right approaches?

 

Cloudera Employee
Posts: 144
Registered: ‎08-21-2013

Re: HowTo test morphline.conf with saxon using java?

Expert Contributor
Posts: 162
Registered: ‎07-29-2013

Re: HowTo test morphline.conf with saxon using java?

Hi, thanks for the reply. It really looks like more "debug", than "test".

I do expect something like:

 

//groovy-like pseudocode using hamcrest

@Test

void testParseSmthUsingMorphline(){

    def aResult = doSomeTrickyStuff('a_path_to_morphline_config', 'a_path_to_input_dataset')

    assertThat(result, hasSize(3))

    assertThat(result.get(0).get('myProperty'), equalTo('some cool value'))

}

 

P.S.

Please add code highlighting!

 

 

 

Cloudera Employee
Posts: 144
Registered: ‎08-21-2013

Re: HowTo test morphline.conf with saxon using java?

That?s pretty much what it already does. Read the first paragraph at https://github.com/kite-sdk/kite-examples/tree/master/kite-examples-morphlines again and look into the dirs mentioned there.


Expert Contributor
Posts: 162
Registered: ‎07-29-2013

Re: HowTo test morphline.conf with saxon using java?

Oh, I've used wrong artifact, here is the right with <type>test-jar</type>:

Thanks!

 

 

<dependency>
<groupId>org.kitesdk</groupId>
<artifactId>kite-morphlines-core</artifactId>
<type>test-jar</type>
<scope>test</scope>
<version>${kite-version}</version>
</dependency>
Expert Contributor
Posts: 162
Registered: ‎07-29-2013

Re: HowTo test morphline.conf with saxon using java?

[ Edited ]

Is there any possibility to contribute to project?

It would be great to decouple "test basement".

 

I see these major problems:

1. tightly coupled with junit

2. i have to download dozens of deps to make it run

3.   protected static final java.lang.String RESOURCES_DIR = "target/test-classes"; 

forces me to put configs under test resource. What is the reason to hardcode it.

 

I do get 

java.io.FileNotFoundException: File not found: target/test-classes/dummy-xml.conf while trying to run my test :(

 

I did put config to desired place then it just throws NPE

 

java.lang.NullPointerException: null
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:187)
at org.kitesdk.morphline.base.AbstractCommand.<init>(AbstractCommand.java:71)
at org.kitesdk.morphline.stdlib.Pipe.<init>(Pipe.java:38)
at org.kitesdk.morphline.stdlib.PipeBuilder.build(PipeBuilder.java:40)

 

checkNotNull what? Not to much info to make it work :(

Expert Contributor
Posts: 162
Registered: ‎07-29-2013

Re: HowTo test morphline.conf with saxon using java?

NPE reason is in wrong test initialization order. I'ev found the problem

 

Announcements
The Kite SDK is a collection of docs, sample code, APIs, and tools to make Hadoop application development faster. Learn more at http://kitesdk.org.