question Re: HowTo test morphline.conf with saxon using java? in Archives of Support Questions (Read Only)

HowTo test morphline.conf with saxon using java?

sergey.sheypak566881637 — Mon, 13 Oct 2014 17:38:19 GMT

Hi, I've seen this:

https://github.com/cloudera/cdk/blob/master/cdk-morphlines/cdk-morphlines-core/src/test/java/com/cloudera/cdk/morphline/api/MorphlineDemo.java

And I have no idea how to get access to parsed records.

I've seen this:

https://github.com/cloudera/cdk/blob/master/cdk-morphlines/cdk-morphlines-saxon/src/test/java/com/cloudera/cdk/morphline/saxon/SaxonMorphlineTest.java

and I can't use it, because

1. it uses junit

2. because can't get access to com.cloudera.cdk.morphline.api.Collector, I don't see where artifact with classifier "test" is published.

What are the right approaches?

Re: HowTo test morphline.conf with saxon using java?

whosch — Mon, 13 Oct 2014 20:12:15 GMT

This is described here: https://github.com/kite-sdk/kite-examples/tree/master/kite-examples-morphlines

Re: HowTo test morphline.conf with saxon using java?

sergey.sheypak566881637 — Mon, 13 Oct 2014 20:26:02 GMT

Hi, thanks for the reply. It really looks like more "debug", than "test".

I do expect something like:

//groovy-like pseudocode using hamcrest

@Test

void testParseSmthUsingMorphline(){

def aResult = doSomeTrickyStuff('a_path_to_morphline_config', 'a_path_to_input_dataset')

assertThat(result, hasSize(3))

assertThat(result.get(0).get('myProperty'), equalTo('some cool value'))

}

P.S.

Please add code highlighting!

Re: HowTo test morphline.conf with saxon using java?

whosch — Mon, 13 Oct 2014 20:38:15 GMT

That?s pretty much what it already does. Read the first paragraph at https://github.com/kite-sdk/kite-examples/tree/master/kite-examples-morphlines again and look into the dirs mentioned there.

Re: HowTo test morphline.conf with saxon using java?

sergey.sheypak566881637 — Tue, 14 Oct 2014 08:04:21 GMT

Oh, I've used wrong artifact, here is the right with <type>test-jar</type>:

Thanks!

<groupId>org.kitesdk</groupId>

<artifactId>kite-morphlines-core</artifactId>

<version>${kite-version}</version>

</dependency>

Re: HowTo test morphline.conf with saxon using java?

sergey.sheypak566881637 — Tue, 14 Oct 2014 16:45:04 GMT

Is there any possibility to contribute to project?

It would be great to decouple "test basement".

I see these major problems:

1. tightly coupled with junit

2. i have to download dozens of deps to make it run

3. protected static final java.lang.String RESOURCES_DIR = "target/test-classes";

forces me to put configs under test resource. What is the reason to hardcode it.

I do get

java.io.FileNotFoundException: File not found: target/test-classes/dummy-xml.conf while trying to run my test 😞

I did put config to desired place then it just throws NPE

java.lang.NullPointerException: null
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:187)
at org.kitesdk.morphline.base.AbstractCommand.<init>(AbstractCommand.java:71)
at org.kitesdk.morphline.stdlib.Pipe.<init>(Pipe.java:38)
at org.kitesdk.morphline.stdlib.PipeBuilder.build(PipeBuilder.java:40)

checkNotNull what? Not to much info to make it work 😞

Re: HowTo test morphline.conf with saxon using java?

sergey.sheypak566881637 — Tue, 14 Oct 2014 18:14:31 GMT

NPE reason is in wrong test initialization order. I'ev found the problem