Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

addValues after readJson fails - is our understanding of this correct?

addValues after readJson fails - is our understanding of this correct?

Explorer

We are testing a morphline with JUnit and finding that we are getting unexpected behavior.  To simplify the explanation, and because it's difficult to

copy the code from an isolated network, I'm showing pseudo-code:

 

Morphline:

addValues { "hello" : "world" }

readJson ...

extractJsonPaths ...

toAvro ....

writeAvroToByteArray ...

addValues { "foo" : "bar" }

logInfo entire record   // this properly shows 'hello' and 'foo' keys in record

 

JUnit test:

Various setup

Record input = new Record()

Command morphline = ...

Put json byte[] in record ATTACHMENT_BODY

morphline.process(input)

assert that input.getFields().containsKey("hello")   //   PASSES

assert that input.getFields().containsKey("foo")   //   FAILS

 

We think this second assertion is failing because jsonRead can process multiple records and so in essence the first addValues is adding the value to a 'different' record.  Is this true?

 

 

 

 

1 REPLY 1
Highlighted

Re: addValues after readJson fails

Expert Contributor
readJson copies the input record and adds to these new record objects.

Don't have an account?
Coming from Hortonworks? Activate your account here