Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Chaining morphlines

avatar
Explorer

Hi...

 

What is the recommended way of chaining independent morphlines together?

 

For example, assume there are multiple such morphlines, each capable of running on its own to implement a well-defined function, and the Java application determines that two or more such functions are successively needed end-to-end (e.g., one to read froma data source and the next to extract the values from the input).

 

One approach of course would be to execute each morphline from within Java and pass the record to the next, until the chain is completed.

 

Is there another way to do so (e.g., somehow compiling together the independent conf files)?

 

 

1 ACCEPTED SOLUTION

avatar
Super Collaborator

Arrange it such that morphlineA pipe records into morphlineB:

 

Command morphlineB = new Compiler().compile(morphlineFileB, morphlineIdB, morphlineContextB, null);
Command morphlineA = new Compiler().compile(morphlineFileA, morphlineIdA, morphlineContextA, morphlineB);

View solution in original post

4 REPLIES 4

avatar
Super Collaborator

Arrange it such that morphlineA pipe records into morphlineB:

 

Command morphlineB = new Compiler().compile(morphlineFileB, morphlineIdB, morphlineContextB, null);
Command morphlineA = new Compiler().compile(morphlineFileA, morphlineIdA, morphlineContextA, morphlineB);

avatar
Explorer

Yes, that seems to be the best strategy.

 

I was experimenting with putting the text from different morphline files, each containing a single morphline, via string substitution (e.g., using the Apache Commons Lang StrSubstitutor class).

 

That approach does not work if you use text substitution in the morphline file, because in that case the "declarations' have to be outside the morphline, at the top of the file, so when the files are merged compilation fails.

 

It would have been nice if HOCON supported a C-style #include command. 🙂

 

Thanks for the advice!

avatar
Super Collaborator

avatar
Explorer

This does not work for merging with text substitution for the same reason I mentioned in my previous comment, but you are right, it is include functionality.

 

I was probably biased towards the use case for which I created this thread. 🙂

 

Thanks.