question Re: Chaining morphlines in Archives of Support Questions (Read Only)

Chaining morphlines

PNS — Tue, 09 Dec 2014 16:59:15 GMT

Hi...

What is the recommended way of chaining independent morphlines together?

For example, assume there are multiple such morphlines, each capable of running on its own to implement a well-defined function, and the Java application determines that two or more such functions are successively needed end-to-end (e.g., one to read froma data source and the next to extract the values from the input).

One approach of course would be to execute each morphline from within Java and pass the record to the next, until the chain is completed.

Is there another way to do so (e.g., somehow compiling together the independent conf files)?

Re: Chaining morphlines

whosch — Tue, 09 Dec 2014 17:55:13 GMT

Arrange it such that morphlineA pipe records into morphlineB:

Command morphlineB = new Compiler().compile(morphlineFileB, morphlineIdB, morphlineContextB, null);
Command morphlineA = new Compiler().compile(morphlineFileA, morphlineIdA, morphlineContextA, morphlineB);

Re: Chaining morphlines

PNS — Tue, 09 Dec 2014 18:01:08 GMT

Yes, that seems to be the best strategy.

I was experimenting with putting the text from different morphline files, each containing a single morphline, via string substitution (e.g., using the Apache Commons Lang StrSubstitutor class).

That approach does not work if you use text substitution in the morphline file, because in that case the "declarations' have to be outside the morphline, at the top of the file, so when the files are merged compilation fails.

It would have been nice if HOCON supported a C-style #include command. 🙂

Thanks for the advice!

Re: Chaining morphlines

whosch — Tue, 09 Dec 2014 18:06:30 GMT

FWIW, also see https://github.com/typesafehub/config/blob/master/HOCON.md#includes

Re: Chaining morphlines

PNS — Tue, 09 Dec 2014 18:27:21 GMT

This does not work for merging with text substitution for the same reason I mentioned in my previous comment, but you are right, it is include functionality.

I was probably biased towards the use case for which I created this thread. 🙂

Thanks.