Reply
New Contributor
Posts: 1
Registered: ‎02-20-2017

call/invoke a pipe from morphline

[ Edited ]

Hi,

I have a morphline file which has two morphlines; lets say id:morphline1 and id:morphline:2. Both of these morphline has its specific task to do but there is a set of commands which is common to both the morphlines

 

So, instead of repeating the same commands in two morphline blocks, I want to have a pipe defined outside of these morphlines and would like to include this pipe based on condition check.

Is there a way, we can call the pipe in a morphline using its id.

Thanks,
Sam


Highlighted
jrg
New Contributor
Posts: 1
Registered: ‎07-20-2018

Re: call/invoke a pipe from morphline

I've been trying to answer this same question myself, this morning.

 

After digging around the morphlines code for PipeBuilder and Pipe I've come to the conclusion that there is no way to do this - nothing actually makes the "id" of the pipe available or stores it anywhere for reference by other commands.

 

The only way to achieve the same ends seems to be to define a single pipe in a separate file, and then use { include "path/to/file.conf" } to include it where you want to execute it.

 

Here are examples of defining the pipe and including the file from the morphlines source code test cases.

 

(Note how the substitution of the include statement for the file contents means that you don't need outer brackets around the pipe command.)

 

If anyone knows otherwise, or there are plans to improve this, please pipe (sorry...) up!

Announcements
The Kite SDK is a collection of docs, sample code, APIs, and tools to make Hadoop application development faster. Learn more at http://kitesdk.org.