Support Questions
Find answers, ask questions, and share your expertise
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

call/invoke a pipe from morphline

call/invoke a pipe from morphline

New Contributor


I have a morphline file which has two morphlines; lets say id:morphline1 and id:morphline:2. Both of these morphline has its specific task to do but there is a set of commands which is common to both the morphlines


So, instead of repeating the same commands in two morphline blocks, I want to have a pipe defined outside of these morphlines and would like to include this pipe based on condition check.

Is there a way, we can call the pipe in a morphline using its id.



Re: call/invoke a pipe from morphline

New Contributor

I've been trying to answer this same question myself, this morning.


After digging around the morphlines code for PipeBuilder and Pipe I've come to the conclusion that there is no way to do this - nothing actually makes the "id" of the pipe available or stores it anywhere for reference by other commands.


The only way to achieve the same ends seems to be to define a single pipe in a separate file, and then use { include "path/to/file.conf" } to include it where you want to execute it.


Here are examples of defining the pipe and including the file from the morphlines source code test cases.


(Note how the substitution of the include statement for the file contents means that you don't need outer brackets around the pipe command.)


If anyone knows otherwise, or there are plans to improve this, please pipe (sorry...) up!