Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Dynamic morphline alteration

Solved Go to solution

Dynamic morphline alteration

Explorer

Hi...

 

What would be the best way of changing parts of a morphline at runtime, i.e. after it has been build from the CONF file?

 

A classic use case for this, is applications running many morphlines that differ only in a few commands. For example, a TryRules pipeline with multiple commands, out of which only 1 or 2 get modified every time, depending on the input.

 

In such cases, the "base" morphline can be built with "empty" (no-op) commands as placeholders from the main CONF file and the applicable commands can then be compiled from small CONF files and inserted in the position of the placeholders, before the whole pipeline can be executed.

 

Has anyone done this? Any code examples?

 

Thanks.

 

 

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Dynamic morphline alteration

Expert Contributor
Basically, yes. Beyon see there are a gazillion text templating tools already out there, e.g. freemarker et al. No need to reinvent the wheel.

4 REPLIES 4

Re: Dynamic morphline alteration

Expert Contributor
Seems like sed or similar on the text of the morphline config file are your friends here.

Highlighted

Re: Dynamic morphline alteration

Explorer

Hmmm...

 

You mean, read the morphline config file, do a text "search and replace" and then compile it?

 

The problem I described (having lots of config code replication) is very common, maybe you should look at providing a more "programatic" solution. :-)

Re: Dynamic morphline alteration

Expert Contributor
Basically, yes. Beyon see there are a gazillion text templating tools already out there, e.g. freemarker et al. No need to reinvent the wheel.

Re: Dynamic morphline alteration

Explorer

Sure.

 

FreeMarker is a good idea, especially now that they have switched to the Apache 2.0 License.

 

Don't have an account?
Coming from Hortonworks? Activate your account here