Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Append Flow unique id

Highlighted

Append Flow unique id

I have a requirement where I want to generate a unique id once the flow is triggered and this will be used as a sessionid of the flow So I have used GenerateFlowFile processor with ${UUID()} so I want to append this uuid to each flowfile so that I can use it in sitetosite provenance for monitoring.

How can I store or share this in entire flow for each trigger?

DistributedMapCache can be used here? I am not sure if this will be able to handle session wise and in concurrent request?

3 REPLIES 3
Highlighted

Re: Append Flow unique id

Super Guru

Each flow file in the system automatically has an attribute "uuid" set to a unique identifier, this attribute is available in provenance events, via Expression Language (use "uuid" instead of "UUID()"). Is this what you are looking for? Or do you want a separate attribute that would remain the same even if the flow changes flow files (Split, Merge, e.g.)?

Highlighted

Re: Append Flow unique id

Master Guru

Or are you looking for a unique ID that is assigned to every FlowFile that is traverses a specific set of processors (dataflow) on the NiFi canvas. That way you can track which FlowFiles were processed by which dataflow?

If so, rather then using the UUID() NiFi Expression Language (EL) function , why not just have an updateAttribute processor in each on of these unique dataflows set a static value on each FlowFile? for exampe:

sessionID = dataflow1




Highlighted

Re: Append Flow unique id

No I want whenever the nifi flow is triggered there should be a uniqueid across all processors.

Don't have an account?
Coming from Hortonworks? Activate your account here