Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to use epoch in Enforce order when its data type is int?

Highlighted

How to use epoch in Enforce order when its data type is int?

Explorer

Hi there,

I'm trying to use the enforce order processor and use epoch to be the order attribute. However, each time I tried to do this, i got the following error:

Failed to parse order attribute due to java.lang.NumberFormatException: For input string: "103317120000
e7fe73b,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1548066389226-175, container=default, section=175], offset=755611, length=200],offset=0,name=40b54e6a-32c6-4065-8c79-ccff66313b2f.avro,size=200]:

Looking into the code of the enforceOrder attribute, it turns out that the processor is using the Integer type, and its a known issue / fact that current epochs' wont fit in the JAVA 32bit integer.

   orderAttribute = processContext.getProperty(ORDER_ATTRIBUTE).getValue();
            waitTimeoutMillis = processContext.getProperty(WAIT_TIMEOUT).asTimePeriod(TimeUnit.MILLISECONDS);
            getOrder = flowFile -> Integer.parseInt(flowFile.getAttribute(orderAttribute));

Question: Why did the team decide on int? Can this be changed to Long as that would be the way to go? Is there a different strategy to use the epoch with this processor?

2 REPLIES 2
Highlighted

Re: How to use epoch in Enforce order when its data type is int?

Hi @Yasir Khokhar

EnforceOrder processor is not designed to order FlowFiles using epoch. It's designed to enforce order using a continuous numbers, such as sequence. Skipping numbers such as epoch cannot be used EnforceOrder.

I think you can achieve what you expect with PriorityAttributePrioritizer instead. Please check NiFi docs on prioritizers.

https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#prioritization

Re: How to use epoch in Enforce order when its data type is int?

Explorer

Thanks @kkawamura we're testing that approach now!

Don't have an account?
Coming from Hortonworks? Activate your account here