Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Max file cout in PutFile processor does not work

Solved Go to solution
Highlighted

Max file cout in PutFile processor does not work

New Contributor

I have putfile processor. Input directory "X:\ab\${path}" for recurse puting files and set max file count in input directory 10000, but it still writing there, not going to failed. Why? But when setting X:\ab\ whithout ${path} it works, but i need it. Any Ideas??

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Max file cout in PutFile processor does not work

Make sure you define the Auto terminate relationship properly and handled the exception in the PutFile processor. Make sure to route the failed message to failure queue.

6613-screen-shot-2016-08-12-at-110609-am.gif

You should able to see the following exception in the log.

2016-08-12 11:02:41,628 INFO [Timer-Driven Process Thread-10] o.a.nifi.processors.standard.PutFile PutFile[id=4540b2c4-9425-4817-a7e1-d9d2930a2d5b] Penalizing StandardFlowFileRecord[uuid=51e1f6cb-3a31-42e3-bd65-1f6a1f20eff0,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1471013498033-1, container=default, section=1], offset=23989, length=806],offset=0,name=903851051716952,size=806] and routing to 'failure' because the output directory /Users/mpandit/test has 2 files, which exceeds the configured maximum number of files

View solution in original post

2 REPLIES 2
Highlighted

Re: Max file cout in PutFile processor does not work

Make sure you define the Auto terminate relationship properly and handled the exception in the PutFile processor. Make sure to route the failed message to failure queue.

6613-screen-shot-2016-08-12-at-110609-am.gif

You should able to see the following exception in the log.

2016-08-12 11:02:41,628 INFO [Timer-Driven Process Thread-10] o.a.nifi.processors.standard.PutFile PutFile[id=4540b2c4-9425-4817-a7e1-d9d2930a2d5b] Penalizing StandardFlowFileRecord[uuid=51e1f6cb-3a31-42e3-bd65-1f6a1f20eff0,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1471013498033-1, container=default, section=1], offset=23989, length=806],offset=0,name=903851051716952,size=806] and routing to 'failure' because the output directory /Users/mpandit/test has 2 files, which exceeds the configured maximum number of files

View solution in original post

Highlighted

Re: Max file cout in PutFile processor does not work

New Contributor

Yes, I did this, but my output directory is d:/abc/${path}

Don't have an account?
Coming from Hortonworks? Activate your account here