Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.
narasimha_varma
Explorer

Filter csv(without header) file based on values us...

by Explorer in Support Questions
‎07-06-2018 05:31 AM
‎07-06-2018 05:31 AM
In nifi, I need to filter csv(without header) files based on values. I used RouteOnAttribute processor but I can't able to filter because those CSV files don't have a header. Anyone help me to solve this issue thanks ... View more
Labels:

ConvertAvrotoJSON processor problem

by Explorer in Support Questions
‎11-07-2017 06:14 AM
‎11-07-2017 06:14 AM
My ConvertAvroToJSON processor is not working problem also i can't stop them without restrarting NIFI app. I attach my app.log files below ERROR [Timer-Driven Process Thread-6] o.a.n.processors.avro.ConvertAvroToJSON ConvertAvroToJSON[id=015e1000-89c2-18cc-0df4-55d016c88d45] ConvertAvroToJSON[id=015e1000-89c2-18cc-0df4-55d016c88d45] failed to process due to java.lang.IllegalStateException: Partition is closed; rolling back session: java.lang.IllegalStateException: Partition is closed 2017-11-03 16:00:08,173 ERROR [Timer-Driven Process Thread-6] o.a.n.processors.avro.ConvertAvroToJSON java.lang.IllegalStateException: Partition is closed at org.wali.MinimalLockingWriteAheadLog$Partition.update(MinimalLockingWriteAheadLog.java:945) ~[nifi-write-ahead-log-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.wali.MinimalLockingWriteAheadLog.update(MinimalLockingWriteAheadLog.java:238) ~[nifi-write-ahead-log-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.controller.repository.WriteAheadFlowFileRepository.updateRepository(WriteAheadFlowFileRepository.java:210) ~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.controller.repository.WriteAheadFlowFileRepository.updateRepository(WriteAheadFlowFileRepository.java:178) ~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:363) ~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:305) ~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:28) ~[nifi-api-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1099) [nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) [nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) [nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) [nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_151] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_151] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_151] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_151] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_151] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_151] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_151] 2017-11-03 16:00:08,174 ERROR [Timer-Driven Process Thread-6] o.a.n.processors.avro.ConvertAvroToJSON ConvertAvroToJSON[id=015e1000-89c2-18cc-0df4-55d016c88d45] ConvertAvroToJSON[id=015e1000-89c2-18cc-0df4-55d016c88d45] failed to process session due to java.lang.IllegalStateException: Partition is closed: java.lang.IllegalStateException: Partition is closed 2017-11-03 16:00:08,174 ERROR [Timer-Driven Process Thread-6] o.a.n.processors.avro.ConvertAvroToJSON java.lang.IllegalStateException: Partition is closed at org.wali.MinimalLockingWriteAheadLog$Partition.update(MinimalLockingWriteAheadLog.java:945) ~[nifi-write-ahead-log-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.wali.MinimalLockingWriteAheadLog.update(MinimalLockingWriteAheadLog.java:238) ~[nifi-write-ahead-log-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.controller.repository.WriteAheadFlowFileRepository.updateRepository(WriteAheadFlowFileRepository.java:210) ~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.controller.repository.WriteAheadFlowFileRepository.updateRepository(WriteAheadFlowFileRepository.java:178) ~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:363) ~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:305) ~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:28) ~[nifi-api-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1099) ~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) [nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) [nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) [nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_151] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_151] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_151] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_151] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_151] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_151] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_151] ... View more
Labels:

Re: how to use File filter in GetFile processor at...

by Explorer in Support Questions
‎10-25-2017 07:52 AM
‎10-25-2017 07:52 AM
Thanks Adrian Oprea ... View more

Re: how to use File filter in GetFile processor at...

by Explorer in Support Questions
‎10-25-2017 07:51 AM
‎10-25-2017 07:51 AM
Thanks Wynner ... View more

Re: how to use File filter in GetFile processor at...

by Explorer in Support Questions
‎10-25-2017 07:40 AM
‎10-25-2017 07:40 AM
Thanks Shu I successfully got correct data ... View more

how to use File filter in GetFile processor at Nif...

by Explorer in Support Questions
‎10-24-2017 11:19 AM
‎10-24-2017 11:19 AM
I have lot of log files I need to get some specific file(based on date) for that I use regular expression in GetFile(file filter option). I have file with name nifi-app.log nifi-app_2017-10-23_10.0.log nifi-app_2017-10-23_20.0.log nifi-setup.log nifi-user_2017-10-06.log In above files I need only nifi-app_2017* files only for I use regular expression ".*app_2017\.log". After I start flow in result no data is flowed. Can anyone help me whether my regular expression is correct or wrong? ... View more
Labels:

Re: Error in PutHDFS while adding Time to file

by Explorer in Support Questions
‎10-04-2017 04:58 AM
‎10-04-2017 04:58 AM
Thanks Shu ... View more

Error in PutHDFS while adding Time to file

by Explorer in Support Questions
‎10-03-2017 09:24 AM
‎10-03-2017 09:24 AM
I am using HDP 2.5. I try to add time for file which is locate in HDFS file. For that I use GetHDFS->UpdateAttribute->PutHDFS. First I get file from HDFS through GetHDFS processor and then I change format of file in UpdateAttribute by adding property " ${filename}.${now():format("yyyy-MM-dd-HH:mm:ss.SSS'z'")}". Finally I try to put file in HDFS that time I got issue " Failed to write to HDFS due to java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: .test.csv.2017-10-03:03:19.814z: java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: .test.csv.2017-10-03:03:19.814z" . UpdateAttribute PutHDFS ... View more
Labels:

Re: Data replicated in postgreSQL while inject th...

by Explorer in Support Questions
‎08-21-2017 01:56 PM
‎08-21-2017 01:56 PM
Thanks Quentin T ... View more

Data replicated in postgreSQL while inject throug...

by Explorer in Support Questions
‎08-21-2017 05:00 AM
‎08-21-2017 05:00 AM
I am using HDP2.5 . I try to inject data into postgresql through Nifi. I attach my process here When I start my Nifi flow, I got issue on " Data is replicate in postgresql". I just pass 5 records but it shows me 300 records in postgres database ... View more
Labels:
Contact Me
Online
Offline
Last Visited
‎06-17-2019 09:13 AM
Public Statistics
Date Registered ‎12-17-2016 04:21 AM
Last Visited ‎06-17-2019 09:13 AM
Total Messages Posted 106
Total Kudos Received 11