Member since
12-17-2016
106
Posts
11
Kudos Received
0
Solutions
07-06-2018
05:31 AM
In nifi, I need to filter csv(without header) files based on values. I used RouteOnAttribute processor but I can't able to filter because those CSV files don't have a header. Anyone help me to solve this issue thanks
... View more
Labels:
- Labels:
-
Apache NiFi
11-07-2017
06:14 AM
My ConvertAvroToJSON processor is not working problem also i can't stop them without restrarting NIFI app. I attach my app.log files below ERROR [Timer-Driven Process Thread-6] o.a.n.processors.avro.ConvertAvroToJSON ConvertAvroToJSON[id=015e1000-89c2-18cc-0df4-55d016c88d45] ConvertAvroToJSON[id=015e1000-89c2-18cc-0df4-55d016c88d45] failed to process due to java.lang.IllegalStateException: Partition is closed; rolling back session: java.lang.IllegalStateException: Partition is closed 2017-11-03 16:00:08,173 ERROR [Timer-Driven Process Thread-6] o.a.n.processors.avro.ConvertAvroToJSON java.lang.IllegalStateException: Partition is closed at org.wali.MinimalLockingWriteAheadLog$Partition.update(MinimalLockingWriteAheadLog.java:945) ~[nifi-write-ahead-log-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.wali.MinimalLockingWriteAheadLog.update(MinimalLockingWriteAheadLog.java:238) ~[nifi-write-ahead-log-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.controller.repository.WriteAheadFlowFileRepository.updateRepository(WriteAheadFlowFileRepository.java:210) ~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.controller.repository.WriteAheadFlowFileRepository.updateRepository(WriteAheadFlowFileRepository.java:178) ~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:363) ~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:305) ~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:28) ~[nifi-api-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1099) [nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) [nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) [nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) [nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_151] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_151] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_151] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_151] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_151] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_151] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_151] 2017-11-03 16:00:08,174 ERROR [Timer-Driven Process Thread-6] o.a.n.processors.avro.ConvertAvroToJSON ConvertAvroToJSON[id=015e1000-89c2-18cc-0df4-55d016c88d45] ConvertAvroToJSON[id=015e1000-89c2-18cc-0df4-55d016c88d45] failed to process session due to java.lang.IllegalStateException: Partition is closed: java.lang.IllegalStateException: Partition is closed 2017-11-03 16:00:08,174 ERROR [Timer-Driven Process Thread-6] o.a.n.processors.avro.ConvertAvroToJSON java.lang.IllegalStateException: Partition is closed at org.wali.MinimalLockingWriteAheadLog$Partition.update(MinimalLockingWriteAheadLog.java:945) ~[nifi-write-ahead-log-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.wali.MinimalLockingWriteAheadLog.update(MinimalLockingWriteAheadLog.java:238) ~[nifi-write-ahead-log-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.controller.repository.WriteAheadFlowFileRepository.updateRepository(WriteAheadFlowFileRepository.java:210) ~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.controller.repository.WriteAheadFlowFileRepository.updateRepository(WriteAheadFlowFileRepository.java:178) ~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:363) ~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:305) ~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:28) ~[nifi-api-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1099) ~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) [nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) [nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) [nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_151] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_151] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_151] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_151] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_151] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_151] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_151]
... View more
Labels:
- Labels:
-
Apache NiFi
10-25-2017
07:52 AM
Thanks Adrian Oprea
... View more
10-25-2017
07:51 AM
Thanks Wynner
... View more
10-25-2017
07:40 AM
Thanks Shu I successfully got correct data
... View more
10-24-2017
11:19 AM
I have lot of log files I need to get some specific file(based on date) for that I use regular expression in GetFile(file filter option). I have file with name nifi-app.log nifi-app_2017-10-23_10.0.log nifi-app_2017-10-23_20.0.log nifi-setup.log nifi-user_2017-10-06.log In above files I need only nifi-app_2017* files only for I use regular expression ".*app_2017\.log". After I start flow in result no data is flowed. Can anyone help me whether my regular expression is correct or wrong?
... View more
Labels:
- Labels:
-
Apache NiFi
10-04-2017
04:58 AM
Thanks Shu
... View more
10-03-2017
09:24 AM
I am using HDP 2.5. I try to add time for file which is locate in HDFS file. For that I use GetHDFS->UpdateAttribute->PutHDFS. First I get file from HDFS through GetHDFS processor and then I change format of file in UpdateAttribute by adding property " ${filename}.${now():format("yyyy-MM-dd-HH:mm:ss.SSS'z'")}". Finally I try to put file in HDFS that time I got issue " Failed to write to HDFS due to java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: .test.csv.2017-10-03:03:19.814z: java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: .test.csv.2017-10-03:03:19.814z" . UpdateAttribute PutHDFS
... View more
Labels:
- Labels:
-
Apache NiFi
08-21-2017
01:56 PM
Thanks Quentin T
... View more
08-21-2017
05:00 AM
I am using HDP2.5 . I try to inject data into postgresql through Nifi. I attach my process here When I start my Nifi flow, I got issue on " Data is replicate in postgresql". I just pass 5 records but it shows me 300 records in postgres database
... View more
Labels:
- Labels:
-
Apache NiFi