Member since
04-27-2016
13
Posts
1
Kudos Received
0
Solutions
05-19-2016
08:53 AM
@mclark @bbende I tried this solution and now it works! But...what if I need a compression not listed in the Processor's property "Compression codec"? For example if I need to compress with LZO codec? Do I have to create my own processor for this purpose and send the outputted compressed file (flowfile) to PutHdfs processor with "none" value in Compression codec property, or an easier way to obtain this behaviour exists? Thank you very much
... View more
05-18-2016
08:15 AM
Sure, preceding lines are
2016-05-18 11:10:18,646 INFO [StandardProcessScheduler Thread-1] o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled PutHDFS[id=e985d9ea-b0f7-4e05-ad1e-f7e0c2df5c7c] to run with 1 threads
2016-05-18 11:10:18,646 INFO [Timer-Driven Process Thread-6] o.apache.nifi.processors.hadoop.PutHDFS PutHDFS[id=e985d9ea-b0f7-4e05-ad1e-f7e0c2df5c7c] Kerberos ticket age exceeds threshold [14400 seconds] attempting to renew ticket for user mclaw
2016-05-18 11:10:18,646 INFO [Timer-Driven Process Thread-6] o.apache.nifi.processors.hadoop.PutHDFS PutHDFS[id=e985d9ea-b0f7-4e05-ad1e-f7e0c2df5c7c] Kerberos relogin successful or ticket still valid
2016-05-18 11:10:18,647 ERROR [Timer-Driven Process Thread-6] o.apache.nifi.processors.hadoop.PutHDFS PutHDFS[id=e985d9ea-b0f7-4e05-ad1e-f7e0c2df5c7c] PutHDFS[id=e985d9ea-b0f7-4e05-ad1e-f7e0c2df5c7c] failed to process due to java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configurable; rolling back session: java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configurable
2016-05-18 11:10:18,649 ERROR [Timer-Driven Process Thread-6] o.apache.nifi.processors.hadoop.PutHDFS
... View more
05-18-2016
08:11 AM
As I answered above, I tried adding hdfs-site.xml but still not working. Thank you
... View more
05-18-2016
08:10 AM
The version is 0.6.1, the latest.
... View more
05-18-2016
08:09 AM
@Pierre Villard I posted a wrong picture, I tried with both core-site and hdfs-site, but it still doesn't work. Thank you for your reply.
... View more
05-17-2016
03:52 PM
1 Kudo
Hi, I configured the PUTHDFS processor to write on Hadoop as shown in the image, but it doesn't work. In the log I see the following error: java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configurable
at java.lang.ClassLoader.defineClass1(Native Method) ~[na:1.8.0_45]
at java.lang.ClassLoader.defineClass(ClassLoader.java:760) ~[na:1.8.0_45]
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[na:1.8.0_45]
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467) ~[na:1.8.0_45]
at java.net.URLClassLoader.access$100(URLClassLoader.java:73) ~[na:1.8.0_45]
at java.net.URLClassLoader$1.run(URLClassLoader.java:368) ~[na:1.8.0_45]
at java.net.URLClassLoader$1.run(URLClassLoader.java:362) ~[na:1.8.0_45]
at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_45]
at java.net.URLClassLoader.findClass(URLClassLoader.java:361) ~[na:1.8.0_45]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[na:1.8.0_45]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) ~[na:1.8.0_45]
at java.lang.ClassLoader.loadClass(ClassLoader.java:411) ~[na:1.8.0_45]
at java.lang.ClassLoader.loadClass(ClassLoader.java:411) ~[na:1.8.0_45]
at java.lang.ClassLoader.loadClass(ClassLoader.java:411) ~[na:1.8.0_45]
at java.lang.ClassLoader.loadClass(ClassLoader.java:411) ~[na:1.8.0_45]
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[na:1.8.0_45]
at java.lang.Class.forName0(Native Method) ~[na:1.8.0_45]
at java.lang.Class.forName(Class.java:348) ~[na:1.8.0_45]
at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:2093) ~[na:na]
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2058) ~[na:na]
at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:128) ~[na:na]
at org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFactory.java:175) ~[na:na]
at org.apache.nifi.processors.hadoop.AbstractHadoopProcessor.getCompressionCodec(AbstractHadoopProcessor.java:375) ~[na:na]
at org.apache.nifi.processors.hadoop.PutHDFS.onTrigger(PutHDFS.java:220) ~[na:na]
at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) ~[nifi-api-0.6.1.jar:0.6.1]
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1059) ~[nifi-framework-core-0.6.1.jar:0.6.1]
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) [nifi-framework-core-0.6.1.jar:0.6.1]
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) [nifi-framework-core-0.6.1.jar:0.6.1]
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:123) [nifi-framework-core-0.6.1.jar:0.6.1]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_45]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_45]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_45]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_45]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_45]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_45]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.conf.Configurable
at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[na:1.8.0_45]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[na:1.8.0_45]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) ~[na:1.8.0_45]
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[na:1.8.0_45]
... 36 common frames omitted Can anyone help me to make the processor working? Thank you
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache NiFi