Failed to write to parent HDFS directory.

[#1]

error-puthdfs1.pngambari-main.jpgerror.jpgerror-puthdfs.pngI am not able to write my files to the parent HDFS directory /tmp/tweets_staging. Please have a look at it, I am attaching the snapshot of the error which I am facing.

Though, I am able to see the data on solr as well as Banana. And in the NiFi template, PutHDFS is the only processor which is showing error.

Note: I have followed every bit of this page.

http://hortonworks.com/hadoop-tutorial/how-to-refine-and-visualize-sentiment-data/

[#2 · Accepted solution]

@Ronak Jangir

The HDFS client does not currently support the LzoCodec and the core-site.xml file you are using includes it.

It should work after you remove “com.hadoop.compression.lzo.LzoCodec,com.hadoop.compression.lzo.LzopCodec” from the “io.compression.codecs” property in the “core-site.xml” file you have referenced in your putHDFS processor.

Thanks,

Matt

[#3]

Right on! Thanks @mclark that did the trick.

[#4]

@Matt Clarke

i have the same issue. how can i go to this file ? i dont find the way to go to this path and this file.

Best Regards,

Martin

[#5]

@Martin van Husen

The core-site.xml file is copied from your HDFS node to your NiFi node. You modify the local copy on NiFi as described above and point to this file using the the "Hadoop Configuration Resources" property in the NiFi HDFS processor.

[#6]

Thanks for your fast answer. yes the steps to modify and where to put the path in the processor i understand. But where can i find this xml file. What is the NiFi node ? iam working in the sandbox. Can i reach this file from the ambari manager ?

[#7]

I don't know where HDFS files are placed in the sandbox, but I know you cannot copy the file from HDFS to NiFi from within Ambari. You will need to do this via a command/terminal window. you could use the "locate" command if you are running on linux and have the "mlocate" package installed.

# yum -y install mlocate # updatedb # locate core-ste.xml

[#8]

I found the file now, downloaded it and changed the entry. How do i get this file now back into my sandbox to the right place ?

[#9]

No iam running the sandbox in win 7. ok i will keep searching for the xml file. Thank you.