Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Failed to write to parent HDFS directory.

avatar
Explorer

error-puthdfs1.pngambari-main.jpgerror.jpgerror-puthdfs.pngI am not able to write my files to the parent HDFS directory /tmp/tweets_staging. Please have a look at it, I am attaching the snapshot of the error which I am facing.

Though, I am able to see the data on solr as well as Banana. And in the NiFi template, PutHDFS is the only processor which is showing error.
1 ACCEPTED SOLUTION

avatar
Master Mentor

@Ronak Jangir

The HDFS client does not currently support the LzoCodec and the core-site.xml file you are using includes it.

It should work after you remove “com.hadoop.compression.lzo.LzoCodec,com.hadoop.compression.lzo.LzopCodec” from the “io.compression.codecs” property in the “core-site.xml” file you have referenced in your putHDFS processor.

Thanks,

Matt

View solution in original post

8 REPLIES 8

avatar
Master Mentor

@Ronak Jangir

The HDFS client does not currently support the LzoCodec and the core-site.xml file you are using includes it.

It should work after you remove “com.hadoop.compression.lzo.LzoCodec,com.hadoop.compression.lzo.LzopCodec” from the “io.compression.codecs” property in the “core-site.xml” file you have referenced in your putHDFS processor.

Thanks,

Matt

avatar
Explorer

Right on! Thanks @mclark that did the trick.

avatar

@Matt Clarke

i have the same issue. how can i go to this file ? i dont find the way to go to this path and this file.

Best Regards,

Martin

avatar
Master Mentor

@Martin van Husen

The core-site.xml file is copied from your HDFS node to your NiFi node. You modify the local copy on NiFi as described above and point to this file using the the "Hadoop Configuration Resources" property in the NiFi HDFS processor.

avatar

Thanks for your fast answer. yes the steps to modify and where to put the path in the processor i understand. But where can i find this xml file. What is the NiFi node ? iam working in the sandbox. Can i reach this file from the ambari manager ?

avatar
Master Mentor

I don't know where HDFS files are placed in the sandbox, but I know you cannot copy the file from HDFS to NiFi from within Ambari. You will need to do this via a command/terminal window. you could use the "locate" command if you are running on linux and have the "mlocate" package installed.

# yum -y install mlocate # updatedb # locate core-ste.xml

avatar

I found the file now, downloaded it and changed the entry. How do i get this file now back into my sandbox to the right place ?

avatar

No iam running the sandbox in win 7. ok i will keep searching for the xml file. Thank you.