Reply
Highlighted
Explorer
Posts: 8
Registered: ‎01-11-2017
Accepted Solution

HDFS Directory Structure Best Practices

Hi-

   Can someone point me to a good resource for "best practices" for a hadoop directory structure for storing raw data, intermediate files, output files, metadata etc in HDFS?   Do you segregate different data types into different directory structures?   Are the directory structures labeled per YYMMDD?  What would a typical HDFS directory structure look like when setting up to store data? 

Cloudera Employee
Posts: 42
Registered: ‎08-16-2016

Re: HDFS Directory Structure Best Practices

Eric Sammer (author of Hadoop Operations) has written a great answer about the same here:

https://www.quora.com/What-is-the-best-directory-structure-to-store-different-types-of-logs-in-HDFS-...

Hadoop Operations is a great book and has quite a few good tricks.
Announcements