- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
is it possible to set the hadoop.tmp.dir value to HDFS path instead of local directory?
- Labels:
-
Apache Hadoop
Created ‎08-04-2017 06:22 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
is it possible to set the hadoop.tmp.dir value to HDFS path instead of local directory? thank you.
Created ‎08-04-2017 02:34 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Setting this property creates a tmp directory in BOTH local and HDFS.
It does so in HDFS because other properties use hadoop.tmp.dir as a base path to store data in HDFS. Example: dfs.name.dir=${hadoop.tmp.dir}/dfs/name creates this path in hdfs.
There is no way to have this property NOT create a path locally.
See these links for a good discussion:
https://stackoverflow.com/questions/2354525/what-should-be-hadoop-tmp-dir
Created ‎08-04-2017 02:34 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Setting this property creates a tmp directory in BOTH local and HDFS.
It does so in HDFS because other properties use hadoop.tmp.dir as a base path to store data in HDFS. Example: dfs.name.dir=${hadoop.tmp.dir}/dfs/name creates this path in hdfs.
There is no way to have this property NOT create a path locally.
See these links for a good discussion:
https://stackoverflow.com/questions/2354525/what-should-be-hadoop-tmp-dir
