Reply
Champion Alumni
Posts: 196
Registered: ‎11-18-2014

HADOOP_HOME/HADOOP_PREFIX is not set - Cloudera 5

Hello,

 

I'm having an  HADOOP_HOME is not set error: 

 

DEBUG Shell: Failed to detect a valid hadoop home directory
java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
	at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:302)
	at org.apache.hadoop.util.Shell.<clinit>(Shell.java:327)
	at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:79)
	at org.apache.hadoop.security.Groups.parseStaticMapping(Groups.java:104)
	at org.apache.hadoop.security.Groups.<init>(Groups.java:86)
	at org.apache.hadoop.security.Groups.<init>(Groups.java:66)
	at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:280)
	at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:269)
	at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:246)
	at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:775)
	at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:760)
	at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:633)
	at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:2753)
	at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:2745)
	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2611)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:169)

 

However, I know that HADOOP_HOME is depreciated. Also, there is no mapReduce installed on my cluster (only map reduce 2 -yarn is installed).

Therefore I thought I should have an HADOOP_PREFIX error. (I do not understand) 

 

The error is triggered at the second line of the folowing code. 

 

	Configuration conf = new Configuration();
		FileSystem hdfs = FileSystem.get(conf);
		Path pathInputEvent= new Path(FOLDER);
		Path localFilePath = new Path(LOCAL_FOLDER);
		
		if(hdfs.exists(pathInputEvent)){
			hdfs.delete(pathInputEvent,false);
			hdfs.copyFromLocalFile(localFilePath, pathInputEvent);
		}

 

 I tried to set HADOOP_HOME in eclipse by setting the value /opt/cloudera/parcels/CDH/lib/hadoop to the HADOOP_HOME variable in run configurations->environement

 

Note: I got the value from the forum http://community.cloudera.com/t5/Cloudera-Manager-Installation/Where-is-HADOOP-HOME-lib-on-CDH-5-0-1...

 

This it didn't worked :I had an error that said that hadoop home directory is an absolute path or it doesn't exist - an IOException.

 

Why is the error asking for HADOOP_HOME and not HADOOP_PREFIX? And what value should I set for which variable?

 

Thank you!

 

 

 

GHERMAN Alina
krb
New Contributor
Posts: 5
Registered: ‎07-25-2018

Re: HADOOP_HOME/HADOOP_PREFIX is not set - Cloudera 5

solved: I restarted the cloudera agent where the namenode is running..

>service  cloudera-scm-agent restart

 

it fixed the issue -- HADOOP_HOME/HADOOP_PREFIX is not set -----in cloudera 5.15

Announcements