Reply
New Contributor
Posts: 7
Registered: ‎09-19-2013
Accepted Solution

MR Job using Hive libraries

Hi all,

 

I am using Cloudera manager

 

 Cloudera Standard 4.7.2 (#135 built by jenkins on 20130918-2007 git: 72d3f9dfa797fe2c627d00dc6414a1e0151b91d6)

 

CDH 4.4.0-1.cdh4.4.0.p0.39

 

I have deployed hive and the hive2 server to one of the boxes on my cluster, along with it being a Journal Node and HDFS Balencer. When I do a hadoop fs -ls /user/, i get a bunch of directories; but if I kick off a MR job via the hive --service jar ... I am getting a

 

Exception in thread "main" java.io.FileNotFoundException: File /user does not exist
at org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:334)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1462)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1502)
at org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:570)

 

because it looks like it is not picking up the right core-site.xml, etc. I have deployed the client configuration for both hive and hdfs. Anyone have an idea?

 

thanks

 

dave

New Contributor
Posts: 7
Registered: ‎09-19-2013

Re: MR Job using Hive libraries

Nevermind, figured it out. This job used to run on a box with hdfs and MR installed. Only runs on a box with hdfs config. I was asking for the file system via the JobClient and was getting back the local file system.
New Contributor
Posts: 7
Registered: ‎09-19-2013

Re: MR Job using Hive libraries

Is there a way in Cloudera Manager to put MR configs on a box that isn't a TT or Job tracker?

Highlighted
Cloudera Employee
Posts: 508
Registered: ‎07-30-2013

Re: MR Job using Hive libraries

Click on your MapReduce service, go to the instances tab, and add a Gateway to the host where you want configs. Then deploy client configuration.

 

In general, you can use gateway roles to deploy client configs to hosts. This works for services like Hive and HBase as well.

 

Thanks,

Darren

Announcements