Support Questions
Find answers, ask questions, and share your expertise

MR Job using Hive libraries

Solved Go to solution

MR Job using Hive libraries

Explorer

Hi all,

 

I am using Cloudera manager

 

 Cloudera Standard 4.7.2 (#135 built by jenkins on 20130918-2007 git: 72d3f9dfa797fe2c627d00dc6414a1e0151b91d6)

 

CDH 4.4.0-1.cdh4.4.0.p0.39

 

I have deployed hive and the hive2 server to one of the boxes on my cluster, along with it being a Journal Node and HDFS Balencer. When I do a hadoop fs -ls /user/, i get a bunch of directories; but if I kick off a MR job via the hive --service jar ... I am getting a

 

Exception in thread "main" java.io.FileNotFoundException: File /user does not exist
at org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:334)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1462)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1502)
at org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:570)

 

because it looks like it is not picking up the right core-site.xml, etc. I have deployed the client configuration for both hive and hdfs. Anyone have an idea?

 

thanks

 

dave

1 ACCEPTED SOLUTION

Accepted Solutions

Re: MR Job using Hive libraries

Click on your MapReduce service, go to the instances tab, and add a Gateway to the host where you want configs. Then deploy client configuration.

 

In general, you can use gateway roles to deploy client configs to hosts. This works for services like Hive and HBase as well.

 

Thanks,

Darren

View solution in original post

3 REPLIES 3

Re: MR Job using Hive libraries

Explorer
Nevermind, figured it out. This job used to run on a box with hdfs and MR installed. Only runs on a box with hdfs config. I was asking for the file system via the JobClient and was getting back the local file system.

Re: MR Job using Hive libraries

Explorer

Is there a way in Cloudera Manager to put MR configs on a box that isn't a TT or Job tracker?

Re: MR Job using Hive libraries

Click on your MapReduce service, go to the instances tab, and add a Gateway to the host where you want configs. Then deploy client configuration.

 

In general, you can use gateway roles to deploy client configs to hosts. This works for services like Hive and HBase as well.

 

Thanks,

Darren

View solution in original post