Reply
New Contributor
Posts: 2
Registered: ‎01-13-2014

Re: HFile PriviledgedActionException - cause missing JAR?

I am not sure why/if i had to do this (didn't see anything like this in any of the startup docs anywhere)

but I ran one of these:

 

hdfs dfs -put /hadoop/hbase-0.96.1.1-hadoop2/lib/*.jar /hadoop/hbase-0.96.1.1-hadoop2/lib


And....my MR job seems to run now

 

thanks,
Mark

Highlighted
New Contributor
Posts: 2
Registered: ‎01-13-2014

HFile PriviledgedActionException - cause missing JAR?

Hello,

 

I have a single node setup in pseudodistributed mode and am attempting to create HFiles for bulk loading

I have successfully been able to run simple MR inserts and lookups, however when submitting the MR job to create HFiles I am getting an PriviledgedActionException that indicates dependent JARs cannot be found in HDFS

 

I am not clear what the reference to HDFS here is for (i.e. why is the submit looking for the JARS in HDFS?)

I suspect I have a configuration error?

 

Any advice here is sincerely appreaciated - let me know if there is any other detail i can provide.


Thanks,
Mark

 

Here's my exception:

 

2014-01-13 13:39:40,582 INFO  [main] mapreduce.HFileOutputFormat: Incremental table pcoa output configured.
2014-01-13 13:39:40,608 INFO  [main] Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2014-01-13 13:39:40,609 INFO  [main] jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
2014-01-13 13:39:40,862 WARN  [main] mapreduce.JobSubmitter: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
2014-01-13 13:39:40,877 WARN  [main] mapreduce.JobSubmitter: No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
2014-01-13 13:39:40,880 INFO  [main] mapreduce.JobSubmitter: Cleaning up the staging area file:/tmp/hadoop-mbeening/mapred/staging/mbeening2031672872/.staging/job_local2031672872_0001
2014-01-13 13:39:40,880 ERROR [main] security.UserGroupInformation: PriviledgedActionException as:mbeening (auth:SIMPLE) cause:java.io.FileNotFoundException: File does not exist: hdfs://localhost/local/mbeening/hbase/hbase-0.96.1.1-hadoop2/lib/protobuf-java-2.5.0.jar
Exception in thread "main" java.io.FileNotFoundException: File does not exist: hdfs://localhost/local/mbeening/hbase/hbase-0.96.1.1-hadoop2/lib/protobuf-java-2.5.0.jar
        at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1110)
        at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1102)
        at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
        at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1102)
        at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.getFileStatus(ClientDistributedCacheManager.java:288)
        at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.getFileStatus(ClientDistributedCacheManager.java:224)
        at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.determineTimestamps(ClientDistributedCacheManager.java:93)
        at org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.determineTimestampsAndCacheVisibilities(ClientDistributedCacheManager.java:57)
        at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:264)
        at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:300)
        at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:387)
        at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1268)
        at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:1265)
        at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1286)
        at com.hadoop.BulkImportDriver.main(BulkImportDriver.java:63)

I have confirmed that the JAR is question does exist on my system

 

-rw-r--r-- 1 mbeening ACL_UNIX_flprod 533455 Aug 21 12:47 /local/mbeening/hbase/hbase-0.96.1.1-hadoop2/lib/protobuf-java-2.5.0.jar

Here is my driver routine:

 

public static void main(String[] args) throws Exception {

    Configuration conf = new Configuration();
    Job job = new Job(conf, "HBase Bulk Import");

    job.setJarByClass(HBaseKVMapper.class);
    job.setMapperClass(HBaseKVMapper.class);
    job.setMapOutputKeyClass(ImmutableBytesWritable.class);
    job.setMapOutputValueClass(KeyValue.class);

    job.setInputFormatClass(TextInputFormat.class);

    HTable hTable = new HTable(conf, args[2]);
    HFileOutputFormat.configureIncrementalLoad(job, hTable);

    FileInputFormat.addInputPath(job, new Path(args[0]));
    FileOutputFormat.setOutputPath(job, new Path(args[1]));

    job.waitForCompletion(true);
}

 

Posts: 416
Topics: 51
Kudos: 85
Solutions: 49
Registered: ‎06-26-2013

Re: HFile PriviledgedActionException - cause missing JAR?

Mark,

 

  I have moved this thread to our Mapreduce board in hopes that one of the experts in here might have some insight into why that happened.  It does sound odd to me that you had to do that.

 

Regards,

 

Clint

Announcements