Created 03-07-2018 08:56 AM
Hi experts,
When I ran a program for map-reduce in eclipse, I met an issue here. Pls help on it. Thanks.
Exception in thread "main" org.apache.hadoop.security.AccessControlException: SIMPLE authentication is not enabled. Available:[TOKEN] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73) at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2118) at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1305) at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1301) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1301) at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1424) at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:145) at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:266) at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:139) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287) at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1308) at WordCount.main(WordCount.java:131)
Created 03-07-2018 08:59 AM
Please check the "core-site.xml" that you are using in the classpath of your MapReduce code and grep for the property "hadoop.security.authentication" if you see that it is "kerberos" then it means you are trying to access a secure cluster So in that case you should have a valid kerberos ticket.
Please see: https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/SecureMode.html
The value of this property decides how will it behave.
"simple" : No authentication. (default)
"kerberos" : Enable authentication by Kerberos.
.
Created 03-07-2018 09:04 AM
If the cluster you are trying to connect is secure (means Kerberos Enabled) then you can refer to the following article to know about how to connect to it : https://community.hortonworks.com/articles/56702/a-secure-hdfs-client-example.html
.
If the cluster that you are trying to connect to , is not secure then you might be using incorrect (core-site.xml / hdfs-site.xml) or incorrect property like "hadoop.security.authentication" which might be creating confusion here. Please check the classpath resources/configs.