Member since
11-07-2019
71
Posts
1
Kudos Received
0
Solutions
02-17-2020
04:02 AM
Hello Thank you for your reply. Actually , it is not exact what you wrote. Here are the steps: Last login: Thu Feb 13 16:24:13 2020 [root@server~]# su hdfs [hdfs@server root]$ hdfs dfs -ls 20/02/17 12:54:29 WARN ipc.Client: Exception encountered while connecting to the server : org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS] ls: DestHost:destPort nameNodeServer:8020 , LocalHost:localPort server/10.48.142.32:0. Failed on local exception: java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS] [hdfs@server root]$ klist klist: No credentials cache found (filename: /tmp/krb5cc_1005) [hdfs@server root]$ [hdfs@server root]$ kinit -kt /etc/security/keytabs/hdfs.headless.keytab hdfs-analytics_hadoop@REALM.COM [hdfs@server root]$ klist Ticket cache: FILE:/tmp/krb5cc_1005 Default principal: hdfs-analytics_hadoop@REALM.COM Valid starting Expires Service principal 02/17/2020 12:56:11 02/17/2020 22:56:11 krbtgt/REALM.COM@REALM.COM renew until 02/24/2020 12:56:11 [hdfs@serverroot]$ hdfs dfs -ls Found 7 items drwx------ - hdfs mapred 0 2020-02-06 19:00 .Trash drwxr-xr-x - hdfs mapred 0 2019-11-21 15:50 .sparkStaging drwx------ - hdfs mapred 0 2019-11-19 17:17 .staging drwxr-xr-x - hdfs mapred 0 2019-11-19 16:33 QuasiMonteCarlo_1574177607896_1218363848 drwxr-xr-x - hdfs mapred 0 2019-11-19 16:41 QuasiMonteCarlo_1574178102082_2043590941 drwxr-xr-x - hdfs mapred 0 2019-11-19 16:50 QuasiMonteCarlo_1574178649791_1714364119 drwxr-xr-x - hdfs mapred 0 2019-11-19 17:17 QuasiMonteCarlo_1574180274656_991656908 [hdfs@server root]$ hdfs dfs -ls / Found 13 items drwxrwxrwt - yarn hadoop 0 2020-02-05 14:40 /app-logs drwxr-xr-x - hdfs hdfs 0 2019-11-18 18:16 /apps drwxr-xr-x - yarn hadoop 0 2019-11-15 17:58 /ats drwxr-xr-x - hdfs hdfs 0 2019-11-15 18:02 /atsv2 drwxr-xr-x - hdfs hdfs 0 2019-11-15 17:58 /hdp drwxr-xr-x - mapred hdfs 0 2019-11-15 17:58 /mapred drwxrwxrwx - mapred hadoop 0 2019-11-18 11:03 /mr-history drwxr-xr-x - hdfs hdfs 0 2019-11-15 17:58 /services drwxrwxrwx - edgeuser hdfs 0 2020-02-06 11:25 /share drwxrwxrwx - spark hadoop 0 2020-02-17 12:56 /spark2-history drwxrwxrwx - mapred mapred 0 2020-01-31 16:10 /tmp drwxr-xr-x - mapred mapred 0 2020-02-06 17:19 /user drwxr-xr-x - hdfs hdfs 0 2019-11-18 18:08 /warehouse The problem that it i written that this principle is valid until 17/02/2020. So when I will test again I will get the same error when trying to browse the hdfs How Could I odt in order to maintain the principle not to be expired I am using hadoop cluster with kerberos enabled and I have integrated linux machine with Active directory: samba + winbind Thanks and Regards
... View more
02-12-2020
06:54 AM
Dear Community
I used to configure Hadoop HortonWorks Cluster with Active directory : Samba and Winbind
I also configure Kerberos
Now when I add kerberos tickets for both root and AD users, t hey are getting expired after one day (24h) As configured in krb5.conf
I saw that in order to configure automatically tickets, I should use SSSD.
Coud this possible with winbind? If yes, is there any procedure to do it?
Many thanks in advance
... View more
Labels:
- Labels:
-
Kerberos
02-12-2020
03:39 AM
This what I did : [hdfs@server root]$ kdestroy [hdfs@serverroot]$ klist klist: No credentials cache found (filename: /tmp/krb5cc_1005) [hdfs@server root]$ kinit -kt /etc/security/keytabs/hdfs.headless.keytab hdfs-analytics_hadoop@realm.COM [hdfs@luapp698 root]$ klist Ticket cache: FILE:/tmp/krb5cc_1005 Default principal: hdfs-analytics_hadoop@realm.COM Valid starting Expires Service principal 02/12/2020 12:36:13 02/12/2020 22:36:13 krbtgt/realm.COM@realm.COM renew until 02/19/2020 12:36:13 Why does the principle expires ?? 😞 Valid starting Expires Service principal 02/12/2020 12:36:13 02/12/2020 22:36:13 krbtgt/realm.COM@realm.COM How am I supposed to configure it please? Thanks
... View more
02-12-2020
02:51 AM
one last question please should I use both kerberos and SSL for a secured cluster? thanks
... View more
- Tags:
- ne lads
02-10-2020
01:54 PM
Thank you for your pertinent answers as usual so there is no another option except configuring browser with SPENGo for browsers?! Is it recommander to configure the Hadoop cluster with https? If yes do you have a procedure please ? very appreciated 🙂
... View more
02-10-2020
08:10 AM
Hello Community
My actual HDP version is : HDP-3.1.4.0
I have enabled Kerberos on the hadoop cluster but still the security officer checked a lot of vulnerabilitites
1. we could access via these URLs the content without any authentication :
http://ambarinode:61310/conf
http://ambarinode:10002/conf
P.S : I did not install Hbase as a service
2. This URL also is accessible without authentication from all datanodes:
http://datanode1:8042/conf
3. Grafana si also accessible without authentication
http://namenode:3000
http://namenode:9060
Any ideas please
Thanks
Asma
... View more
Labels:
02-07-2020
08:49 AM
Dear Community,
I logged as user root and I set kinint for root user to browse hdfs = > Ok
I changed user as hdfs
su hdfs
[hdfs@luapp698 root]$ klist
Ticket cache: FILE:/tmp/krb5cc_1005
Default principal: hdfs-analytics_hadoop@LU.EMA.AD.ANEXAMPLE.COM
Valid starting Expires Service principal
02/05/2020 11:28:21 02/05/2020 21:28:21 krbtgt/LU.EMA.AD.ANEXAMPLE.COM@LU.EMA.AD.ANEXAMPLE.COM
renew until 02/12/2020 11:28:21
= > The principle is well set
However :
I am getting this
[hdfs@server root]$ hdfs dfs -ls /
20/02/07 17:44:24 WARN ipc.Client: Exception encountered while connecting to the server : org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]
ls: DestHost:destPort server:8020 , LocalHost:localPort luapp698.lu.ema.ad.anexample.com/10.48.142.27:0. Failed on local exception: java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]
#kdestroy
#[hdfs@lserver root]$ kinit -kt /etc/security/keytabs/hdfs.headless.keytab hdfs-analytics_hadoop@REALM.COM
[hdfs@server root]$ hdfs dfs -ls /
Found 13 items
drwxrwxrwt - yarn hadoop 0 2020-02-05 14:40 /app-logs
drwxr-xr-x - hdfs hdfs 0 2019-11-18 18:16 /apps
drwxr-xr-x - yarn hadoop 0 2019-11-15 17:58 /ats
drwxr-xr-x - hdfs hdfs 0 2019-11-15 18:02 /atsv2
drwxr-xr-x - hdfs hdfs 0 2019-11-15 17:58 /hdp
drwxr-xr-x - mapred hdfs 0 2019-11-15 17:58 /mapred
drwxrwxrwx - mapred hadoop 0 2019-11-18 11:03 /mr-history
drwxr-xr-x - hdfs hdfs 0 2019-11-15 17:58 /services
drwxrwxrwx - edgeuser hdfs 0 2020-02-06 11:25 /share
drwxrwxrwx - spark hadoop 0 2020-02-07 17:47 /spark2-history
drwxrwxrwx - mapred mapred 0 2020-01-31 16:10 /tmp
drwxr-xr-x - mapred mapred 0 2020-02-06 17:19 /user
drwxr-xr-x - hdfs hdfs 0 2019-11-18 18:08 /warehouse
=> any explanation please?
This issue occurs every day 😞
Thanks
... View more
02-06-2020
12:57 PM
In the second link that you have attached there is a link to centrifugal and it is about adding ticket for users , example we user .. it should be done manually for each user ..
... View more
02-06-2020
12:38 PM
Thank you for your reply I have configured first only ambari server and edge node with active directory (samba) then I have enabled kerberos i got a lot of principles generated on the Hadoop but for spark , yarn ... Users from AD could not connect and execute their jobs Then I configured others nodes name nodes and Dara nodes with Active directory and I added users manually and then they were able to use the cluster So I don’t understand why I did not get the automatically thing for AD users ? Should I regenerate keytabs from the ambari server ? It is really a blocking issue for me .. thanks a lot in advance
... View more
02-06-2020
08:58 AM
Dear Community,
Actually, I have Hadoop Hortonworks Cluster with Kerberos enabled.
Users to access are in AD repository.
I can add some users and test that users could submit their jobs successfully.
But the issue here that I have to add every user on all nodes and also register every user keytab on all nodes.
This is ok for 1 or 2 users but not for 50 or 100 😞
I am creating keytab using kutil , so i have to ask every user to tape his password!!
Is there any other tip to do this?
How can I register keytabs for a domain not per user?
Should I do it per user?
Any idea will be appreciated.
thanks
Asma
... View more
Labels:
- Labels:
-
Kerberos
02-06-2020
05:27 AM
Dear Community,
I have set the Kerberos on Hadoop Cluster Ambari on 8 nodes.
I was getting errors like
20/02/06 11:09:13 WARN ipc.Client: Exception encountered while connecting to the server : org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS] mkdir: DestHost:destPort namenode:8020 , LocalHost:localPort edgenode/10.48.142.32:0 . Failed on local exception: java.io.IOException : org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]
When I registered the principles on all nodes like :
kinit -kt /etc/security/keytabs/hdfs.headless.keytab hdfs-cluster@REALM.COM
and the error getting disappeared
I have integrated hadoop with active directory, so I also do this
su user1
kinit -kt user1.keytab user1@REALM.COM
otherwise user1 could not sent jobs to yarn
Should I create keytab per user? (active directory ones?)
The main issue that when I reconnect to the cluster the day after I get the same issue of kerberos token!!
I have to set again principals on all nodes and for all users 😞
Is there any way to set permanently these principles please?
Thanks a lot in advance
Asma
... View more
Labels:
- Labels:
-
Apache Ambari
-
HDFS
-
Kerberos
-
Security
02-04-2020
02:10 PM
Hello 🙂 I have the same issue! I have integrated the edge node with Active directory users could connect and submit theirs jobs to yarn before enabling Kerberos on the cluster. Actually I have used samba on edge node to create users folders and get information about users Now I configured the Kerberos and so I am getting the same error user1 not found , user1 is in AD should I now add this user with normal command add user on alll nodes ? How could it be as AD user and not local one ? I did not configured samba on others nodes may I do it ? thanks a lot in advance
... View more
02-04-2020
09:17 AM
Hello Please how did u add users? Actually i am using the active directory users and I just add them into Edge node using samba + kerberos Now I have enabled kerberos on the hadoop hortonworks cluster => I got the same issue as yours So may I add the same user to all nodes? adduser? which group? how could it be resolved as an AD user? Thanks
... View more
02-04-2020
08:55 AM
when i updated the yarn conf yarn.scheduler.capacity.root.default.acl_administer_jobs=yarn,*,user1 yarn.scheduler.capacity.root.default.acl_administer_queue=yarn,*,user1 yarn.scheduler.capacity.root.default.acl_submit_applications=yarn,yarn-ats,*,user1 => user1 is able to authenticate However he is getting this error now Error in rxRemoteExecute(computeContext, shellCmd, schedulerJobInstance) : /var/RevoShare/user1/cluster-127006E48F49439EA1A090A78C9851C9/ start-job.sh : line 97: export: `-Xrs': not a valid identifier /var/RevoShare/fcuni001/cluster-127006E48F49439EA1A090A78C9851C9/ start-job.sh : line 97: export: `-Xss4m': not a valid identifier ERROR: Fail to execute spark-submit. Last 20 lines' log: at org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:934) at org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:925) at scala.Option.getOrElse(Option.scala:121) at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:925) at com.microsoft.scaler.spark.api.SparkApp$.main(SparkApp.scala:28) at com.microsoft.scaler.spark.api.SparkApp.main(SparkApp.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.re On the namenode web interface Application application_1580827205892_0001 failed 2 times due to AM Container for appattempt_1580827205892_0001_000002 exited with exitCode: -1000 Failing this attempt.Diagnostics: [2020-02-04 15:43:08.042]Application application_1580827205892_0001 initialization failed (exitCode=255) with output: main : command provided 0 main : run as user is fcuni001 main : requested yarn user is user1 User user1 not found For more detailed output, check the application tracking page: http://namenode:8088/cluster/app/application_1580827205892_0001 Then click on links to logs of each attempt. . Failing the application. Any idea please? Thanks a lot
... View more
02-04-2020
08:50 AM
when i updated the yarn conf yarn.scheduler.capacity.root.default.acl_administer_jobs=yarn,*,user1 yarn.scheduler.capacity.root.default.acl_administer_queue=yarn,*,user1 yarn.scheduler.capacity.root.default.acl_submit_applications=yarn,yarn-ats,*,user1 => user1 is able to authenticate However he is getting this error now Error in rxRemoteExecute(computeContext, shellCmd, schedulerJobInstance) : /var/RevoShare/user1/cluster-127006E48F49439EA1A090A78C9851C9/ start-job.sh : line 97: export: `-Xrs': not a valid identifier /var/RevoShare/fcuni001/cluster-127006E48F49439EA1A090A78C9851C9/ start-job.sh : line 97: export: `-Xss4m': not a valid identifier ERROR: Fail to execute spark-submit. Last 20 lines' log: at org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:934) at org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:925) at scala.Option.getOrElse(Option.scala:121) at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:925) at com.microsoft.scaler.spark.api.SparkApp$.main(SparkApp.scala:28) at com.microsoft.scaler.spark.api.SparkApp.main(SparkApp.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.re On the namenode web interface Application application_1580827205892_0001 failed 2 times due to AM Container for appattempt_1580827205892_0001_000002 exited with exitCode: -1000 Failing this attempt.Diagnostics: [2020-02-04 15:43:08.042]Application application_1580827205892_0001 initialization failed (exitCode=255) with output: main : command provided 0 main : run as user is fcuni001 main : requested yarn user is user1 User user1 not found For more detailed output, check the application tracking page: http://namenode:8088/cluster/app/application_1580827205892_0001 Then click on links to logs of each attempt. . Failing the application. Any idea please? Thanks a lot
... View more
02-04-2020
05:40 AM
Dear Community,
After enabling Kerberos with Active Directory on HortonWorks Hadoop Cluster (Ambari), users are unable to submit jobs on yarn
Error
Error in rxRemoteExecute(computeContext, shellCmd, schedulerJobInstance) : /var/RevoShare/aduser/cluster-7B39D0A894BC4F73ABC73D192697AFC3/ start-job.sh : line 97: export: `-Xrs': not a valid identifier /var/RevoShare/aduser/cluster-7B39D0A894BC4F73ABC73D192697AFC3/ start-job.sh : line 97: export: `-Xss4m': not a valid identifier ERROR: Fail to execute spark-submit. Last 20 lines' log: at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2682) Caused by: org.apache.hadoop.security.AccessControlException: User aduser does not have permission to submit application_1580742122197_0003 to queue default at org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.createAndPopulateNewRMApp(RMAppManager.java:429) ... 12 more at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1498) at org.apache.hadoop.ipc.Client.call(Client.java:1444) at org.ap
I set these properties like
yarn.scheduler.capacity.root.default.acl_submit_applications=yarn,yarn-ats,*
yarn.scheduler.capacity.root.acl_submit_applications=yarn,ambari-qa,*
Please advice
Asma
... View more
Labels:
- Labels:
-
Apache YARN
02-03-2020
09:23 AM
should i create principle for each user in the AD ? We are using active directory users? If yes how so? Many thanks Asma
... View more
02-03-2020
08:55 AM
Actually, for more details: In my ambari server machine I have this ticket: [root@ambariserver ~]# klist Ticket cache: FILE:/tmp/krb5cc_0 Default principal: spark-analytics_hadoop@REALM.COM Valid starting Expires Service principal 02/03/2020 13:31:21 02/03/2020 23:31:21 krbtgt/REALM.COM@REALM.COM renew until 02/10/2020 13:31:21 When i connect with spark user : HADOOP_ROOT_LOGGER=DEBUG,console /usr/hdp/3.1.4.0-315/spark2/bin/spark-submit --class org.apache.spark.examples.SparkPi --master spark://Edgenode:7077 --num-executors 4 --driver-memory 512m --executor-memory 512m --executor-cores 1 /usr/hdp/3.1.4.0-315/spark2/examples/jars/spark-examples_2.11-2.3.2.3.1.4.0-315.jar => OK Now if I connect from the Edge Node [root@EdgeNode~]# klist Ticket cache: FILE:/tmp/krb5cc_0 Default principal: spark/EdgeNode@REALM.COM Valid starting Expires Service principal 02/03/2020 16:52:12 02/04/2020 02:52:12 krbtgt/REALM.COM@REALM.COM renew until 02/10/2020 16:52:12 But when I connect with user spark HADOOP_ROOT_LOGGER=DEBUG,console /usr/hdp/3.1.4.0-315/spark2/bin/spark-submit --class org.apache.spark.examples.SparkPi --master spark://Edgenode:7077 --num-executors 4 --driver-memory 512m --executor-memory 512m --executor-cores 1 /usr/hdp/3.1.4.0-315/spark2/examples/jars/spark-examples_2.11-2.3.2.3.1.4.0-315.jar => I got error : 20/02/03 17:53:01 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@69cac930{/metrics/json,null,AVAILABLE,@Spark} 20/02/03 17:53:01 WARN Client: Exception encountered while connecting to the server : org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS] 20/02/03 17:53:01 ERROR SparkContext: Error initializing SparkContext. java.io.IOException: DestHost:destPort NameNode:8020 , LocalHost:localPort EdgeNode/10.48.142.32:0. Failed on local exception: java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:4 Did I miss something please? Users from their laptop launch this commands cluster = RxSpark(sshHostname = " EdgeNode ", sshUsername = "username") rxSetComputeContext(cluster) source = c("~/AirlineDemoSmall.csv") dest_file = "/share" rxHadoopMakeDir(dest_file) They are getting thr same issue On all node cluster hdfs dfs -ls / is working well Please advise Thanks Asma
... View more
02-03-2020
06:23 AM
Thanks a lot Now the problem for hdfs is fixed however when i try to launch a script from an edge node , i am getting the same issue /usr/hdp/3.1.4.0-315/spark2/bin/spark-submit --class org.apache.spark.examples.SparkPi --master spark://edgenode.servername:7077 --num-executors 4 --driver-memory 512m --executor-memory 512m --executor-cores 1 /usr/hdp/3.1.4.0-315/spark2/examples/jars/spark-examples_2.11-2.3.2.3.1.4.0-315.jar Results : 20/02/03 15:13:41 INFO StandaloneAppClient$ClientEndpoint: Executor updated: app-20200203151341-0000/79 is now RUNNING 20/02/03 15:13:41 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@69cac930{/metrics/json,null,AVAILABLE,@Spark} 20/02/03 15:13:42 WARN Client: Exception encountered while connecting to the server : org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS] 20/02/03 15:13:42 ERROR SparkContext: Error initializing SparkContext. java.io.IOException: DestHost:destPort namenode.servername:8020 , LocalHost:localPort edgenodeaddress:0. Failed on local exception: java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:831) at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:806) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1502) at org.apache.hadoop.ipc.Client.call(Client.java:1444) at org.apache.hadoop.ipc.Client.call(Client.java:1354) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116) at com.sun.proxy.$Proxy11.getFileInfo(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:900) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) at com.sun.proxy.$Proxy12.getFileInfo(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1660) at org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1577) at org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1574) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1589) at org.apache.spark.scheduler.EventLoggingListener.start(EventLoggingListener.scala:100) at org.apache.spark.SparkContext.<init>(SparkContext.scala:522) at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2498) at org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:934) at org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:925) at scala.Option.getOrElse(Option.scala:121) at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:925) at org.apache.spark.examples.SparkPi$.main(SparkPi.scala:31) at org.apache.spark.examples.SparkPi.main(SparkPi.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:904) at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:198) at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:228) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:137) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused by: java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS] at org.apache.hadoop.ipc.Client$Connection$1.run(Client.java:758) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730) at org.apache.hadoop.ipc.Client$Connection.handleSaslConnectionFailure(Client.java:721) at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:814) at org.apache.hadoop.ipc.Client$Connection.access$3600(Client.java:411) at org.apache.hadoop.ipc.Client.getConnection(Client.java:1559) at org.apache.hadoop.ipc.Client.call(Client.java:1390)
... View more
01-31-2020
08:31 AM
Thanks a lot 🙂 I have configured the Cluster with Kerberos using Active Directory but i got some issues when connecting [root@server keytabs]# hdfs dfs -ls / 20/01/31 16:31:19 WARN ipc.Client: Exception encountered while connecting to the server : org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS] ls: DestHost:destPort namenode:8020 , LocalHost:localPort ambari/ip:0. Failed on local exception: java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS] Any idea please? looks like the 8020 ports is also blocked Thanks Asma
... View more
01-30-2020
09:27 AM
thak you for your help I tried to restart th ambari server but in vain . I got this error 2020-01-30 18:20:21,866 INFO [main] KerberosChecker:64 - Checking Ambari Server Kerberos credentials. 2020-01-30 18:20:22,052 ERROR [main] KerberosChecker:120 - Client not found in Kerberos database (6) 2020-01-30 18:20:22,052 ERROR [main] AmbariServer:1119 - Failed to run the Ambari Server org.apache.ambari.server.AmbariException: Ambari Server Kerberos credentials check failed. Check KDC availability and JAAS configuration in /etc/ambari-server/conf/krb5JAASLogin.conf at org.apache.ambari.server.controller.utilities.KerberosChecker.checkJaasConfiguration(KerberosChecker.java:121) at org.apache.ambari.server.controller.AmbariServer.main(AmbariServer.java:1110) ht JAASLogin is configured like this com.sun.security.jgss.krb5.initiate { com.sun.security.auth.module.Krb5LoginModule required renewTGT=false doNotPrompt=true useKeyTab=true keyTab="/etc/security/ambariservername.keytab" principal="ambariservername@REALM.COM" storeKey=true useTicketCache=false; }; I tried to follow these links https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.5/authentication-with-kerberos/content/kerberos_optional_install_a_new_mit_kdc.html https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.5/authentication-with-kerberos/content/set_up_kerberos_for_ambari_server.html Any suggestion please? 😞 Thanks
... View more
01-27-2020
08:48 AM
Dear community I have installed a hadoop cluster on 8 servers using Ambari Hortonworks. I am able to access webhdfs using the ip address and the default port 50070 without authentication. How can I secure Webhdfs? P.S I did not enable using kerberos in Ambari > Enable kerberos , should I do it? Any suggestion will be appreciated Thanks Asma
... View more
Labels:
01-10-2020
08:13 AM
Thank you again!! 1) I have installed these client on my edge node 🙂 2) For instance, we decide that the cluster is not configured with kerberos 3) Actually, I want that end users could submit spark-shell 🙂 4) For this , I created a local user called "sparkuser" on the edge node 5) with which tool could client use the spark-shell? API? application? All I want that users can use the sparkuser that I have created on edge nôde to submit their scripts but I don't not want that these users access the edge node server directly (like remotelx or via an API? or else? I hope that I explained more 😄 Many thanks for your help and patience Asma
... View more
01-10-2020
06:25 AM
In a nutshell, you don't need a Master process the edge node but Client to initiate communication with the Cluster => Actually this is my question 🙂 Which client? Should I create a local user like spark and then ask the end users to use it in order to launch this command for example from their machines?? spark-sumbit --master spark://edgenode:7077 calculPi.jar? All that I need for now that the end users could execute their scripts spark or Python ... from their side, how could we do this? do they need to access the edge server? Thanks Asma
... View more
01-10-2020
02:17 AM
Thank you for your reply 🙂 I will explain my actual situation. I have installed Hadoop Cluster using ambari HortonWorks. One of nodes is an Edge node, I defined it as a spark master and I could then run spark-sumbit from this linux server (edge node) with spark user (spark) on 4 workers ( datanodes) My question now, I will have developers and end users who should execute scripts from their local machine on the edge node. These users should not access directly the linux server (Edge node) but they will have to launch scripts (spark-sumbit). How can I create accounts for them? How could they access the edge node ? Thanks Asma
... View more
01-09-2020
06:38 AM
Is it possible to add a second user on hadoop cluster like the spark user?
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache Spark
12-04-2019
07:30 AM
My bad ! Thank you very much for your answer. Now I can see the sun 🙂 So basically, the access to ambari server is that developers for example may monitor in read only of services are ok. My question is now that how those developers could access the hadoop for their development? I can test spark submit by connecting as a root user to one of the spark nodes. In order to give access to developers to use the hadoop platform for their calculation , what should I configure? /users are in AD Thank you very much in advance for your help ! Asma
... View more
12-04-2019
05:14 AM
Thank you for your answer Actually, in Ambari , I could not see USers and Groups page .. (Please check the screenshoot) As I am the administrator of the platform, I use admin/admin I will have developers who will use the hadoop platform. As the users are in AD , I think that I should integrate Hadoop with AD? The strange thing that I don't have access to users and groups What do you think please? Should developers have access to Ambari? Thanks and Regards Asma
... View more
12-02-2019
09:08 AM
Hello Community,
I want to add users and groups but when i connect to ambari settings i could see only this.
Any idea please?
The version is 2.7.4
Many thanks in advance
Asma
... View more
- Tags:
- Ambari
Labels:
- Labels:
-
Apache Ambari