Hi Folks, I have 3 node cluster in my environment, where i have configured kerberos & ranger integrated with Active Directory. I have setup one way trust MIT KDC. Now while i'm running hive query, i'm getting below logs: Diagnostics: Application application_1528875723692_0001 initialization failed (exitCode=255) with output: main : command provided 0 main : run as user is asif main : requested yarn user is asif User asif not found If i create user on OS server where yarn install, I'm able to run query. But i don't want to create user on OS because i have setup with AD. How yarn understand about AD user? ... View more

Hi Folks, I have 3 node cluster in my environment, where i have configured kerberos & ranger integrated with Active Directory. I have setup one way trust MIT KDC. Users are able to get ticket. Now while i'm running hive query, i'm getting failed query and getting below logs: Application application_1528875723692_0001 failed 2 times due to AM Container for appattempt_1528875723692_0001_000002 exited with exitCode: -1000 For more detailed output, check the application tracking page: http://security-test3.example.com:8088/cluster/app/application_1528875723692_0001 Then click on links to logs of each attempt. Diagnostics: Application application_1528875723692_0001 initialization failed (exitCode=255) with output: main : command provided 0 main : run as user is asif main : requested yarn user is asif User asif not found Failing this attempt. Failing the application. Please advice what can i do for solve it. Regards, Vinay ... View more

HI Team, I have three node Cluster. I have enabled kerberos in cluster environment. While i'm running any hive query from different users, I'm getting only hive username in history server username who is running query. How can i enable owner name in application? sample screenshot attached. Regards, Vinay capture1.png ... View more

Hi All, I was running command hadoop fs -get /user/centos/dist/a.txt I got error during command java.net.SocketException: error creating UNIX domain socket with SOCK_STREAM: Too many open files Open file limit is default limit. I didn't change. Can some one suggest should i have to increase open file limit in sysctl and limit.conf? Thanks vinay ... View more

Hi Folks, I have configured MIT kdc which is integrated with Active directory according to referred link: https://community.hortonworks.com/articles/59635/one-way-trust-mit-kdc-to-active-directory.html My question is: 1. How can i test one way trust is successfully created or not? 2. Users will persist on AD server and services will persist on hadoop cluster. Should i have to create user principal in kerberos database? 3. If yes, Should be have to add principal in kerberos manually whenever new user created in AD server? Regards, Vinay ... View more

Hi Folks, Hope all are doing well. I'm newer in spark. I have installed HDP 2.6.2. i have added spark as a service. Before start the spark, There was no job running. But when i had started spark service, i have found two jobs are running continuously in UNDEFINED state. #yarn application -list 18/05/28 15:07:51 INFO client.AHSProxy: Connecting to Application History server at 10.10.10.16:10200 18/05/28 15:07:51 INFO client.RequestHedgingRMFailoverProxyProvider: Looking for the active RM in [rm1, rm2]... 18/05/28 15:07:51 INFO client.RequestHedgingRMFailoverProxyProvider: Found active RM [rm2] Total number of applications (application-types: [] and states: [SUBMITTED, ACCEPTED, RUNNING]):4 Application-Id Application-Name Application-Type User Queue State Final-State Progress Tracking-URL application_1527494556086_0039 org.apache.spark.sql.hive.thriftserver.HiveThriftServer2 SPARK hive admin RUNNING UNDEFINED 10% http://10.10.10.8:4040 application_1527494556086_0038 org.apache.spark.sql.hive.thriftserver.HiveThriftServer2 SPARK hive admin RUNNING UNDEFINED 10% http://10.10.10.12:4040 Thrift server is installed on both Server having IP are: 10.10.10.8 10.10.10.12 Can you please help me to clearify? ... View more

Hi Folks, Hope all are doing well. I've dev setup of 11 node(2x NN, 8x DN, 1x edge node). Basically All nodes are connected by Cluster private(only 11 machine can access to each other) networking. We are using floating IP(public in only organization) on data node as well as on Edge node for access the data from DB. If we keep floating ip only on Edge node and run MapReduce job from edge node which is importing data from DB to HDFS, i'm getting error that DataNode IP is not able to access to source DB. Can someone suggest does we require Floating IP on all DataNode machine? Or we can use any other solution? Will be very thankful to you. Regards, Vinay K ... View more

we are having HDP 2.6.2 running properly. We are looking to install nifi in our environment. Is it recommended to install HDF on existing HDP? Have there any HDF version compatibility with HDP 2.6.2? ... View more

Hi, we are using hdp 2.5.2 I'm newer in security part. I have integrated Kerberos unix based authentication(we are not using LDAP) where we have also configured SPNEGO for restrict the UI access, which is working fine. Now We are looking for knox, As per documentation, Knox provide SSO facility. I have few query based on knox 1. Is it mandatory to install AD/LDAP with KNOX? we want to run knox without AD/LDAP. 2. How know is useful in web based URL? Have there any specific link and configuration which i can understand. 3. Does we have to remove SPNEGO if we want to configure KNOX with SSO? Please assist.. Will be very helpful.. ... View more