@Matt FoleyThanks for additional information. This is very helpful ... View more

@Divakar Annapureddy Does this mean that we can't restore the cluster with the meta-data/data backups we take before the cluster re-install? ... View more

@jramakrishnan This looks more like authentication solution than securing the data channels to eliminate the risk of having your data transfer in clear text. Am i missing anything here? ... View more

@Kuldeep Kulkarni Thanks for the update.This issue was resolved by Hortonworks Support Engineering. I am trying to catching up with what they have done to fix the issue. ... View more

Thats interesting .. can someone confirm that we can't use cp or mv on encrypted zone files? ... View more

yes. user not found issue is gone after i created the user on all the nodes. Do you know where I can look for which classpath/jars that has permissions issue? ... View more

@Vikas Gadade I created the user on all the nodes but the job is still failing with the following output xxxxx:/#yarn jar /usr/hdp/2.3.2.0-2950/hadoop-mapreduce/hadoop-mapreduce-examples.jar teragen 10000 /user/xxxxx/teraout8 16/02/26 10:52:18 INFO impl.TimelineClientImpl: Timeline service address: http://timelineuri:8188/ws/v1/timeline/ 16/02/26 10:52:18 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 37 for rbalam on ha-hdfs:testnnhasvc 16/02/26 10:52:19 INFO security.TokenCache: Got dt for hdfs://testnnhasvc; Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:testnnhasvc, Ident: (HDFS_DELEGATION_TOKEN token 37 for rbalam) 16/02/26 10:52:19 INFO client.ConfiguredRMFailoverProxyProvider: Failing over to rm2 16/02/26 10:52:20 INFO terasort.TeraSort: Generating 10000 using 2 16/02/26 10:52:21 INFO mapreduce.JobSubmitter: number of splits:2 16/02/26 10:52:22 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1456512672399_0001 16/02/26 10:52:22 INFO mapreduce.JobSubmitter: Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:testnnhasvc, Ident: (HDFS_DELEGATION_TOKEN token 37 for rbalam) 16/02/26 10:52:24 INFO impl.YarnClientImpl: Submitted application application_1456512672399_0001 16/02/26 10:52:24 INFO mapreduce.Job: The url to track the job: http://timelineuri:8188/ws/v1/timeline/ 16/02/26 10:52:24 INFO mapreduce.Job: Running job: job_1456512672399_0001 16/02/26 10:52:29 INFO mapreduce.Job: Job job_1456512672399_0001 running in uber mode : false 16/02/26 10:52:29 INFO mapreduce.Job: map 0% reduce 0% 16/02/26 10:52:29 INFO mapreduce.Job: Job job_1456512672399_0001 failed with state FAILED due to: Application application_1456512672399_0001 failed 2 times due to AM Container for appattempt_1456512672399_0001_000002 exited with exitCode: -1000 For more detailed output, check application tracking page:http://timlineserveruri:8088/cluster/app/application_1456512672399_0001Then, click on links to logs of each attempt. Diagnostics: Application application_1456512672399_0001 initialization failed (exitCode=255) with output: main : command provided 0 main : run as user is xxxxx main : requested yarn user is xxxxx Failing this attempt. Failing the application. 16/02/26 10:52:29 INFO mapreduce.Job: Counters: 0 ... View more

Could you please confirm this again? if i need to have users on all the nodes in the cluster to run jobs successfully.. i could end up with quite a few users on all the nodes which may become a maintenance head-ache down the line .. ... View more

@Neeraj Sabharwal I ran the job again and tried to get yarn logs .... here is what i see xxxxx:~#yarn logs -applicationId application_1456457210711_0002 16/02/26 03:44:26 INFO impl.TimelineClientImpl: Timeline service address: http://yarntimelineserveraddress:8188/ws/v1/timeline/ /app-logs/xxxxx/logs/application_1456457210711_0002 does not have any log files. Here is what I see on the ResourceManager UI Application application_1456457210711_0002 failed 2 times due to AM Container for appattempt_1456457210711_0002_000002 exited with exitCode: -1000 For more detailed output, check application tracking page:http://resourcemanageruri:8088/cluster/app/application_1456457210711_0002Then, click on links to logs of each attempt. Diagnostics: Application application_1456457210711_0002 initialization failed (exitCode=255) with output: main : command provided 0 main : run as user is xxxxx main : requested yarn user is xxxxx User xxxxx not found Failing this attempt. Failing the application. ... View more

I followed the same steps on another cluster where we dont have hue https and kerberos and it is working as expected there. So i think there is a problem with either https and/or kerberos settings. ... View more

Hi @Neeraj Sabharwal I followed the same steps and restarted httpfs and hue services but when I try to access hue filebrowser it is throwing exceptions. The only difference in this environment is hue is running on https and cluster is kerberized but not sure if it makes any difference. Can you please let me know how to trouble shoot this issue? WebHdfsException at /filebrowser/ StandbyException: Operation category READ is not supported in state standby (error 403) Request Method: GET Request URL: https://falbdcdd0001v:8000/filebrowser/ Django Version: Exception Type: WebHdfsException Exception Value: StandbyException: Operation category READ is not supported in state standby (error 403) Exception Location: /usr/lib/hue/desktop/libs/hadoop/src/hadoop/fs/webhdfs.py in _stats, line 205 Python Executable: /usr/bin/python2.6 Python Version: ... View more

I have a follow up question on this. Lets say I removed all the users from Ranger which were synced from a local unix server and then re-configured to sync users from an AD domain/group. In this case, do II need to create "hive" user on that particular AD group before I can create a policy to let hive queries run as hive user instead of end users on the cluster? what about other service accounts like mapred, yarn etc .. do I need to create all those accounts on AD? please advise. ... View more

I am using Firefox 44.0 and here is the trace from it. TRACE: The url is: /api/v1/clusters/hwtest?fields=Clusters/health_report,Clusters/total_hosts,alerts_summary_hosts&minimal_response=true app.js:154162:7 Status code 200: Success. app.js:54173:3 App.componentsStateMapper execution time: timer started app.js:55315 App.componentsStateMapper execution time: 29.04ms app.js:55360 TRACE: The url is: /api/v1/clusters/hwtest/host_components?fields=HostRoles/component_name,HostRoles/host_name&minimal_response=true app.js:154162:7 Status code 200: Success. app.js:54173:3 TRACE: The url is: /api/v1/clusters/hwtest/configurations?type=cluster-env app.js:154162:7 Status code 200: Success. app.js:54173:3 App.componentConfigMapper execution time: timer started app.js:55076 App.componentConfigMapper execution time: 5.95ms app.js:55125 Config validation failed: Object { readyState: 4, setRequestHeader: .ajax/v.setRequestHeader(), getAllResponseHeaders: .ajax/v.getAllResponseHeaders(), getResponseHeader: .ajax/v.getResponseHeader(), overrideMimeType: .ajax/v.overrideMimeType(), abort: .ajax/v.abort(), done: f.Callbacks/p.add(), fail: f.Callbacks/p.add(), progress: f.Callbacks/p.add(), state: .Deferred/h.state(), 14 more… } error Internal Server Error Object { type: "POST", timeout: 180000, dataType: "json", statusCode: Object, headers: Object, url: "/api/v1/stacks/HDP/versions/2.3/val…", data: "{"hosts":["falbdcdq0001v.farmersinsurance.com","falbdcdq0002v.farmersinsurance.com","falbdcdq0003v.farmersinsurance.com","falbdcdq0004v.farmersinsurance.com","falbdcdq0005v.farmersinsurance.com","falbdcdq0006v.farmersinsurance.com","falbdcdq0007v.farmersinsurance.com"],"services":["AMBARI_METRICS","ATLAS","FALCON","FLUME","HBASE","HDFS","HIVE","KAFKA","KNOX","MAPREDUCE2","OOZIE","PIG","RANGER","RANGER_KMS","SPARK","SQOOP","STORM","TEZ","YARN","ZOOKEEPER"],"validate":"configurations","recommendations":{"blueprint":{"host_groups":[{"name":"host-group-1","components":[{"name":"FALCON_CLIENT"},{"name":"HBASE_CLIENT"},{"name":"HDFS_CLIENT"},{"name":"HCAT"},{"name":"HIVE_CLIENT"},{"name":"MAPREDUCE2_CLIENT"},{"name":"OOZIE_CLIENT"},{"name":"PIG"},{"name":"SPARK_CLIENT"},{"name":"SQOOP"},{"name":"TEZ_CLIENT"},{"name":"YARN_CLIENT"},{"name":"ZOOKEEPER_CLIENT"},{"name":"METRICS_MONITOR"},{"name":"FALCON_CLIENT"},{"name":"HBASE_CLIENT"},{"name":"HCAT"},{"name":"HDFS_CLIENT"},{"name":"HIVE_CLIENT"[…], context: Object, beforeSend: ajax<.send/opt.beforeSend(), success: ajax<.send/opt.success(), 2 more… } app.js:64175:5 Error code 500: Internal Error on server side. app.js:54195:3 Config validation failed. Going ahead with saving of configs app.js:64188:7 TRACE: The url is: /api/v1/clusters?fields=Clusters/provisioning_state app.js:154162:7 Status code 200: Success. app.js:54173:3 TRACE: The url is: /api/v1/clusters/hwtest/requests?to=end&page_size=10&fields=Requests app.js:154162:7 Status code 200: Success. app.js:54173:3 ... View more

I dont any option to remove this hadoop.security.key.provider.path. please see the attached image ... View more

sorry if it is a dumb question but how do i remove those entries from Ambari? I dont see any option to delete. ... View more