Community Articles

Find and share helpful community-sourced technical articles.
Announcements
Celebrating as our community reaches 100,000 members! Thank you!
avatar

1. As a first step, enable HTTPS for HDFS, you can follow the article https://community.hortonworks.com/articles/52875/enable-https-for-hdfs.html

2. Add/Update below configurations in "Custom mapred-site" mapred-site.xml

mapreduce.jobhistory.http.policy=HTTPS_ONLY
mapreduce.jobhistory.webapp.https.address=<JHS>:<JHS_HTTPS_PORT> 
mapreduce.ssl.enabled=true
mapreduce.shuffle.ssl.enabled=true

Ex: mapreduce.jobhistory.webapp.https.address=apappu-hdp234-2.openstacklocal:19889

3. Add/update below configurations under "Advanced yarn-site" ( yarn-site.xml)

yarn.http.policy=HTTPS_ONLY
yarn.log.server.url=https://JHS:JHS_HTTPS_PORT/jobhistory/logs
yarn.resourcemanager.webapp.https.address=RM:RM_HTTPS_PORT
yarn.nodemanager.webapp.https.address=0.0.0.0:NM_HTTPS_PORT

Ex: 
yarn.log.server.url=https://apappu-hdp234-2.ambari.org:19889/jobhistory/logs
yarn.resourcemanager.webapp.https.address=apappu-hdp234-2.ambari.org:8090
yarn.nodemanager.webapp.https.address=0.0.0.0:8042

4. Add/update below property in hdfs-site (hdfs-site.xml) under HDFS service.

dfs.https.enable=true

5. Restart HDFS, YARN, MAPREDUCE services

6. should be able to access the URLs now,

YARN: https://HT_HOST:19889/

MAPREDUCE:https://YARN-RM-HOST:19889/jobhistory

More articles

*. To enable HTTPS for HBASE - https://community.hortonworks.com/articles/51165/enable-httpsssl-for-hbase-master-ui.html

10,221 Views
Comments
avatar

@apappu Great article dfs.https.enable=true is needed if you are using HA, otherwise ambari fails to check HDFS. without that parameter ambari tries to connect to HTTP port (50070) instead of (50470).

if you see error below, it's because you are missing dfs.https.enable=true:

2016-06-02 23:07:22,221 - Getting jmx metrics from NN failed. URL: http://test.support.com:50070/jmx?qry=Hadoop:service=NameNode,name=FSNamesys...
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/jmx.py", line 38, in get_value_from_jmx
    _, data, _ = get_user_call_output(cmd, user=run_user, quiet=False)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/get_user_call_output.py", line 61, in get_user_call_output
    raise Fail(err_msg)
Fail: Execution of 'curl --negotiate -u : -s 'http://test.support.com:50070/jmx?qry=Hadoop:service=NameNode,name=FSNamesys...' 1>/tmp/tmpOGkdi9 2>/tmp/tmpCUfr6B' returned 7.
avatar
Explorer

Who knows what the reasoning behind the https port numbering is? resource manager webapp https port 8090 instead of 8088 for http, but nodemanager webapp 8042 for both http and https.

avatar
Super Collaborator

is the step #4 necessary for HDP-2.6.4 version?
btw, I had some error after enabling SSL on Mapred

(<urlopen error [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:590)>)
Version history
Last update:
‎08-22-2016 11:22 PM
Updated by:
Contributors