Enable HTTPS for YARN and MAPREDUCE2

[#1]

1. As a first step, enable HTTPS for HDFS, you can follow the article https://community.hortonworks.com/articles/52875/enable-https-for-hdfs.html

2. Add/Update below configurations in "Custom mapred-site" mapred-site.xml

mapreduce.jobhistory.http.policy=HTTPS_ONLY
mapreduce.jobhistory.webapp.https.address=<JHS>:<JHS_HTTPS_PORT> 
mapreduce.ssl.enabled=true
mapreduce.shuffle.ssl.enabled=true

Ex: mapreduce.jobhistory.webapp.https.address=apappu-hdp234-2.openstacklocal:19889

3. Add/update below configurations under "Advanced yarn-site" ( yarn-site.xml)

yarn.http.policy=HTTPS_ONLY
yarn.log.server.url=https://JHS:JHS_HTTPS_PORT/jobhistory/logs
yarn.resourcemanager.webapp.https.address=RM:RM_HTTPS_PORT
yarn.nodemanager.webapp.https.address=0.0.0.0:NM_HTTPS_PORT

Ex: 
yarn.log.server.url=https://apappu-hdp234-2.ambari.org:19889/jobhistory/logs
yarn.resourcemanager.webapp.https.address=apappu-hdp234-2.ambari.org:8090
yarn.nodemanager.webapp.https.address=0.0.0.0:8042

4. Add/update below property in hdfs-site (hdfs-site.xml) under HDFS service.

dfs.https.enable=true

5. Restart HDFS, YARN, MAPREDUCE services

6. should be able to access the URLs now,

YARN: https://HT_HOST:19889/

MAPREDUCE:https://YARN-RM-HOST:19889/jobhistory

More articles

*. To enable HTTPS for HBASE - https://community.hortonworks.com/articles/51165/enable-httpsssl-for-hbase-master-ui.html

[#2]

@apappu Great article dfs.https.enable=true is needed if you are using HA, otherwise ambari fails to check HDFS. without that parameter ambari tries to connect to HTTP port (50070) instead of (50470).

if you see error below, it's because you are missing dfs.https.enable=true:

2016-06-02 23:07:22,221 - Getting jmx metrics from NN failed. URL: http://test.support.com:50070/jmx?qry=Hadoop:service=NameNode,name=FSNamesys...
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/jmx.py", line 38, in get_value_from_jmx
    _, data, _ = get_user_call_output(cmd, user=run_user, quiet=False)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/get_user_call_output.py", line 61, in get_user_call_output
    raise Fail(err_msg)
Fail: Execution of 'curl --negotiate -u : -s 'http://test.support.com:50070/jmx?qry=Hadoop:service=NameNode,name=FSNamesys...' 1>/tmp/tmpOGkdi9 2>/tmp/tmpCUfr6B' returned 7.

[#3]

Who knows what the reasoning behind the https port numbering is? resource manager webapp https port 8090 instead of 8088 for http, but nodemanager webapp 8042 for both http and https.

[#4]

is the step #4 necessary for HDP-2.6.4 version?
btw, I had some error after enabling SSL on Mapred

(<urlopen error [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:590)>)