Re: Nginx config for Hue - reverse proxy

AKB · ‎01-25-2019

Can anyone please look at my last post (configs) and let me know if I missed anything else? Thanks. I need to stop using both of my accounts.

AKB · ‎12-17-2018

After making the changes, I see this in the log: 2018/12/17 15:47:50 [warn] 20199#20199: could not build optimal proxy_headers_hash, you should increase either proxy_headers_hash_max_size: 512 or proxy_headers_hash_bucket_size: 64; ignoring proxy_headers_hash_bucket_size 2018/12/17 17:49:19 [crit] 20348#20348: *16 SSL_do_handshake() failed (SSL: error:1417D0A0:SSL routines:tls_process_client_hello:length too short) while SSL handshaking, client: <CLIENT_IP>, server: 0.0.0.0:443 Config file now: server { server_name _; charset utf-8; #or if running hue on https:// listen 8001 ssl; ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_certificate /ssl/grav.crt; ssl_certificate_key /ssl/grav.key; ssl_prefer_server_ciphers on; ssl_session_timeout 1d; ssl_session_cache shared:SSL:50m; ssl_session_tickets off; client_max_body_size 0; location / { proxy_pass http://hue; proxy_set_header Host $http_host; proxy_set_header X-Forwarded-For $remote_addr; } location /static/ { # Uncomment to expose the static file directories. #autoindex on; # Or if on a parcel install: proxy_pass http://hue; proxy_set_header Host $http_host; proxy_set_header X-Forwarded-For $remote_addr; alias /opt/cloudera/parcels/CDH/lib/hue/build/static/; expires 30d; add_header Cache-Control public; } } upstream hue { ip_hash; # List all the Hue instances here for high availability. server <HUE_NODE_1>:8888 max_fails=3; server <HUE_NODE_2>:8888 max_fails=3; }

AKB · ‎12-17-2018

Changed what you requested. Did not help. 400 Bad Request The plain HTTP request was sent to HTTPS port nginx/1.14.2 On Inspection: :8001/favicon.ico:1 Failed to load resource: the server responded with a status of 400 (Bad Request) login:1 Failed to load resource: the server responded with a status of 400 (Bad Request) From logs: 2018/12/17 15:15:34 [error] 19342#19342: *99 connect() failed (111: Connection refused) while connecting to upstream, client: <My-client-ip>, server: _, request: "GET / HTTP/1.1", upstream: "http://<hue-node-ip>:8888/", host: "gravalytics.com:8001", referrer: "https://gravalytics.com/explore/home" 2018/12/17 15:47:50 [warn] 20199#20199: could not build optimal proxy_headers_hash, you should increase either proxy_headers_hash_max_size: 512 or proxy_headers_hash_bucket_size: 64; ignoring proxy_headers_hash_bucket_size

AKB · ‎12-13-2018

Also another thing to note. This config works for http (nginx) to http (hue), but fails for https (nginx) to http (hue).

AKB · ‎12-13-2018

Firewall rules are setup to allow traffic from nginx vm to cluster. Hue is running on that node and port.

AKB · ‎12-11-2018

Nginx log: 2018/12/11 19:44:51 [warn] 19347#19347: could not build optimal proxy_headers_hash, you should increase either proxy_headers_hash_max_size: 512 or proxy_headers_hash_bucket_size: 64; ignoring proxy_headers_hash_bucket_size 2018/12/11 19:51:44 [error] 19352#19352: *3 open() "/www/gravalytics/public/favicon.ico" failed (2: No such 2018/12/11 20:04:18 [error] 19352#19352: *14 connect() failed (111: Connection refused) while connecting to upstream, client: <client_ip> , server: gravalytics.com, request: "GET / HTTP/1.1", upstream: "http://<node1_ip>:8888/", host: "gravalytics.com:8001", referrer: "https://gravalytics.com/" 2018/12/11 20:04:18 [error] 19352#19352: *14 connect() failed (111: Connection refused) while connecting to upstream, client: <client_ip>, server: gravalytics.com, request: "GET /favicon.ico HTTP/1.1", upstream: "http://<node1_ip>:8888/favicon.ico", host: "gravalytics.com:8001", referrer: "https://gravalytics.com:8001/" There is no entry in the Hue instance log in /var/log/hue-httpd/ folder.

AKB · ‎12-11-2018

Thanks, will post details later today.

AKB · ‎12-11-2018

First of all this is a DEV server, just trying to figure out the configuration. From what I read, this should work, but it is not. I have a Nginx server running on SSL and trying to do a reverse proxy to a non-ssl cluster (5.15) running Hue. Why does this config not work? server { server_name myhostname.com; charset utf-8; listen 8001; ssl on; ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_certificate /ssl/grav.crt; ssl_certificate_key /ssl/grav.key; client_max_body_size 0; location / { proxy_pass http://hue; proxy_set_header Host $http_host; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto https; } location /static/ { # Uncomment to expose the static file directories. #autoindex on; # Or if on a parcel install: proxy_pass http://hue; proxy_set_header Host $http_host; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header X-Forwarded-Proto https; #alias /opt/cloudera/parcels/CDH/lib/hue/build/static/; expires 30d; add_header Cache-Control public; } } upstream hue { ip_hash; # List all the Hue instances here for high availability. server node1_IP:8888 max_fails=3; server node2_IP:8888 max_fails=3; } Get error message - login:1 Failed to load resource: the server responded with a status of 404 (Not Found)

AKB · ‎12-04-2018

Other than being on the latest version, what features make the difference between choosing 6.x at this time, especially since CDH and HDP are going to come back with a common product? Thanks.

AKB · ‎11-28-2018

Another question here, is it easier to setup Nginx reverse proxy with SSL certs to point to Hadoop services we need to expose like Hue and CM? If so, any CDH specific docs for that. Thanks

AKB · ‎11-14-2018

Thanks I will look into all of these. Is there a plan for CM automating a lot of these steps in the future? In summary, what does Auto-TLS in CDH 6.x provide that is not there in 5.15?

AKB · ‎11-12-2018

Thanks, that book and additional docs were very helpful. Another question, how do I generate test certs for a test cluster to see the workflow end to end. Was reading this doc -> https://www.cloudera.com/documentation/enterprise/5-15-x/topics/how_to_configure_cm_tls.html#concept_wk4_jlx_qw and stuck on step 3 because I dont want to buy a CA cert yet, rather generate a local one and test. Any help is appreciated.

AKB · ‎11-08-2018

Can anyone please suggest me a book to understand and implement Kerberos auth and SSL configs for cluster, Cloudera specific would be even better? I have not involved myself with this task before so it feels daunting after reading all the docs.

AKB · ‎11-06-2018

I am not using Kerberos on this cluster, it is a CDH 5.15 install.

AKB · ‎11-05-2018

Did not work. [centos@ip-172-31-81-167 ~]$ curl -i --retry 5 -s -L -k --negotiate -u : http://ip-172-31-82-140.ec2.internal:8983/solr curl: option --negotiate: the installed libcurl version doesn't support this curl: try 'curl --help' or 'curl --manual' for more information

AKB · ‎11-01-2018

Yes, running all commands on NN2 which happens to be a Solr node also. I was able to create the collections using the collections api, but solrctl works for some commands and just not the collections command. Maybe some bug, please check the code. Thanks

AKB · ‎11-01-2018

[centos@ip-172-31-81-167 data]$ solrctl --trace --zk ip-172-31-90-165.ec2.internal:2181,ip-172-31-81-167.ec2.internal:2181,ip-172-31-92-169.ec2.internal:2181/solr collection --delete ds-collection + shift 1 + test 5 '!=' 0 + case "$1" in + '[' 5 -gt 1 ']' + SOLR_ZK_ENSEMBLE=ip-172-31-90-165.ec2.internal:2181,ip-172-31-81-167.ec2.internal:2181,ip-172-31-92-169.ec2.internal:2181/solr + shift 2 + test 3 '!=' 0 + case "$1" in + break + '[' -n '' ']' + '[' -n '' ']' + '[' -n '' ']' + '[' -z ip-172-31-90-165.ec2.internal:2181,ip-172-31-81-167.ec2.internal:2181,ip-172-31-92-169.ec2.internal:2181/solr ']' + SOLR_ADMIN_ZK_CMD='ZKCLI_JVM_FLAGS=${ZKCLI_JVM_FLAGS} LOG4J_PROPS=${SOLR_CONF_DIR}/log4j.properties ${SOLR_HOME}/bin/zkcli.sh -zkhost $SOLR_ZK_ENSEMBLE 2>&3' + '[' 3 -gt 0 ']' + test 3 '!=' 0 + case "$1" in + '[' --delete = --list ']' + '[' --delete = --list-snapshots ']' + '[' 3 -gt 2 ']' + case "$2" in ++ echo --delete ++ tr '[a-z]-' '[A-Z] ' + COL_ACTION=' DELETE' ++ echo DELETE + eval solr_webapi ''\''/admin/collections?action=DELETE&name=ds-collection'\''' ++ solr_webapi '/admin/collections?action=DELETE&name=ds-collection' ++ '[' -z '' ']' +++ get_solr_protocol ++++ eval 'ZKCLI_JVM_FLAGS=${ZKCLI_JVM_FLAGS}' 'LOG4J_PROPS=${SOLR_CONF_DIR}/log4j.properties' '${SOLR_HOME}/bin/zkcli.sh' -zkhost '$SOLR_ZK_ENSEMBLE' '2>&3' -cmd get /clusterprops.json +++++ ZKCLI_JVM_FLAGS= +++++ LOG4J_PROPS=/etc/solr/conf/log4j.properties +++++ /opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/bin/../lib/solr//bin/zkcli.sh -zkhost ip-172-31-90-165.ec2.internal:2181,ip-172-31-81-167.ec2.internal:2181,ip-172-31-92-169.ec2.internal:2181/solr -cmd get /clusterprops.json +++ SOLR_STATE='{"urlScheme":"http"}' +++ echo '{"urlScheme":"http"}' +++ grep -i urlScheme +++ grep -q -i https +++ echo http ++ local SOLR_PROTOCOL=http +++ get_solr_state /live_nodes +++ sed -ne 's#/live_nodes/$.*:[0-9][0-9]*$.*$#\1#p' ++++ eval 'ZKCLI_JVM_FLAGS=${ZKCLI_JVM_FLAGS}' 'LOG4J_PROPS=${SOLR_CONF_DIR}/log4j.properties' '${SOLR_HOME}/bin/zkcli.sh' -zkhost '$SOLR_ZK_ENSEMBLE' '2>&3' -cmd ls /live_nodes +++++ ZKCLI_JVM_FLAGS= +++++ LOG4J_PROPS=/etc/solr/conf/log4j.properties +++++ /opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/bin/../lib/solr//bin/zkcli.sh -zkhost ip-172-31-90-165.ec2.internal:2181,ip-172-31-81-167.ec2.internal:2181,ip-172-31-92-169.ec2.internal:2181/solr -cmd ls /live_nodes +++ SOLR_STATE='/live_nodes (5) /live_nodes/ip-172-31-81-167.ec2.internal:8983_solr (0) /live_nodes/ip-172-31-90-165.ec2.internal:8983_solr (0) /live_nodes/ip-172-31-82-140.ec2.internal:8983_solr (0) /live_nodes/ip-172-31-92-169.ec2.internal:8983_solr (0) /live_nodes/ip-172-31-93-54.ec2.internal:8983_solr (0)' +++ echo '/live_nodes (5) /live_nodes/ip-172-31-81-167.ec2.internal:8983_solr (0) /live_nodes/ip-172-31-90-165.ec2.internal:8983_solr (0) /live_nodes/ip-172-31-82-140.ec2.internal:8983_solr (0) /live_nodes/ip-172-31-92-169.ec2.internal:8983_solr (0) /live_nodes/ip-172-31-93-54.ec2.internal:8983_solr (0)' +++ grep -v '^/ ' ++ for node in '`get_solr_state '\''/live_nodes'\'' | sed -ne '\''s#/live_nodes/$.*:[0-9][0-9]*$.*$#\1#p'\''`' 'localhost:$SOLR_PORT' ++ curl -i --retry 5 -s -L -k --negotiate -u : http://ip-172-31-81-167.ec2.internal:8983/solr ++ for node in '`get_solr_state '\''/live_nodes'\'' | sed -ne '\''s#/live_nodes/$.*:[0-9][0-9]*$.*$#\1#p'\''`' 'localhost:$SOLR_PORT' ++ curl -i --retry 5 -s -L -k --negotiate -u : http://ip-172-31-90-165.ec2.internal:8983/solr ++ for node in '`get_solr_state '\''/live_nodes'\'' | sed -ne '\''s#/live_nodes/$.*:[0-9][0-9]*$.*$#\1#p'\''`' 'localhost:$SOLR_PORT' ++ curl -i --retry 5 -s -L -k --negotiate -u : http://ip-172-31-82-140.ec2.internal:8983/solr ++ for node in '`get_solr_state '\''/live_nodes'\'' | sed -ne '\''s#/live_nodes/$.*:[0-9][0-9]*$.*$#\1#p'\''`' 'localhost:$SOLR_PORT' ++ curl -i --retry 5 -s -L -k --negotiate -u : http://ip-172-31-92-169.ec2.internal:8983/solr ++ for node in '`get_solr_state '\''/live_nodes'\'' | sed -ne '\''s#/live_nodes/$.*:[0-9][0-9]*$.*$#\1#p'\''`' 'localhost:$SOLR_PORT' ++ curl -i --retry 5 -s -L -k --negotiate -u : http://ip-172-31-93-54.ec2.internal:8983/solr ++ for node in '`get_solr_state '\''/live_nodes'\'' | sed -ne '\''s#/live_nodes/$.*:[0-9][0-9]*$.*$#\1#p'\''`' 'localhost:$SOLR_PORT' ++ curl -i --retry 5 -s -L -k --negotiate -u : http://localhost:/solr ++ '[' -n '' ']' ++ die 'Error: can'\''t discover Solr URI. Please specify it explicitly via --solr.' ++ '[' '!' -z 'Error: can'\''t discover Solr URI. Please specify it explicitly via --solr.' ']' ++ echo Error: 'can'\''t' discover Solr URI. Please specify it explicitly via --solr. Error: can't discover Solr URI. Please specify it explicitly via --solr. ## Seems like the collection command is not working in CDH 5.15. Instancedir works fine with zk quoram.

AKB · ‎11-01-2018

solrctl --zk ip-172-31-90-165.ec2.internal:2181,ip-172-31-81-167.ec2.internal:2181,ip-172-31-92-169.ec2.internal:2181/solr collection --delete ds-collection --trace Error: can't discover Solr URI. Please specify it explicitly via --solr.

AKB · ‎11-01-2018

Anyone?

AKB · ‎10-30-2018

@bgooley Please see my post above. Thanks.

AKB · ‎10-30-2018

Is this a bug in CDH 5.15? The same happens for me. This works solrctl --zk ip-172-31-90-165.ec2.internal:2181,ip-172-31-81-167.ec2.internal:2181,ip-172-31-92-169.ec2.internal:2181/solr instancedir --create ds-collection /tmp/ds-collection This gives me Solr url not found error: solrctl --zk ip-172-31-90-165.ec2.internal:2181,ip-172-31-81-167.ec2.internal:2181,ip-172-31-92-169.ec2.internal:2181/solr collection --create ds-collection -s 2 -r 2 The above command used to work in older 5.1x versions. Please advise.

AKB · ‎10-25-2018

I have created a slef-signed cert using this: Where do I find the other files mentioned in step 7 - https://www.cloudera.com/documentation/enterprise/5-15-x/topics/how_to_configure_cm_tls.html#concept_wk4_jlx_qw ? Now onwards trying to use CM for configs: https://www.cloudera.com/documentation/enterprise/5-15-x/topics/how_to_configure_cm_tls.html#concept_wk4_jlx_qw

AKB · ‎10-23-2018

But a wildcard domain cert can be used, correct?

AKB · ‎10-23-2018

Can one SSL certificate be used on all nodes of the cluster? Sorry for the questions, I am not familiar with doing this.

AKB · ‎10-19-2018

What if I needed to setup SSL only and then use our corporate SSO mechanism for authentication? Any hints on that?

AKB · ‎10-19-2018

Is Kerberos authentication mandatory before setting up SSL on Hadoop nodes? Also, moving to CDH 6 is not an option for us at the moment.

AKB · ‎10-16-2018

I am trying to enable SSL for all nodes and services in the cluster. How can I do this? Can someone please point me to some concise documentation where I can do this quickly and safely. Bit confused about Sentry and how it plays into authorization etc. Thanks.

AKB · ‎08-23-2018

Solution found. In the hosts file of the python client machine, add public IP and private host name This is appropriate for a cloud service like AWS. Python lib works fine now. Thanks for @bgooley help on another thread that resolved this too.

AKB · ‎08-22-2018

@bgooley Thanks for the tip. That worked. So, putting the public IP and private hostname in hosts file on client did the trick. 🙂

AKB · ‎08-22-2018

Thanks for the comment. Not sure how to do this private to public mapping. Any help is appreciated. Thanks. This is what the hosts file on client looks like right now: 127.0.0.1 localhost.localdomain localhost 192.168.157.154 edgenode.local.com edgenode 18.215.25.118 ec2-18-215-25-118.compute-1.amazonaws.com ec2-18-215-25-118 <-- Public IP of server

Online	Offline
Last Visited	‎02-12-2019 05:00 PM

Date Registered	‎04-11-2018 12:23 PM
Last Visited	‎02-12-2019 05:00 PM
Total Messages Posted	55
Total Kudos Received	1

Re: Nginx config for Hue - reverse proxy

Re: Nginx config for Hue - reverse proxy

Re: Nginx config for Hue - reverse proxy

Re: Nginx config for Hue - reverse proxy

Re: Nginx config for Hue - reverse proxy

Re: Nginx config for Hue - reverse proxy

Re: Nginx config for Hue - reverse proxy

Nginx config for Hue - reverse proxy

Why install CDH 6.x?

Re: Enable SSL for CDH 5.15 Cluster

Re: Enable SSL for CDH 5.15 Cluster

Re: Enable SSL for CDH 5.15 Cluster

Re: Enable SSL for CDH 5.15 Cluster

Re: unable to create Solr collection / index

Re: unable to create Solr collection / index

Re: unable to create Solr collection / index

Re: unable to create Solr collection / index

Re: unable to create Solr collection / index

Re: unable to create Solr collection / index

Re: unable to create Solr collection / index

Re: unable to create Solr collection / index

Re: Enable SSL for CDH 5.15 Cluster

Re: Enable SSL for CDH 5.15 Cluster

Re: Enable SSL for CDH 5.15 Cluster

Re: Enable SSL for CDH 5.15 Cluster

Re: Enable SSL for CDH 5.15 Cluster

Enable SSL for CDH 5.15 Cluster

Re: Unable to access HDFS Namenode from Python lib...

Re: install hadoop client on unmanaged host

Re: install hadoop client on unmanaged host