Member since
04-11-2018
55
Posts
1
Kudos Received
0
Solutions
01-25-2019
07:49 AM
Can anyone please look at my last post (configs) and let me know if I missed anything else? Thanks. I need to stop using both of my accounts.
... View more
12-17-2018
11:15 AM
After making the changes, I see this in the log: 2018/12/17 15:47:50 [warn] 20199#20199: could not build optimal proxy_headers_hash, you should increase either proxy_headers_hash_max_size: 512 or proxy_headers_hash_bucket_size: 64; ignoring proxy_headers_hash_bucket_size 2018/12/17 17:49:19 [crit] 20348#20348: *16 SSL_do_handshake() failed (SSL: error:1417D0A0:SSL routines:tls_process_client_hello:length too short) while SSL handshaking, client: <CLIENT_IP>, server: 0.0.0.0:443 Config file now: server { server_name _; charset utf-8; #or if running hue on https:// listen 8001 ssl; ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_certificate /ssl/grav.crt; ssl_certificate_key /ssl/grav.key; ssl_prefer_server_ciphers on; ssl_session_timeout 1d; ssl_session_cache shared:SSL:50m; ssl_session_tickets off; client_max_body_size 0; location / { proxy_pass http://hue; proxy_set_header Host $http_host; proxy_set_header X-Forwarded-For $remote_addr; } location /static/ { # Uncomment to expose the static file directories. #autoindex on; # Or if on a parcel install: proxy_pass http://hue; proxy_set_header Host $http_host; proxy_set_header X-Forwarded-For $remote_addr; alias /opt/cloudera/parcels/CDH/lib/hue/build/static/; expires 30d; add_header Cache-Control public; } } upstream hue { ip_hash; # List all the Hue instances here for high availability. server <HUE_NODE_1>:8888 max_fails=3; server <HUE_NODE_2>:8888 max_fails=3; }
... View more
12-17-2018
07:19 AM
Changed what you requested. Did not help. 400 Bad Request The plain HTTP request was sent to HTTPS port nginx/1.14.2 On Inspection: :8001/favicon.ico:1 Failed to load resource: the server responded with a status of 400 (Bad Request) login:1 Failed to load resource: the server responded with a status of 400 (Bad Request) From logs: 2018/12/17 15:15:34 [error] 19342#19342: *99 connect() failed (111: Connection refused) while connecting to upstream, client: <My-client-ip>, server: _, request: "GET / HTTP/1.1", upstream: "http://<hue-node-ip>:8888/", host: "gravalytics.com:8001", referrer: "https://gravalytics.com/explore/home" 2018/12/17 15:47:50 [warn] 20199#20199: could not build optimal proxy_headers_hash, you should increase either proxy_headers_hash_max_size: 512 or proxy_headers_hash_bucket_size: 64; ignoring proxy_headers_hash_bucket_size
... View more
12-13-2018
11:17 AM
Also another thing to note. This config works for http (nginx) to http (hue), but fails for https (nginx) to http (hue).
... View more
12-13-2018
11:11 AM
Firewall rules are setup to allow traffic from nginx vm to cluster. Hue is running on that node and port.
... View more
12-11-2018
12:15 PM
Nginx log: 2018/12/11 19:44:51 [warn] 19347#19347: could not build optimal proxy_headers_hash, you should increase either proxy_headers_hash_max_size: 512 or proxy_headers_hash_bucket_size: 64; ignoring proxy_headers_hash_bucket_size 2018/12/11 19:51:44 [error] 19352#19352: *3 open() "/www/gravalytics/public/favicon.ico" failed (2: No such 2018/12/11 20:04:18 [error] 19352#19352: *14 connect() failed (111: Connection refused) while connecting to upstream, client: <client_ip> , server: gravalytics.com, request: "GET / HTTP/1.1", upstream: "http://<node1_ip>:8888/", host: "gravalytics.com:8001", referrer: "https://gravalytics.com/" 2018/12/11 20:04:18 [error] 19352#19352: *14 connect() failed (111: Connection refused) while connecting to upstream, client: <client_ip>, server: gravalytics.com, request: "GET /favicon.ico HTTP/1.1", upstream: "http://<node1_ip>:8888/favicon.ico", host: "gravalytics.com:8001", referrer: "https://gravalytics.com:8001/" There is no entry in the Hue instance log in /var/log/hue-httpd/ folder.
... View more
12-11-2018
08:50 AM
First of all this is a DEV server, just trying to figure out the configuration. From what I read, this should work, but it is not. I have a Nginx server running on SSL and trying to do a reverse proxy to a non-ssl cluster (5.15) running Hue. Why does this config not work? server { server_name myhostname.com; charset utf-8; listen 8001; ssl on; ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_certificate /ssl/grav.crt; ssl_certificate_key /ssl/grav.key; client_max_body_size 0; location / { proxy_pass http://hue; proxy_set_header Host $http_host; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto https; } location /static/ { # Uncomment to expose the static file directories. #autoindex on; # Or if on a parcel install: proxy_pass http://hue; proxy_set_header Host $http_host; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header X-Forwarded-Proto https; #alias /opt/cloudera/parcels/CDH/lib/hue/build/static/; expires 30d; add_header Cache-Control public; } } upstream hue { ip_hash; # List all the Hue instances here for high availability. server node1_IP:8888 max_fails=3; server node2_IP:8888 max_fails=3; } Get error message - login:1 Failed to load resource: the server responded with a status of 404 (Not Found)
... View more
Labels:
12-04-2018
09:37 AM
1 Kudo
Other than being on the latest version, what features make the difference between choosing 6.x at this time, especially since CDH and HDP are going to come back with a common product? Thanks.
... View more
Labels:
11-28-2018
07:38 AM
Another question here, is it easier to setup Nginx reverse proxy with SSL certs to point to Hadoop services we need to expose like Hue and CM? If so, any CDH specific docs for that. Thanks
... View more
11-14-2018
06:58 AM
Thanks I will look into all of these. Is there a plan for CM automating a lot of these steps in the future? In summary, what does Auto-TLS in CDH 6.x provide that is not there in 5.15?
... View more
11-12-2018
12:54 PM
Thanks, that book and additional docs were very helpful. Another question, how do I generate test certs for a test cluster to see the workflow end to end. Was reading this doc -> https://www.cloudera.com/documentation/enterprise/5-15-x/topics/how_to_configure_cm_tls.html#concept_wk4_jlx_qw and stuck on step 3 because I dont want to buy a CA cert yet, rather generate a local one and test. Any help is appreciated.
... View more
11-08-2018
09:34 AM
Can anyone please suggest me a book to understand and implement Kerberos auth and SSL configs for cluster, Cloudera specific would be even better? I have not involved myself with this task before so it feels daunting after reading all the docs.
... View more
11-06-2018
07:06 AM
Is there a special type of field called <url> that can be added to the index schema to get clickable URLs to show up in Solr search results in Hue dashboard? Thanks
... View more
Labels:
11-06-2018
07:01 AM
I am not using Kerberos on this cluster, it is a CDH 5.15 install.
... View more
11-05-2018
01:15 PM
Did not work. [centos@ip-172-31-81-167 ~]$ curl -i --retry 5 -s -L -k --negotiate -u : http://ip-172-31-82-140.ec2.internal:8983/solr curl: option --negotiate: the installed libcurl version doesn't support this curl: try 'curl --help' or 'curl --manual' for more information
... View more
11-01-2018
10:05 AM
Yes, running all commands on NN2 which happens to be a Solr node also. I was able to create the collections using the collections api, but solrctl works for some commands and just not the collections command. Maybe some bug, please check the code. Thanks
... View more
11-01-2018
09:35 AM
[centos@ip-172-31-81-167 data]$ solrctl --trace --zk ip-172-31-90-165.ec2.internal:2181,ip-172-31-81-167.ec2.internal:2181,ip-172-31-92-169.ec2.internal:2181/solr collection --delete ds-collection + shift 1 + test 5 '!=' 0 + case "$1" in + '[' 5 -gt 1 ']' + SOLR_ZK_ENSEMBLE=ip-172-31-90-165.ec2.internal:2181,ip-172-31-81-167.ec2.internal:2181,ip-172-31-92-169.ec2.internal:2181/solr + shift 2 + test 3 '!=' 0 + case "$1" in + break + '[' -n '' ']' + '[' -n '' ']' + '[' -n '' ']' + '[' -z ip-172-31-90-165.ec2.internal:2181,ip-172-31-81-167.ec2.internal:2181,ip-172-31-92-169.ec2.internal:2181/solr ']' + SOLR_ADMIN_ZK_CMD='ZKCLI_JVM_FLAGS=${ZKCLI_JVM_FLAGS} LOG4J_PROPS=${SOLR_CONF_DIR}/log4j.properties ${SOLR_HOME}/bin/zkcli.sh -zkhost $SOLR_ZK_ENSEMBLE 2>&3' + '[' 3 -gt 0 ']' + test 3 '!=' 0 + case "$1" in + '[' --delete = --list ']' + '[' --delete = --list-snapshots ']' + '[' 3 -gt 2 ']' + case "$2" in ++ echo --delete ++ tr '[a-z]-' '[A-Z] ' + COL_ACTION=' DELETE' ++ echo DELETE + eval solr_webapi ''\''/admin/collections?action=DELETE&name=ds-collection'\''' ++ solr_webapi '/admin/collections?action=DELETE&name=ds-collection' ++ '[' -z '' ']' +++ get_solr_protocol ++++ eval 'ZKCLI_JVM_FLAGS=${ZKCLI_JVM_FLAGS}' 'LOG4J_PROPS=${SOLR_CONF_DIR}/log4j.properties' '${SOLR_HOME}/bin/zkcli.sh' -zkhost '$SOLR_ZK_ENSEMBLE' '2>&3' -cmd get /clusterprops.json +++++ ZKCLI_JVM_FLAGS= +++++ LOG4J_PROPS=/etc/solr/conf/log4j.properties +++++ /opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/bin/../lib/solr//bin/zkcli.sh -zkhost ip-172-31-90-165.ec2.internal:2181,ip-172-31-81-167.ec2.internal:2181,ip-172-31-92-169.ec2.internal:2181/solr -cmd get /clusterprops.json +++ SOLR_STATE='{"urlScheme":"http"}' +++ echo '{"urlScheme":"http"}' +++ grep -i urlScheme +++ grep -q -i https +++ echo http ++ local SOLR_PROTOCOL=http +++ get_solr_state /live_nodes +++ sed -ne 's#/live_nodes/\(.*:[0-9][0-9]*\).*$#\1#p' ++++ eval 'ZKCLI_JVM_FLAGS=${ZKCLI_JVM_FLAGS}' 'LOG4J_PROPS=${SOLR_CONF_DIR}/log4j.properties' '${SOLR_HOME}/bin/zkcli.sh' -zkhost '$SOLR_ZK_ENSEMBLE' '2>&3' -cmd ls /live_nodes +++++ ZKCLI_JVM_FLAGS= +++++ LOG4J_PROPS=/etc/solr/conf/log4j.properties +++++ /opt/cloudera/parcels/CDH-5.15.1-1.cdh5.15.1.p0.4/bin/../lib/solr//bin/zkcli.sh -zkhost ip-172-31-90-165.ec2.internal:2181,ip-172-31-81-167.ec2.internal:2181,ip-172-31-92-169.ec2.internal:2181/solr -cmd ls /live_nodes +++ SOLR_STATE='/live_nodes (5) /live_nodes/ip-172-31-81-167.ec2.internal:8983_solr (0) /live_nodes/ip-172-31-90-165.ec2.internal:8983_solr (0) /live_nodes/ip-172-31-82-140.ec2.internal:8983_solr (0) /live_nodes/ip-172-31-92-169.ec2.internal:8983_solr (0) /live_nodes/ip-172-31-93-54.ec2.internal:8983_solr (0)' +++ echo '/live_nodes (5) /live_nodes/ip-172-31-81-167.ec2.internal:8983_solr (0) /live_nodes/ip-172-31-90-165.ec2.internal:8983_solr (0) /live_nodes/ip-172-31-82-140.ec2.internal:8983_solr (0) /live_nodes/ip-172-31-92-169.ec2.internal:8983_solr (0) /live_nodes/ip-172-31-93-54.ec2.internal:8983_solr (0)' +++ grep -v '^/ ' ++ for node in '`get_solr_state '\''/live_nodes'\'' | sed -ne '\''s#/live_nodes/\(.*:[0-9][0-9]*\).*$#\1#p'\''`' 'localhost:$SOLR_PORT' ++ curl -i --retry 5 -s -L -k --negotiate -u : http://ip-172-31-81-167.ec2.internal:8983/solr ++ for node in '`get_solr_state '\''/live_nodes'\'' | sed -ne '\''s#/live_nodes/\(.*:[0-9][0-9]*\).*$#\1#p'\''`' 'localhost:$SOLR_PORT' ++ curl -i --retry 5 -s -L -k --negotiate -u : http://ip-172-31-90-165.ec2.internal:8983/solr ++ for node in '`get_solr_state '\''/live_nodes'\'' | sed -ne '\''s#/live_nodes/\(.*:[0-9][0-9]*\).*$#\1#p'\''`' 'localhost:$SOLR_PORT' ++ curl -i --retry 5 -s -L -k --negotiate -u : http://ip-172-31-82-140.ec2.internal:8983/solr ++ for node in '`get_solr_state '\''/live_nodes'\'' | sed -ne '\''s#/live_nodes/\(.*:[0-9][0-9]*\).*$#\1#p'\''`' 'localhost:$SOLR_PORT' ++ curl -i --retry 5 -s -L -k --negotiate -u : http://ip-172-31-92-169.ec2.internal:8983/solr ++ for node in '`get_solr_state '\''/live_nodes'\'' | sed -ne '\''s#/live_nodes/\(.*:[0-9][0-9]*\).*$#\1#p'\''`' 'localhost:$SOLR_PORT' ++ curl -i --retry 5 -s -L -k --negotiate -u : http://ip-172-31-93-54.ec2.internal:8983/solr ++ for node in '`get_solr_state '\''/live_nodes'\'' | sed -ne '\''s#/live_nodes/\(.*:[0-9][0-9]*\).*$#\1#p'\''`' 'localhost:$SOLR_PORT' ++ curl -i --retry 5 -s -L -k --negotiate -u : http://localhost:/solr ++ '[' -n '' ']' ++ die 'Error: can'\''t discover Solr URI. Please specify it explicitly via --solr.' ++ '[' '!' -z 'Error: can'\''t discover Solr URI. Please specify it explicitly via --solr.' ']' ++ echo Error: 'can'\''t' discover Solr URI. Please specify it explicitly via --solr. Error: can't discover Solr URI. Please specify it explicitly via --solr. ## Seems like the collection command is not working in CDH 5.15. Instancedir works fine with zk quoram.
... View more
11-01-2018
09:34 AM
solrctl --zk ip-172-31-90-165.ec2.internal:2181,ip-172-31-81-167.ec2.internal:2181,ip-172-31-92-169.ec2.internal:2181/solr collection --delete ds-collection --trace Error: can't discover Solr URI. Please specify it explicitly via --solr.
... View more
10-30-2018
08:32 AM
Is this a bug in CDH 5.15? The same happens for me. This works solrctl --zk ip-172-31-90-165.ec2.internal:2181,ip-172-31-81-167.ec2.internal:2181,ip-172-31-92-169.ec2.internal:2181/solr instancedir --create ds-collection /tmp/ds-collection This gives me Solr url not found error: solrctl --zk ip-172-31-90-165.ec2.internal:2181,ip-172-31-81-167.ec2.internal:2181,ip-172-31-92-169.ec2.internal:2181/solr collection --create ds-collection -s 2 -r 2 The above command used to work in older 5.1x versions. Please advise.
... View more
10-25-2018
09:12 AM
I have created a slef-signed cert using this: Where do I find the other files mentioned in step 7 - https://www.cloudera.com/documentation/enterprise/5-15-x/topics/how_to_configure_cm_tls.html#concept_wk4_jlx_qw ? Now onwards trying to use CM for configs: https://www.cloudera.com/documentation/enterprise/5-15-x/topics/how_to_configure_cm_tls.html#concept_wk4_jlx_qw
... View more
10-23-2018
12:11 PM
Can one SSL certificate be used on all nodes of the cluster? Sorry for the questions, I am not familiar with doing this.
... View more
10-19-2018
10:39 AM
What if I needed to setup SSL only and then use our corporate SSO mechanism for authentication? Any hints on that?
... View more
10-19-2018
09:34 AM
Is Kerberos authentication mandatory before setting up SSL on Hadoop nodes? Also, moving to CDH 6 is not an option for us at the moment.
... View more
10-16-2018
07:32 AM
I am trying to enable SSL for all nodes and services in the cluster. How can I do this? Can someone please point me to some concise documentation where I can do this quickly and safely. Bit confused about Sentry and how it plays into authorization etc. Thanks.
... View more
Labels:
08-23-2018
06:27 AM
Solution found. In the hosts file of the python client machine, add public IP and private host name This is appropriate for a cloud service like AWS. Python lib works fine now. Thanks for @bgooley help on another thread that resolved this too.
... View more
08-22-2018
08:08 PM
@bgooley Thanks for the tip. That worked. So, putting the public IP and private hostname in hosts file on client did the trick. 🙂
... View more