Member since
08-16-2019
38
Posts
8
Kudos Received
4
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
444 | 05-21-2018 06:32 PM | |
1721 | 01-18-2018 06:51 PM | |
933 | 10-25-2017 06:28 PM | |
1631 | 08-25-2017 07:09 PM |
01-14-2021
08:42 AM
Starting CDP 7.2.7, Knox can be configured to load-balance backend service instances as well as support sticky sessions in HA mode. Prior to this when HA was configured for Knox (using the HaProvider in Knox topology) Knox would route requests to a specific backend service, except in case of failover where Knox would pick the next backend service from the configured list. This approach had some limitations, Knox would only serve one backend, even when there were multiple backends available. This especially was a problem for stateless services that needed even load distribution. Knox also had no support for sticky sessions. With this release, Knox can be configured to load-balance requests for multiple backend service instances configured with HaProvider. Currently, load-balancing is done in a simple round-robin fashion. Following is an example of how to configure only load-balancing for an example HA service WHOAMI <provider>
<role>ha</role>
<name>HaProvider</name>
<enabled>true</enabled>
<param>
<name>WHOAMI</name>
<value>enabled=true;maxFailoverAttempts=3;failoverSleep=1000;enableLoadBalancing=true</value>
</param>
</provider>
...
<service>
<role>WHOAMI</role>
<url>http://localhost:50070</url>
<url>http://localhost:50071</url>
</service> Sticky sessions can be used for services such as Hive where session state matters. Sticky sessions can be turned on by using the property enableStickySession. When sticky sessions are on, Knox uses a cookie to match the client requests with the backend host, and as a result, cookies are required for this feature. Cookie name used for sticky session is KNOX_BACKEND-{service-name}. Sticky session cookie name can be changed by using the property stickySessionCookieName. With sticky session turned on, loadbalancing is turned on automatically. If sticky session is turned on and there is a failover, Knox will choose a new backend and route the request to it. This can be undesirable in cases where session state is critical. Knox can be configured to not failover when sticky sessions are used by using the flag noFallback. When this flag (noFallback=true) is set, Knox will return a 502 (Bad Gateway) if a request comes with a cookie and the corresponding backend is unavailable. Example of sticky session configuration: <provider>
<role>ha</role>
<name>HaProvider</name>
<enabled>true</enabled>
<param>
<name>WHOAMI</name>
<value>enabled=true;maxFailoverAttempts=3;failoverSleep=1000;enableStickySession=true;noFallback=true</value>
</param>
</provider> Summary of HaProvider properties discussed above: enableStickySession - Enables sticky sessions stickySessionCookieName - Customize sticky session cookie name, default is KNOX_BACKEND-{serviceName} noFallback - When this flag is set, Knox will return a 502 (Bad Gateway) if a request comes with a cookie and the corresponding backend is unavailable enableLoadBalancing - Enable loadbalancing, roundrobin requests to backend HA urls
... View more
Labels:
01-21-2019
01:54 AM
Hello Doug, Try setting message size using the following properties gateway.websocket.max.text.size
gateway.websocket.max.binary.size
gateway.websocket.max.text.buffer.size
gateway.websocket.max.binary.buffer.size
gateway.websocket.input.buffer.size
... View more
09-24-2018
06:27 PM
There is some documentation on how this can be done, http://knox.apache.org/books/knox-1-1-0/user-guide.html#High+Availability and https://risdenk.github.io/2018/03/11/load-balancing-behind-apache-knox.html
... View more
08-22-2018
10:33 PM
Looking at the documentation (https://knox.apache.org/books/knox-1-1-0/user-guide.html#Gateway+Server+Configuration), try removing gateway.dispatch.whitelist: DEFAULT property.
... View more
08-07-2018
08:00 PM
@Danilo Sousa Looks like you are using demo ldap that ships with Knox, if you are using Ambari then start Demo ldap using the quicklinks on Knox components.
... View more
07-26-2018
01:29 PM
If you are using kerberos then you do not need LDAP, this looks like an issue with your configuration. This is an example https://community.hortonworks.com/articles/192759/knox-with-kerberos-authentication-to-proxy-to-hive.html The topology file is not formatted properly but you can compare it with yours and add the necessary configuration.
... View more
07-25-2018
08:49 PM
What do you see in gateway.log ? Also, is your sandbox is kerberized ? if not check whether demo ldap is running, sandbox might be using demo ldap.
... View more
06-06-2018
08:19 PM
I am not sure if Ambari supports HeaderPreAuth yet, it supports KnoxSSO though but it needs to be configured.
... View more
05-21-2018
06:32 PM
2 Kudos
Hello @Dominique De Vito Yes, Knox does work with unsecure Hadoop custer and yes you can use KnoxSSO with unsecure cluster as well.
... View more
04-13-2018
06:33 PM
This is most likely because you are using a SSOCookie provider which checks for cookies, this is why your browser based scenario works. You can try sending the token as a cookie using curl , something like curl -ivk --cookie "hadoop-jwt=
eyJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJndWVzdCIsImF1ZCI6InRva2VuYmFzZWQiLCJpc3MiOiJLTk9YU1NPIiwiZXhwIjoxNTIzNTY2NzEyfQ.KwKUKLbC6nYtRSjqfbGkLSY_wUdqChLcLEgRhL5Qvsu1L0eiG-RUdKaafeIPwZ8LA-7nL7Z0CjoSjUiHGBk93cwqGOwL9hpp1_-THx8l9ingZoTw-oj1rFbL8ZWoGbhBbNox6dXfwKj3cB1Qh8iI9rpwCra4VE0ndivUyTtNkpQ"
https://192.168.154.194:8443/gateway/tokenbased/webhdfs/v1/tmp?op=LISTSTATUS Here I am assuming your cookie name is "hadoop-jwt"
... View more
02-27-2018
06:35 PM
Hello Nick, I am not Ranger expert but I think you should be able to achieve with the Knox Ranger Plugin. If you can browse your data sets using REST APIs you can certainly use Knox to proxy them.
... View more
02-15-2018
02:00 PM
@Nisha that is by design, try passing cookies through Curl with the CookieProvider, I think this should work !
... View more
02-12-2018
07:07 PM
ah, I see, I have had issues with using localhost, to get around this in my etc/host file I add an entry for www.local.com -> 127.0.0.1 and in your topology add it to the whitelist <param> <name>knoxsso.redirect.whitelist.regex</name> <value>^https?:\/\/(www\.local\.com|localhost|127\.0\.0\.1|0:0:0:0:0:0:0:1|::1):[0-9].*$</value> </param> Then try to access it using https://www.local.com:8443/. This should help, let me know if this does not help. Best, Sandeep
... View more
02-09-2018
06:33 PM
Hello @Nisha It is difficult to say anything without seeing your topology, but I am assuming you are using "SSOCookieProvider" in your topology. Try using "JWTProvider" https://knox.apache.org/books/knox-1-0-0/user-guide.html#JWT+Provider Also check this out https://knox.apache.org/books/knox-1-0-0/user-guide.html#KnoxToken+Configuration Best, Sandeep
... View more
01-18-2018
06:51 PM
1 Kudo
Hello @Theyaa Matti this issue should be fixed in the current 0.14.0 Knox release. In case you do not want to upgrade you can set the following properties in gateway-site.xml "gateway.httpclient.connectionTimeout" and "gateway.httpclient.socketTimeout" to appropriate value, say 300.
... View more
11-17-2017
07:03 PM
Interesting, thanks for updating !
... View more
11-16-2017
06:45 PM
This seems like an OpenJDK and OS problem. What OS are you using ? there are a bunch of such bugs with OpenJDK and Fedora/RHEL
... View more
10-25-2017
06:31 PM
Hello @Neha G, you cannot access "webhdfs://" protocol using Knox, Knox only supports Http, Https and WS protocols.
... View more
10-25-2017
06:28 PM
This is because 'guest' user in the Demo Ldap does not belong to group "users". For this to work you should create a group "users" in the demo ldap (users.ldif) and add guest user to it. You could try to impersonate 'guest' to a user who belongs to 'users' group on the machine.
... View more
10-17-2017
06:24 PM
Hello @Neha G Sure, this guide should help you http://knox.apache.org/books/knox-0-13-0/user-guide.html#Secure+Clusters
... View more
10-11-2017
06:40 PM
Did you Login using kinit ?
... View more
09-27-2017
06:25 PM
Looks like you are missing the variable zeppelin_master_hosts
... View more
08-25-2017
07:09 PM
Do you know what version of Knox you are running, there was major work done around this area in the recent 0.13.0 release https://issues.apache.org/jira/browse/KNOX-949
... View more
07-31-2017
06:39 PM
Hello @sharad vishe to use Knox you do not have to make changes to other hadoop config files. Changes to your Knox topology (sandbox.xml) should be enough for the commonly used hadoop services (There are exceptions to this though but I do not believe that is the case here.)
... View more
07-11-2017
02:47 PM
You should have something logged in the gateway.log file. If you do not see anything meaningful there you can turn up the debug log (by update the conf/gateway-log4j.properties file, uncomment log4j.logger.org.apache.hadoop.gateway=DEBUG property)
... View more
07-09-2017
07:42 PM
Knox Master Secret that is used to protect the key and credential stores for the gateway instance.
To test the demo Ldap use "bin/ldap.sh start" the demo Ldap runs on localhost and port 33389 which is the default port for the ApacheDS LDAP server. That the LDAP directory in use has a set of demo users provisioned with
the convention of username and username“-password” as the password. Most
of the samples have some variation of this pattern with “guest” and
“guest-password”.
... View more
06-24-2017
02:55 AM
Ah, looks like a Cognos issue to me. Did you try using Congnos without Knox ? if you see that without Knox (and with Cognos) you have improved speeds then the issue might be with Knox.
... View more
06-23-2017
02:45 PM
Hello @Debodipto Roy Do you have problems retrieving data only when Apache Knox is used with IBM Cognos ?
... View more
05-26-2017
06:32 PM
Can you access Hbase Rest API without Knox ? is the hbase-deamon running ? you can find more info at Knox user guide.
If you still run into issues then something funny is going around, can you post the relevant log snippet from gateway.log. Thanks.
... View more