Member since
01-12-2022
19
Posts
2
Kudos Received
0
Solutions
08-10-2023
11:05 PM
Hi, Scharan, thisi is the command's return: [root@hdp002 ~]# openssl s_client -connect hdp002.datasw.com:50470 -showcerts
CONNECTED(00000003)
depth=1 C = CN, ST = ShenZhen, L = GuangDong, O = DATASW, OU = PlatformTeam, CN = datsw
verify error:num=19:self signed certificate in certificate chain
---
Certificate chain
0 s:/C=CN/ST=GuangDong/L=ShenZhen/O=DATASW/OU=PlatformTeam/CN=hdp002.datasw
i:/C=CN/ST=ShenZhen/L=GuangDong/O=DATASW/OU=PlatformTeam/CN=datsw
-----BEGIN CERTIFICATE-----
MIIDXDCCAkQCCQDNmxfKgcxaOjANBgkqhkiG9w0BAQsFADBsMQswCQYDVQQGEwJD
TjERMA8GA1UECAwIU2hlblpoZW4xEjAQBgNVBAcMCUd1YW5nRG9uZzEPMA0GA1UE
CgwGREFUQVNXMRUwEwYDVQQLDAxQbGF0Zm9ybVRlYW0xDjAMBgNVBAMMBWRhdHN3
MB4XDTIxMDkxNDE3MjcwM1oXDTMxMDkxMjE3MjcwM1owdDELMAkGA1UEBhMCQ04x
EjAQBgNVBAgTCUd1YW5nRG9uZzERMA8GA1UEBxMIU2hlblpoZW4xDzANBgNVBAoT
BkRBVEFTVzEVMBMGA1UECxMMUGxhdGZvcm1UZWFtMRYwFAYDVQQDEw1oZHAwMDIu
ZGF0YXN3MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAgUquEQOPlyZs
vqfNZCB6qXh+wDoQlOMTPWfoHAtk5c041LwDrtkjF3I+uXntqcV/aB0ufSLA3M/j
2iiPt+8sosjp0cOakWXpOGx3Vv7MXteF/c8HMav+9yheY8qcoHfrRwPqpq9v0Ysz
31v1l6zJiuubF9JJpYMMmBkfwd0RUyLD079VkKAHtq6zAlpBm+zKVc6B0Xiddvw6
La2PB/c+vqVGVKsI+0RqiMdM1IXCuV76CT47riQ8G0PPs1OSu4HVI+J1j6P3R6He
rpIG1GstmcGjbR10qo/MDg1svwxiGOJxw1sN+65LQPLB3cTw4JmUMIJq8F1REiAZ
azP8I1oTsQIDAQABMA0GCSqGSIb3DQEBCwUAA4IBAQA4B1NwbQ0jBBccOfX0zCoQ
mHiIQvpMiiUYJRTAfUqueIFMwRF5aw85u2PAMnkgxqUmRG87RTgOg5dx3hPswS83
FQOeG2kUPXIR9HRaJZirRZ0vZByzsHl77N4Bb0y0Z0m1svbSczxH/RCEY5L7NvMR
fUSZjYW8LFk8nl3tH60u4f7cDJ+bTiDA37B7tUqvwrRHsvSJY9Ndp52QRUHxST3o
iutBSEOWfhBvPkv7U/B3WAnT9Dp0aSur2jIr5BP99qs13cy4qK0h5OAB+lgCEBt5
L4JKqWOv3W33j4vtNeYOSzfHgoi2JgKDP+imoVgnGeK1GNJsHMxecw4ef0Eik9sW
-----END CERTIFICATE-----
1 s:/C=CN/ST=ShenZhen/L=GuangDong/O=DATASW/OU=PlatformTeam/CN=datsw
i:/C=CN/ST=ShenZhen/L=GuangDong/O=DATASW/OU=PlatformTeam/CN=datsw
-----BEGIN CERTIFICATE-----
MIIDqzCCApOgAwIBAgIJAMmdxf5CbH3BMA0GCSqGSIb3DQEBCwUAMGwxCzAJBgNV
BAYTAkNOMREwDwYDVQQIDAhTaGVuWmhlbjESMBAGA1UEBwwJR3VhbmdEb25nMQ8w
DQYDVQQKDAZEQVRBU1cxFTATBgNVBAsMDFBsYXRmb3JtVGVhbTEOMAwGA1UEAwwF
ZGF0c3cwHhcNMjEwOTE0MDE1MTQwWhcNMzEwOTEyMDE1MTQwWjBsMQswCQYDVQQG
EwJDTjERMA8GA1UECAwIU2hlblpoZW4xEjAQBgNVBAcMCUd1YW5nRG9uZzEPMA0G
A1UECgwGREFUQVNXMRUwEwYDVQQLDAxQbGF0Zm9ybVRlYW0xDjAMBgNVBAMMBWRh
dHN3MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAuDsAyDscHIGXRmTz
EKzWjenR8c2f6hpLEyRTqvlI9AzTd5gZRMMWg7ax4erC7BPwva+RHZoeug9kE2HC
UHoGCP4YIZdEux5phPqv1vP/CBvbXnYZ4olMRSJDuf57TpAZjMTy5FHgs7QDzpCk
9Ez7CQWeXaaAaqnGo8SUWLATLadudSgkPDLSJL/h2IGjhxKPMyaHODxXQRxUIRbr
tI+8+9+siRLi+3EIhMXLT1oEOnsB/BQmawbNjyLtuZZoH8pyGJ3ByoM06zLWWMGI
eujFCOlSRMMzpEr/xLhJQQDFBLEFJKYOD0Z5QbqgrFkQtLOFpEnDbTXasQvlk4on
nLVemQIDAQABo1AwTjAdBgNVHQ4EFgQUUkBUUrsXSVF7MQDnB0hKjtpiOt0wHwYD
VR0jBBgwFoAUUkBUUrsXSVF7MQDnB0hKjtpiOt0wDAYDVR0TBAUwAwEB/zANBgkq
hkiG9w0BAQsFAAOCAQEAdU7iAr/F5lCLvPMfo9LA7JhI7IQdic/EhxELLuUELF7c
UBOOlJbWFxLYaZ6SwZ9lGa4d+wjNWoX+QvLt02PGZV3h0aB6O8E0827jjgI61r0C
UNSD3N3KadbK52st5W34sIssXqBNIga1w9knfWouiqNcHBixyZdYfWOwGLPSAbpC
K4os4yi5QU4YSvNwLO9GAYgem0p0Uel9By3m0cFmyFr+GcA+VAWltk7xBKOsCxam
nnQJE+djbMekmXW6cmujbqh02Q6LF0/6wNDMRnRFkDvF5WnT1XxQ7O+HFkeQXPED
qCkcKcHLqMxhK72iVlLgCq6n+oYLDxeODfHEjvo3sg==
-----END CERTIFICATE-----
---
Server certificate
subject=/C=CN/ST=GuangDong/L=ShenZhen/O=DATASW/OU=PlatformTeam/CN=hdp002.datasw
issuer=/C=CN/ST=ShenZhen/L=GuangDong/O=DATASW/OU=PlatformTeam/CN=datsw
---
No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 2350 bytes and written 483 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES256-SHA384
Session-ID: 64D5CE575ACD383C1B9BED92D5F2FDC1C63308098FD241173411E62C2E5E0395
Session-ID-ctx:
Master-Key: D73511C7D981C1A2F7813E02F102BD23057A5A79C5E9E75C3BCE870AA40D7CE4F02E41115F28510CE7AF85C6F6675BE6
Key-Arg : None
Krb5 Principal: None
PSK identity: None
PSK identity hint: None
Start Time: 1691733590
Timeout : 300 (sec)
Verify return code: 19 (self signed certificate in certificate chain)
--- KIT is not installed on my Windows machine, so I use curl on the Linux server to access the http request of the namenode: curl -i --insecure --negotiate -u: "https://hdp002.datasw.com:50470/jmx?qry=Hadoop:service=NameNode,name=RpcActivityForPort8020"
... View more
08-07-2023
08:18 PM
I have a secure hadoop cluster with HDP3.1， I recently tried to interconnect this cluster with the Knox component to implement a secure proxy. This cluster has kerberos, Ldap, and https enabled, I create one config like this: <topology>
<gateway>
<provider>
<role>authentication</role>
<name>ShiroProvider</name>
<enabled>true</enabled>
<param>
<name>main.ldapRealm</name>
<value>org.apache.knox.gateway.shirorealm.KnoxLdapRealm</value>
</param>
<param>
<name>main.ldapContextFactory</name>
<value>org.apache.knox.gateway.shirorealm.KnoxLdapContextFactory</value>
</param>
<param>
<name>main.ldapRealm.contextFactory</name>
<value>$ldapContextFactory</value>
</param>
<param>
<name>main.ldapRealm.userDnTemplate</name>
<value>cn=admin,dc=datasw,dc=com</value>
</param>
<param>
<name>main.ldapRealm.contextFactory.url</name>
<value>ldap://hdp001.datasw.com:389</value>
</param>
<param>
<name>main.ldapRealm.contextFactory.authenticationMechanism</name>
<value>simple</value>
</param>
<param>
<name>urls./**</name>
<value>authcBasic</value>
</param>
</provider>
<provider>
<role>authentication</role>
<name>HadoopAuth</name>
<enabled>true</enabled>
<param>
<name>config.prefix</name>
<value>hadoop.auth.config</value>
</param>
<param>
<name>hadoop.auth.config.type</name>
<value>kerberos</value>
</param>
<param>
<name>hadoop.auth.config.simple.anonymous.allowed</name>
<value>false</value>
</param>
<param>
<name>hadoop.auth.config.token.validity</name>
<value>1800</value>
</param>
<param>
<name>hadoop.auth.config.cookie.domain</name>
<value>datasw.com</value>
</param>
<param>
<name>hadoop.auth.config.cookie.path</name>
<value>gateway/default</value>
</param>
<param>
<name>hadoop.auth.config.kerberos.principal</name>
<value>HTTP/hdp003.datasw@DATASW.COM</value>
</param>
<param>
<name>hadoop.auth.config.kerberos.keytab</name>
<value>/etc/security/keytabs/spnego.service.keytab</value>
</param>
<param>
<name>hadoop.auth.config.kerberos.name.rules</name>
<value>DEFAULT</value>
</param>
<param>
<name>fs.defaultFS</name>
<value>hdfs://hdfsCluster</value>
</param>
<param>
<name>dfs.internal.nameservices</name>
<value>hdfsCluster</value>
</param>
<param>
<name>dfs.ha.namenodes.hdfsCluster</name>
<value>nn1,nn2</value>
</param>
<param>
<name>dfs.nameservices</name>
<value>hdfsCluster</value>
</param>
<param>
<name>dfs.namenode.https-address</name>
<value>hdp001.datasw:50470</value>
</param>
<param>
<name>dfs.namenode.https-address.hdfsCluster.nn1</name>
<value>hdp001.datasw:50470</value>
</param>
<param>
<name>dfs.namenode.https-address.hdfsCluster.nn2</name>
<value>hdp002.datasw:50470</value>
</param>
</provider>
</gateway>
<service>
<role>HDFSUI</role>
<url>https://hdp002.datasw.com:50470</url>
</service>
</topology> and I copy the hadoop cluster's truststore.jks file to the $GATEWAY_HOME/data/security/keystores/ and set gateway.httpclient.truststore.path param in gateway-stie.xml: <property>
<name>gateway.httpclient.truststore.path</name>
<value>/usr/local/knox/data/security/keystores/truststore.jks</value>
</property>
<property>
<name>gateway.httpclient.truststore.type</name>
<value>JKS</value>
</property>
<property>
<name>gateway.httpclient.truststore.password.alias</name>
<value>pthdp</value>
</property> Then I restart the Knox gateway，but w hen I access the NameNode webUi, I receive the following error message: 2023-08-08 11:14:38,050 58fc3dbf-4c6e-4684-860d-0a4e443f85d2 WARN knox.gateway (DefaultDispatch.java:executeOutboundRequest(183)) - Connection exception dispatching request: https://hdp002.datasw.com:50470/?user.name=admin javax.net.ssl.SSLPeerUnverifiedException: Certificate for <hdp002.datasw.com> doesn't match any of the subject alternative names: []
javax.net.ssl.SSLPeerUnverifiedException: Certificate for <hdp002.datasw.com> doesn't match any of the subject alternative names: []
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.verifyHostname(SSLConnectionSocketFactory.java:507) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:437) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:384) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.knox.gateway.dispatch.DefaultDispatch.executeOutboundRequest(DefaultDispatch.java:166) ~[gateway-spi-2.0.0.jar:2.0.0]
at org.apache.knox.gateway.dispatch.DefaultDispatch.executeRequest(DefaultDispatch.java:152) ~[gateway-spi-2.0.0.jar:2.0.0]
at org.apache.knox.gateway.dispatch.DefaultDispatch.executeRequestWrapper(DefaultDispatch.java:135) ~[gateway-spi-2.0.0.jar:2.0.0]
at org.apache.knox.gateway.dispatch.DefaultDispatch.doGet(DefaultDispatch.java:300) ~[gateway-spi-2.0.0.jar:2.0.0]
at org.apache.knox.gateway.dispatch.GatewayDispatchFilter$GetAdapter.doMethod(GatewayDispatchFilter.java:183) ~[gateway-spi-2.0.0.jar:2.0.0]
at org.apache.knox.gateway.dispatch.GatewayDispatchFilter.doFilter(GatewayDispatchFilter.java:127) ~[gateway-spi-2.0.0.jar:2.0.0]
at org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:58) ~[gateway-spi-2.0.0.jar:2.0.0]
at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:377) ~[gateway-server-2.0.0.jar:2.0.0]
at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:291) ~[gateway-server-2.0.0.jar:2.0.0]
at org.apache.knox.gateway.identityasserter.common.filter.AbstractIdentityAssertionFilter.doFilterInternal(AbstractIdentityAssertionFilter.java:193) ~[gateway-provider-identity-assertion-common-2.0.0.jar:2.0.0]
at org.apache.knox.gateway.identityasserter.common.filter.AbstractIdentityAssertionFilter.access$000(AbstractIdentityAssertionFilter.java:55) ~[gateway-provider-identity-assertion-common-2.0.0.jar:2.0.0]
at org.apache.knox.gateway.identityasserter.common.filter.AbstractIdentityAssertionFilter$1.run(AbstractIdentityAssertionFilter.java:161) ~[gateway-provider-identity-assertion-common-2.0.0.jar:2.0.0]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_291]
at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_291]
at org.apache.knox.gateway.identityasserter.common.filter.AbstractIdentityAssertionFilter.doAs(AbstractIdentityAssertionFilter.java:156) ~[gateway-provider-identity-assertion-common-2.0.0.jar:2.0.0]
at org.apache.knox.gateway.identityasserter.common.filter.AbstractIdentityAssertionFilter.continueChainAsPrincipal(AbstractIdentityAssertionFilter.java:146) ~[gateway-provider-identity-assertion-common-2.0.0.jar:2.0.0]
at org.apache.knox.gateway.identityasserter.common.filter.CommonIdentityAssertionFilter.doFilter(CommonIdentityAssertionFilter.java:241) ~[gateway-provider-identity-assertion-common-2.0.0.jar:2.0.0]
at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:377) ~[gateway-server-2.0.0.jar:2.0.0]
at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:291) ~[gateway-server-2.0.0.jar:2.0.0]
at org.apache.knox.gateway.filter.rewrite.api.UrlRewriteServletFilter.doFilter(UrlRewriteServletFilter.java:57) ~[gateway-provider-rewrite-2.0.0.jar:2.0.0]
at org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:58) ~[gateway-spi-2.0.0.jar:2.0.0]
at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:377) ~[gateway-server-2.0.0.jar:2.0.0]
at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:291) ~[gateway-server-2.0.0.jar:2.0.0]
at org.apache.knox.gateway.filter.ShiroSubjectIdentityAdapter$CallableChain$1.run(ShiroSubjectIdentityAdapter.java:93) ~[gateway-provider-security-shiro-2.0.0.jar:2.0.0]
at org.apache.knox.gateway.filter.ShiroSubjectIdentityAdapter$CallableChain$1.run(ShiroSubjectIdentityAdapter.java:90) ~[gateway-provider-security-shiro-2.0.0.jar:2.0.0]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_291]
at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_291]
at org.apache.knox.gateway.filter.ShiroSubjectIdentityAdapter$CallableChain.call(ShiroSubjectIdentityAdapter.java:146) ~[gateway-provider-security-shiro-2.0.0.jar:2.0.0]
at org.apache.knox.gateway.filter.ShiroSubjectIdentityAdapter$CallableChain.call(ShiroSubjectIdentityAdapter.java:76) ~[gateway-provider-security-shiro-2.0.0.jar:2.0.0]
at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) ~[shiro-core-1.10.0.jar:1.10.0]
at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) ~[shiro-core-1.10.0.jar:1.10.0]
at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:387) ~[shiro-core-1.10.0.jar:1.10.0]
at org.apache.knox.gateway.filter.ShiroSubjectIdentityAdapter.doFilter(ShiroSubjectIdentityAdapter.java:73) ~[gateway-provider-security-shiro-2.0.0.jar:2.0.0]
at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:377) ~[gateway-server-2.0.0.jar:2.0.0]
at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:291) ~[gateway-server-2.0.0.jar:2.0.0]
at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:61) ~[shiro-web-1.10.0.jar:1.10.0]
at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108) ~[shiro-web-1.10.0.jar:1.10.0]
at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137) ~[shiro-web-1.10.0.jar:1.10.0]
at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:154) ~[shiro-web-1.10.0.jar:1.10.0]
at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66) ~[shiro-web-1.10.0.jar:1.10.0]
at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108) ~[shiro-web-1.10.0.jar:1.10.0]
at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137) ~[shiro-web-1.10.0.jar:1.10.0]
at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:154) ~[shiro-web-1.10.0.jar:1.10.0]
at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66) ~[shiro-web-1.10.0.jar:1.10.0]
at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:458) ~[shiro-web-1.10.0.jar:1.10.0]
at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:373) ~[shiro-web-1.10.0.jar:1.10.0] In order to achieve Knox proxy, What else do I need to do?
... View more
Labels:
- Labels:
-
Apache Knox
-
Kerberos
07-27-2023
01:01 AM
Hi, My hadoop cluster use HDP3.1.5-152.0, I start httpfs in cli command, and my webhdfs is work fine. But when i send request to the httpfs service , i got 500 code error like this: 15:50:39,064 WARN ServletHandler:632 - /webhdfs/v1/user
java.lang.IllegalArgumentException: Empty key
at javax.crypto.spec.SecretKeySpec.<init>(SecretKeySpec.java:96)
at org.apache.hadoop.security.authentication.util.Signer.computeSignature(Signer.java:93)
at org.apache.hadoop.security.authentication.util.Signer.sign(Signer.java:59)
at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:587)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1751)
at org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1619)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:539)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:745) This error occurs regardless of whether my cluster has kerberos and ssl enabled, Please give me some help, thanks!
... View more
Labels:
- Labels:
-
HDFS
12-06-2022
07:34 PM
Hi,r ecently I have been trying to track my spark application using SparkListener. Now I have a problem: There is a application, the result of its execution is SUCCESSED on Yarn WebUI,but it's actually a failed application. And my listener class can not get the error message,the application log only show the Start events for failed jobs and tasks. Some can help me ?
... View more
Labels:
- Labels:
-
Apache Spark
12-06-2022
07:04 PM
1 Kudo
Thanks,but this api is used to delete the entity,I'm afraid it can't be used to delete a deleted Relationship hardly.
... View more
11-03-2022
02:25 AM
You can try this using the Rest api: # DELETE the Client
curl -k -u admin:admin -H "X-Requested-By: ambari" -X DELETE
"$HTTP_PROTOCOL://$CONSOLE_NODE:$PORT/api/v1/clusters/$CLUSTER/host_components?HostRoles/component_name.in(HDFS_CLIENT)&HostRoles/host_name.in(hostname01,hostname02)"
# INSTAL the Client
curl -v -k -i -u 'admin:admin' -H 'X-Requested-By: ambari' -X POST -d '
{
"RequestInfo":{
"query":"Hosts/host_name.in(hostname01)"
},
"Body":{
"host_components":[
{
"HostRoles":{
"component_name":"HDFS_CLIENT"
}
}
]
}
}' '$HTTP_PROTOCOL://$CONSOLE_NODE:$PORT/api/v1/clusters/$CLUSTER/hosts'
... View more
11-03-2022
02:00 AM
Hi, I'm trying to configure the spark-connector for apache atlas. When I execute this statement "create table dept_three_spark as select * from dept_four_ext_hive;" ,the table created successed ,but I get the following error: And when I visited atlas, a lot of information was missing. I tried to add the number of kafka topic ATLAS_HOOK and ATLAS_ENITIES partitions, but no use. What can I do next to solve this problem? Spark version: 2.3.0 Atlas version: 2.2.0 Kafka version: 2.0.0 HDP version: 3.1.5.0-152 Spark-connector version: 0.1.0.3.1.5.0-152
... View more
Labels:
11-01-2022
12:52 AM
1 Kudo
Hi: Is it currently possible to hard delete (purge) Relationships and RelationshipDefs via the API? I use the api /api/atlas/admin/purge/ to delete a DELETED Relationship but no use. Any help will be useful,Thanks. The Atlas version is 2.2.0
... View more
Labels:
- Labels:
-
Apache Atlas
07-23-2022
08:51 PM
Hi, Recently, in our production environment, there was an abnormal exit of the Hbase Master. In the log, I found that the ZK connection had some problems. According to the sessionid, I found that the sessionid of one of our applications was the same as the sessionid established by the master at the time in question, and then the application side closed the session, and then the master hung. Is this possible? Why does a session with the same ZooKeeper sessionid appear?
... View more
Labels:
- Labels:
-
Apache HBase
-
Apache Zookeeper
07-08-2022
11:22 PM
I know.Need FQDN likes testhost.magg.com,Thanks
... View more