Member since
01-29-2018
6
Posts
0
Kudos Received
0
Solutions
06-20-2019
09:28 AM
Getting below error while Start making an entry for Version Control in NiFi Registry: Version Details: HDF - 3.3.0 NiFi - 1.8.0 NiFi Registry - 0.3.0 NiFi Console Log: Unable to obtain listing of buckets: org.apache.nifi.registry.client.NiFiRegistryException: Error retrieving all buckets: Untrusted proxy [CN=node1.domain.com, OU=IT, O=COMPANY, L=Area, ST=State, C=US]. Contact the system administrator. NiFi Registry Log: 2019-06-19 06:04:03,248 INFO [NiFi Registry Web Server-18] o.a.n.r.w.m.IllegalStateExceptionMapper java.lang.IllegalStateException: Kerberos service ticket login not supported by this NiFi Registry. Returning Conflict response.
2019-06-19 06:04:03,620 INFO [NiFi Registry Web Server-17] o.a.n.r.w.s.NiFiRegistrySecurityConfig Client could not be authenticated due to: org.springframework.security.authentication.AuthenticationCredentialsNotFoundException: An Authentication object was not found in the SecurityContext Returning 401 response.
2019-06-19 06:22:49,013 INFO [NiFi Registry Web Server-63] o.a.n.r.w.m.IllegalStateExceptionMapper java.lang.IllegalStateException: Kerberos service ticket login not supported by this NiFi Registry. Returning Conflict response.
2019-06-19 06:22:49,263 INFO [NiFi Registry Web Server-17] o.a.n.r.w.s.NiFiRegistrySecurityConfig Client could not be authenticated due to: org.springframework.security.authentication.AuthenticationCredentialsNotFoundException: An Authentication object was not found in the SecurityContext Returning 401 response.
2019-06-19 06:27:01,720 INFO [NiFi Registry Web Server-68] o.a.n.r.w.s.NiFiRegistrySecurityConfig Identity in proxy chain not trusted to act as a proxy: org.apache.nifi.registry.web.security.authentication.exception.UntrustedProxyException: Untrusted proxy [CN=node1.domain.com, OU=IT, O=COMPANY, L=Area, ST=State, C=US]. Returning 403 response.
2019-06-19 06:28:10,402 INFO [NiFi Registry Web Server-68] o.a.n.r.w.s.NiFiRegistrySecurityConfig Identity in proxy chain not trusted to act as a proxy: org.apache.nifi.registry.web.security.authentication.exception.UntrustedProxyException: Untrusted proxy [CN=node1.domain.com, OU=IT, O=COMPANY, L=Area, ST=State, C=US]. Returning 403 response. Configuration: SSL - For both NiFi and NiFi Registry, we are using a host-specific self-signed certificate. And both services refer to the same Java Keystores. Authentication Used - LDAP Kerberos - No Ranger Plugin to NiFi - Yes NiFi Service: Authorization - Ranger Initial Admin Identity - admin nifi.security.user.login.identity.provider - ldap-provider Node Identities <property name="Node Identity 1">CN=node1.domain.com, OU=IT, O=COMPANY, L=Area, ST=State, C=US</property>
<property name="Node Identity 2">CN=node2.domain.com, OU=IT, O=COMPANY, L=Area, ST=State, C=US</property> NiFi Registry Service: Initial Admin Identity CN=admin, OU=Hadoop, OU=Accounts-Service, DC=Domain, DC=com nifi.registry.security.identity.provider=ldap-identity-provider NiFi Identities <property name="NiFi Identity 1">CN=node1.domain.com, OU=IT, O=COMPANY, L=Area, ST=State, C=US</property>
<property name="NiFi Identity 2">CN=node2.domain.com, OU=IT, O=COMPANY, L=Area, ST=State, C=US</property> authorizers.xml <userGroupProvider>
<identifier>file-user-group-provider</identifier>
<class>org.apache.nifi.registry.security.authorization.file.FileUserGroupProvider</class>
<property name="Users File">{{nifi_registry_internal_config_dir}}/users.xml</property>
<property name="Initial User Identity 1">CN=node1.domain.com, OU=IT, O=COMPANY, L=Area, ST=State, C=US</property>
<property name="Initial User Identity 2">CN=node2.domain.com, OU=IT, O=COMPANY, L=Area, ST=State, C=US</property>
<property name="Initial User Identity 3">CN=admin, OU=Hadoop, OU=Accounts-Service, DC=Domain, DC=com</property>
{{nifi_registry_ssl_config_content | replace("NiFi","Initial User")}}
</userGroupProvider> <accessPolicyProvider>
<identifier>file-access-policy-provider</identifier>
<class>org.apache.nifi.registry.security.authorization.file.FileAccessPolicyProvider</class>
<property name="User Group Provider">file-user-group-provider</property>
<property name="Authorizations File">{{nifi_registry_internal_config_dir}}/authorizations.xml</property>
<property name="Initial Admin Identity">CN=admin, OU=Hadoop, OU=Accounts-Service, DC=Domain, DC=com</property>
{{nifi_registry_ssl_config_content}}
</accessPolicyProvider> Note: I am able to login to both the services using LDAP user credentials. Also, created Ranger Policy: /proxy - Read & Write - CN=node1.domain.com, OU=IT, O=COMPANY, L=Area, ST=State, C=US Read & Write - CN=node2.domain.com, OU=IT, O=COMPANY, L=Area, ST=State, C=US Kindly help me to resolve this issue.
... View more
Labels:
06-20-2019
09:28 AM
I tried to configure LDAP with NiFi and NiFi Registry. And LDAP authentication is working fine in both the services. Here, the thing is I am not able to get into NiFi Registry Administration Console. Below are the configurations made to NiFi Registry Service: SSL For both NiFi and NiFi Registry, we are using a host-specific self-signed certificate. And both services refer to the same Java Keystores. Initial Admin Identity CN=admin, OU=Hadoop, OU=Accounts-Service, DC=Domain, DC=com NiFi Identities <property name="NiFi Identity 1">CN=node1.domain.com, OU=IT, O=COMPANY, L=Area, ST=State, C=US</property>
<property name="NiFi Identity 2">CN=node2.domain.com, OU=IT, O=COMPANY, L=Area, ST=State, C=US</property> authorizers.xml <userGroupProvider>
<identifier>file-user-group-provider</identifier>
<class>org.apache.nifi.registry.security.authorization.file.FileUserGroupProvider</class>
<property name="Users File">{{nifi_registry_internal_config_dir}}/users.xml</property>
<property name="Initial User Identity 1">CN=node1.domain.com, OU=IT, O=COMPANY, L=Area, ST=State, C=US</property>
<property name="Initial User Identity 2">CN=node2.domain.com, OU=IT, O=COMPANY, L=Area, ST=State, C=US</property>
<property name="Initial User Identity 3">CN=admin, OU=Hadoop, OU=Accounts-Service, DC=Domain, DC=com</property>
{{nifi_registry_ssl_config_content | replace("NiFi","Initial User")}}
</userGroupProvider> <accessPolicyProvider>
<identifier>file-access-policy-provider</identifier>
<class>org.apache.nifi.registry.security.authorization.file.FileAccessPolicyProvider</class>
<property name="User Group Provider">file-user-group-provider</property>
<property name="Authorizations File">{{nifi_registry_internal_config_dir}}/authorizations.xml</property>
<property name="Initial Admin Identity">CN=admin, OU=Hadoop, OU=Accounts-Service, DC=Domain, DC=com</property>
{{nifi_registry_ssl_config_content}}
</accessPolicyProvider> Please look below screenshot for the reference. Kindly help me to resolve this issue.
... View more
Labels:
08-06-2018
11:22 AM
We need to take backup all the topics in Kafka to the file named in respective topic names and need to restore the topic as per user requirement. Note: This script needs to be run in the Kerberized environment. kafkabackup.sh Making required directories monyear=`date | awk '{print $2$6}'`
dat=`date| awk '{print $2$3$6}'`
export BACKUPDIR=/root/backup/$monyear
mkdir -p $BACKUPDIR
mkdir -p $BACKUPDIR/$dat
cd $BACKUPSDIR
BKDIR=$BACKUPDIR/$dat
Log into Kafka Get topics from Kafka Broker kinit -kt /etc/security/keytabs/kafka.service.keytab kafka/node1.localdomaino@domain.co
cd /usr/hdp/current/kafka-broker/bin/
export KAFKA_CLIENT_KERBEROS_PARAMS="-Djava.security.auth.login.config=/etc/kafka/conf/kafka_client_jaas.conf"
./kafka-topics.sh --zookeeper adminnode.localdomain:2181 --list > $BKDIR/listtopics.txt
Remove if any mark of deletion topics exists sed -i.bak '/deletion/d' $BKDIR/listtopics.txt Starting kill script in parallel bash checkandkill.sh& Reading the file contents for topics for line in $(cat $BKDIR/listtopics.txt)
do
echo $line
./test.sh --bootstrap-server node1.localdomain:6668 --topic $line --consumer.config /home/kafka/conf.properties --from-beginning --security-protocol SASL_SSL > $BKDIR/$line
done
Delete empty files /usr/bin/find . -size 0 -delete Killing checkandkill daemon and exit ps -ef |grep -i checkandkill.sh| grep -v grep | awk '{print $2}' | xargs kill
exit
When consumer runs, it constantly waits for messages to receive. We need to kill the process. checkandkill.sh sleep 0.5m
for line in $(cat /root/backup/listtopics.txt)
do
echo $line
sleep 1m
ps -ef |grep -i $line| grep -v grep | awk '{print $2}' | xargs kill
done Need your help to complete restoration script.
... View more
Labels:
08-06-2018
11:17 AM
Need to connect and send the messages from Kafka Client(windows) to Kerberized environment on Hortonworks Platform using python. We have client certificate cacerts which in wildcard cert with an entry *.localdomain . Below possibilities are tried and results are unsuccessful. Please help me to resolve it. We referred to this link to connect. Using pykafka: Code import logging as log
log.basicConfig(level=log.DEBUG)
from pykafka import KafkaClient,SslConfig
config = SslConfig(cafile='D:\cacerts',password='password')
config._wrap_socket = config._legacy_wrap_socket()
client = KafkaClient(hosts='node1.localdomain:6668,node2.localdomain:6668',
ssl_config=config,
socket_timeout_ms=10000,
zookeeper_hosts="zknode.localdomain:2181")
client.topics
print (client.topics) Note: It is working fine in non-kerberized environment. Error DEBUG:pykafka.cluster:Updating cluster, attempt 1/3
INFO:kazoo.client:Connecting to 192.168.xx.xxx:2181
DEBUG:kazoo.client:Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=10000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None)
INFO:kazoo.client:Zookeeper connection established, state: CONNECTED
DEBUG:kazoo.client:Sending request(xid=1): GetChildren(path='/brokers/ids', watcher=None)
DEBUG:kazoo.client:Received response(xid=1): ['1002']
DEBUG:kazoo.client:Sending request(xid=2): GetData(path='/brokers/ids/1002', watcher=None)
DEBUG:kazoo.client:Received response(xid=2): (b'{"jmx_port":-1,"timestamp":"1531102985486","endpoints":["SASL_PLAINTEXT://node1.localdomain:6667","SASL_SSL://node2.localdomain:6668"],"host":null,"version":3,"port":-1,"rack":"/default-rack"}', ZnodeStat(czxid=34360596628, mzxid=34360596628, ctime=1531102985489, mtime=1531102985489, version=0, cversion=0, aversion=0, ephemeralOwner=244410183054001653, dataLength=202, numChildren=0, pzxid=34360596628))
DEBUG:kazoo.client:Sending request(xid=3): Close()
INFO:kazoo.client:Closing connection to 192.168.xx.xxx:2181
INFO:kazoo.client:Zookeeper session lost, state: CLOSED
DEBUG:pykafka.connection:Connecting to None:-1
INFO:pykafka.connection:Failed to connect to None:-1
INFO:pykafka.connection:[WinError 10061] No connection could be made because the target machine actively refused it
WARNING:pykafka.broker:Failed to connect newly created broker for None:-1
ERROR:pykafka.cluster:Socket disconnected during request for broker None:-1. Continuing.
DEBUG:pykafka.connection:Connecting to None:-1
INFO:pykafka.connection:Failed to connect to None:-1
INFO:pykafka.connection:[WinError 10061] No connection could be made because the target machine actively refused it
WARNING:pykafka.broker:Failed to connect newly created broker for None:-1
ERROR:pykafka.cluster:Socket disconnected during request for broker None:-1. Continuing.
DEBUG:pykafka.connection:Connecting to None:-1
INFO:pykafka.connection:Failed to connect to None:-1
INFO:pykafka.connection:[WinError 10061] No connection could be made because the target machine actively refused it
WARNING:pykafka.broker:Failed to connect newly created broker for None:-1
ERROR:pykafka.cluster:Socket disconnected during request for broker None:-1. Continuing. Using kafka-python: Code import logging as log
log.basicConfig(level=log.DEBUG)
from __future__ import print_function # python 2/3 compatibility
import sys # used to exit
from kafka import KafkaConsumer
import ssl
KAFKA_TOPIC = 'test'
KAFKA_BROKERS = 'node1.localdomain:6668,node2.localdomain:6668'
consumer = KafkaConsumer(KAFKA_TOPIC,
bootstrap_servers=KAFKA_BROKERS,
auto_offset_reset='earliest',
security_protocol='SSL',
ssl_certfile='D:\cacerts',
ssl_password='password',
ssl_check_hostname=True
)
try:
for message in consumer:
print(message.value)
except KeyboardInterrupt:
sys.exit() Error Traceback (most recent call last):
File "C:\Users\user\AppData\Local\Programs\Python\Python37\Lib\site-packages\kafkapython.py", line 64, in <module>
ssl_check_hostname=True
File "C:\Users\user\AppData\Local\Programs\Python\Python37\Lib\site-packages\kafka\consumer\group.py", line 340, in __init__
self._client = KafkaClient(metrics=self._metrics, **self.config)
File "C:\Users\user\AppData\Local\Programs\Python\Python37\Lib\site-packages\kafka\client_async.py", line 214, in __init__
self._bootstrap(collect_hosts(self.config['bootstrap_servers']))
File "C:\Users\user\AppData\Local\Programs\Python\Python37\Lib\site-packages\kafka\client_async.py", line 245, in _bootstrap
if not bootstrap.connect_blocking():
File "C:\Users\user\AppData\Local\Programs\Python\Python37\Lib\site-packages\kafka\conn.py", line 301, in connect_blocking
self.connect()
File "C:\Users\user\AppData\Local\Programs\Python\Python37\Lib\site-packages\kafka\conn.py", line 354, in connect
ret = self._sock.connect_ex(self._sock_addr)
File "C:\Users\user\AppData\Local\Programs\Python\Python37\lib\ssl.py", line 1146, in connect_ex
return self._real_connect(addr, True)
File "C:\Users\user\AppData\Local\Programs\Python\Python37\lib\ssl.py", line 1118, in _real_connect
raise ValueError("attempt to connect already-connected SSLSocket!")
ValueError: attempt to connect already-connected SSLSocket! Code import logging as log
log.basicConfig(level=log.DEBUG)
from __future__ import print_function # python 2/3 compatibility
import sys # used to exit
from kafka import KafkaConsumer
import ssl
KAFKA_TOPIC = 'test'
KAFKA_BROKERS = 'node1.localdomain:6668,node2.localdomain:6668'
consumer = KafkaConsumer(KAFKA_TOPIC,
bootstrap_servers=KAFKA_BROKERS,
auto_offset_reset='earliest',
security_protocol='SSL',
ssl_cafile='D:\cacerts',
ssl_certfile='D:\certificate.pem',
ssl_keyfile='D:\key.pem',
ssl_password='password',
ssl_check_hostname=True
)
try:
for message in consumer:
print(message.value)
except KeyboardInterrupt:
sys.exit() FYI, these pem files are generated from cacerts(Alias: *.localdomain ) Error Traceback (most recent call last):
File "C:\Users\user\AppData\Local\Programs\Python\Python37\Lib\site-packages\kafkapython.py", line 40, in <module>
ssl_check_hostname=True
File "C:\Users\user\AppData\Local\Programs\Python\Python37\Lib\site-packages\kafka\consumer\group.py", line 340, in __init__
self._client = KafkaClient(metrics=self._metrics, **self.config)
File "C:\Users\user\AppData\Local\Programs\Python\Python37\Lib\site-packages\kafka\client_async.py", line 214, in __init__
self._bootstrap(collect_hosts(self.config['bootstrap_servers']))
File "C:\Users\user\AppData\Local\Programs\Python\Python37\Lib\site-packages\kafka\client_async.py", line 245, in _bootstrap
if not bootstrap.connect_blocking():
File "C:\Users\user\AppData\Local\Programs\Python\Python37\Lib\site-packages\kafka\conn.py", line 301, in connect_blocking
self.connect()
File "C:\Users\user\AppData\Local\Programs\Python\Python37\Lib\site-packages\kafka\conn.py", line 340, in connect
self._wrap_ssl()
File "C:\Users\user\AppData\Local\Programs\Python\Python37\Lib\site-packages\kafka\conn.py", line 426, in _wrap_ssl
self._ssl_context.load_verify_locations(self.config['ssl_cafile'])
OSError: [Errno 22] Invalid argument
... View more
Labels:
01-29-2018
02:32 PM
When I save the workflow, it shows the below error.
Error occurred while saving workflow
Access Error to file due to access control
org.apache.oozie.ambari.view.exception.WfmWebException at org.apache.oozie.ambari.view.OozieProxyImpersonator.saveWorkflow(OozieProxyImpersonator.java:257) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205) at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733) at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1507) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.apache.ambari.server.security.authorization.AmbariAuthorizationFilter.doFilter(AmbariAuthorizationFilter.java:287) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.apache.ambari.server.security.authentication.AmbariDelegatingAuthenticationFilter.doFilter(AmbariDelegatingAuthenticationFilter.java:132) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.apache.ambari.server.security.authorization.AmbariUserAuthorizationFilter.doFilter(AmbariUserAuthorizationFilter.java:91) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192) at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478) at org.apache.ambari.server.api.MethodOverrideFilter.doFilter(MethodOverrideFilter.java:72) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478) at org.apache.ambari.server.api.AmbariPersistFilter.doFilter(AmbariPersistFilter.java:47) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478) at org.apache.ambari.server.view.AmbariViewsMDCLoggingFilter.doFilter(AmbariViewsMDCLoggingFilter.java:54) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478) at org.apache.ambari.server.view.ViewThrottleFilter.doFilter(ViewThrottleFilter.java:161) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478) at org.apache.ambari.server.security.AbstractSecurityHeaderFilter.doFilter(AbstractSecurityHeaderFilter.java:125) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478) at org.apache.ambari.server.security.AbstractSecurityHeaderFilter.doFilter(AbstractSecurityHeaderFilter.java:125) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478) at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:82) at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:294) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:499) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:427) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.apache.ambari.server.controller.AmbariHandlerList.processHandlers(AmbariHandlerList.java:212) at org.apache.ambari.server.controller.AmbariHandlerList.processHandlers(AmbariHandlerList.java:201) at org.apache.ambari.server.controller.AmbariHandlerList.handle(AmbariHandlerList.java:150) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:370) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494) at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:984) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1045) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:861) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:236) at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696) at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.hadoop.security.authorize.AuthorizationException: Unauthorized connection for super-user: root from IP 192.168.0.56 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.toIOException(WebHdfsFileSystem.java:519) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileSystem.java:497) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.access$200(WebHdfsFileSystem.java:114) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$FsPathOutputStreamRunner$1.close(WebHdfsFileSystem.java:950) at org.apache.oozie.ambari.view.HDFSFileUtils.writeToFile(HDFSFileUtils.java:89) at org.apache.oozie.ambari.view.WorkflowFilesService.createFile(WorkflowFilesService.java:42) at org.apache.oozie.ambari.view.OozieProxyImpersonator.saveWorkflowXml(OozieProxyImpersonator.java:274) at org.apache.oozie.ambari.view.OozieProxyImpersonator.saveWorkflow(OozieProxyImpersonator.java:244) ... 97 more Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): Unauthorized connection for super-user: root from IP 192.168.0.56 at org.apache.hadoop.hdfs.web.JsonUtil.toRemoteException(JsonUtil.java:119) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileSystem.java:480) ... 103 more<br>
... View more
Labels:
01-29-2018
02:32 PM
When I save the workflow, it shows the below error. Error occurred while saving workflow
Access Error to file due to access control
org.apache.oozie.ambari.view.exception.WfmWebException at org.apache.oozie.ambari.view.OozieProxyImpersonator.saveWorkflow(OozieProxyImpersonator.java:257) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205) at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733) at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1507) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.apache.ambari.server.security.authorization.AmbariAuthorizationFilter.doFilter(AmbariAuthorizationFilter.java:287) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.apache.ambari.server.security.authentication.AmbariDelegatingAuthenticationFilter.doFilter(AmbariDelegatingAuthenticationFilter.java:132) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.apache.ambari.server.security.authorization.AmbariUserAuthorizationFilter.doFilter(AmbariUserAuthorizationFilter.java:91) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192) at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478) at org.apache.ambari.server.api.MethodOverrideFilter.doFilter(MethodOverrideFilter.java:72) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478) at org.apache.ambari.server.api.AmbariPersistFilter.doFilter(AmbariPersistFilter.java:47) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478) at org.apache.ambari.server.view.AmbariViewsMDCLoggingFilter.doFilter(AmbariViewsMDCLoggingFilter.java:54) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478) at org.apache.ambari.server.view.ViewThrottleFilter.doFilter(ViewThrottleFilter.java:161) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478) at org.apache.ambari.server.security.AbstractSecurityHeaderFilter.doFilter(AbstractSecurityHeaderFilter.java:125) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478) at org.apache.ambari.server.security.AbstractSecurityHeaderFilter.doFilter(AbstractSecurityHeaderFilter.java:125) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478) at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:82) at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:294) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:499) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:427) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.apache.ambari.server.controller.AmbariHandlerList.processHandlers(AmbariHandlerList.java:212) at org.apache.ambari.server.controller.AmbariHandlerList.processHandlers(AmbariHandlerList.java:201) at org.apache.ambari.server.controller.AmbariHandlerList.handle(AmbariHandlerList.java:150) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:370) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494) at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:984) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1045) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:861) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:236) at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696) at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.hadoop.security.authorize.AuthorizationException: Unauthorized connection for super-user: root from IP 192.168.0.56 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.toIOException(WebHdfsFileSystem.java:519) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileSystem.java:497) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.access$200(WebHdfsFileSystem.java:114) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$FsPathOutputStreamRunner$1.close(WebHdfsFileSystem.java:950) at org.apache.oozie.ambari.view.HDFSFileUtils.writeToFile(HDFSFileUtils.java:89) at org.apache.oozie.ambari.view.WorkflowFilesService.createFile(WorkflowFilesService.java:42) at org.apache.oozie.ambari.view.OozieProxyImpersonator.saveWorkflowXml(OozieProxyImpersonator.java:274) at org.apache.oozie.ambari.view.OozieProxyImpersonator.saveWorkflow(OozieProxyImpersonator.java:244) ... 97 more Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): Unauthorized connection for super-user: root from IP 192.168.0.56 at org.apache.hadoop.hdfs.web.JsonUtil.toRemoteException(JsonUtil.java:119) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileSystem.java:480) ... 103 more<br>
... View more
Labels:
