Member since
11-08-2016
10
Posts
2
Kudos Received
0
Solutions
05-18-2017
12:42 PM
Hi @dvillarreal the problem was the domain: I entered FIELD instead of FIELD.HORTONWORKS.COM. I thought I should enter the windows domain name (not FQDN) and FIELD.HORTONWORKS.COM is only the realm. In Ranger, I entered just FIELD and it works. I think Ranger make no use of the domain parameter, Atlas (or Shiro?) on the contrary append it to the username: looking at the tcpdump with Wireshark I discovered that it tried to login with "sales1@field". Many thanks!
... View more
05-18-2017
12:42 PM
@dvillarreal I'm trying with the Atlas version provided by HDP 2.5.3 using Active Directory with both atlas.authentication.method.ldap.type=ldap
atlas.authentication.method.ldap.type=ad It doesn't work either way. In the logs I see: DEBUG - [main:] ~ atlas.authentication.method.ldap.ad.base.dn = [DC=field, DC=hortonworks, DC=com]
DEBUG - [main:] ~ atlas.authentication.method.ldap.ad.bind.dn = [CN=binduser, OU=ServiceUsers, DC=field, DC=hortonworks, DC=com]
DEBUG - [main:] ~ atlas.authentication.method.ldap.ad.bind.password = BadPass#1
DEBUG - [main:] ~ atlas.authentication.method.ldap.ad.default.role = ROLE_USER
DEBUG - [main:] ~ atlas.authentication.method.ldap.ad.domain = FIELD
DEBUG - [main:] ~ atlas.authentication.method.ldap.ad.referral = follow
DEBUG - [main:] ~ atlas.authentication.method.ldap.ad.url = ldaps://ad01.field.hortonworks.com:636
DEBUG - [main:] ~ atlas.authentication.method.ldap.ad.user.searchfilter = (sAMAccountName={0})
DEBUG - [main:] ~ atlas.authentication.method.ldap.base.dn = [DC=field, DC=hortonworks, DC=com]
DEBUG - [main:] ~ atlas.authentication.method.ldap.bind.dn = [CN=binduser, OU=ServiceUsers, DC=field, DC=hortonworks, DC=com]
DEBUG - [main:] ~ atlas.authentication.method.ldap.bind.password = BadPass#1
DEBUG - [main:] ~ atlas.authentication.method.ldap.default.role = ROLE_USER
DEBUG - [main:] ~ atlas.authentication.method.ldap.groupRoleAttribute = cn
DEBUG - [main:] ~ atlas.authentication.method.ldap.groupSearchBase = [DC=field, DC=hortonworks, DC=com]
DEBUG - [main:] ~ atlas.authentication.method.ldap.groupSearchFilter = [(member=CN={0}, OU=CorpUsers, DC=field, DC=hortonworks, DC=com)]
DEBUG - [main:] ~ atlas.authentication.method.ldap.referral = follow
DEBUG - [main:] ~ atlas.authentication.method.ldap.type = ad
DEBUG - [main:] ~ atlas.authentication.method.ldap.url = ldaps://ad01.field.hortonworks.com:636
DEBUG - [main:] ~ atlas.authentication.method.ldap.user.searchfilter = (sAMAccountName={0})
DEBUG - [main:] ~ atlas.authentication.method.ldap.userDNpattern = [CN={0}, OU=CorpUsers, DC=field, DC=hortonworks, DC=com] What I notice is that the DNs are embedded in [].
... View more
01-18-2017
10:32 AM
@Nixon Rodrigues putting jceks://file before the path solved the problem, thanks!
... View more
01-18-2017
10:31 AM
That was the problem. Now it works! Thanks!
... View more
01-18-2017
09:45 AM
@Ayub Khan it is sufficient to enter in jceks://file/etc/atlas/conf/stores.jceks the config, no need to execute cputil.py again. Many thanks!
... View more
01-18-2017
09:20 AM
@Ayub Khan I notice only now this other log entry in a separate file /var/log/atlas/atlas.20170118-091030.err: Exception in thread "main" java.io.IOException: No CredentialProviderFactory for /etc/atlas/conf/stores.jceks in hadoop.security.credential.provider.path
at org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:66)
at org.apache.atlas.web.service.SecureEmbeddedServer.getPassword(SecureEmbeddedServer.java:121)
at org.apache.atlas.web.service.SecureEmbeddedServer.getConnector(SecureEmbeddedServer.java:69)
at org.apache.atlas.web.service.EmbeddedServer.<init>(EmbeddedServer.java:45)
at org.apache.atlas.web.service.SecureEmbeddedServer.<init>(SecureEmbeddedServer.java:60)
at org.apache.atlas.web.service.EmbeddedServer.newServer(EmbeddedServer.java:60)
at org.apache.atlas.Atlas.main(Atlas.java:117)
... View more
01-18-2017
08:47 AM
@Ayub Khan I recreated all mentioned files chown also the hidden file (it was still belonging to root) because chmod with * doesn't consider hidden files. The problem persist. However, I noticed that after running /usr/hdp/current/atlas-server/bin/cputil.py, 3 passwords are asked: Please enter the full path to the credential provider:/etc/atlas/conf/stores.jceks
Please enter the password value for keystore.password:
Please enter the password value for keystore.password again:
Please enter the password value for truststore.password:
Please enter the password value for truststore.password again:
Please enter the password value for password:
Please enter the password value for password again: For what is the last password? is it the password for accessing stores.jceks itself?
... View more
01-17-2017
06:33 PM
@Ayub Khan thanks for the help! I put export HADOOP_HOME=/usr/hdp/current/hadoop-client in "atlas-env template" and the error about HADOOP_HOME has gone. I also copied the hdfs-site.xml to /etc/atlas/conf and chown atlas:hadoop everything there Atlas fails to start in any case. Previously i did: /usr/hdp/current/atlas-server/bin/cputil.py and entered /etc/atlas/conf/stores.jceks as file many times <password>. Then: sudo keytool -noprompt \
-genkey -alias atlasssl -keyalg RSA -keysize 2048 -keypass <password> \
-keystore /etc/atlas/conf/keystore.jks -storepass <password> \
-dname "CN=Nicola Marangoni, OU=PS, O=Hortonworks, L=Munich, ST=BY, C=DE"
sudo cp /etc/atlas/conf/keystore.jks /etc/atlas/conf/truststore.jks
sudo chown atlas:hadoop /etc/atlas/conf/*
sudo chmod 400 /etc/atlas/conf/*.jks Passwords are the same everywhere. Should I retry these last steps?
... View more
01-17-2017
04:02 PM
After enabling TLS with the following properties in Atlas on HDP 2.5.3: keystore.file=/etc/atlas/conf/keystore.jks
truststore.file=/etc/atlas/conf/truststore.jks
cert.stores.credential.provider.path=/etc/atlas/conf/stores.jceks Atlas server doesn't start anymore. Logs: 2017-01-17 15:35:46,681 DEBUG - [main:] ~ cert.stores.credential.provider.path = /etc/atlas/conf/stores.jceks (ApplicationProperties:102)
2017-01-17 15:35:46,682 DEBUG - [main:] ~ keystore.file = /etc/atlas/conf/keystore.jks (ApplicationProperties:102)
2017-01-17 15:35:46,682 DEBUG - [main:] ~ truststore.file = /etc/atlas/conf/truststore.jks (ApplicationProperties:102)
2017-01-17 15:35:46,684 DEBUG - [main:] ~ ==> InMemoryJAASConfiguration.init() (InMemoryJAASConfiguration:168)
2017-01-17 15:35:46,695 DEBUG - [main:] ~ ==> InMemoryJAASConfiguration.init() (InMemoryJAASConfiguration:181)
2017-01-17 15:35:46,716 DEBUG - [main:] ~ ==> InMemoryJAASConfiguration.initialize() (InMemoryJAASConfiguration:220)
2017-01-17 15:35:46,889 DEBUG - [main:] ~ Setting hadoop.security.token.service.use_ip to true (SecurityUtil:116)
2017-01-17 15:35:46,898 DEBUG - [main:] ~ Failed to detect a valid hadoop home directory (Shell:477)
java.io.FileNotFoundException: HADOOP_HOME and hadoop.home.dir are unset.
at org.apache.hadoop.util.Shell.checkHadoopHomeInner(Shell.java:425)
at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:396)
at org.apache.hadoop.util.Shell.<clinit>(Shell.java:473)
at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:79)
at org.apache.hadoop.conf.Configuration.getBoolean(Configuration.java:1443)
at org.apache.hadoop.security.SecurityUtil.setConfigurationInternal(SecurityUtil.java:96)
at org.apache.hadoop.security.SecurityUtil.<clinit>(SecurityUtil.java:80)
at org.apache.atlas.security.InMemoryJAASConfiguration.initialize(InMemoryJAASConfiguration.java:312)
at org.apache.atlas.security.InMemoryJAASConfiguration.<init>(InMemoryJAASConfiguration.java:216)
at org.apache.atlas.security.InMemoryJAASConfiguration.init(InMemoryJAASConfiguration.java:184)
at org.apache.atlas.security.InMemoryJAASConfiguration.init(InMemoryJAASConfiguration.java:172)
at org.apache.atlas.ApplicationProperties.get(ApplicationProperties.java:60)
at org.apache.atlas.Atlas.main(Atlas.java:107)
2017-01-17 15:35:47,015 DEBUG - [main:] ~ setsid exited with exit code 0 (Shell:768)
2017-01-17 15:35:47,041 DEBUG - [main:] ~ Adding client: [KafkaClient{-1}]
loginModule: [com.sun.security.auth.module.Krb5LoginModule]
controlFlag: [LoginModuleControlFlag: required]
Options: [principal] => [atlas/nmara-hdp-m4.field.hortonworks.com@FIELD.HORTONWORKS.COM]
Options: [storeKey] => [true]
Options: [keyTab] => [/etc/security/keytabs/atlas.service.keytab]
Options: [useKeyTab] => [true]
Options: [serviceName] => [kafka]
(InMemoryJAASConfiguration:334)
2017-01-17 15:35:47,041 DEBUG - [main:] ~ <== InMemoryJAASConfiguration.initialize() (InMemoryJAASConfiguration:347)
2017-01-17 15:35:47,042 DEBUG - [main:] ~ <== InMemoryJAASConfiguration.init() (InMemoryJAASConfiguration:190)
2017-01-17 15:35:47,042 DEBUG - [main:] ~ <== InMemoryJAASConfiguration.init() (InMemoryJAASConfiguration:177)
2017-01-17 15:35:47,044 INFO - [main:] ~ Not running setup per configuration atlas.server.run.setup.on.start. (Atlas:134)
2017-01-17 15:35:47,044 INFO - [main:] ~
########################################################################################
Atlas Server (STARTUP)
project.name: apache-atlas
project.description: Metadata Management and Data Governance Platform over Hadoop
build.user: jenkins
build.epoch: 1480481030662
project.version: 0.7.0.2.5.3.0-37
build.version: 0.7.0.2.5.3.0-37-rf427fc5f5b82c6582d1520a279f523d1b1c874f6
vc.revision: f427fc5f5b82c6582d1520a279f523d1b1c874f6
vc.source.url: scm:git:git://git.apache.org/incubator-atlas.git/atlas-webapp
######################################################################################## (Atlas:202)
2017-01-17 15:35:47,045 INFO - [main:] ~ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (Atlas:203)
2017-01-17 15:35:47,045 INFO - [main:] ~ Server starting with TLS ? true on port 21443 (Atlas:204)
2017-01-17 15:35:47,045 INFO - [main:] ~ <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< (Atlas:205)
2017-01-17 15:35:47,079 DEBUG - [main:] ~ Logging to org.slf4j.impl.Log4jLoggerAdapter(org.eclipse.jetty.util.log) via org.eclipse.jetty.util.log.Slf4jLog (log:176)
2017-01-17 15:35:47,088 INFO - [main:] ~ Logging initialized @1508ms (log:186)
2017-01-17 15:35:47,107 DEBUG - [main:] ~ org.eclipse.jetty.server.Server@1b68b9a4 added {qtp878274034{STOPPED,8<=0<=200,i=0,q=0},AUTO} (ContainerLifeCycle:324)
2017-01-17 15:35:47,112 INFO - [main:] ~ Attempting to retrieve password from configured credential provider path (SecureEmbeddedServer:118)
2017-01-17 15:35:47,195 INFO - [pool-1-thread-1:] ~ ==> Shutdown of Atlas (Atlas:60)
2017-01-17 15:35:47,195 ERROR - [pool-1-thread-1:] ~ Failed to shutdown (Atlas:64)
java.lang.NullPointerException
at org.apache.atlas.Atlas.shutdown(Atlas.java:73)
at org.apache.atlas.Atlas.access$100(Atlas.java:42)
at org.apache.atlas.Atlas$1.run(Atlas.java:62)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2017-01-17 15:35:47,196 INFO - [pool-1-thread-1:] ~ <== Shutdown of Atlas (Atlas:66)
2017-01-17 15:35:47,197 DEBUG - [Thread-0:] ~ ShutdownHookManger complete shutdown. (ShutdownHookManager:84)
It shutdown just after attempting to get passwords from the jceks file.
... View more
Labels: