ATLAS logs showing errors

[#1]

even though HDP2.5 console shows ATLAS server and clients as started and no errors but I can't reach the atlas URL hadoop1:21000

I am seeing errors in /var/log/atlas/application.log and /var/log/atlas/audit.log as follows:

[root@hadoop1 atlas]# pwd
/var/log/atlas
[root@hadoop1 atlas]# tail -100f application.log
2016-11-16 12:35:06,875 INFO  - [qtp964346941-15 - 7e9b2f05-70b3-49eb-b782-fb602a194250:] ~ Audit: UNKNOWN/127.0.0.1-127.0.0.1 performed request GET http://localhost:21000/api/atlas/admin/status (127.0.0.1) at time 2016-11-16T17:35Z (AUDIT:104)
2016-11-16 12:36:06,832 INFO  - [qtp964346941-17 - d8126dcf-bafe-4ea5-8169-853fb70b89dd:] ~ Audit: UNKNOWN/127.0.0.1 performed request GET http://localhost:21000/api/atlas/admin/status (127.0.0.1) at time 2016-11-16T17:36Z (AuditFilter:91)
2016-11-16 12:36:06,832 INFO  - [qtp964346941-17 - d8126dcf-bafe-4ea5-8169-853fb70b89dd:] ~ Audit: UNKNOWN/127.0.0.1-127.0.0.1 performed request GET http://localhost:21000/api/atlas/admin/status (127.0.0.1) at time 2016-11-16T17:36Z (AUDIT:104)
2016-11-16 12:37:06,911 INFO  - [qtp964346941-16 - e51663cb-3132-4bfd-abc7-28c941c1ae29:] ~ Audit: UNKNOWN/127.0.0.1 performed request GET http://localhost:21000/api/atlas/admin/status (127.0.0.1) at time 2016-11-16T17:37Z (AuditFilter:91)
2016-11-16 12:37:06,911 INFO  - [qtp964346941-16 - e51663cb-3132-4bfd-abc7-28c941c1ae29:] ~ Audit: UNKNOWN/127.0.0.1-127.0.0.1 performed request GET http://localhost:21000/api/atlas/admin/status (127.0.0.1) at time 2016-11-16T17:37Z (AUDIT:104)
tail -100f audit.log
2016-11-16 12:37:06,911  Audit: UNKNOWN/127.0.0.1-127.0.0.1 performed request GET http://localhost:21000/api/atlas/admin/status (127.0.0.1) at time 2016-11-16T17:37Z
2016-11-16 12:38:06,913  Audit: UNKNOWN/127.0.0.1-127.0.0.1 performed request GET http://localhost:21000/api/atlas/admin/status (127.0.0.1) at time 2016-11-16T17:38Z

[#2]

atlas-install-screenshot.jpg

please see the attached picture showing successful installation of Atlas .

also curl is returning 504 error

[root@hadoop1 atlas]# curl -sL -w "%{http_code} %{url_effective}\\n" "http://hadoop1:21000" -o /dev/null
504 http://hadoop1:21000/
[root@hadoop1 atlas]#

[#3]

@Sami Ahmad

Can you check if the process is listening on port 21000

lsof -i :21000

then ps -ef | grep <pid> (pid from output of above command)

Also attach complete application.log

[#4]

application.zipyes its listening and the process is up , attaching the log file .

[root@hadoop1 atlas]# lsof -i :21000
COMMAND   PID  USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
java    13343 atlas  251u  IPv4 251676      0t0  TCP *:irtrans (LISTEN)
[root@hadoop1 atlas]#
[root@hadoop1 atlas]# ps -ef | grep 13343
atlas    13343     1  0 13:37 ?        00:00:49 /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.111.x86_64/bin/java -Datlas.log.dir=/var/log/atlas -Datlas.log.file=application.log -Datlas.home=/usr/hdp/2.5.0.0-1245/atlas -Datlas.conf=/etc/atlas/conf -Xms2048m -Xmx2048m -XX:MaxNewSize=600m -XX:MaxPermSize=512m -server -XX:SoftRefLRUPolicyMSPerMB=0 -XX:+CMSClassUnloadingEnabled -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:+PrintTenuringDistribution -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/log/atlas/atlas_server.hprof -Xloggc:-worker.log -verbose:gc -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1m -XX:+PrintGCDetails -XX:+PrintHeapAtGC -XX:+PrintGCTimeStamps -Dlog4j.configuration=atlas-log4j.xml -classpath /etc/atlas/conf:/usr/hdp/current/atlas-server/server/webapp/atlas/WEB-INF/classes:/usr/hdp/current/atlas-server/server/webapp/atlas/WEB-INF/lib/atlas-titan-0.7.0.2.5.0.0-1245.jar:/usr/hdp/current/atlas-server/server/webapp/atlas/WEB-INF/lib/*:/usr/hdp/2.5.0.0-1245/atlas/libext/*:/etc/hbase/conf org.apache.atlas.Atlas -app /usr/hdp/current/atlas-server/server/webapp/atlas
root     23419 22082  0 22:58 pts/0    00:00:00 grep 13343
[root@hadoop1 atlas]#

[#5]

can someone help please?

[#6 · Accepted solution]

@Sami AhmadFrom the logs, looks like some of configuration files are missing in your installation. Below is the log snapshot for the missing files.

• /etc/atlas/conf/users-credentials.properties

2016-11-16 13:37:19,351 ERROR - [main:] ~ Error while reading user.properties file, filepath=/etc/atlas/conf/users-credentials.properties (UserDao:75)
java.io.FileNotFoundException: /etc/atlas/conf/users-credentials.properties (No such file or directory)
	at java.io.FileInputStream.open(Native Method)
	at java.io.FileInputStream.<init>(FileInputStream.java:146)
	at java.io.FileInputStream.<init>(FileInputStream.java:101)
	at org.apache.atlas.web.dao.UserDao.loadFileLoginsDetails(UserDao.java:67)
	at org.apache.atlas.web.dao.UserDao.init(UserDao.java:53)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:346)
	at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:299)
	at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:132)
	at org.springframework.beans.factory.support.AbstractAuto

• /etc/atlas/conf/policy-store.txt

2016-11-16 13:37:19,574 ERROR - [main:] ~ SimpleAtlasAuthorizer could not be initialized properly due to :  (SimpleAtlasAuthorizer:114)
java.nio.file.NoSuchFileException: /etc/atlas/conf/policy-store.txt
	at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
	at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
	at java.nio.file.Files.newByteChannel(Files.java:317)
	at java.nio.file.Files.newByteChannel(Files.java:363)
	at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:380)
	at java.nio.file.Files.newInputStream(Files.java:108)
	at java.nio.file.Files.newBufferedReader(Files.java:2677)
	at java.nio.file.Files.readAllLines(Files.java:3033)
	at org.apache.atlas.authorize.simple.FileReaderUtil.readFile(FileReaderUtil.java:41)
	at org.apache.atlas.authorize.simple.SimpleAtlasAuthorizer.init(SimpleAtlasAuthorizer.java:92)
	at org.apache.atlas.web.filters.AtlasAuthorizationFilter.<init>(AtlasAuthorizationFilter.java:65)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

Make sure that, the above files are present and restart Atlas server. This should fix the issue.

[#7]

application.zip

added both the files , bounced the Atlas server but still seeing same errors. Uploading the new application.log file.

There are no Errors in the application log and the strange thing is now its not reading the users-credentials file

[root@hadoop1 atlas]# grep Error application.log
[root@hadoop1 atlas]# grep policy-store.txt  application.log
2016-11-17 10:58:32,134 INFO  - [main:] ~ reading the file/etc/atlas/conf/policy-store.txt (FileReaderUtil:40)
[root@hadoop1 atlas]# grep users-credentials.properteies  application.log
[root@hadoop1 atlas]#

[root@hadoop1 conf]# pwd
/etc/atlas/conf
[root@hadoop1 conf]# ls -ltr
total 32
-rw-r--r-- 1 root  root   1336 Nov 16 11:18 application.properties
-rwxr-xr-x 1 root  root   1265 Nov 16 11:18 client.properties
drwxr-xr-x 3 atlas hadoop 4096 Nov 16 11:18 solr
-rw-r--r-- 1 atlas hadoop 3259 Nov 16 11:54 atlas-log4j.xml
-rwxr-xr-x 1 atlas hadoop 1611 Nov 16 12:24 atlas-env.sh
-rw-r--r-- 1 atlas hadoop  325 Nov 17 10:33 policy-store.txt
-rw-r--r-- 1 atlas hadoop   81 Nov 17 10:34 users-credentials.properties
-rw-r--r-- 1 atlas hadoop 3247 Nov 17 10:58 atlas-application.properties
[root@hadoop1 conf]#
[root@hadoop1 conf]# more policy-store.txt
adminPolicy;;admin:rwud;;ROLE_ADMIN:rwud;;type:*,entity:*,operation:*,taxonomy:*,term:*
userReadPolicy;;readUser1:r,readUser2:r;;DATA_SCIENTIST:r;;type:*,entity:*,operation:*,taxonomy:*,term:*
userWritePolicy;;writeUser1:rwu,writeUser2:rwu;;BUSINESS_GROUP:rwu,DATA_STEWARD:rwud;;type:*,entity:*,operation:*,taxonomy:*,term:*
[root@hadoop1 conf]#
[root@hadoop1 conf]# more users-credentials.properties
admin=ADMIN::8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918  -
[root@hadoop1 conf]#

[#8]

I really need to get Atlas up and running , can anyone please help ?

[#9]

@Sami Ahmad

I dont see any errors in the log. Atlas startup seems to be proper. Can you try to clear the browser cache and try again? Also, which browser you are using? IE has some incompatibility issues.

[#10]

I can't telnet to port 21000 from my laptop does it mean this port could be blocked by firewall ?