question Re: ATLAS logs showing errors in Archives of Support Questions (Read Only)

ATLAS logs showing errors

aliyesami — Thu, 17 Nov 2016 01:53:15 GMT

even though HDP2.5 console shows ATLAS server and clients as started and no errors but I can't reach the atlas URL hadoop1:21000

I am seeing errors in /var/log/atlas/application.log and /var/log/atlas/audit.log as follows:

[root@hadoop1 atlas]# pwd
/var/log/atlas
[root@hadoop1 atlas]# tail -100f application.log
2016-11-16 12:35:06,875 INFO  - [qtp964346941-15 - 7e9b2f05-70b3-49eb-b782-fb602a194250:] ~ Audit: UNKNOWN/127.0.0.1-127.0.0.1 performed request GET http://localhost:21000/api/atlas/admin/status (127.0.0.1) at time 2016-11-16T17:35Z (AUDIT:104)
2016-11-16 12:36:06,832 INFO  - [qtp964346941-17 - d8126dcf-bafe-4ea5-8169-853fb70b89dd:] ~ Audit: UNKNOWN/127.0.0.1 performed request GET http://localhost:21000/api/atlas/admin/status (127.0.0.1) at time 2016-11-16T17:36Z (AuditFilter:91)
2016-11-16 12:36:06,832 INFO  - [qtp964346941-17 - d8126dcf-bafe-4ea5-8169-853fb70b89dd:] ~ Audit: UNKNOWN/127.0.0.1-127.0.0.1 performed request GET http://localhost:21000/api/atlas/admin/status (127.0.0.1) at time 2016-11-16T17:36Z (AUDIT:104)
2016-11-16 12:37:06,911 INFO  - [qtp964346941-16 - e51663cb-3132-4bfd-abc7-28c941c1ae29:] ~ Audit: UNKNOWN/127.0.0.1 performed request GET http://localhost:21000/api/atlas/admin/status (127.0.0.1) at time 2016-11-16T17:37Z (AuditFilter:91)
2016-11-16 12:37:06,911 INFO  - [qtp964346941-16 - e51663cb-3132-4bfd-abc7-28c941c1ae29:] ~ Audit: UNKNOWN/127.0.0.1-127.0.0.1 performed request GET http://localhost:21000/api/atlas/admin/status (127.0.0.1) at time 2016-11-16T17:37Z (AUDIT:104)
tail -100f audit.log
2016-11-16 12:37:06,911  Audit: UNKNOWN/127.0.0.1-127.0.0.1 performed request GET http://localhost:21000/api/atlas/admin/status (127.0.0.1) at time 2016-11-16T17:37Z
2016-11-16 12:38:06,913  Audit: UNKNOWN/127.0.0.1-127.0.0.1 performed request GET http://localhost:21000/api/atlas/admin/status (127.0.0.1) at time 2016-11-16T17:38Z

Re: ATLAS logs showing errors

aliyesami — Thu, 17 Nov 2016 02:06:03 GMT

atlas-install-screenshot.jpg

please see the attached picture showing successful installation of Atlas .

also curl is returning 504 error

[root@hadoop1 atlas]# curl -sL -w "%{http_code} %{url_effective}\\n" "http://hadoop1:21000" -o /dev/null
504 http://hadoop1:21000/
[root@hadoop1 atlas]#

Re: ATLAS logs showing errors

mugdha — Thu, 17 Nov 2016 11:02:02 GMT

@Sami Ahmad

Can you check if the process is listening on port 21000

lsof -i :21000

then ps -ef | grep <pid> (pid from output of above command)

Also attach complete application.log

Re: ATLAS logs showing errors

aliyesami — Thu, 17 Nov 2016 12:00:03 GMT

application.zipyes its listening and the process is up , attaching the log file .

[root@hadoop1 atlas]# lsof -i :21000
COMMAND   PID  USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
java    13343 atlas  251u  IPv4 251676      0t0  TCP *:irtrans (LISTEN)
[root@hadoop1 atlas]#
[root@hadoop1 atlas]# ps -ef | grep 13343
atlas    13343     1  0 13:37 ?        00:00:49 /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.111.x86_64/bin/java -Datlas.log.dir=/var/log/atlas -Datlas.log.file=application.log -Datlas.home=/usr/hdp/2.5.0.0-1245/atlas -Datlas.conf=/etc/atlas/conf -Xms2048m -Xmx2048m -XX:MaxNewSize=600m -XX:MaxPermSize=512m -server -XX:SoftRefLRUPolicyMSPerMB=0 -XX:+CMSClassUnloadingEnabled -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:+PrintTenuringDistribution -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/log/atlas/atlas_server.hprof -Xloggc:-worker.log -verbose:gc -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1m -XX:+PrintGCDetails -XX:+PrintHeapAtGC -XX:+PrintGCTimeStamps -Dlog4j.configuration=atlas-log4j.xml -classpath /etc/atlas/conf:/usr/hdp/current/atlas-server/server/webapp/atlas/WEB-INF/classes:/usr/hdp/current/atlas-server/server/webapp/atlas/WEB-INF/lib/atlas-titan-0.7.0.2.5.0.0-1245.jar:/usr/hdp/current/atlas-server/server/webapp/atlas/WEB-INF/lib/*:/usr/hdp/2.5.0.0-1245/atlas/libext/*:/etc/hbase/conf org.apache.atlas.Atlas -app /usr/hdp/current/atlas-server/server/webapp/atlas
root     23419 22082  0 22:58 pts/0    00:00:00 grep 13343
[root@hadoop1 atlas]#

Re: ATLAS logs showing errors

aliyesami — Thu, 17 Nov 2016 22:02:34 GMT

can someone help please?

Re: ATLAS logs showing errors

apathan — Thu, 17 Nov 2016 22:29:54 GMT

@Sami AhmadFrom the logs, looks like some of configuration files are missing in your installation. Below is the log snapshot for the missing files.

/etc/atlas/conf/users-credentials.properties

2016-11-16 13:37:19,351 ERROR - [main:] ~ Error while reading user.properties file, filepath=/etc/atlas/conf/users-credentials.properties (UserDao:75)
java.io.FileNotFoundException: /etc/atlas/conf/users-credentials.properties (No such file or directory)
	at java.io.FileInputStream.open(Native Method)
	at java.io.FileInputStream.<init>(FileInputStream.java:146)
	at java.io.FileInputStream.<init>(FileInputStream.java:101)
	at org.apache.atlas.web.dao.UserDao.loadFileLoginsDetails(UserDao.java:67)
	at org.apache.atlas.web.dao.UserDao.init(UserDao.java:53)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:346)
	at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:299)
	at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:132)
	at org.springframework.beans.factory.support.AbstractAuto

/etc/atlas/conf/policy-store.txt

2016-11-16 13:37:19,574 ERROR - [main:] ~ SimpleAtlasAuthorizer could not be initialized properly due to :  (SimpleAtlasAuthorizer:114)
java.nio.file.NoSuchFileException: /etc/atlas/conf/policy-store.txt
	at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
	at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
	at java.nio.file.Files.newByteChannel(Files.java:317)
	at java.nio.file.Files.newByteChannel(Files.java:363)
	at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:380)
	at java.nio.file.Files.newInputStream(Files.java:108)
	at java.nio.file.Files.newBufferedReader(Files.java:2677)
	at java.nio.file.Files.readAllLines(Files.java:3033)
	at org.apache.atlas.authorize.simple.FileReaderUtil.readFile(FileReaderUtil.java:41)
	at org.apache.atlas.authorize.simple.SimpleAtlasAuthorizer.init(SimpleAtlasAuthorizer.java:92)
	at org.apache.atlas.web.filters.AtlasAuthorizationFilter.<init>(AtlasAuthorizationFilter.java:65)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

Make sure that, the above files are present and restart Atlas server. This should fix the issue.

Re: ATLAS logs showing errors

aliyesami — Fri, 18 Nov 2016 00:08:13 GMT

application.zip

added both the files , bounced the Atlas server but still seeing same errors. Uploading the new application.log file.

There are no Errors in the application log and the strange thing is now its not reading the users-credentials file

[root@hadoop1 atlas]# grep Error application.log
[root@hadoop1 atlas]# grep policy-store.txt  application.log
2016-11-17 10:58:32,134 INFO  - [main:] ~ reading the file/etc/atlas/conf/policy-store.txt (FileReaderUtil:40)
[root@hadoop1 atlas]# grep users-credentials.properteies  application.log
[root@hadoop1 atlas]#

[root@hadoop1 conf]# pwd
/etc/atlas/conf
[root@hadoop1 conf]# ls -ltr
total 32
-rw-r--r-- 1 root  root   1336 Nov 16 11:18 application.properties
-rwxr-xr-x 1 root  root   1265 Nov 16 11:18 client.properties
drwxr-xr-x 3 atlas hadoop 4096 Nov 16 11:18 solr
-rw-r--r-- 1 atlas hadoop 3259 Nov 16 11:54 atlas-log4j.xml
-rwxr-xr-x 1 atlas hadoop 1611 Nov 16 12:24 atlas-env.sh
-rw-r--r-- 1 atlas hadoop  325 Nov 17 10:33 policy-store.txt
-rw-r--r-- 1 atlas hadoop   81 Nov 17 10:34 users-credentials.properties
-rw-r--r-- 1 atlas hadoop 3247 Nov 17 10:58 atlas-application.properties
[root@hadoop1 conf]#
[root@hadoop1 conf]# more policy-store.txt
adminPolicy;;admin:rwud;;ROLE_ADMIN:rwud;;type:*,entity:*,operation:*,taxonomy:*,term:*
userReadPolicy;;readUser1:r,readUser2:r;;DATA_SCIENTIST:r;;type:*,entity:*,operation:*,taxonomy:*,term:*
userWritePolicy;;writeUser1:rwu,writeUser2:rwu;;BUSINESS_GROUP:rwu,DATA_STEWARD:rwud;;type:*,entity:*,operation:*,taxonomy:*,term:*
[root@hadoop1 conf]#
[root@hadoop1 conf]# more users-credentials.properties
admin=ADMIN::8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918  -
[root@hadoop1 conf]#

Re: ATLAS logs showing errors

aliyesami — Fri, 18 Nov 2016 06:11:48 GMT

I really need to get Atlas up and running , can anyone please help ?

Re: ATLAS logs showing errors

apathan — Sat, 19 Nov 2016 09:30:16 GMT

@Sami Ahmad

I dont see any errors in the log. Atlas startup seems to be proper. Can you try to clear the browser cache and try again? Also, which browser you are using? IE has some incompatibility issues.

Re: ATLAS logs showing errors

aliyesami — Sat, 19 Nov 2016 12:35:52 GMT

I can't telnet to port 21000 from my laptop does it mean this port could be blocked by firewall ?

Re: ATLAS logs showing errors

aliyesami — Sat, 19 Nov 2016 12:37:04 GMT

I tried all browsers and I cant get all from anyone

Re: ATLAS logs showing errors

apathan — Sat, 19 Nov 2016 13:04:53 GMT

Can you check on the atlas server machine if the Atlas process is running and 21000 port is used? If it is running fine, check if it is acessible from your laptop.

Re: ATLAS logs showing errors

aliyesami — Sat, 19 Nov 2016 14:44:47 GMT

Ah it was the firewall , I installed KDE on the server it self and I can connect to the Atlas server from the local browswer, so port 21000 is being blocked from my laptop to the node running apache server.

thanks for all your help

Re: ATLAS logs showing errors

apathan — Sat, 19 Nov 2016 14:47:29 GMT

Glad that my suggestion were helpful 🙂

Re: ATLAS logs showing errors

apathan — Sat, 19 Nov 2016 15:53:03 GMT

@Sami Ahmad Can you accept my reply as an answer instead of this one?

Re: ATLAS logs showing errors

aliyesami — Sat, 19 Nov 2016 21:50:37 GMT

I would love to but its not showing the "accept as answer" option on your message .

if the moderators are listening .. can you please accept Ayub's comment as answer ?