question Re: Atlas installation does not work in Support Questions

Atlas installation does not work

f_rey — Mon, 09 Oct 2017 21:13:29 GMT

Hello,

Under Ambari 2.5.1 Atlas has been installed just after Hbase and Kafka (required modules)

Atlas Metadata Server does not start and I don't see any logs or messages (/var/log/atlas is emty)

Does anyone has an idea on how to resolve this kind of issue ?

Thanks in advance

Regards

Re: Atlas installation does not work

manish1 — Mon, 09 Oct 2017 23:21:18 GMT

@Lou Richard Can you share the Ambari log after starting Atlas Metadata server? It must have some information about the failure.

Re: Atlas installation does not work

amestry — Mon, 09 Oct 2017 23:42:05 GMT

If you could let us know these details, it will help in getting towards a solution:

What are the deployment environment details?
How was the build done? Information about Maven parameters will help.
Is it SSL or non-SSL environment? Is Kerboros used?

When atlas_start.py is executed, do you see new process getting created? If yes, what are the classpath details?

Re: Atlas installation does not work

f_rey — Tue, 10 Oct 2017 13:12:27 GMT

Hello guys,

Thanks for your reply.

I am beginner so there arre some terms I don't understand well, sorry...

Deployment env : Linux cent OS 7.3 64 bits with ambari 2.5.1 with 3 nodes and each 8GB of RAM
Installation by HDP package with Yum. I don't know Maven
Non SSL with Kerberos

I finally found this error logs in the ops popup window

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py", line 181, in <module>
    MetadataServer().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 329, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py", line 117, in start
    not_if=no_op_test
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 262, in action_run
    tries=self.resource.tries, try_sleep=self.resource.try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
    result = function(command, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
    tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
    result = _call(command, **kwargs_copy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
    raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of 'source /usr/hdp/current/atlas-server/conf/atlas-env.sh ; /usr/hdp/current/atlas-server/bin/atlas_start.py' returned 255. Exception: [Errno 2] No such file or directory 
Traceback (most recent call last):
  File "/usr/hdp/current/atlas-server/bin/atlas_start.py", line 149, in <module>
    returncode = main()
  File "/usr/hdp/current/atlas-server/bin/atlas_start.py", line 73, in main
    mc.expandWebApp(atlas_home)
  File "/usr/hdp/2.6.1.0-129/atlas/bin/atlas_config.py", line 143, in expandWebApp
    jar(atlasWarPath)
  File "/usr/hdp/2.6.1.0-129/atlas/bin/atlas_config.py", line 196, in jar
    process = runProcess(commandline)
  File "/usr/hdp/2.6.1.0-129/atlas/bin/atlas_config.py", line 232, in runProcess
    p = subprocess.Popen(commandline, stdout=stdoutFile, stderr=stderrFile, shell=shell)
  File "/usr/lib64/python2.7/subprocess.py", line 711, in __init__
    errread, errwrite)
  File "/usr/lib64/python2.7/subprocess.py", line 1327, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

Thanks

Re: Atlas installation does not work

jsensharma — Tue, 10 Oct 2017 13:25:38 GMT

@Lou Richard

We see the cause of failure as following:

File "/usr/lib64/python2.7/subprocess.py", line 711, in __init__    errread, errwrite)  
File "/usr/lib64/python2.7/subprocess.py", line 1327, in _execute_child    
raise child_exceptionOSError: [Errno 2] No such file or directory

So can you please check the following directory to see if the "atlas.war" is extracted there or not? And the directory permission is set to "atlas:hadoop"

# ls /usr/hdp/2.6.1.0-129/atlas/server/webapp/
atlas  atlas.war

### Permission atlas:hadoop
# ls -ld  /usr/hdp/2.6.1.0-129/atlas/server/webapp
drwxr-xr-x 1 atlas hadoop 4096 xxx 19 18:51 /usr/hdp/2.6.1.0-129/atlas/server/webapp

If it is not extracted there properly then please try to extract the war and then see if it works.

Re: Atlas installation does not work

f_rey — Tue, 10 Oct 2017 13:50:55 GMT

This is the check :

[root@hadoop001 ~]# ls /usr/hdp/2.6.1.0-129/atlas/server/webapp/
atlas  atlas.war
[root@hadoop001 ~]# ls -ld  /usr/hdp/2.6.1.0-129/atlas/server/webapp
drwxr-xr-x 3 atlas hadoop 4096 Oct  9 15:41 /usr/hdp/2.6.1.0-129/atlas/server/webapp

Re: Atlas installation does not work

jsensharma — Tue, 10 Oct 2017 13:58:43 GMT

@Lou Richard

Do you have the "JAVA_HOME" property set properly?

After reviewing the script "atlas_config.py" the only thing that i am suspecting at this point ... maybe the JAVA_HOME is not properly set. As the execution "commandline" variable in the script looks for it.

I will also suggest if you can try manually extracting the file to see if it works:

# cd /usr/hdp/2.6.1.0-129/atlas/server/webapp/
# mv atlas  ../
# $JAVA_HOME/bin/jar -xf atlas.war

Then try restarting the Atlas.

Re: Atlas installation does not work

f_rey — Tue, 10 Oct 2017 14:24:56 GMT

I suppose JAVA_HOME is set properly :

[root@hadoop001 webapp]# echo $JAVA_HOME
/usr/lib/jvm/jre-1.8.0-openjdk

I did what you suggested and :

[root@hadoop001 webapp]# $JAVA_HOME/bin/jar -xf atlas.war
-bash: /usr/lib/jvm/jre-1.8.0-openjdk/bin/jar: No such file or directory

There is no bin/jar on the server (I searched with the locate command)

Re: Atlas installation does not work

jsensharma — Tue, 10 Oct 2017 14:41:09 GMT

@Lou Richard

You are using JRE (Not JDK): "/usr/lib/jvm/jre-1.8.0-openjdk"

I am suspecting that it might be because of missing "jar" utility which is used to extract the war.

https://github.com/apache/incubator-atlas/blob/master/distro/src/bin/atlas_config.py#L193-L196

Can you please install a JDK (not JRE) on your host and then set the JAVA_HOME properly.

Re: Atlas installation does not work

f_rey — Tue, 10 Oct 2017 14:58:47 GMT

Sorry to ask that but how to install JDK ?

I don't understand because I have the following packages :

[root@hadoop001]# rpm -qa|grep jre
[root@hadoop001]# rpm -qa|grep jdk
java-1.8.0-openjdk-headless-1.8.0.141-1.b16.el7_3.x86_64
java-1.8.0-openjdk-1.8.0.141-1.b16.el7_3.x86_64
copy-jdk-configs-1.2-1.el7.noarch

Re: Atlas installation does not work

jsensharma — Tue, 10 Oct 2017 15:24:25 GMT

@Lou Richard

JDK requirement: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_support-matrices/content/ch_matrices-hdp.html#hdp_jdk

One option is to set the java_home for the cluster using setup command as following: , Using this option when you plan to use a JDK other than the default Oracle JDK 1.8. If you are using an alternate JDK, you must manually install the same JDK on all hosts and specify the Java Home path during Ambari Server setup.

# ambari-server setup --java-home=/usr/jdk64/jdk1.8.0_121

https://docs.hortonworks.com/HDPDocuments/Ambari-2.5.1.0/bk_ambari-installation/content/setup_options.html

Re: Atlas installation does not work

jsensharma — Tue, 10 Oct 2017 15:28:50 GMT

@Lou Richard

Looks like you already have "java-1.8.0-openjdk-1.8.0.141-1.b16.el7_3.x86_64" present in your atlas host.

So all you need to do is that locate that JDK and then set the JAVA_HOME some thing like :

export JAVA_HOME=/PATH/TO/java-1.8.0-openjdk-1.8.0.141

Example: In the file "/etc/profile" we can set the JAVA_HOME for all the users globally. You will need to find the exact location of your JDK 1.8.0.141 installation on your file system and then set it accordingly.

export JAVA_HOME=/usr/jdk64/java-1.8.0-openjdk-1.8.0.141
export PATH=$JAVA_HOME/bin:$PATH

Re: Atlas installation does not work

f_rey — Tue, 10 Oct 2017 15:38:38 GMT

Well, JAVA_HOME is already set in /etc/profile

The actual value is /usr/lib/jvm/jre-1.8.0-openjdk-1.8.0.141-1.b16.el7_3.x86_64

The value given above

/usr/lib/jvm/jre-1.8.0-openjdk

is a link to /usr/lib/jvm/jre-1.8.0-openjdk-1.8.0.141-1.b16.el7_3.x86_64

The 3 nodes are exactly the same they have the same version of all components

Re: Atlas installation does not work

jsensharma — Tue, 10 Oct 2017 15:47:01 GMT

@Lou Richard

Please try this:

# cd /usr/hdp/2.6.1.0-129/atlas/server/webapp/
# mv atlas  ../
# /usr/lib/jvm/jre-1.8.0-openjdk/bin/jar -xf atlas.war

And then try restarting the atlas.

Re: Atlas installation does not work

f_rey — Tue, 10 Oct 2017 15:54:42 GMT

I already tried it :

 /usr/lib/jvm/jre-1.8.0-openjdk/bin/jar: No such file or directory

So we tried using unzip but it is the same

Re: Atlas installation does not work

jsensharma — Tue, 10 Oct 2017 17:42:53 GMT

@Lou Richard

You are using JRE (/usr/lib/jvm/jre-1.8.0-openjdk/bin/) which does not have "jar" utility inside the it's "bin" directory.

You need JDK (not JRE).

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.0/bk_command-line-installation/content/meet-min-system-requirements.html

Re: Atlas installation does not work

f_rey — Tue, 10 Oct 2017 18:56:42 GMT

Ok thanks, I'm going to try "Manually Installing Oracle JDK 1.7 or 1.8"..

Re: Atlas installation does not work

f_rey — Tue, 10 Oct 2017 19:44:58 GMT

Ok so JDK installed and jar on the machine.

But still the same error .... 😞

Re: Atlas installation does not work

f_rey — Tue, 10 Oct 2017 21:48:24 GMT

Sorry we forgot to modify the environment variable and now it works !

Thanks a lot for your help 🙂

Re: Atlas installation does not work

jsensharma — Tue, 10 Oct 2017 22:01:48 GMT

@Lou Richard

Good to know that the issue is resolved. It will be great if you can mark this HCC thread as Answered by clicking on the "Accept" Button. That way other HCC users can quickly find the solution when they encounter the same issue.

As it was a long thread hence i am writing a brief summary for the HCC users who might encounter this issue and can quickly find the answer.

Issue: Atlas Installation Was failing with the following error:

File "/usr/hdp/2.6.1.0-129/atlas/bin/atlas_config.py", line 232, in runProcess  p = subprocess.Popen(commandline, stdout=stdoutFile, stderr=stderrFile, shell=shell)  
File "/usr/lib64/python2.7/subprocess.py", line 711, in __init__  errread, errwrite)  
File "/usr/lib64/python2.7/subprocess.py", line 1327, in _execute_child   
raise child_exceptionOSError: [Errno 2] No such file or director

.
Solution: Making sure that the JAVA_HOME is set correctly on the host and it is pointing to a valid JDK (not JRE) and setting the JAVA_HOME properly inside the global environment variable.

# cd /usr/hdp/2.6.1.0-129/atlas/server/webapp/
# mv atlas  ../
# /usr/lib/jvm/jre-1.8.0-openjdk/bin/jar -xf atlas.war

.
Cause: "jar" utility comes with JDK (inside $JAVA_HOME/bin) which is being used by the "atlas_config.py" script to extract the atlas.war.