How to install Apache Atlas on cloudera on VM?

[#1]

Hello,

I try to install Apache Atlas on cloudera 5.12.X version on my VM. It giving lot of error while i'm build project (clean and install Maven). So, is it possibale to install Atlas on cloudera without using cloudera manager? If, can you please provide any relavent documentation for installation.

Thank you in advance!!

[#2]

Atlas is dependent on HBase, Solr, Kafka, Zookeeper. If these components are available then Atlas can work on CDH.

http://atlas.apache.org/InstallationSteps.html

[#3]

@Nixon Rodrigues Thank you for response. In my CDH i have every component installed but still i'm getting lot of errors. Specially when i'm installing i'm getting error on Apache Atlas UI package problem.

[#4]

@mohsu

How are you installing Atlas in CDH? , please provide this steps. Also provide the error logs which you are facing ?

[#5]

@Nixon Rodrigues i just follow the steps same as documented on http://atlas.apache.org/InstallationSteps.html. I'm getting below errors:

[INFO] Tests run: 64, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Apache Atlas Repository ............................ FAILURE [17:47 min] [INFO] Apache Atlas Authorization ......................... SKIPPED [INFO] Apache Atlas UI .................................... SKIPPED [INFO] Apache Atlas Web Application ....................... SKIPPED [INFO] Apache Atlas Documentation ......................... SKIPPED [INFO] Apache Atlas FileSystem Model ...................... SKIPPED [INFO] Apache Atlas Plugin Classloader .................... SKIPPED [INFO] Apache Atlas Hive Bridge Shim ...................... SKIPPED [INFO] Apache Atlas Hive Bridge ........................... SKIPPED [INFO] Apache Atlas Falcon Bridge Shim .................... SKIPPED [INFO] Apache Atlas Falcon Bridge ......................... SKIPPED [INFO] Apache Atlas Sqoop Bridge Shim ..................... SKIPPED [INFO] Apache Atlas Sqoop Bridge .......................... SKIPPED [INFO] Apache Atlas Storm Bridge Shim ..................... SKIPPED [INFO] Apache Atlas Storm Bridge .......................... SKIPPED [INFO] Apache Atlas Hbase Bridge Shim ..................... SKIPPED [INFO] Apache Atlas Hbase Bridge .......................... SKIPPED [INFO] Apache Atlas Distribution .......................... SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 17:51 min [INFO] Finished at: 2018-01-07T23:55:26-08:00 [INFO] Final Memory: 57M/435M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.20.1:test (default-test) on project atlas-repository: There are test failures. [ERROR] [ERROR] Please refer to /home/cloudera/shares/vm_shared_folder/ATLAS/atlas/repository/target/surefire-reports for the individual test results. [ERROR] Please refer to dump files (if any exist) [date]-jvmRun[N].dump, [date].dumpstream and [date]-jvmRun[N].dumpstream. [ERROR] ExecutionException org.apache.maven.surefire.report.SafeThrowable.<init>(Ljava/lang/String;)V [ERROR] ExecutionException org.apache.maven.surefire.report.SafeThrowable.<init>(Ljava/lang/String;)V [ERROR] ExecutionException org.apache.maven.surefire.report.SafeThrowable.<init>(Ljava/lang/String;)V [ERROR] ExecutionException org.apache.maven.surefire.report.SafeThrowable.<init>(Ljava/lang/String;)V [ERROR] ExecutionException org.apache.maven.surefire.report.SafeThrowable.<init>(Ljava/lang/String;)V [ERROR] ExecutionException org.apache.maven.surefire.report.SafeThrowable.<init>(Ljava/lang/String;)V [ERROR] ExecutionException org.apache.maven.surefire.report.SafeThrowable.<init>(Ljava/lang/String;)V

org.apache.maven.surefire.report.SafeThrowable.<init>(Ljava/lang/String;)V [ERROR] ExecutionException org.apache.maven.surefire.report.SafeThrowable.<init>(Ljava/lang/String;)V [ERROR] ExecutionException org.apache.maven.surefire.report.SafeThrowable.<init>(Ljava/lang/String;)V [ERROR] ExecutionException org.apache.maven.surefire.report.SafeThrowable.<init>(Ljava/lang/String;)V [ERROR] ExecutionException org.apache.maven.surefire.report.SafeThrowable.<init>(Ljava/lang/String;)V [ERROR] ExecutionException org.apache.maven.surefire.report.SafeThrowable.<init>(Ljava/lang/String;)V [ERROR] ExecutionException org.apache.maven.surefire.report.SafeThrowable.<init>(Ljava/lang/String;)V [ERROR] ExecutionException org.apache.maven.surefire.report.SafeThrowable.<init>(Ljava/lang/String;)V [ERROR] ExecutionException org.apache.maven.surefire.report.SafeThrowable.<init>(Ljava/lang/String;)V [ERROR] ExecutionException org.apache.maven.surefire.report.SafeThrowable.<init>(Ljava/lang/String;)V [ERROR] ExecutionException org.apache.maven.surefire.report.SafeThrowable.<init>(Ljava/lang/String;)V [ERROR] ExecutionException org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:832) [ERROR] at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134) [ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208) [ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:154) [ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:146) [ERROR] at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117) [ERROR] at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81) [ERROR] at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51) [ERROR] at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128) [ERROR] at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:309) [ERROR] at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:194) [ERROR] at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:107) [ERROR] at org.apache.maven.cli.MavenCli.execute(MavenCli.java:955) [ERROR] at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:290) [ERROR] at org.apache.maven.cli.MavenCli.main(MavenCli.java:194) [ERROR] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [ERROR] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [ERROR] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [ERROR] at java.lang.reflect.Method.invoke(Method.java:498)

[#6]

@moshu

please try with -DskipTests option while executing maven command, the testcase failing on your env

eg

mvn clean package -Pdist,embedded-hbase-solr -DskipTests

[#7]

@Nixon Rodrigues

Here are some more errors. It shows some thing regarding mojoFRailuer exception. I try to resolve but still it not resolved.

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.7.0:compile (default-compile) on project atlas-intg: Compilation failure: Compilation failure: [ERROR] /home/cloudera/shares/vm_shared_folder/ATLAS/atlas/intg/src/main/java/org/apache/atlas/model/typedef/AtlasEnumDef.java:[158,33] cannot find symbol [ERROR] symbol: method equalsIgnoreCase(java.lang.String,java.lang.String) [ERROR] location: class org.apache.hadoop.util.StringUtils [ERROR] /home/cloudera/shares/vm_shared_folder/ATLAS/atlas/intg/src/main/java/org/apache/atlas/model/typedef/AtlasEnumDef.java:[177,33] cannot find symbol [ERROR] symbol: method equalsIgnoreCase(java.lang.String,java.lang.String) [ERROR] location: class org.apache.hadoop.util.StringUtils [ERROR] /home/cloudera/shares/vm_shared_folder/ATLAS/atlas/intg/src/main/java/org/apache/atlas/model/typedef/AtlasEnumDef.java:[199,32] cannot find symbol [ERROR] symbol: method equalsIgnoreCase(java.lang.String,java.lang.String) [ERROR] location: class org.apache.hadoop.util.StringUtils [ERROR] /home/cloudera/shares/vm_shared_folder/ATLAS/atlas/intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java:[155,33] cannot find symbol [ERROR] symbol: method equalsIgnoreCase(java.lang.String,java.lang.String) [ERROR] location: class org.apache.hadoop.util.StringUtils [ERROR] /home/cloudera/shares/vm_shared_folder/ATLAS/atlas/intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java:[173,33] cannot find symbol [ERROR] symbol: method equalsIgnoreCase(java.lang.String,java.lang.String) [ERROR] location: class org.apache.hadoop.util.StringUtils [ERROR] /home/cloudera/shares/vm_shared_folder/ATLAS/atlas/intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java:[195,32] cannot find symbol [ERROR] symbol: method equalsIgnoreCase(java.lang.String,java.lang.String) [ERROR] location: class org.apache.hadoop.util.StringUtils [ERROR] /home/cloudera/shares/vm_shared_folder/ATLAS/atlas/intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java:[553,31] cannot find symbol [ERROR] symbol: method equalsIgnoreCase(java.lang.String,java.lang.String) [ERROR] location: class org.apache.hadoop.util.StringUtils

[#8]

@mohsu, can you confirm if mvn is configured properly mvn version 3.5.2 is required.

secondly, what is hbase version you are using? if it is less than 2.0.0 then you need to use Atlas 0.8.1 release and not the master checkout.

[#9]

@Nixon Rodrigues Yes Mvn configured latest version 3.5.2.

Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T00:58:13-07:00) Maven home: /usr/local/apache-maven-3.5.2 Java version: 1.8.0_151, vendor: Oracle Corporation Java home: /usr/java/jdk1.8.0_151/jre Default locale: en_US, platform encoding: UTF-8 OS name: "linux", version: "2.6.32-573.el6.x86_64", arch: "amd64", family: "unix"

Hbase version is 1.2.0..

Java HotSpot(TM) 64-Bit Server VM warning: Using incremental CMS is deprecated and will likely be removed in a future release HBase 1.2.0-cdh5.12.0 Source code repository file:///data/jenkins/workspace/generic-package-rhel64-6-0/topdir/BUILD/hbase-1.2.0-cdh5.12.0 revision=Unknown Compiled by jenkins on Thu Jun 29 04:42:07 PDT 2017 From source with checksum 6834049453a9459ccaf4cadbf9a54b2c

So how and where i can download the 0.8.1 release and not the master checkout.?

[#10]

And new error while installing 0.8 version.

[ERROR] /home/cloudera/shares/vm_shared_folder/atlas_0.8/atlas/typesystem/src/main/scala/org/apache/atlas/typesystem/json/InstanceSerialization.scala:262: File name too long This can happen on some encrypted or legacy file systems. Please see SI-3623 for more details. [ERROR] ts <- traits [ERROR] ^ [ERROR] /home/cloudera/shares/vm_shared_folder/atlas_0.8/atlas/typesystem/src/main/scala/org/apache/atlas/typesystem/json/InstanceSerialization.scala:263: File name too long This can happen on some encrypted or legacy file systems. Please see SI-3623 for more details. [ERROR] s_attr <- Some(convertSystemAttributes) [ERROR] ^ [ERROR] two errors found [DEBUG] Compilation failed (CompilerInterface) [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Apache Atlas Typesystem ............................ FAILURE [ 54.231 s]

[#11]

@moshu,

I think there is some security constraint for building code on Atlas. It would be good if you build it locally and copy the Atlas distribution on VM.

https://stackoverflow.com/questions/28565837/filename-too-long-sbt