@Pradeep kumar @Benjamin Leonhardi Guys I have installed Ambari cluster and Atlas service on it successfully. Now I am trying to install Hive. I am unable to start Hive service as metadata service is not running i have attached snapshots and log here with plz guide me. Metastore on ip-172-31-69-106.ec2.internal failed (Traceback (most recent call last): File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_metastore.py", line 200, in execute timeout_kill_strategy=TerminateStrategy.KILL_PROCESS_TREE, File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__ self.env.run() File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run self.run_action(resource, action) File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action provider_action() File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 262, in action_run tries=self.resource.tries, try_sleep=self.resource.try_sleep) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner result = function(command, **kwargs) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper result = _call(command, **kwargs_copy) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call raise ExecutionFailed(err_msg, code, out, err) ExecutionFailed: Execution of 'export HIVE_CONF_DIR='/usr/hdp/current/hive-metastore/conf' ; hive --hiveconf hive.metastore.uris=thrift://ip-172-31-69-106.ec2.internal:9083 --hiveconf hive.metastore.client.connect.retry.delay=1 --hiveconf hive.metastore.failure.retries=1 --hiveconf hive.metastore.connect.retries=1 --hiveconf hive.metastore.client.socket.timeout=14 --hiveconf hive.execution.engine=mr -e 'show databases;'' returned 1. log4j:WARN No such property [maxFileSize] in org.apache.log4j.DailyRollingFileAppender. Logging initialized using configuration in file:/etc/hive/2.6.0.3-8/0/hive-log4j.properties Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:547) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:233) at org.apache.hadoop.util.RunJar.main(RunJar.java:148) Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1565) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:92) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:138) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:110) at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3303) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3322) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:528) ... 8 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1563) ... 14 more Caused by: MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused (Connection refused) at org.apache.thrift.transport.TSocket.open(TSocket.java:226) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:486) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:281) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1563) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:92) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:138) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:110) at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3303) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3322) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:528) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:233) at org.apache.hadoop.util.RunJar.main(RunJar.java:148) Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at org.apache.thrift.transport.TSocket.open(TSocket.java:221) ... 22 more ) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:532) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:281) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74) ... 19 more ) Thanks, ... View more

@Dan Rice Yes i have gone through that and i was able to install using Hortonwork's guide... but not with the Apache's guide. ... View more

@Nixon Rodrigues You mean currently I can only specify * I can't authorize a user to access to a specific type right? Version I am using is apache-atlas-0.9-incubating-SNAPSHOT. ... View more

Hello @Ayub Khan I am trying to add custom policy like dataAnalyistPolicy;;;;DATA_ANALYST:r;;type:*,entity:contents,taxonomy:*,term:* I am assuming that if I mention specific type or entity or term it will allow to perform set actions to a particular. I have tried mentioning entity:content but it block all operations. Can you guide on this. Thanks, ... View more

Hello @Nixon Rodrigues Thanks for your help, it works with mvn clean package -Pdist,berkeley-elasticsearch Now I am trying to load code base in eclipse and all dependencies have been resolved but in atlas-webapp. In setup code in eclipse guide following line is mention: a. File - Import - Maven - Existing Maven Projects b. Browse to your Atlas folder c. Uncheck the root project and non-Java projects such as dashboardv2, docs and distro, then click Finish But if I don't import dashboardv2 in eclipse atlas-webapp gives following error: Failed to execute goal on project atlas-webapp: Could not resolve dependencies for project org.apache.atlas:atlas-webapp:war:0.9-incubating-SNAPSHOT: Failure to find org.apache.atlas:atlas-dashboardv2:war:0.9-incubating-SNAPSHOT Can you guide me on this? ... View more

@Nixon Rodrigues Yes I am able to create tables and insert data in hbase. Yes currently all ports are open, but not able to open port 21000. ... View more

@Nixon Rodrigues There is progress in the setup, now I am getting following logs when I execute bin/atlas-start.py configured for local hbase. hbase started. configured for local solr. solr started. setting up solr collections... Apache Atlas Server started!!! But now port 21000 is not open. I have attached the log file, can you look at that. bin/quick_start.py Enter username for atlas :- admin Enter password for atlas :- admin Exception in thread "main" java.lang.IllegalArgumentException: None of the passed URLs are active org.apache.atlas.AtlasServerEnsemble@40ef3420 at org.apache.atlas.AtlasBaseClient.determineActiveServiceURL(AtlasBaseClient.java:207) at org.apache.atlas.AtlasBaseClient.initializeState(AtlasBaseClient.java:152) at org.apache.atlas.AtlasBaseClient.initializeState(AtlasBaseClient.java:140) at org.apache.atlas.AtlasBaseClient.<init>(AtlasBaseClient.java:96) at org.apache.atlas.AtlasClientV2.<init>(AtlasClientV2.java:105) at org.apache.atlas.examples.QuickStartV2.<init>(QuickStartV2.java:179) at org.apache.atlas.examples.QuickStartV2.runQuickstart(QuickStartV2.java:141) at org.apache.atlas.examples.QuickStartV2.main(QuickStartV2.java:132) Caused by: org.apache.atlas.AtlasServiceException: Metadata service API GET : api/atlas/admin/status failed at org.apache.atlas.AtlasBaseClient.selectActiveServerAddress(AtlasBaseClient.java:228) at org.apache.atlas.AtlasBaseClient.determineActiveServiceURL(AtlasBaseClient.java:204) ... 7 more Caused by: java.lang.RuntimeException: Could not find any active instance ... 9 more No sample data added to Apache Atlas Server. application-log-1.txt ... View more

@Nixon Rodrigues Do I need to install Ambari cluster on EC2 as well in order to make Atlas work or it can work without Ambari? ... View more

@Nixon Rodrigues Also look into atlas-propertiws and atlas-env file might be i am missing something there.atlas-applicationproperties.txt, atlas-env.txt ... View more

@Nixon Rodrigues Please find attached application log file. Following is the output if echo. echo $MANAGE_LOCAL_SOLR : true echo $MANAGE_LOCAL_HBASE : true echo $JAVA_HOME : /usr/lib/jvm/java-8-oracle/jre/bin/java application-log.txt ... View more

@Nixon Rodrigues These vars are already added in the script. I have re build the whole process but still getting same error on bin/quick_start.py command also please tell me what username and password should I set? Enter username for atlas :- Enter password for atlas :- Exception in thread "main" java.lang.IllegalArgumentException: None of the passed URLs are active org.apache.atlas.AtlasServerEnsemble@40ef3420 at org.apache.atlas.AtlasBaseClient.determineActiveServiceURL(AtlasBaseClient.java:207) at org.apache.atlas.AtlasBaseClient.initializeState(AtlasBaseClient.java:152) at org.apache.atlas.AtlasBaseClient.initializeState(AtlasBaseClient.java:140) at org.apache.atlas.AtlasBaseClient.<init>(AtlasBaseClient.java:96) at org.apache.atlas.AtlasClientV2.<init>(AtlasClientV2.java:105) at org.apache.atlas.examples.QuickStartV2.<init>(QuickStartV2.java:179) at org.apache.atlas.examples.QuickStartV2.runQuickstart(QuickStartV2.java:141) at org.apache.atlas.examples.QuickStartV2.main(QuickStartV2.java:132) Caused by: org.apache.atlas.AtlasServiceException: Metadata service API GET : api/atlas/admin/status failed at org.apache.atlas.AtlasBaseClient.selectActiveServerAddress(AtlasBaseClient.java:228) at org.apache.atlas.AtlasBaseClient.determineActiveServiceURL(AtlasBaseClient.java:204) ... 7 more Caused by: java.lang.RuntimeException: Could not find any active instance ... 9 more No sample data added to Apache Atlas Server. ... View more

@Nixon Rodrigues Thanks for following up. Can you tell me that do i need to install standalone Solr instance while using embeded-hbas-solr packaging. ... View more

@Nixon Rodrigues Yes you are right, I am using embedded-hbase-solr option. and I have stopped external zookeeper and restart the atlas server. This time logs are different. It keep trying to connect zookeeper and than through an exception after every 4 tries, have a look at the log file atached herewith. Do I need to change some configurations in applicatoin.properties file? logs1.txt ... View more

@Nixon Rodrigues I have stopped and started the server many times but same problem. What can be possible reasons that hbase is not initiated? Is it packaging issue? What is the structure of apache-atlas-0.9-incubating-SNAPSHOT directory after installation? I have installed zookeeper on the same machine as atlas is installed on. Thanks. ... View more

@Nixon Rodrigues Following is the log file. logs.txt ... View more

Tried the following. apache-atlas-0.9-incubating-SNAPSHOT$ curl -v http://localhost:21000/login.jsp * Trying 127.0.0.1... * Connected to localhost (127.0.0.1) port 21000 (#0) > GET /login.jsp HTTP/1.1 > Host: localhost:21000 > User-Agent: curl/7.47.0 > Accept: */* > * Empty reply from server * Connection #0 to host localhost left intact curl: (52) Empty reply from server Can you check my applicatio.properties attached in the comments. ... View more

@Nixon Rodrigues apache-atlas-0.9-incubating-SNAPSHOT$ bin/quick_start.py http://localhost:21000 Enter username for atlas :- sam Enter password for atlas :- 123 Creating sample types: Exception in thread "main" com.sun.jersey.api.client.ClientHandlerException: java.net.SocketException: Unexpected end of file from server at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:155) at com.sun.jersey.api.client.filter.HTTPBasicAuthFilter.handle(HTTPBasicAuthFilter.java:105) at com.sun.jersey.api.client.Client.handle(Client.java:652) at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682) at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:634) at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:300) at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:287) at org.apache.atlas.AtlasBaseClient.callAPI(AtlasBaseClient.java:429) at org.apache.atlas.AtlasClientV2.createAtlasTypeDefs(AtlasClientV2.java:217) at org.apache.atlas.examples.QuickStartV2.createTypes(QuickStartV2.java:191) at org.apache.atlas.examples.QuickStartV2.runQuickstart(QuickStartV2.java:147) at org.apache.atlas.examples.QuickStartV2.main(QuickStartV2.java:132) Caused by: java.net.SocketException: Unexpected end of file from server at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:792) at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647) at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:789) at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647) at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1569) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480) at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:253) at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:153) ... 12 more No sample data added to Apache Atlas Server. ... View more

@Nixon Rodrigues which end point do i need to provide here in following format? in guide it is mentioned as optional. <http/https>://<atlas-fqdn>:<atlas port> ... View more

@Ayub Khan Following is my properties file. atlas-applicationproperties.txt ... View more

@Ayub Khan I have tested that server is running by using following command: curl -v http://localhost:21000/api/atlas/admin/version Following is the result: * Trying 127.0.0.1... * Connected to localhost (127.0.0.1) port 21000 (#0) > GET /api/atlas/admin/version HTTP/1.1 > Host: localhost:21000 > User-Agent: curl/7.47.0 > Accept: */* > * Empty reply from server * Connection #0 to host localhost left intact curl: (52) Empty reply from server what else I need to test in order to make sure that my server and all of its components are running fine. Thanks, ... View more