Member since
03-24-2017
24
Posts
0
Kudos Received
0
Solutions
04-13-2017
11:06 AM
Hi, I have set ambari 2.5.0.3 on EC2 instance, I didn't make multiple nodes, I am working on single node. I have setup mysql and installed Ranger but it is not getting started. It is showing me errors as shown in attached images. Please look into it and guide me if i am doing any thing wrong?
... View more
Labels:
- Labels:
-
Apache Ambari
-
Apache Ranger
04-12-2017
02:17 PM
@Pradeep kumar @Benjamin Leonhardi Guys I have installed Ambari cluster and Atlas service on it successfully. Now I am trying to install Hive. I am unable to start Hive service as metadata service is not running i have attached snapshots and log here with plz guide me. Metastore on ip-172-31-69-106.ec2.internal failed (Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_metastore.py", line 200, in execute
timeout_kill_strategy=TerminateStrategy.KILL_PROCESS_TREE,
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
self.env.run()
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
self.run_action(resource, action)
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
provider_action()
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 262, in action_run
tries=self.resource.tries, try_sleep=self.resource.try_sleep)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
result = function(command, **kwargs)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
result = _call(command, **kwargs_copy)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
raise ExecutionFailed(err_msg, code, out, err)
ExecutionFailed: Execution of 'export HIVE_CONF_DIR='/usr/hdp/current/hive-metastore/conf' ; hive --hiveconf hive.metastore.uris=thrift://ip-172-31-69-106.ec2.internal:9083 --hiveconf hive.metastore.client.connect.retry.delay=1 --hiveconf hive.metastore.failure.retries=1 --hiveconf hive.metastore.connect.retries=1 --hiveconf hive.metastore.client.socket.timeout=14 --hiveconf hive.execution.engine=mr -e 'show databases;'' returned 1. log4j:WARN No such property [maxFileSize] in org.apache.log4j.DailyRollingFileAppender.
Logging initialized using configuration in file:/etc/hive/2.6.0.3-8/0/hive-log4j.properties
Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:547)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:233)
at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1565)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:92)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:138)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:110)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3303)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3322)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:528)
... 8 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1563)
... 14 more
Caused by: MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused (Connection refused)
at org.apache.thrift.transport.TSocket.open(TSocket.java:226)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:486)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:281)
at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1563)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:92)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:138)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:110)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3303)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3322)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:528)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:233)
at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.thrift.transport.TSocket.open(TSocket.java:221)
... 22 more
)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:532)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:281)
at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
... 19 more
) Thanks,
... View more
04-12-2017
12:09 PM
@Dan Rice Yes i have gone through that and i was able to install using Hortonwork's guide... but not with the Apache's guide.
... View more
04-11-2017
12:39 PM
I am installing Ambari from following guide: https://cwiki.apache.org/confluence/display/AMBARI/Installation+Guide+for+Ambari+2.4.2 When I execute following command mvn versions:set -DnewVersion= 2.4 . 2.0 . 0 following report summary generated. It skips every thing. [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Ambari Main ........................................ SUCCESS [ 0.780 s] [INFO] Apache Ambari Project POM .......................... SKIPPED [INFO] Ambari Web ......................................... SKIPPED [INFO] Ambari Views ....................................... SKIPPED [INFO] Ambari Admin View .................................. SKIPPED [INFO] ambari-metrics ..................................... SKIPPED [INFO] Ambari Metrics Common .............................. SKIPPED [INFO] Ambari Metrics Hadoop Sink ......................... SKIPPED [INFO] Ambari Metrics Flume Sink .......................... SKIPPED [INFO] Ambari Metrics Kafka Sink .......................... SKIPPED [INFO] Ambari Metrics Storm Sink .......................... SKIPPED [INFO] Ambari Metrics Storm Sink (Legacy) ................. SKIPPED [INFO] Ambari Metrics Collector ........................... SKIPPED [INFO] Ambari Metrics Monitor ............................. SKIPPED [INFO] Ambari Metrics Grafana ............................. SKIPPED [INFO] Ambari Metrics Assembly ............................ SKIPPED [INFO] Ambari Server ...................................... SKIPPED [INFO] Ambari Functional Tests ............................ SKIPPED [INFO] Ambari Agent ....................................... SKIPPED [INFO] Ambari Client ...................................... SKIPPED [INFO] Ambari Python Client ............................... SKIPPED [INFO] Ambari Groovy Client ............................... SKIPPED [INFO] Ambari Shell ....................................... SKIPPED [INFO] Ambari Python Shell ................................ SKIPPED [INFO] Ambari Groovy Shell ................................ SKIPPED [INFO] ambari-logsearch ................................... SKIPPED [INFO] Ambari Logsearch Appender .......................... SKIPPED [INFO] Ambari Logsearch Solr Client ....................... SKIPPED [INFO] Ambari Logsearch Portal ............................ SKIPPED [INFO] Ambari Logsearch Log Feeder ........................ SKIPPED [INFO] Ambari Logsearch Assembly .......................... SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 1.837 s [INFO] Finished at: 2017-04-11T09:32:01+00:00 [INFO] Final Memory: 22M/482M [INFO] ------------------------------------------------------------------------
... View more
Labels:
- Labels:
-
Apache Ambari
04-06-2017
12:29 PM
@Nixon Rodrigues You mean currently I can only specify * I can't authorize a user to access to a specific type right? Version I am using is apache-atlas-0.9-incubating-SNAPSHOT.
... View more
04-06-2017
10:51 AM
Hello @Ayub Khan I am trying to add custom policy like dataAnalyistPolicy;;;;DATA_ANALYST:r;;type:*,entity:contents,taxonomy:*,term:* I am assuming that if I mention specific type or entity or term it will allow to perform set actions to a particular. I have tried mentioning entity:content but it block all operations. Can you guide on this. Thanks,
... View more
03-31-2017
01:58 PM
Hello @Nixon Rodrigues Thanks for your help, it works with mvn clean package -Pdist,berkeley-elasticsearch Now I am trying to load code base in eclipse and all dependencies have been resolved but in atlas-webapp. In setup code in eclipse guide following line is mention: a. File - Import - Maven - Existing Maven Projects b. Browse to your Atlas folder c. Uncheck the root project and non-Java projects such as dashboardv2, docs and distro, then click Finish
But if I don't import dashboardv2 in eclipse atlas-webapp gives following error: Failed to execute goal on project atlas-webapp: Could not resolve dependencies for project org.apache.atlas:atlas-webapp:war:0.9-incubating-SNAPSHOT: Failure to find org.apache.atlas:atlas-dashboardv2:war:0.9-incubating-SNAPSHOT
Can you guide me on this?
... View more
03-29-2017
12:09 PM
@Nixon Rodrigues Yes I am able to create tables and insert data in hbase. Yes currently all ports are open, but not able to open port 21000.
... View more
03-29-2017
09:41 AM
@Nixon Rodrigues There is progress in the setup, now I am getting following logs when I execute bin/atlas-start.py configured for local hbase. hbase started. configured for local solr. solr started. setting up solr collections... Apache Atlas Server started!!! But now port 21000 is not open. I have attached the log file, can you look at that. bin/quick_start.py Enter username for atlas :- admin Enter password for atlas :- admin Exception in thread "main" java.lang.IllegalArgumentException: None of the passed URLs are active org.apache.atlas.AtlasServerEnsemble@40ef3420 at org.apache.atlas.AtlasBaseClient.determineActiveServiceURL(AtlasBaseClient.java:207) at org.apache.atlas.AtlasBaseClient.initializeState(AtlasBaseClient.java:152) at org.apache.atlas.AtlasBaseClient.initializeState(AtlasBaseClient.java:140) at org.apache.atlas.AtlasBaseClient.<init>(AtlasBaseClient.java:96) at org.apache.atlas.AtlasClientV2.<init>(AtlasClientV2.java:105) at org.apache.atlas.examples.QuickStartV2.<init>(QuickStartV2.java:179) at org.apache.atlas.examples.QuickStartV2.runQuickstart(QuickStartV2.java:141) at org.apache.atlas.examples.QuickStartV2.main(QuickStartV2.java:132) Caused by: org.apache.atlas.AtlasServiceException: Metadata service API GET : api/atlas/admin/status failed at org.apache.atlas.AtlasBaseClient.selectActiveServerAddress(AtlasBaseClient.java:228) at org.apache.atlas.AtlasBaseClient.determineActiveServiceURL(AtlasBaseClient.java:204) ... 7 more Caused by: java.lang.RuntimeException: Could not find any active instance ... 9 more No sample data added to Apache Atlas Server. application-log-1.txt
... View more
03-27-2017
03:02 PM
@Nixon Rodrigues Do I need to install Ambari cluster on EC2 as well in order to make Atlas work or it can work without Ambari?
... View more
03-27-2017
12:07 PM
@Nixon Rodrigues Also look into atlas-propertiws and atlas-env file might be i am missing something there.atlas-applicationproperties.txt, atlas-env.txt
... View more
03-27-2017
11:46 AM
@Nixon Rodrigues Please find attached application log file. Following is the output if echo. echo $MANAGE_LOCAL_SOLR : true echo $MANAGE_LOCAL_HBASE : true echo $JAVA_HOME : /usr/lib/jvm/java-8-oracle/jre/bin/java application-log.txt
... View more
03-27-2017
09:59 AM
@Nixon Rodrigues
These vars are already added in the script. I have re build the whole process but still getting same error on bin/quick_start.py command also please tell me what username and password should I set? Enter username for atlas :- Enter password for atlas :- Exception in thread "main" java.lang.IllegalArgumentException: None of the passed URLs are active org.apache.atlas.AtlasServerEnsemble@40ef3420 at org.apache.atlas.AtlasBaseClient.determineActiveServiceURL(AtlasBaseClient.java:207) at org.apache.atlas.AtlasBaseClient.initializeState(AtlasBaseClient.java:152) at org.apache.atlas.AtlasBaseClient.initializeState(AtlasBaseClient.java:140) at org.apache.atlas.AtlasBaseClient.<init>(AtlasBaseClient.java:96) at org.apache.atlas.AtlasClientV2.<init>(AtlasClientV2.java:105) at org.apache.atlas.examples.QuickStartV2.<init>(QuickStartV2.java:179) at org.apache.atlas.examples.QuickStartV2.runQuickstart(QuickStartV2.java:141) at org.apache.atlas.examples.QuickStartV2.main(QuickStartV2.java:132) Caused by: org.apache.atlas.AtlasServiceException: Metadata service API GET : api/atlas/admin/status failed at org.apache.atlas.AtlasBaseClient.selectActiveServerAddress(AtlasBaseClient.java:228) at org.apache.atlas.AtlasBaseClient.determineActiveServiceURL(AtlasBaseClient.java:204) ... 7 more Caused by: java.lang.RuntimeException: Could not find any active instance ... 9 more No sample data added to Apache Atlas Server.
... View more
03-27-2017
07:20 AM
@Nixon Rodrigues Thanks for following up. Can you tell me that do i need to install standalone Solr instance while using embeded-hbas-solr packaging.
... View more
03-24-2017
12:48 PM
@Nixon Rodrigues Yes you are right, I am using embedded-hbase-solr option. and I have stopped external zookeeper and restart the atlas server. This time logs are different. It keep trying to connect zookeeper and than through an exception after every 4 tries, have a look at the log file atached herewith. Do I need to change some configurations in applicatoin.properties file? logs1.txt
... View more
03-24-2017
12:25 PM
@Nixon Rodrigues I have stopped and started the server many times but same problem. What can be possible reasons that hbase is not initiated? Is it packaging issue? What is the structure of apache-atlas-0.9-incubating-SNAPSHOT directory after installation? I have installed zookeeper on the same machine as atlas is installed on. Thanks.
... View more
03-24-2017
11:29 AM
@Nixon Rodrigues Following is the log file. logs.txt
... View more
03-24-2017
11:14 AM
Tried the following. apache-atlas-0.9-incubating-SNAPSHOT$ curl -v http://localhost:21000/login.jsp * Trying 127.0.0.1... * Connected to localhost (127.0.0.1) port 21000 (#0) > GET /login.jsp HTTP/1.1 > Host: localhost:21000 > User-Agent: curl/7.47.0 > Accept: */* > * Empty reply from server * Connection #0 to host localhost left intact curl: (52) Empty reply from server Can you check my applicatio.properties attached in the comments.
... View more
03-24-2017
11:13 AM
@Nixon Rodrigues apache-atlas-0.9-incubating-SNAPSHOT$ bin/quick_start.py http://localhost:21000 Enter username for atlas :- sam Enter password for atlas :- 123 Creating sample types: Exception in thread "main" com.sun.jersey.api.client.ClientHandlerException: java.net.SocketException: Unexpected end of file from server at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:155) at com.sun.jersey.api.client.filter.HTTPBasicAuthFilter.handle(HTTPBasicAuthFilter.java:105) at com.sun.jersey.api.client.Client.handle(Client.java:652) at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682) at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:634) at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:300) at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:287) at org.apache.atlas.AtlasBaseClient.callAPI(AtlasBaseClient.java:429) at org.apache.atlas.AtlasClientV2.createAtlasTypeDefs(AtlasClientV2.java:217) at org.apache.atlas.examples.QuickStartV2.createTypes(QuickStartV2.java:191) at org.apache.atlas.examples.QuickStartV2.runQuickstart(QuickStartV2.java:147) at org.apache.atlas.examples.QuickStartV2.main(QuickStartV2.java:132) Caused by: java.net.SocketException: Unexpected end of file from server at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:792) at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647) at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:789) at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647) at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1569) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480) at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:253) at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:153) ... 12 more No sample data added to Apache Atlas Server.
... View more
03-24-2017
10:50 AM
@Nixon Rodrigues which end point do i need to provide here in following format? in guide it is mentioned as optional. <http/https>://<atlas-fqdn>:<atlas port>
... View more
03-24-2017
10:15 AM
@Ayub Khan Following is my properties file.
atlas-applicationproperties.txt
... View more
03-24-2017
10:12 AM
@Ayub Khan I have tested that server is running by using following command: curl -v http://localhost:21000/api/atlas/admin/version Following is the result: * Trying 127.0.0.1... * Connected to localhost (127.0.0.1) port 21000 (#0) > GET /api/atlas/admin/version HTTP/1.1 > Host: localhost:21000 > User-Agent: curl/7.47.0 > Accept: */* > * Empty reply from server * Connection #0 to host localhost left intact curl: (52) Empty reply from server what else I need to test in order to make sure that my server and all of its components are running fine. Thanks,
... View more
03-24-2017
10:02 AM
Hello, I am setting apache atlas on EC2. When I tried command " bin/quick_start.py" it ask for username/password. I provide admin/admin. It took a while and than gives this crashlog. apache-atlas-0.9-incubating-SNAPSHOT$ bin/quick_start.py Enter username for atlas :- sam Enter password for atlas :- 123 =========Crash Log================ Exception in thread "main" java.lang.IllegalArgumentException: None of the passed URLs are active org.apache.atlas.AtlasServerEnsemble@40ef3420 at org.apache.atlas.AtlasBaseClient.determineActiveServiceURL(AtlasBaseClient.java:207) at org.apache.atlas.AtlasBaseClient.initializeState(AtlasBaseClient.java:152) at org.apache.atlas.AtlasBaseClient.initializeState(AtlasBaseClient.java:140) at org.apache.atlas.AtlasBaseClient.<init>(AtlasBaseClient.java:96) at org.apache.atlas.AtlasClientV2.<init>(AtlasClientV2.java:105) at org.apache.atlas.examples.QuickStartV2.<init>(QuickStartV2.java:179) at org.apache.atlas.examples.QuickStartV2.runQuickstart(QuickStartV2.java:141) at org.apache.atlas.examples.QuickStartV2.main(QuickStartV2.java:132) Caused by: org.apache.atlas.AtlasServiceException: Metadata service API GET : api/atlas/admin/status failed at org.apache.atlas.AtlasBaseClient.selectActiveServerAddress(AtlasBaseClient.java:228) at org.apache.atlas.AtlasBaseClient.determineActiveServiceURL(AtlasBaseClient.java:204) ... 7 more Caused by: java.lang.RuntimeException: Could not find any active instance ... 9 more No sample data added to Apache Atlas Server. Please guide If any one has any idea about it.
... View more
Labels:
- Labels:
-
Apache Ambari
-
Apache Atlas