Member since
09-15-2015
457
Posts
507
Kudos Received
90
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
15665 | 11-01-2016 08:16 AM | |
11083 | 11-01-2016 07:45 AM | |
8571 | 10-25-2016 09:50 AM | |
1918 | 10-21-2016 03:50 AM | |
3830 | 10-14-2016 03:12 PM |
12-15-2015
02:41 PM
1 Kudo
Regarding 1) If you have Atlas installed, it will hook into the post-execution procedure of Hive (hive.exec.post.hooks). This hook ensures that queries or changes are recorded in Atlas (=>Data Governance)
... View more
12-15-2015
02:30 PM
4 Kudos
This sounds like a bug, and both values are removed in newer Ambari versions. https://issues.apache.org/jira/browse/AMBARI-13232 Could you remove the following two values and try to restart the RM: yarn.scheduler.capacity.root.accessible-node-labels.default.capacity yarn.scheduler.capacity.root.accessible-node-labels.default.maximum-capacity
... View more
12-15-2015
02:19 PM
I have seen this before, but on a HDP 2.3 system. Could you paste the content of your capacity-scheduler.xml (In Ambari within the Yarn config)
... View more
12-15-2015
01:10 PM
Here are some use case for a community like this: Solr to power the search engine of the community Hive to analyze access logs and user interaction Kafka/Storm to prevent spam (kind of like FraudDetection for banks) and process posts (filter bad words, escape posts, etc.) I also like the use case from Facebook, they are using HBase for their messages, chats, etc. (https://www.facebook.com/notes/facebook-engineering/the-underlying-technology-of-messages/454991608919/)
... View more
12-15-2015
09:15 AM
At the moment Hortonworks Community is using AnswerHub as a platform. I believe they are using Django as a backend framework, not sure though. As far as I know we have not connected any HDP tool, @Mark Herring has all the details. We could definitely use some of the tools from our HDP stack to power different services of our community site. Are you working on a similar use case?
... View more
12-15-2015
07:22 AM
1 Kudo
If you have imported and started the sandbox you should be able to access Ambari via sandbox.hortonworks.com:8080 or 127.0.0.1:8080. You can access the sandbox via shell by using Putty (Windows) and the following command: ssh root@sandbox.hortonworks.com -p 2222 Additional information about the installation are available in our Sandbox docs: http://hortonworks.com/wp-content/uploads/2015/07/...
... View more
12-14-2015
09:39 AM
2 Kudos
Glad I could help. Usually these settings are correct in our sandbox and you dont have to reconfigure them after you boot the sandbox. I'll let our sandbox-team know, so they can check if there might be an issue.
... View more
12-14-2015
09:14 AM
2 Kudos
@yan Adiputra Just looking at your logs, looks like the problem is a wrong memory configuration: 2015-12-10 09:57:39,443 INFO service.AbstractService (AbstractService.java:noteFailure(272)) - Service RMActiveServices failed in state INITED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Invalid resource scheduler memory allocation configuration, yarn.scheduler.minimum-allocation-mb=3072, yarn.scheduler.maximum-allocation-mb=2250, min and max should be greater than 0, max should be no smaller than min.
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Invalid resource scheduler memory allocation configuration, yarn.scheduler.minimum-allocation-mb=3072, yarn.scheduler.maximum-allocation-mb=2250, min and max should be greater than 0, max should be no smaller than min.
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.validateConf(CapacityScheduler.java:179)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.initScheduler(CapacityScheduler.java:308)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.serviceInit(CapacityScheduler.java:348)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:572)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:972)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:259)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1202)
According to the logs the following config is set: yarn.scheduler.maximum-allocation-mb=2250 yarn.scheduler.minimum-allocation-mb=3072 This config is impossible min has to be smaller than max Could you please switch the configurations, so that you have:
yarn.scheduler.maximum-allocation-mb=3072 yarn.scheduler.minimum-allocation-mb=2250
... View more
12-14-2015
07:48 AM
Could you please check if the RM is not still running? (process list, ps aux | grep resourcemanager) Also make sure there is no pid-file in /var/run/hadoop-yarn/yarn/
... View more