Member since
11-24-2015
223
Posts
10
Kudos Received
0
Solutions
03-21-2017
09:00 PM
I have a hadoop cluster on 2 nodes. Each node has 2 cpus and 24 gb memory. So I am thinking of having 3 mappers and 3 reducers on each node. That's good? So in Ambari -> Yarn -> Configs, can I have : Memory allocated for all YARN containers on a node : 18 gb Minimum container size (memory) : 1 gb Maximum container size (memory) : 4 gb If I configure as per above, then Ambari suggests other configurations based on these values - is that safe to follow? Likewise in Ambari -> MapReduce2 -> Configs : Not sure what to configure Map Memory, Reduce Memory, Appmaster Memory. Appreciate the insights.
... View more
Labels:
- Labels:
-
Apache YARN
03-15-2017
07:58 PM
Thanks. I was missing the cluster name in the command. Thanks again.
... View more
03-15-2017
03:34 PM
When I ran your command got this : {
"status": 403,
"message": "Full authentication is required to access this resource"
}
... View more
03-15-2017
02:42 PM
Our vendor asked us to use this for removing falcon : https://cwiki.apache.org/confluence/display/AMBARI/Using+APIs+to+delete+a+service+or+all+host+components+on+a+host But the path given in the first command itself I cannot find in my installation : http: //AMBARI_SERVER_HOST:8080/api/v1/clusters/c1/services/SERVICENAME Is the above path local to the installation given in the example? By "SERVICE" I am equating with falcon as that is the service I am trying to remove. When I searched for falcon in the Ambari host I got a bunch of directories - but substituting any of the directories in the above command only got me : {
"status": 404,
"message": "Not Found"
}
... View more
03-14-2017
02:00 PM
Thanks for the response. And BTW how to reinstall Falcon? Appreciate the feedback.
... View more
03-14-2017
01:59 PM
Yes I did check that but the said jar file is already available in our installation. Thanks
... View more
03-13-2017
08:37 PM
Given the 503 error we are getting when trying to open the Falcon Web UI, our support engineer has recommended that we uninstall and reinstall falcon. So I did a bit of research and found this command : curl -u $USER:$PASS -i -H 'X-Requested-By: ambari'-X DELETE http://$AMBARI_HOST:$AMBARI_PORT/api/v1/clusters/<cluster-name>/services/<service-name>; So the $AMBARI_HOST in above command is the host on which Ambari is running - and not the host Falcon is running? And what is the $AMBARI_PORT/port number to be used in the above command - 8080? The USER/PASS is root and its password or falcon and its password? (I had done the HDP/Ambari installation as root). Appreciate the clarification.
... View more
Labels:
- Labels:
-
Apache Falcon
03-09-2017
01:55 PM
Hi, I am having similar issue with falcon replication not working. The permissions, ownership, oozie wildcard are all as per this page in my environment. But still replication is not going through. Any insights?
... View more
03-07-2017
02:52 PM
Hello, any feedback on the Falcon issues? I have configured the Falcon clusters and also the job. But replication is not happening. I enabled Ozzie log as shown in the Hortonworks site (including restarting Oozie) but still I can see the log as disabled in the job error log. Not sure why. Also in Falcon UI if I click on the job I can see one or two instances (?) of the job running, while the rest are WAITING. So to restart the job I just have to Suspend and Resume? So when will the RUNNING jobs end and when will the WAITING jobs start? Appreciate the feedback.
... View more
03-06-2017
05:27 PM
I had setup Falcon mirroring based on this document https://hortonworks.com/hadoop-tutorial/mirroring-datasets-between-hadoop-clusters-with-apache-falcon/ The one single difference is that unlike the example in the document which seems to be using a single hdfs cluster, I am using two different clusters - each with its set of nodes. At the outset after running the mirror job it is NOT working for me. Some initial questions are : 1. After creating a Falcon Cluster, how do we view it(s) definition? I had created two falcon clusters - one for the primary and another for the target. I would like to see the information falcon holds for them. 2. After executing a Falcon job how do we check the logs to see what it did and where it failed? Appreciate the insights.
... View more