Member since
09-24-2015
144
Posts
72
Kudos Received
8
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1175 | 08-15-2017 08:15 AM | |
5681 | 01-24-2017 06:58 AM | |
1451 | 08-03-2016 06:45 AM | |
2626 | 06-01-2016 10:08 PM | |
2276 | 04-07-2016 10:30 AM |
02-11-2016
12:39 AM
2 Kudos
I think at this moment, controlling priority by capacity and preemption would be the only way to start and finish high priority jobs faster, am I correct? Ideally, I would like to set some priority *per* a job/application, but I found https://issues.apache.org/jira/browse/YARN-1963 . So I guess this is not possible. I also found http://blog.sequenceiq.com/blog/2014/03/14/yarn-capacity-scheduler/ which queue names are "highPriority" and "lowPriority", but if my reading is correct, this is not actually setting any priority but because the high has more capacity, the job finish faster. Until YARN-1963 is released, I would like to always start jobs in highPrority queue before any jobs in lowPriority queue, and if possible, I would like low priority jobs to wait until high priority jobs finishes. Any advice/hint is welcome.
... View more
Labels:
- Labels:
-
Apache YARN
02-04-2016
07:15 PM
1 Kudo
If any rule matches, does it stop processing other rules? Stop or not wouldn't be issue in most of the case but in case someone put completely wrong rules...
... View more
02-03-2016
05:46 PM
Looks like if I wanted to use HCat, the column names need to match...
... View more
02-02-2016
07:42 PM
1 Kudo
so... answer is it won't be fixed or middle of restoring process?
... View more
01-25-2016
02:07 AM
1 Kudo
After reading above, I'm just curious to know what would happen in the following scenario: 1) Create queues (ex: Rack1, Rack2, Rack3...) 2) Create (exclusive=true) Node Labels and assign to queues per my physical rack layout 3) Didn't set up HDFS rack-awareness (so that replication won't care about rack) 4) Submit a job to the queue "Rack1" but all blocks for this data are in DataNodes in different rack (ex: Rack2) Would YARN AM try to create a remote container in a NameNode in Rack2?
Or keep using container in Rack1 but fetch the data from a remote DataNode?
... View more
01-21-2016
09:54 PM
I read the 2nd link and couldn't find DockerFIle. Did I miss something? I just wanted to compare with mine to improve...
... View more
01-19-2016
11:53 PM
1 Kudo
I would like to read http://hortonworks.com/kb/how-to-connect-tableau-to-hortonworks-sandbox/ which link is in http://hortonworks.com/hadoop-tutorial/making-things-tick-with-tableau/ Does anyone know what happened to this link?
... View more
Labels:
- Labels:
-
Apache Hadoop
01-14-2016
05:17 AM
If I don't mind some down time, can I skip decommission / recommission process? I have 12 DataNode locations and 12 disks.
I would like to replace only one disk.
I can schedule maintenance window.
... View more
01-13-2016
11:28 PM
Is there any possibility that this headless keytab is used when spark submits a job (to YARN or hive, maybe?) to identify itself? Not for ambari to start Spark service, maybe?
... View more
01-07-2016
01:29 AM
1 Kudo
Is "/mnt/hadoop/storm" or "hadoop/storm" decided by the value of "storm.local.dir" parameter?
... View more