Member since
05-16-2014
7
Posts
0
Kudos Received
0
Solutions
06-26-2018
06:33 PM
Few things to check 1) Are the datanodes actually running 2) Are those datanodes setup with a different port for some reason? 3) Double check your dfs.exclude and dfs.include files too , Are you sure that other 1/3 of datanodes are running? You may also want to make sure the NN isn't rejecting those datanodes from joining the cluster. Check the dfs.include and dfs.exclude files
... View more
03-27-2018
09:04 PM
Hi, Can we get a roadmap for when Debian 8, 9, etc will get a release from Hortonworks? I'd hate to have to switch to a different distro! Thanks!
... View more
Labels:
03-27-2018
08:57 PM
Any idea if Hortonworks plans on supporting HDFS Federation or are they waiting for the new HDFS architecture to come out (object store)?
... View more
07-07-2014
09:15 AM
We found this Jira: https://issues.apache.org/jira/browse/MAPREDUCE-5537 Which was to fix a bug that seems similar to what we saw, howver, that was back in Hive .8. Perhaps it made it back into Hive .10?
... View more
07-07-2014
08:23 AM
Has anyone been using hive.hadoop.supports.splittable.combineinputformat with Hive .10 (CDH4.3.2) and non-indexed lzo (non-splittable) compressed files? We recently tried out this parameter with non-indexed lzo files and at first it appeared to be great, reducing the number of mappers to read the input data and doing a great job of merging data into files that were about the size of the max input split size. Unfortunately, it was found that results from some hive queries with this parameter in place and using non-indexed lzo file started to give different results. Here are a couple of older Jira's that seemed to indicate we could use this parameter with non-splittable compressed files. https://issues.apache.org/jira/browse/MAPREDUCE-1597 https://issues.apache.org/jira/browse/HIVE-2089 Perhaps this is a new bug or we did something wrong.
... View more
06-30-2014
02:29 PM
Is anyone using control groups (cgroups) to manage memory on debian (squeeze or wheezy)? If so, what version of debian and what do you think of it so far? I'm looking into additional options to control map and reduce memory usage and control groups looks to be a great fit, especially whenever we add Impala. If not...any other options out there? My main course of action now is using mapred.map.child.limit and mapred.reduce.child limit to put a lid on child JVM size and spawned processes for streaming in addition to readjusting the map and reduce slots. YARN is some day in our future, but not soon enough.
... View more