Member since
01-09-2019
401
Posts
163
Kudos Received
80
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1539 | 06-21-2017 03:53 PM | |
2183 | 03-14-2017 01:24 PM | |
1454 | 01-25-2017 03:36 PM | |
2455 | 12-20-2016 06:19 PM | |
1150 | 12-14-2016 05:24 PM |
05-05-2016
09:30 PM
You can take a look at this review board discussion (https://reviews.apache.org/r/43557/) That corresponding change in ambari will only work from HDP-2.3.4+ . But since ambari couldn't support 2 different properties for third level version number (2.3.x change), they ended up leaving as is and calling it a known issue. If you upgrade to 2.3.4, you won't see this issue. But for ones before that, this is being called out as known issue which is not going to be fixed.
... View more
05-05-2016
09:04 PM
This is one of the known issues. Take a look at this and other known issues and fixes here
... View more
05-05-2016
08:28 PM
2 Kudos
Take a look at KMS documentation If you are sandbox, login to ranger as keyadmin/keyadmin, select sandbox_kms and add a key from there.
... View more
05-05-2016
07:06 PM
1 Kudo
Adding all details from other answer here to consolidate. Try to keep fs.trash.interval longer (I prefer to keep it as one week). For fs.trashcheckpoint.interval, this is the interval of the thread that run to clean up all the trash that is longer than the fs.trash.interval. Keep this shorter, like twice a day or more. If you leave it at 0, cleanup happens every 7 days, so there can be some files that can stay for upto 14 days. @Arul Ramachandran @Sean Creedon @Saumil Mayani if fs.trash.checkpoint.interval < fs.trash.interval or == 0, fs.trash.interval is used as checkpoint interval. So, you can leave it as default 0, as long as you are ok leaving some data for longer in trash. You can take a look at TrashIntervalDefault.java code that has the details. Emptier(Configuration conf, long emptierInterval) throws IOException {
this.conf = conf;
this.emptierInterval = emptierInterval;
if (emptierInterval > deletionInterval || emptierInterval == 0) {
LOG.info("The configured checkpoint interval is " +
(emptierInterval / MSECS_PER_MINUTE) + " minutes." +
" Using an interval of " +
(deletionInterval / MSECS_PER_MINUTE) +
" minutes that is used for deletion instead");
this.emptierInterval = deletionInterval;
}
... View more
05-05-2016
06:06 PM
If it is a small cluster, you can skip passwordless ssh and use manual ambari-agent install. Steps for that are here. While this is not a solution to your passwordless ssh issue, this works very well on smaller clusters (I did manual registration on larger clusters too since there were policies of no passwordless ssh for su user).
... View more
05-05-2016
05:54 PM
This points to a jar version mismatch. That class is in hadoop-common.jar. Have you manually copied any jars? Please check if all hadoop-common jars are at version 2.7.1.2.4.0.0-169
... View more
05-05-2016
05:43 PM
Not sure how your sandbox missed that folder in the first place. I sent you the steps from manual install. Should have been there in the first place, but since it was not there, we followed manual steps to put them there. Tez and mapreduce use the tar.gz files from /hdp/apps/<hdp_version>/ for submitting applications.
... View more
05-05-2016
05:38 PM
Spark Thrift Server is similar to HiveServer2, you will install it on a node and use that as JDBC URL. You could have more than one in HA and load balancing scenarios but other than that, you will only have one of it.
... View more
05-05-2016
05:13 PM
2 Kudos
Are you able to run any other queries with Tez? Looking at exceptions, you seem to run into some environment issue with wrong versions of jars. Please give more details about your environment (HDP version, OS details etc). Is this the state after an upgrade or were you never able to run a Tez query on this environment?
... View more
05-05-2016
05:10 PM
1 Kudo
If you are using latest sandbox, you should see /hdp/apps/2.4.0.0-169/tez/tez.tar.gz folder in HDFS But if you are using a 2.3.4 version of sandbox, you should see the version in HDFS. Can you look at /hdp/apps/ in hdfs and paste the contents? If the file does not exist, you can manually put it there by following below steps su - $HDFS_USER
hdfs dfs -mkdir -p /hdp/apps/<hdp_version>/tez/
hdfs dfs -put /usr/hdp/<hdp_version>/tez/lib/tez.tar.gz /hdp/apps/<hdp_version>/tez/
hdfs dfs -chown -R $HDFS_USER:$HADOOP_USER /hdp
hdfs dfs -chmod -R 555 /hdp/apps/<hdp_version>/tez
hdfs dfs -chmod -R 444 /hdp/apps/<hdp_version>/tez/tez.tar.gz
... View more