Member since
03-23-2015
1288
Posts
114
Kudos Received
98
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 4412 | 06-11-2020 02:45 PM | |
| 6037 | 05-01-2020 12:23 AM | |
| 3861 | 04-21-2020 03:38 PM | |
| 4071 | 04-14-2020 12:26 AM | |
| 3044 | 02-27-2020 05:51 PM |
08-09-2019
01:49 AM
Hi, NN log is under /var/log/hadoop-hdfs and HS2 under /var/log/hive by default. If you use Cloudera Manager, you can check in CM's configuration page for both of them. To determine why the directory was not removed: - check if the table is managed or external table, as external table will not delete HDFS files - check the HMS server log, also under /var/log/hive on the HMS host, to check if there is any permission errors, as it is HMS's responsibility to create and delete HDFS paths - please also confirm if you have Hive Impersonation enabled? (under CM > Hive > Configuration > "HiveServer2 Enable Impersonation" if you are using CM) Cheers Eric
... View more
08-09-2019
01:41 AM
Hi, Without the actual error message, it would be quite hard to troubleshoot issues. For future reference, please share the error log if possible, it helps to determine the cause. If you want me to guess, I would think it might be caused by the fact that you create same table every time when workflow is triggered. Do you intend to overwrite the table or append data to the table? Cheers Eric
... View more
08-09-2019
01:36 AM
Hi Zane, Please share the failed query PROFILE as attachment to the thread if possible. Also, what's the default MEMORY_LIMIT for Impala? Have you tried to increase this limit to see if it can help with the query? SET MEMORY_LIMIT=20g; Of course the value will depend on how much memory you have on your impala daemon host. Cheers Eric
... View more
08-09-2019
01:32 AM
Hi Vishal, Did you mean you just finished your certification exam or you got your certification? If you have your certification, then result should be sent to you already. If you just finished the exam, then expect the result in a couple of days max. Cheers Eric
... View more
07-28-2019
08:52 PM
Hi Tim, The doc says: This optimization does not apply if the queries contain any WHERE, GROUP BY, or HAVING clause. Do you know if it still applies if WHERE condition only has partition column as well? Cheers Eric
... View more
07-25-2019
04:21 PM
1 Kudo
Please use commands mentioned here: https://oozie.apache.org/docs/4.3.0/DG_CommandLineTool.html#Getting_list_of_available_sharelib to check if your sharedlib was installed correctly. Cheers Eric
... View more
07-24-2019
04:48 PM
1 Kudo
Thanks Lars for pointing it out. So solution is to disable HDFS shortcircuit read for coordinator only impala daemons: a) create a new role group and add all coordinator only impala daemon hosts to this group b) Go to "CM -> Cluster -> Impala services -> Configuration"; c) Add the following property into "Impala Daemon HDFS Advanced Configuration Snippet (Safety Valve)" for the new role group that you just created: <property>
<name>dfs.client.read.shortcircuit</name>
<value>false</value>
</property> d) Save the changes and restart the Impala Daemon instance that are affected. Hope above can help. Cheers Eric
... View more
07-24-2019
04:39 AM
What do you mean by "not so early", did you mean by "not so easy"? You should be able to test out to confirm if it is version issue. Cheers Eric
... View more
07-24-2019
04:29 AM
Hi, COORDINATOR_ONLY impala daemon will not perform data reads, rather it only coordinates query execution by distributing jobs to executors. Why do you need to enable this for COORDINATOR_ONLY impala daemons? Or is it that all your COORDINATOR_ONLY impala daemons fail with this error? Also, have you checked whether socket /var/run/hdfs-sockets/dn exist on that impala daemon host? Cheers Eric
... View more