My expertise is not in hadoop but rather online communities, support and social media. Interests include: photography, travel, movies and watching sports.
Welcome to the community @HamTera. While you wait for a more knowledge member to respond, I'll ask if the size of the table is perhaps maxing out your memory? I did some looking around and that came up as a possibility in large tables. I hope it helps.
... View more
@ryu If @steven-matison answered your question, please mark his reply as the solution, as it will make it easier for others to find the answer in the future.
... View more
@orodriguesrenan Have you resolved your issue? If so please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.
... View more
Hi, Is Impala session hang means not able to run any queries? Is this issue observed via hue or impala-shell? repro the issue via impala-shell and check the coordinator logs to get any suspected error messages. At the time of issue check the cluster load how may queries running concurrently, Issue may arise if any expensive/complex operations are running. Do not frequently run "invalidate metadata" query in the cluster it is one of the expensive operations and it pressurises the catalog service. Regards, Chethan YM
... View more
@Dave0x1 have you been able to resolved your issue. If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.
... View more
Hi @mrTao It is not good idea to use entire memory Cluster YARN memory. You can tune the memory from spark side by adjusting the memory using following parameters: --conf spark.executor.instances=5
--conf spark.driver.memory=10g
--conf spark.driver.memoryOverhead=1g
--conf spark.executor.memory=10g
--conf spark.executor.memoryOverhead=1g With the above memory configuration, YARN will allocate 66gb(Executor memory 11gb * Executor Instances(5) + Driver memory 11gb = 55g + 11g = 66g). Better check your spark-submit once again and tune the above parameters according to your requirement.
... View more
@Manju1993 Welcome to our community! To help you get the best possible answer, I have tagged in our Spark experts @RangaReddy @Babasaheb who may be able to assist you further. Please feel free to provide any additional information or details about your query, and we hope that you will find a satisfactory solution to your question.
... View more
Welcome to the community @prabhu456. Hopefully you have already resolved this, but if not, perhaps this older solved question may get you closer.
Cloudera Navigator - Identify unused db, tables, files, folders, etc
... View more
