Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Hadoop enviroment performance root cause??

Highlighted

Hadoop enviroment performance root cause??

We are working on some proof of concepts on the Hadoop dev environments and are running into perceived performance and memory issues using Hive and SQL. Is there a way we can run something like "explains" on the SQL or assess the environment. Need to determine where the bottlenecks might be. It takes about about 20 minutes to do an average calculation in SQL for about 26 million rows, when we increase that volume we run of of memory. We need to take a look at what the issue might be at root cause.

2 REPLIES 2
Highlighted

Re: Hadoop enviroment performance root cause??

@kishore sanchina

Yes, Hive has the EXPLAIN operator: https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Explain

Can you provide more information on your environment? How many nodes do you have? What is the server configuration (CPU, memory)? Is Hive on Tez enabled?

Re: Hadoop enviroment performance root cause??

@Michael Young Hadoop: 6 Nodes On Premise - SWHadoop 2.2.4.2 Storage Hadoop: 1TB / node Internet Bandwidth25MBS

Don't have an account?
Coming from Hortonworks? Activate your account here