Member since
01-14-2016
10
Posts
3
Kudos Received
0
Solutions
05-16-2016
02:01 PM
Also, please share the best practices on how to use spark sql configurations properties
... View more
05-16-2016
01:59 PM
Hi, I am using Spark Sql(ver 1.5.2) to read data from hive tables. It took 21 min to get the results where as the same query in hive cli took only 25 sec. Below are my HDP cluster details and spark query execution details.Please help me on this Cluster Details: HDP 2.3 30 Nodes(each node has 251 GB RAM) 10TB Memory 2610 Vcores Spark SQL execution --> spark-sql --master yarn-client --num-executors 40 --executor-cores 5 --executor-memory 4G
... View more
Labels:
- Labels:
-
Apache Spark