03-01-2019 03:10 PM - last edited on 03-02-2019 05:49 AM by cjervis
Spark newbie. need to test Hadoop-Cloudera Cluster for performance under concurrent queries
so I'm looking to send multiple queries concurrently via IntelliJ/Scala...
i.e. something like this pseudo code
for ctr in 5000:
resultDf = sparkSession.sql(...)
just to see how the cluster would handle multiple concurrent requests
Dev IDE: intelliJ with Scala.
Not sure how to do this, looking for guidance
also, can I quickly simulate this concurrent query in the HUE browser?