Reply
Contributor
Posts: 25
Registered: ‎02-11-2019

Concurent SQL query via Scala

[ Edited ]

Hi,

 

Spark newbie.  need to test Hadoop-Cloudera Cluster for performance under concurrent queries

 

so I'm looking to send multiple queries concurrently via IntelliJ/Scala...

 

i.e. something like this pseudo code

 

     for ctr in 5000:

          resultDf = sparkSession.sql(...)

          ...

    

just to see how the cluster would handle multiple concurrent requests

 

PS:

Dev IDE: intelliJ with Scala.

 

Not sure how to do this, looking for guidance

also, can I quickly simulate this concurrent query in the HUE browser?