Created 12-08-2015 05:16 AM
Whats the difference between the two?
Created 12-08-2015 05:23 AM
There are many differences between the two.
Spark:
Tez
Bottom line, if are asking for the difference between Spark & Tez, consider using Spark.
Created 12-08-2015 05:23 AM
There are many differences between the two.
Spark:
Tez
Bottom line, if are asking for the difference between Spark & Tez, consider using Spark.
Created 12-08-2015 05:26 AM
From what we have witnessed in the field and during some customers testing, SparkSQL (1.4.x) at the time of testing was generally 50% - %200 faster when querying small datasets, by small we mean anywhere < 100GB datasets, which is usually great for data discovery, data wrangling, testing stuff out, or even running a production usecase where the datasets tend to be a lot but relatively small.
the bigger the table especially when joins are not effectively used or we are scanning a single one big table, and if you are in the BI space, and SLAs are required and you cant afford a query to break and start over, Tez was able to shine, its rigid stable, and the bigger the datasets the better the performance gets compared to Spark, at a 250GB datasets you will see a lot of similarities on the execution time, of course this will depend on how big is the cluster, how much memory allocated..etc
in general, my personal opinion we shouldn't compare both at this time as both shine in seperate contexts, at some stage Tez might be needed but maybe more Spark would be required in smaller datasets, and as I mentioned that was based on Spark 1.4.x , would love to re-run the testings again especially after the new cube functionalities in Spark 1.5.
hope this was helpful.
Created 12-09-2015 02:44 PM
Spark is a framework and written in Scala, and richer support for Python and Java API's. Scala is based on
functional programming and easy for applications written in Scala.
Created 12-13-2015 05:49 PM
Spark is meant for application development. Tez is a library which is used by tools such as Hive to speed things up. Tez isn't suitable for end-user programming.