Thanks for answering , However I have one more question here, In the existed HiveQL we are creating some temporary hive work tables to store the intermediate data and dropping those off at the end. In Spark QL instead of those temporary tables I can use createOrReplaceTempView("XXXX") to create a temporary in memory view . At any point while my data is growing what happens if this TempView can't fit in the memory. Will my job fails ? What I need to do to tackle these kind of scenarios? Appreciate your reply !!!
... View more
