question Re: Spark SQL as a Federated DB in Production? in Archives of Support Questions (Read Only) https://community.cloudera.com/t5/Archives-of-Support-Questions/Spark-SQL-as-a-Federated-DB-in-Production/m-p/147734#M23913 <P><A href="https://community.hortonworks.com/questions/24935/spark-sql-as-a-federated-db-in-production.html#">@azeltov</A> How large is the implementation (How many tables, how much is cached vs read through to source)? In the case where they are caching the entire table, how do they ensure the data is not stale? Are the tables temporary or saved in the hive meta store?</P> Tue, 29 Mar 2016 23:27:17 GMT vvaks 2016-03-29T23:27:17Z Spark SQL as a Federated DB in Production? https://community.cloudera.com/t5/Archives-of-Support-Questions/Spark-SQL-as-a-Federated-DB-in-Production/m-p/147732#M23911 <P>There are some great articles and threads here on HCC about using Spark to query data from other JDBC sources and mash them up with anything else you can get into an RDD. Has anyone seen this pattern (Spark as a Federated DB including JDBC sources) actually used in Production (with JDBC thrift server)? What is the right configuration within a secure, multi-tenant Hadoop cluster?</P> Tue, 29 Mar 2016 19:44:44 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/Spark-SQL-as-a-Federated-DB-in-Production/m-p/147732#M23911 vvaks 2016-03-29T19:44:44Z Re: Spark SQL as a Federated DB in Production? https://community.cloudera.com/t5/Archives-of-Support-Questions/Spark-SQL-as-a-Federated-DB-in-Production/m-p/147733#M23912 <P><A rel="user" href="https://community.cloudera.com/users/3656/vvaks.html" nodeid="3656">@Vadim</A> I have seen SparkSQL used in production for pulling RDBMS data. I have not seen it yet using kerberos environment, will follow thread for secure configurations.</P> Tue, 29 Mar 2016 23:14:30 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/Spark-SQL-as-a-Federated-DB-in-Production/m-p/147733#M23912 azeltov 2016-03-29T23:14:30Z Re: Spark SQL as a Federated DB in Production? https://community.cloudera.com/t5/Archives-of-Support-Questions/Spark-SQL-as-a-Federated-DB-in-Production/m-p/147734#M23913 <P><A href="https://community.hortonworks.com/questions/24935/spark-sql-as-a-federated-db-in-production.html#">@azeltov</A> How large is the implementation (How many tables, how much is cached vs read through to source)? In the case where they are caching the entire table, how do they ensure the data is not stale? Are the tables temporary or saved in the hive meta store?</P> Tue, 29 Mar 2016 23:27:17 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/Spark-SQL-as-a-Federated-DB-in-Production/m-p/147734#M23913 vvaks 2016-03-29T23:27:17Z