Support Questions
Find answers, ask questions, and share your expertise

Disabling SparkSQL in HDP 2.4.2 Or work around to secure databases

Expert Contributor

Hi All,

In our secure cluster, we have configured Spark. For PoC it ran fine, we could run SparkSQL, Spark-shell, could run queries using sqlContext.sql("mySQLHere").

Folloing a successful PoC we setup the Spark in 40 nodes production cluster, and just before unavailing the curtains of Spark mystery we learn that end users (after getting their own user kerb TGT) can get hold of sqlContext (in program or in shell) and can run any query against any secure databases.

We are looking forward to a solution or a work around,

1) By which we can give secure access only to the "deserving" users to "deserved" database(s).

2) If not, can we disable sparkSQLcontext/hiveContext and stop users from accessing it until Spark is ready for production?

Removing hive-site.xml from all the 40 nodes? entering wrong values for meta-store uris? Anything better?

Thanks in advance.


Expert Contributor