Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created 11-14-2016 09:12 PM
Is dynamic query rewrite avialable for hive as it is In HDP 2.5 dynamic query rewrite via Ranger for SparkSQL?
Created on 11-15-2016 03:56 PM - edited 08-18-2019 03:22 AM
@Sunile Manjee Answer is Yes. In HDP 2.5 Spark Column Security is available with LLAP and Ranger integration
You get Fine-Grained Column Level Access Control for SparkSQL. Fully dynamic policies per user. Doesn’t require views. Use Standard Ranger policies and tools to control access and masking policies.
Flow:
1.SparkSQL gets data locations known as “splits” from HiveServer and plans query.
2.HiveServer2 authorizes access using Ranger. Per-user policies like row filtering are applied.
3.Spark gets a modified query plan based on dynamic security policy.
4.Spark reads data from LLAP. Filtering / masking guaranteed by LLAP server.
Created 11-15-2016 12:37 AM
Ranger supports row-level filtering for Hive in 2.5, and accomplishes this by dynamically rewriting the query. I believe LLAP is a dependency for row-level filtering in SparkSQL.
Created on 11-15-2016 03:56 PM - edited 08-18-2019 03:22 AM
@Sunile Manjee Answer is Yes. In HDP 2.5 Spark Column Security is available with LLAP and Ranger integration
You get Fine-Grained Column Level Access Control for SparkSQL. Fully dynamic policies per user. Doesn’t require views. Use Standard Ranger policies and tools to control access and masking policies.
Flow:
1.SparkSQL gets data locations known as “splits” from HiveServer and plans query.
2.HiveServer2 authorizes access using Ranger. Per-user policies like row filtering are applied.
3.Spark gets a modified query plan based on dynamic security policy.
4.Spark reads data from LLAP. Filtering / masking guaranteed by LLAP server.