Here is the reference: https://falcon.apache.org/FalconNativeScheduler.html I am on HDP 2.3.2 which has Falcon 0.6. Is the version ready for Falcon native scheduler? If not, how about HDP 2.4.2? Currently native scheduler doesn't support Oracle. Will this be supported soon? ... View more

Where could I find a complete guide on Falcon configuration for HA services on both source and target clusters? So for NN HA, the solution is clear - define both clusters in hdfs-site.xml in source and target clusters. How about RM HA? With both RM1 and RM2 as active and passive, should I just pick one for source cluster such as in the following: https://community.hortonworks.com/questions/9416/falcon-with-ha-resource-manager.html <interface type="execute" endpoint="RM1:8050" version="2.2.0" /> What should I do for remote cluster RM? Just pick one? How do we define "hive.metastore.kerberos.principal" and "hive.metastore.uris" for multiple Hive metastore instances? How do we define for multiple oozie instances? ... View more

Is there a Hive policy that we can enable a user to read schema only without accessing the data? Since Ranger plugin is in HS2 only, could Hive metastore be a choice? ... View more

My cluster NN HA was implemented w/o securing the access to ZK. I am following the document to enable SASL: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.2/bk_installing_manually_book/content/hdfs_configuration.html Here is the existing Znode used by Hadoop NN for HA. I was ablet to setACL on /hadoop-ha to sasl:nn:rwcda, but haadmin failover command failed. [zk: zkserver1:2181(CONNECTED) 5] getAcl /hadoop-ha 'world,'anyone : cdrwa If I delete the old one, will NN automatically create a new Znode? I also tried to do "hdfs zkfc -formatZK" to create a new one, but the command was not using SASL. ... View more

When a user submits a query via Beeline, HS2 in turn runs the query with user hive as doas is set to false. How could a user kill the job? It should be a command via beeline to ask HS2 to kill on behalf of a user. Snapshot: XYZ $ yarn application -kill application_1461543063850_12340 16/05/16 11:56:13 INFO impl.TimelineClientImpl: Timeline service address: ... 16/05/16 11:56:13 INFO client.RMProxy: Connecting to ResourceManager at ... Killing application application_1461543063850_12340 Exception in thread "main" org.apache.hadoop.yarn.exceptions.YarnException: java.security.AccessControlException: User XYZ cannot perform operation MODIFY_APP on application_1461543063850_12340 ... View more

Here is what I found for YARN and HDFS to access ZK using SASL. YARN configuration makes perfect sense. However, is there something missing for HDFS such as configuring its own hdfs_jaas.conf? How about Hive? https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.2/bk_installing_manually_book/content/securing_zookeeper_access.html ... View more