Hi @Wes Floyd, support for HS2 HA Zookeeper discovery is also available in Knox-0.7, but the version of Knox packaged in the latest HDP-2.4 is still 0.6, and so it's not yet availalbe in HDP. You can consider to install it yourself. ... View more

There is no option to upgrade only individual components in Ambari, like HBase, and it's intentional because each distribution of HDP is created and tested so that components are compatible with each other. So, if you want to upgrade only your custom service I think you need to follow the manual procedure you suggested. ... View more

It can be done by extending the OutputFormat class and overwriting the OutputFormat.checkOutputSpecs method so that it doesn't throw exception when the output path already exists. After that, register the new class using JobConf.setOutputFormatClass method [some more details here]. ... View more

Hi @JOSE GUILLEN, this happens because Flume keeps on writing in a .tmp temporary file, and when it's full the file is renamed so that .tmp suffix is dropped. So, when you start your Hive script it may be there, but when the time comes for Hive to read it, the file may have already been renamed. There is a Flume JIRA FLUME-2458 created to separate tmp files to another directory but it's not resolved yet. In the meanwhile you can try to use a workaround described here by setting hdfs.filePrefix and hdfs.inUsePrefix in your Flume conf file, for example hdfs.path=/user/flume/twitter/landing hdfs.filePrefix=bod/ hdfs.inUsePrefix=tmp/ and pre-create /user/flume/twitter/landing/tmp/bod in HDFS where .tmp files will be stored (please test this since I don't have a Flume setup handy to try). Edit: It might be enough just to set "hdfs.inUsePrefix=.", in this way tmp files will be named like .file101.tmp and will be hidden from Hive. So please try this first, and if you still have issues then try the workaround above. ... View more

Hi @Madhavi Amirneni, Yes, it is possible to store audit records only in HDFS but they cannot be viewed through Ranger UI. The main reason is that search is not supported. To view records in UI, a DB or Solr have to be configured and ranger.audit.source.type set to either db or solr. By the way, audit records in HDFS are stored in text files, as Json objects, see a sample below (audit for HDFS), and can be explored using another tool. The directories are organized by day, for example: /ranger/audit/hdfs/20160404. {"repoType":1,"repo":"Sandbox_hadoop","reqUser":"oozie","evtTime":"2016-04-04 01:27:05.123","access":"READ_EXECUTE","resource":"/user/oozie/share/lib","resType":"path","result":1,"policy":7,"reason":"/user/oozie/share/lib","enforcer":"ranger-acl","cliIP":"10.0.2.15","agentHost":"sandbox.hortonworks.com","logType":"RangerAudit","id":"49abe678-ffa7-46cd-ba1f-de85368dd88c","seq_num":81811,"event_count":1,"event_dur_ms":0} ... View more