How do we check if Hive is running in Http mode? I tried to search the same in hive-site.xml and found that http.mode <binary> be default. In this case , can you please help me with this issue? ... View more

Hello MD ,   Ranger doesn't lock the account, it is AD/LDAP setting to manage.  Basically, you need to check with the AD Admin regarding this policy management. When you give an incorrect password the BadPwdCnt parameter in AD Account Attribute will increase and the account will not lock till it reaches the Account Locked Threshold. Hope it helps! Thanks  Mayank Gupta   ... View more

[hdfs@c****-node* hive-testbench-hive14]$ ./tpcds-build.sh  Building TPC-DS Data Generator make: Nothing to be done for `all’. TPC-DS Data Generator built, you can now use tpcds-setup.sh to generate data. [hdfs@c4237-node2 hive-testbench-hive14]$ ./tpcds-setup.sh 2 TPC-DS text data generation complete. Loading text data into external tables. make: *** [time_dim] Error 1 make: *** Waiting for unfinished jobs.... make: *** [date_dim] Error 1 Data loaded into database tpcds_bin_partitioned_orc_2.   INFO : OK +---------------------+ | database_name | +---------------------+ | default | | information_schema | | sys | +---------------------+ 3 rows selected (1.955 seconds) 0: jdbc:hive2://c4237-node2.coelab.cloudera.c> tpcds_bin_partitioned_orc_2 database is not created, I have some issues in testing the tpcds queries   sudo -u hdfs -s    13   cd /home/hdfs    14   wget https://github.com/hortonworks/hive-testbench/archive/hive14.zip     15   unzip hive14.zip   17   export JAVA_HOME=/usr/jdk64/jdk1.8.0_77    18   export PATH=$JAVA_HOME/bin:$PATH ./tpcds-build.sh beeline -i testbench.settings -u "jdbc:hive2://c****-node9.coe***.*****.com:10500/tpcds_bin_partitioned_orc_2"   I'm not able to test the tpcds queries, any help would be appreciated. ... View more

Hello,    Do you have any test to be sure the cluster is working fine after all those steps ?  I move one node ever 3.  I put some files on the hdfs and i don't see file system in the dfs.journalnode.edits.dir even whether for the old or new Journal Node.    Best Regards Abdou      ... View more

Hi @Michael Dennis "MD" Uanang I have similar issue where i can't find VDF file for HDF 3.1.0 version for SLES12 Operating system Can you please help me to find VDF file ? My Goal is to upgrade the HDF version to 3.1.0 from 3..0.2. Basically i have two separate clusters - HDP and HDF. Each clusters is managed by Ambari (one Ambari to manage HDF and another Ambari to manage HDP) Steps to reproduce the issue: In the Versions tab in Ambari of HDF cluster, i clicked on "Register version" In the drop down > Select "Add Version" I will get a popup to upload VDF file as show below: VDF file for HDF 3.1. for SLES12 is not available in HDF 3.1 release notes https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.1.0/bk_release-notes/content/ch_hdf_relnotes.html Appreciate help to get me the VDF file for HDF 3.1. for SLES12 Thanks Chethan ... View more

@Michael Dennis "MD" Uanang was the above answer helpful to you? Please keep me posted. ... View more

Hi @Michael Dennis "MD" Uanang Could you briefly describe how you dealt with this problem? Regards ... View more

@muanang What can be the other reasons for this error. I faced the same issue as was using aliases. I corrected the same on all servers , stopped and restarted ambari-server but find the same issue ... View more

Hi @Michael Dennis "MD" Uanang Sure it is possible. You just need to customize your template. Regards ... View more

OK, after some digging: Plugin may be visibly enabled in ambari after blueprint install, but it won't be configured and available in Ranger GUI unless the "plugined" service is restarted. Moreover - keep in mind that if you set your own password for amb_ranger_admin user (for example using jinja2 template in blueprint) it has to follow required rules - at least 8 chars and at least one letter and one digit within. Unproperly set password was my case. If you are installing cluster remotely, those API calls below may be usefull for you (for example to call from ansible playbook). Example for HIVE (all components, all hosts). Put the service down curl -i -u admin:admin -H "X-Requested-By: ambari" -X PUT -d '{"RequestInfo":{"context":"stop hive","operation_level":{"level":"CLUSTER","cluster_name":"<cluster_name>"}},"Body":{"ServiceInfo":{"state":"INSTALLED"}}}' http://<ambari_host>:8080/api/v1/clusters/<cluster_name>/services/HIVE Check service state: curl --silent -u admin:admin -X GET http://<ambari_host>:8080/api/v1/clusters/<cluster_name>/services/HIVE?fields=ServiceInfo | grep '"state" :' Bring the service up: curl -i -u admin:admin -H "X-Requested-By: ambari" -X PUT -d '{"RequestInfo":{"context":"start hive","operation_level":{"level":"CLUSTER","cluster_name":"<cluster_name>"}},"Body":{"ServiceInfo":{"state":"STARTED"}}}' http://<ambari_host>:8080/api/v1/clusters/<cluster_name>/services/HIVE ... View more

Hi, @Michael Dennis "MD" Uanang, Did you solve this problem? I have the same issue after installing test cluster. Any help is appreciated. Thanks! HDP-2.6.5.0 ... View more

Will post later, on the road. 🙂 ... View more

Helped me big time! Thank you for sharing! Kafka all-the-way! ... View more

WOW. Thank you for laying it out for me. For #2, how can I recreate the view? Do I need to resolve issue #1 and #3 first? 🙂 ... View more

Anyone did follow this answer and still got the same issue? I added custom hbase-site in Ambari, but timezone wasn't changed. 😞 ... View more

Thanks @Rajeshbabu Chintaguntla After adding the hadoop.http.staticuser.user=yarn (and as also mentioned by @rguruvannagari), solved my issue. ... View more

The phoenix-sqlline command is not using PQS. You want to use /usr/hdp/current/phoenix-client/bin/sqlline-thin.py to interactive with PQS. ... View more

Well documented... Worked well with Postgres 9.4.7... Would like to add that during restore I receieved error message 'Unknown role ambari'. I manually added the role as below after step 7 in the Restore section. CREATE ROLE ambari WITH LOGIN PASSWORD 'bigdata'; ... View more

@Michael Dennis "MD" Uanang - Please accept appropriate answer to close this question. ... View more

Thanks @Kuldeep Kulkarni ... View more

Hi, Is there any steps to take our own kafka server metrics to elasticsearch..because we have grafana which will have all the dashboards but some of our project requirement need to keep few kafka metrics in kibana visualization.so we want to index the kafka metrics logs to elasticsearch. Can we consume kafka metrics into elasticsearch ? ... View more

check the property :kafka.metrics.reporters ... in my case it was blank ... i updated with 'org.apache.hadoop.metrics2.sink.kafka.KafkaTimelineMetricsReporter' and it worked.... ... View more

Thanks @Predrag Minovic! Will try this to our dev cluster. ... View more

As the answer suggests, you are out of memory. You could try breaking up your batch of data using one of the processors designed for splitting up batches. ... View more

if you continue having issues, consider opening a ticket with support. ... View more

Hi @Shishir Saxena I Have 3 brokers. Another info: When 2 brokers are down (logs with ISR Shrinking), the remaining broker will have OOM error. I adjusted the Heap to 2GB. ... View more

We had to add this to custom hive-site (custom hive-env does not work). Once added to custom hive-site, this worked fine. James ... View more