question Re: configure hive.auto.convert.join.noconditionaltask.size via Ambari is not visible in beeline client? in Archives of Support Questions (Read Only)

configure hive.auto.convert.join.noconditionaltask.size via Ambari is not visible in beeline client?

yevgen_shramko — Wed, 11 Oct 2017 17:34:47 GMT

screenshot-from-2017-10-11-11-55-06.png

Hi,

Parameters configured via Ambari is not applied. Why?

Via Amabari It was configured hive.auto.convert.join.noconditionaltask.size=2600000

Connected to: Apache Hive (version 2.1.0.2.6.1.0-129)
Driver: Hive JDBC (version 1.2.1000.2.6.1.0-129)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 1.2.1000.2.6.1.0-129 by Apache Hive
0: jdbc:hive2://ip-172-31-35-100.us-west-2.co> set hive.auto.convert.join.noconditionaltask.size;
+----------------------------------------------------------+--+
|                           set                            |
+----------------------------------------------------------+--+
| hive.auto.convert.join.noconditionaltask.size=858783744  |
+----------------------------------------------------------+--+

    <property>
      <name>hive.auto.convert.join.noconditionaltask</name>
      <value>true</value>
    </property>
   
    <property>
      <name>hive.auto.convert.join.noconditionaltask.size</name>
      <value>858783744</value>
    </property>

Thank you,

Re: configure hive.auto.convert.join.noconditionaltask.size via Ambari is not visible in beeline client?

jsensharma — Sun, 18 Aug 2019 04:00:18 GMT

@Yevgen Shramko

I tried the same on HDP 2.6.1 (Ambari 2.5.1) and i can see that the changes are getting reflected after making the changes via Ambari When we do "Restart All Required" services.

Example:

Connected to: Apache Hive (version 1.2.1000.2.6.1.0-129)
Driver: Hive JDBC (version 1.2.1000.2.6.1.0-129)
Transaction isolation: TRANSACTION_REPEATABLE_READ
1: jdbc:hive2://amb25102.example.com:2181,amb> set hive.auto.convert.join.noconditionaltask.size;
+----------------------------------------------------------+--+
|                           set                            |
+----------------------------------------------------------+--+
| hive.auto.convert.join.noconditionaltask.size=333333333  |
+----------------------------------------------------------+--+
1 row selected (0.302 seconds)
1: jdbc:hive2://amb25102.example.com:2181,amb>

Please check if the client configs are reflected properly or not? And the host which is running the beeline is synced with Ambari (means ambari agent is running fine on that host and able to communication with Ambari)

- Also most important "Restart All Required" is executed".

Re: configure hive.auto.convert.join.noconditionaltask.size via Ambari is not visible in beeline client?

jsensharma — Wed, 11 Oct 2017 17:57:06 GMT

@Yevgen Shramko

Also please check the if you are looking at the right file:

[root@amb25103 etc]#  find /etc -name "hive-site.xml" | xargs grep -A 3 -i "hive.auto.convert.join.noconditionaltask.size"
/etc/hive/conf.backup/hive-site.xml:      <name>hive.auto.convert.join.noconditionaltask.size</name>
/etc/hive/conf.backup/hive-site.xml-      <value>286331153</value>
/etc/hive/conf.backup/hive-site.xml-    </property>
/etc/hive/conf.backup/hive-site.xml-    
--
/etc/hive/2.6.1.0-129/0/conf.server/hive-site.xml:      <name>hive.auto.convert.join.noconditionaltask.size</name>
/etc/hive/2.6.1.0-129/0/conf.server/hive-site.xml-      <value>333333333</value>
/etc/hive/2.6.1.0-129/0/conf.server/hive-site.xml-    </property>
/etc/hive/2.6.1.0-129/0/conf.server/hive-site.xml-    
--
/etc/hive/2.6.1.0-129/0/hive-site.xml:      <name>hive.auto.convert.join.noconditionaltask.size</name>
/etc/hive/2.6.1.0-129/0/hive-site.xml-      <value>333333333</value>
/etc/hive/2.6.1.0-129/0/hive-site.xml-    </property>
/etc/hive/2.6.1.0-129/0/hive-site.xml-    

<br>

And the Symlinks are correct:

[root@amb25103 etc]# ls -l /etc/hive
total 4
drwxr-xr-x. 3 root root   14 Jun  9 11:16 2.6.1.0-129
lrwxrwxrwx. 1 root root   33 Jun  9 11:16 conf -> /usr/hdp/current/hive-client/conf
drwxr-xr-x. 3 root root 4096 Jun  9 11:16 conf.backup

[root@amb25103 etc]# ls -l /usr/hdp/current/hive-client/conf
lrwxrwxrwx. 1 root root 23 Jun  9 11:16 /usr/hdp/current/hive-client/conf -> /etc/hive/2.6.1.0-129/0

Re: configure hive.auto.convert.join.noconditionaltask.size via Ambari is not visible in beeline client?

yevgen_shramko — Wed, 11 Oct 2017 18:34:53 GMT

@Jay SenSharma

Ok. It looks strange as for me.

1. The one is Correct.

$ beeline -u "jdbc:hive2://ip-172-31-35-100.us-west-2.compute.internal:2181,ip-172-31-34-50.us-west-2.compute.internal:2181,ip-172-31-34-136.us-west-2.compute.internal:2181/bench_mtu_p;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2" -n hive -p admin

Connected to: Apache Hive (version 1.2.1000.2.6.1.0-129)
Driver: Hive JDBC (version 1.2.1000.2.6.1.0-129)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 1.2.1000.2.6.1.0-129 by Apache Hive
0: jdbc:hive2://ip-172-31-35-100.us-west-2.co> set hive.auto.convert.join.noconditionaltask.size;
+--------------------------------------------------------+--+
|                          set                           |
+--------------------------------------------------------+--+
| hive.auto.convert.join.noconditionaltask.size=2600000  |
+--------------------------------------------------------+--+

2. But I would like to change here. As I understood this is LLAP server

Connected to: Apache Hive (version 2.1.0.2.6.1.0-129)
Driver: Hive JDBC (version 1.2.1000.2.6.1.0-129)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 1.2.1000.2.6.1.0-129 by Apache Hive
0: jdbc:hive2://ip-172-31-35-100.us-west-2.co> set hive.auto.convert.join.noconditionaltask.size;
+----------------------------------------------------------+--+
|                           set                            |
+----------------------------------------------------------+--+
| hive.auto.convert.join.noconditionaltask.size=858783744  |
+----------------------------------------------------------+--+

Re: configure hive.auto.convert.join.noconditionaltask.size via Ambari is not visible in beeline client?

asirna — Sun, 18 Aug 2019 04:00:10 GMT

@Yevgen Shramko,

For LLAP server , this should be configured under Advanced hive-interactive site.

Check screenshot attached

Thanks,

Aditya

Re: configure hive.auto.convert.join.noconditionaltask.size via Ambari is not visible in beeline client?

yevgen_shramko — Wed, 11 Oct 2017 19:38:09 GMT

@Aditya Sirna That's it. Thank you so much.