Support Questions
Find answers, ask questions, and share your expertise

configure hive.auto.convert.join.noconditionaltask.size via Ambari is not visible in beeline client?

screenshot-from-2017-10-11-11-55-06.png

Hi,

Parameters configured via Ambari is not applied. Why?

Via Amabari It was configured hive.auto.convert.join.noconditionaltask.size=2600000

Connected to: Apache Hive (version 2.1.0.2.6.1.0-129)
Driver: Hive JDBC (version 1.2.1000.2.6.1.0-129)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 1.2.1000.2.6.1.0-129 by Apache Hive
0: jdbc:hive2://ip-172-31-35-100.us-west-2.co> set hive.auto.convert.join.noconditionaltask.size;
+----------------------------------------------------------+--+
|                           set                            |
+----------------------------------------------------------+--+
| hive.auto.convert.join.noconditionaltask.size=858783744  |
+----------------------------------------------------------+--+
    <property>
      <name>hive.auto.convert.join.noconditionaltask</name>
      <value>true</value>
    </property>
   
    <property>
      <name>hive.auto.convert.join.noconditionaltask.size</name>
      <value>858783744</value>
    </property>
   

Thank you,

1 ACCEPTED SOLUTION

Accepted Solutions

Re: configure hive.auto.convert.join.noconditionaltask.size via Ambari is not visible in beeline client?

@Yevgen Shramko,

For LLAP server , this should be configured under Advanced hive-interactive site.

Check screenshot attached

Thanks,

Aditya

40789-screen-shot-2017-10-11-at-53426-pm.png

View solution in original post

5 REPLIES 5

Re: configure hive.auto.convert.join.noconditionaltask.size via Ambari is not visible in beeline client?

Super Mentor

@Yevgen Shramko

I tried the same on HDP 2.6.1 (Ambari 2.5.1) and i can see that the changes are getting reflected after making the changes via Ambari When we do "Restart All Required" services.

Example:

Connected to: Apache Hive (version 1.2.1000.2.6.1.0-129)
Driver: Hive JDBC (version 1.2.1000.2.6.1.0-129)
Transaction isolation: TRANSACTION_REPEATABLE_READ
1: jdbc:hive2://amb25102.example.com:2181,amb> set hive.auto.convert.join.noconditionaltask.size;
+----------------------------------------------------------+--+
|                           set                            |
+----------------------------------------------------------+--+
| hive.auto.convert.join.noconditionaltask.size=333333333  |
+----------------------------------------------------------+--+
1 row selected (0.302 seconds)
1: jdbc:hive2://amb25102.example.com:2181,amb> 

.

39773-beeline-issue.png

.

Please check if the client configs are reflected properly or not? And the host which is running the beeline is synced with Ambari (means ambari agent is running fine on that host and able to communication with Ambari)

- Also most important "Restart All Required" is executed".

Re: configure hive.auto.convert.join.noconditionaltask.size via Ambari is not visible in beeline client?

Super Mentor

@Yevgen Shramko

Also please check the if you are looking at the right file:

[root@amb25103 etc]#  find /etc -name "hive-site.xml" | xargs grep -A 3 -i "hive.auto.convert.join.noconditionaltask.size"
/etc/hive/conf.backup/hive-site.xml:      <name>hive.auto.convert.join.noconditionaltask.size</name>
/etc/hive/conf.backup/hive-site.xml-      <value>286331153</value>
/etc/hive/conf.backup/hive-site.xml-    </property>
/etc/hive/conf.backup/hive-site.xml-    
--
/etc/hive/2.6.1.0-129/0/conf.server/hive-site.xml:      <name>hive.auto.convert.join.noconditionaltask.size</name>
/etc/hive/2.6.1.0-129/0/conf.server/hive-site.xml-      <value>333333333</value>
/etc/hive/2.6.1.0-129/0/conf.server/hive-site.xml-    </property>
/etc/hive/2.6.1.0-129/0/conf.server/hive-site.xml-    
--
/etc/hive/2.6.1.0-129/0/hive-site.xml:      <name>hive.auto.convert.join.noconditionaltask.size</name>
/etc/hive/2.6.1.0-129/0/hive-site.xml-      <value>333333333</value>
/etc/hive/2.6.1.0-129/0/hive-site.xml-    </property>
/etc/hive/2.6.1.0-129/0/hive-site.xml-    

<br>

.

And the Symlinks are correct:

[root@amb25103 etc]# ls -l /etc/hive
total 4
drwxr-xr-x. 3 root root   14 Jun  9 11:16 2.6.1.0-129
lrwxrwxrwx. 1 root root   33 Jun  9 11:16 conf -> /usr/hdp/current/hive-client/conf
drwxr-xr-x. 3 root root 4096 Jun  9 11:16 conf.backup

[root@amb25103 etc]# ls -l /usr/hdp/current/hive-client/conf
lrwxrwxrwx. 1 root root 23 Jun  9 11:16 /usr/hdp/current/hive-client/conf -> /etc/hive/2.6.1.0-129/0

.

Re: configure hive.auto.convert.join.noconditionaltask.size via Ambari is not visible in beeline client?

@Jay SenSharma

Ok. It looks strange as for me.

1. The one is Correct.

$ beeline -u "jdbc:hive2://ip-172-31-35-100.us-west-2.compute.internal:2181,ip-172-31-34-50.us-west-2.compute.internal:2181,ip-172-31-34-136.us-west-2.compute.internal:2181/bench_mtu_p;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2" -n hive -p admin

Connected to: Apache Hive (version 1.2.1000.2.6.1.0-129)
Driver: Hive JDBC (version 1.2.1000.2.6.1.0-129)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 1.2.1000.2.6.1.0-129 by Apache Hive
0: jdbc:hive2://ip-172-31-35-100.us-west-2.co> set hive.auto.convert.join.noconditionaltask.size;
+--------------------------------------------------------+--+
|                          set                           |
+--------------------------------------------------------+--+
| hive.auto.convert.join.noconditionaltask.size=2600000  |
+--------------------------------------------------------+--+ 

2. But I would like to change here. As I understood this is LLAP server

$ beeline -u "jdbc:hive2://ip-172-31-35-100.us-west-2.compute.internal:2181,ip-172-31-34-50.us-west-2.compute.internal:2181,ip-172-31-34-136.us-west-2.compute.internal:2181/bench_mtu_p;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2-hive2" -n hive -p admin

Connected to: Apache Hive (version 2.1.0.2.6.1.0-129)
Driver: Hive JDBC (version 1.2.1000.2.6.1.0-129)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 1.2.1000.2.6.1.0-129 by Apache Hive
0: jdbc:hive2://ip-172-31-35-100.us-west-2.co> set hive.auto.convert.join.noconditionaltask.size;
+----------------------------------------------------------+--+
|                           set                            |
+----------------------------------------------------------+--+
| hive.auto.convert.join.noconditionaltask.size=858783744  |
+----------------------------------------------------------+--+

Re: configure hive.auto.convert.join.noconditionaltask.size via Ambari is not visible in beeline client?

@Yevgen Shramko,

For LLAP server , this should be configured under Advanced hive-interactive site.

Check screenshot attached

Thanks,

Aditya

40789-screen-shot-2017-10-11-at-53426-pm.png

View solution in original post

Re: configure hive.auto.convert.join.noconditionaltask.size via Ambari is not visible in beeline client?

@Aditya Sirna That's it. Thank you so much.