Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Unable to get the record count into the variable after Kerberos Authentication implementation and Ranger Installation

avatar
Contributor

Hi everyone,

 

We are unable to get the record count into the variable  after Kerberos Authentication implementation and Ranger Installation. we are using following command but it's showing count by printing. Following is we used command

I need to get the count into the variable. anyone please help me!

 

count=$(beeline -u "jdbc:hive2://dev-lisa.realm.com:10000/default;principal=hive/dev-lisa.intranet.slt.com.lk@REALM.COM;ssl=true;sslTrustStore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks" -n hadoopuser02 -e 'SELECT count(*) from table_name; ')

1 ACCEPTED SOLUTION

avatar
Super Collaborator
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
7 REPLIES 7

avatar

Hi @Choolake , sorry, maybe it is not clear only for me, have you executed the beeline command alone? What do you exactly get when you do so? 


beeline -u "jdbc:hive2://<lb_or_hs2_hostname>:10000/default;principal=hive/<lbhostname_if_lb_is_enabled_otherwise_hs2_hostname>@REALM.COM;ssl=true;sslTrustStore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks" -e 'SELECT count(*) from table_name; '

 

just to understand if you get any errors. Then repeat with redirecting the stderr to /dev/null

 

beeline -u "jdbc:hive2://<lb_or_hs2_hostname>:10000/default;principal=hive/<lbhostname_if_lb_is_enabled_otherwise_hs2_hostname>@REALM.COM;ssl=true;sslTrustStore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks" -e 'SELECT count(*) from table_name; ' 2>/dev/null

 

Thanks

 Miklos

avatar
Contributor

Hi,@mszurap

Thanks for the reply.!

I not got any errors after end the commands.it was just printing the result as the following.

+-----------+
| _c0 |
+-----------+
| 84300000 |
+-----------+

I need to get the record count into the variable of "count" as following

count=84300000

 

avatar
Cloudera Employee

Just alias the column name.  With aggregations any output automatically gives a value of c + Index number.  In this case it is c0 as it's only one aggregate column.  An example of a query with the above output as count with the number under it is SELECT count)(*) as count from <TABLE_NAME> should give you the c0 column listed as count.  If you are looking to store the output as a variable as a local variable in you could do something like:

count=`hive -S -e "select count(*) from <TABLE_NAME>;"`
echo $count

 

avatar
Super Collaborator

Hi @Choolake 

You can use it as below. 

[root@c3910-node2 1546341386-hive-HIVEMETASTORE]# count=`beeline -u "jdbc:hive2://c3910-node3.coelab.cloudera.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;sslTrustStore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks;trustStorePassword=lpsZVjarkhv7WPONRXNCVscCkntpfjTtilIcrZH8NeP" -e "select count(*) from default.test"`
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-7.1.8-1.cdh7.1.8.p35.41426564/jars/log4j-slf4j-impl-2.18.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-7.1.8-1.cdh7.1.8.p35.41426564/jars/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
WARNING: Use "yarn jar" to launch YARN applications.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-7.1.8-1.cdh7.1.8.p35.41426564/jars/log4j-slf4j-impl-2.18.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-7.1.8-1.cdh7.1.8.p35.41426564/jars/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Connecting to jdbc:hive2://c3910-node3.coelab.cloudera.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;sslTrustStore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks;trustStorePassword=lpsZVjarkhv7WPONRXNCVscCkntpfjTtilIcrZH8NeP
23/07/13 12:56:03 [main]: INFO jdbc.HiveConnection: Connected to c3910-node3.coelab.cloudera.com:10000
Connected to: Apache Hive (version 3.1.3000.7.1.8.35-1)
Driver: Hive JDBC (version 3.1.3000.7.1.8.35-1)
Transaction isolation: TRANSACTION_REPEATABLE_READ
INFO : Compiling command(queryId=hive_20230713125603_1478c0aa-6c44-42b2-a78f-b323ac872df8): select count(*) from default.test
INFO : Semantic Analysis Completed (retrial = false)
INFO : Created Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:bigint, comment:null)], properties:null)
INFO : Completed compiling command(queryId=hive_20230713125603_1478c0aa-6c44-42b2-a78f-b323ac872df8); Time taken: 0.103 seconds
INFO : Executing command(queryId=hive_20230713125603_1478c0aa-6c44-42b2-a78f-b323ac872df8): select count(*) from default.test
INFO : Completed executing command(queryId=hive_20230713125603_1478c0aa-6c44-42b2-a78f-b323ac872df8); Time taken: 0.01 seconds
INFO : OK
1 row selected (0.245 seconds)
Beeline version 3.1.3000.7.1.8.35-1 by Apache Hive
Closing: 0: jdbc:hive2://c3910-node3.coelab.cloudera.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;sslTrustStore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks;trustStorePassword=lpsZVjarkhv7WPONRXNCVscCkntpfjTtilIcrZH8NeP
[root@c3910-node2 1546341386-hive-HIVEMETASTORE]# echo $count
+------+ | _c0 | +------+ | 1 | +------+
[root@c3910-node2 1546341386-hive-HIVEMETASTORE]#

You might need to further put some formatting on the "count" variable, as that contains the entire output of the select query.

 

Let me know if this helps!

 

Cheers!

Let me know if this helps.

avatar
Contributor

Thank you!

avatar
Super Collaborator
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar
Community Manager

@Choolake, Thank you for your participation in Cloudera Community. I'm happy to see you resolved your issue. Please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.

 



Regards,

Vidya Sargur,
Community Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community: