Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Not able to set dfs.replication while running export command for hive table in beeline command.

avatar
Contributor

Hello Team,

I am trying to run beeline commandline to export the hive table to a location. i am trying to set replication factor to 1 but it if failing with following error :
Error: Error while processing statement: Cannot modify dfs.replication at runtime. It is not in list of params that are allowed to be modified at runtime (state=42000,code=1)

Cluster details :
3 nodes cluster 
and in all hvie-site.xml i have set the following property and then restarted hive and hiveserver2 
<property>
<name>hive.security.authorization.sqlstd.confwhitelist.append</name>
<value>mapred.*|hive.*|mapreduce.*|spark.*|dfs.*</value>
</property>

Beeline Command : 

sudo -u hive beeline -u "jdbc:hive2://machine1.dev.domain.com:2181/default;password=hive;principal=hive/_HOST@DEV.domain.COM;serviceDiscoveryMode=zooKeeper;ssl=true;sslTrustStore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks;trustStorePassword=****;user=hive;zooKeeperNamespace=hiveserver2" -n hive--showHeader=false --outputformat=tsv2 -e "Set dfs.replication=1; use testdb; export table newt1 to '/staging/exporttable/testdb/newt1';"

2 REPLIES 2

avatar
Super Collaborator

@jayes You will need to use --hiveconf to specify the hive configuration in the jdbc url.

The below example command can be used to set the replication factor to 1. 

beeline -u "jdbc:hive2://machine1.dev.domain.com:2181/default" --hiveconf dfs.replication=1 -n hive -p hive

Check the replication factor :

> set dfs.replication;

You can modify your beeline command accordingly.

avatar
Contributor

Hi @rki_ ,

i tired that but still it is failing with same error.


sudo -u hive beeline -u "jdbc:hive2://machine1.dev.domain.com:2181/default;password=hive;principal=hive/_HOST@DEV.domain.COM;serviceDiscoveryMode=zooKeeper;ssl=true;sslTrustStore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks;trustStorePassword=****;user=hive;zooKeeperNamespace=hiveserver2"  --hiveconf dfs.replication=1  -n hive--showHeader=false --outputformat=tsv2 -e "use testdb; export table newt1 to '/staging/exporttable/testdb/newt1';"

Error : 

23/10/31 02:01:14 [main]: ERROR jdbc.Utils: Unable to read HiveServer2 configs from ZooKeeper
Error: Could not open client transport for any of the Server URI's in ZooKeeper: Failed to open new session: java.lang.IllegalArgumentException: Cannot modify dfs.replication at runtime. It is not in list of params that are allowed to be modified at runtime (state=08S01,code=0)