Support Questions

Find answers, ask questions, and share your expertise

Not able to set dfs.replication while running export command for hive table in beeline command.

avatar
Rising Star

Hello Team,

I am trying to run beeline commandline to export the hive table to a location. i am trying to set replication factor to 1 but it if failing with following error :
Error: Error while processing statement: Cannot modify dfs.replication at runtime. It is not in list of params that are allowed to be modified at runtime (state=42000,code=1)

Cluster details :
3 nodes cluster 
and in all hvie-site.xml i have set the following property and then restarted hive and hiveserver2 
<property>
<name>hive.security.authorization.sqlstd.confwhitelist.append</name>
<value>mapred.*|hive.*|mapreduce.*|spark.*|dfs.*</value>
</property>

Beeline Command : 

sudo -u hive beeline -u "jdbc:hive2://machine1.dev.domain.com:2181/default;password=hive;principal=hive/_HOST@DEV.domain.COM;serviceDiscoveryMode=zooKeeper;ssl=true;sslTrustStore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks;trustStorePassword=****;user=hive;zooKeeperNamespace=hiveserver2" -n hive--showHeader=false --outputformat=tsv2 -e "Set dfs.replication=1; use testdb; export table newt1 to '/staging/exporttable/testdb/newt1';"

2 REPLIES 2

avatar
Super Collaborator

@jayes You will need to use --hiveconf to specify the hive configuration in the jdbc url.

The below example command can be used to set the replication factor to 1. 

beeline -u "jdbc:hive2://machine1.dev.domain.com:2181/default" --hiveconf dfs.replication=1 -n hive -p hive

Check the replication factor :

> set dfs.replication;

You can modify your beeline command accordingly.

avatar
Rising Star

Hi @rki_ ,

i tired that but still it is failing with same error.


sudo -u hive beeline -u "jdbc:hive2://machine1.dev.domain.com:2181/default;password=hive;principal=hive/_HOST@DEV.domain.COM;serviceDiscoveryMode=zooKeeper;ssl=true;sslTrustStore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks;trustStorePassword=****;user=hive;zooKeeperNamespace=hiveserver2"  --hiveconf dfs.replication=1  -n hive--showHeader=false --outputformat=tsv2 -e "use testdb; export table newt1 to '/staging/exporttable/testdb/newt1';"

Error : 

23/10/31 02:01:14 [main]: ERROR jdbc.Utils: Unable to read HiveServer2 configs from ZooKeeper
Error: Could not open client transport for any of the Server URI's in ZooKeeper: Failed to open new session: java.lang.IllegalArgumentException: Cannot modify dfs.replication at runtime. It is not in list of params that are allowed to be modified at runtime (state=08S01,code=0)