Created 10-17-2023 06:29 AM
Hello Team,
I am trying to run beeline commandline to export the hive table to a location. i am trying to set replication factor to 1 but it if failing with following error :
Error: Error while processing statement: Cannot modify dfs.replication at runtime. It is not in list of params that are allowed to be modified at runtime (state=42000,code=1)
Cluster details :
3 nodes cluster
and in all hvie-site.xml i have set the following property and then restarted hive and hiveserver2
<property>
<name>hive.security.authorization.sqlstd.confwhitelist.append</name>
<value>mapred.*|hive.*|mapreduce.*|spark.*|dfs.*</value>
</property>
Beeline Command :
sudo -u hive beeline -u "jdbc:hive2://machine1.dev.domain.com:2181/default;password=hive;principal=hive/_HOST@DEV.domain.COM;serviceDiscoveryMode=zooKeeper;ssl=true;sslTrustStore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks;trustStorePassword=****;user=hive;zooKeeperNamespace=hiveserver2" -n hive--showHeader=false --outputformat=tsv2 -e "Set dfs.replication=1; use testdb; export table newt1 to '/staging/exporttable/testdb/newt1';"
Created 10-18-2023 11:14 PM
@jayes You will need to use --hiveconf to specify the hive configuration in the jdbc url.
The below example command can be used to set the replication factor to 1.
beeline -u "jdbc:hive2://machine1.dev.domain.com:2181/default" --hiveconf dfs.replication=1 -n hive -p hive
Check the replication factor :
> set dfs.replication;
You can modify your beeline command accordingly.
Created 10-30-2023 11:05 PM
Hi @rki_ ,
i tired that but still it is failing with same error.
sudo -u hive beeline -u "jdbc:hive2://machine1.dev.domain.com:2181/default;password=hive;principal=hive/_HOST@DEV.domain.COM;serviceDiscoveryMode=zooKeeper;ssl=true;sslTrustStore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks;trustStorePassword=****;user=hive;zooKeeperNamespace=hiveserver2" --hiveconf dfs.replication=1 -n hive--showHeader=false --outputformat=tsv2 -e "use testdb; export table newt1 to '/staging/exporttable/testdb/newt1';"
Error :
23/10/31 02:01:14 [main]: ERROR jdbc.Utils: Unable to read HiveServer2 configs from ZooKeeper
Error: Could not open client transport for any of the Server URI's in ZooKeeper: Failed to open new session: java.lang.IllegalArgumentException: Cannot modify dfs.replication at runtime. It is not in list of params that are allowed to be modified at runtime (state=08S01,code=0)