Reply
Contributor
Posts: 126
Registered: ‎10-15-2014

WARN [main] conf.HiveConf (HiveConf.java:initialize(1488))

Hive 0.13 from CDH 5.2.0 has moved error below from STDERR to STDOUT:

2014-10-16 15:08:08,143 WARN  [main] conf.HiveConf (HiveConf.java:initialize(1488)) - DEPRECATED: Configuration property hive.metastore.local no longer has any effect. Make sure to provide a valid value for hive.metastore.uris if you are connecting to a remote metastore.


Does CM has any plan removing hive.metastore.local for hive when deploying client configurations?

in /etc/hive/conf/hive-site.xml

  <property>
    <name>hive.metastore.local</name>
    <value>false</value>
  </property>

 

Can manually remove from each node

Explorer
Posts: 19
Registered: ‎12-18-2014

Re: WARN [main] conf.HiveConf (HiveConf.java:initialize(1488))

 

I have done the same thing, which is removing it manually from each hive nodes.

This is really annoying because the hive 'WARNINGS' produced by that particular messages goes to STDOUT (not to STDERR).

 

So if you are doing something like hive -e "somequery" > something.csv

the warning error msg will go to the something.csv

 

this is confirmed on CDH 5.3.0

Cloudera Employee
Posts: 26
Registered: ‎08-06-2013

Re: WARN [main] conf.HiveConf (HiveConf.java:initialize(1488))

Hi,

Can you create a DISTRO jira for this on issues.cloudera.org? CM and CDH versions along with any custom logging config you've added would be great.

In the meantime I have created HIVE-9221 and place a patch there to remove this old warning.

Brock
Explorer
Posts: 19
Registered: ‎12-18-2014

Re: WARN [main] conf.HiveConf (HiveConf.java:initialize(1488))

ok, i have created the distro: https://issues.cloudera.org/browse/DISTRO-677

Contributor
Posts: 126
Registered: ‎10-15-2014

Re: WARN [main] conf.HiveConf (HiveConf.java:initialize(1488))

Upgrading to CDH 5.3 makes it worse with more deprecated parameters

 

hadoop.ssl.require.client.cert;

mapreduce.job.end-notification.max.retry.interval;

hadoop.ssl.client.conf; 

hadoop.ssl.keystores.factory.class;

hadoop.ssl.server.conf;

mapreduce.job.end-notification.max.attempts;

 

I am guessing that if I enable security 4 of these will go away

 

 [main] conf.Configuration (Configuration.java:loadProperty(2510)) - file:/tmp//hive_2014-12-30_04-52-18_716_7093356830031101390-1/-local-10009/jobconf.xml:an attempt to override final parameter: hadoop.ssl.require.client.cert;  Ignoring.
2014-12-30 04:52:38,878 WARN  [main] conf.Configuration (Configuration.java:loadProperty(2510)) - file:/tmp//hive_2014-12-30_04-52-18_716_7093356830031101390-1/-local-10009/jobconf.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2014-12-30 04:52:38,879 WARN  [main] conf.Configuration (Configuration.java:loadProperty(2510)) - file:/tmp//hive_2014-12-30_04-52-18_716_7093356830031101390-1/-local-10009/jobconf.xml:an attempt to override final parameter: hadoop.ssl.client.conf;  Ignoring.
2014-12-30 04:52:38,881 WARN  [main] conf.Configuration (Configuration.java:loadProperty(2510)) - file:/tmp//hive_2014-12-30_04-52-18_716_7093356830031101390-1/-local-10009/jobconf.xml:an attempt to override final parameter: hadoop.ssl.keystores.factory.class;  Ignoring.
2014-12-30 04:52:38,886 WARN  [main] conf.Configuration (Configuration.java:loadProperty(2510)) - file:/tmp//hive_2014-12-30_04-52-18_716_7093356830031101390-1/-local-10009/jobconf.xml:an attempt to override final parameter: hadoop.ssl.server.conf;  Ignoring.
2014-12-30 04:52:38,907 WARN  [main] conf.Configuration (Configuration.java:loadProperty(2510)) - file:/tmp//hive_2014-12-30_04-52-18_716_7093356830031101390-1/-local-10009/jobconf.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2014-12-30 04:52:39,180 WARN  [main] conf.HiveConf (HiveConf.java:initialize(1507)) - DEPRECATED: Configuration property hive.metastore.local no longer has any effect. Make sure to provide a valid value for hive.metastore.uris if you are connecting to a remote metastore.
Execution log at: /tmp//ub

Posts: 1,886
Kudos: 425
Solutions: 300
Registered: ‎07-31-2013

Re: WARN [main] conf.HiveConf (HiveConf.java:initialize(1488))

Both of these issues have been resolved with the Cloudera Manager 5.3.0 release:

1. Hive gateway (/etc/hive/conf/) is missing a log4j.properties, causing its logging to go into stdout instead of stderr.
2. Hive unnecessarily carries a hive.metastore.local property for the gateway roles, which can be removed for all CDH5 users.

Until you can upgrade, the workaround for (1) is to do the below (after each time you deploy hive client configs):

cp /etc/hadoop/conf/log4j.properties /etc/hive/conf/

And the workaround for (2) is to add the below to the Hive Gateway Advanced Configuration Snippet for hive-site.xml (safety valve):

<property>
<name>hive.metastore.local</name>
<value></value>
</property>
Posts: 311
Kudos: 6
Solutions: 4
Registered: ‎08-14-2013

Re: WARN [main] conf.HiveConf (HiveConf.java:initialize(1488))

Just a note for those still seeing this message with the CM 5.3 release, that the fix for this has been moved up to the CM 5.4 release.  Harsh's workarounds still apply and work in the CM 5.3 release as well.

 

Thanks,

Robert Justice



Robert Justice, Technical Resolution Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community:

Terms of Service