question Re: Hive Replication Failing with Invalid DFS Filename in Archives of Support Questions (Read Only)

Hive Replication Failing with Invalid DFS Filename

Joey_Krabacher — Fri, 16 Sep 2022 10:36:25 GMT

Could not find anything related to this or others that are experiencing the same issue.

java.lang.IllegalArgumentException: Pathname /user/hdfs/.cm/hive/2016-08-24-16-52-30+00:00-8590 from /user/hdfs/.cm/hive/2016-08-24-16-52-30+00:00-8590 is not a valid DFS filename.

I am using CDH 5.4.7.

I can't find a way to change the settings on how this task is generating the log name to remote the "+" from the name.

For now I am just going to use mysql function to backup and restore the Hive Metastore database on another cluster, to sync them.

Thanks,

/* Joey */

Re: Hive Replication Failing with Invalid DFS Filename

bgooley — Wed, 24 Aug 2016 17:46:02 GMT

Hi Joey,

What you describe appears to be a known issue that is fixed in the latest versions of Cloudera Manager 5.5 and above.

If you are using Cloudera Manager 5.4.7 and JDK version 8, then this issue will occur.

The options are to upgrade to a 5.5 or higher Cloudera Manager

revert to using JDK 1.7 (for instance, 1.7.0_67)

Note that the JDK change will need to be on the host where the Hive Export command is running, so that means reverting to JDK 1.7 on the source cluster.

It may be a better idea, then, to upgrade Cloudera Manager to 5.8.x

Regards,

Ben

Re: Hive Replication Failing with Invalid DFS Filename

Joey_Krabacher — Wed, 24 Aug 2016 17:51:38 GMT

Thank you for the quick response Ben.

We will be upgrading soon, so it is good to know that this will be fixed then.

For now, I will just keep using native mysql commands for syncing.

Thanks,

/* Joey */

Re: Hive Replication Failing with Invalid DFS Filename

bgooley — Wed, 24 Aug 2016 18:06:31 GMT

Joey,

That sounds great. I encourage you to consider going to Cloudera Manager 5.8.x so that you can leverage the new Hive Incremental Replication feature. Check it out here:

http://www.cloudera.com/documentation/enterprise/latest/topics/cm_bdr_hive_replication.html#concept_tzh_dwq_fw

Cheers,

Ben