Reply
Explorer
Posts: 14
Registered: ‎08-12-2014
Accepted Solution

Hive Replication Failing with Invalid DFS Filename

Could not find anything related to this or others that are experiencing the same issue.

 

java.lang.IllegalArgumentException: Pathname /user/hdfs/.cm/hive/2016-08-24-16-52-30+00:00-8590 from /user/hdfs/.cm/hive/2016-08-24-16-52-30+00:00-8590 is not a valid DFS filename.

 

I am using CDH 5.4.7.

I can't find a way to change the settings on how this task is generating the log name to remote the "+" from the name.

For now I am just going to use mysql function to backup and restore the Hive Metastore database on another cluster, to sync them.

 

Thanks,

/* Joey */

Posts: 850
Topics: 1
Kudos: 197
Solutions: 105
Registered: ‎04-22-2014

Re: Hive Replication Failing with Invalid DFS Filename

Hi Joey,

 

What you describe appears to be a known issue that is fixed in the latest versions of Cloudera Manager 5.5 and above.

If you are using Cloudera Manager 5.4.7 and JDK version 8, then this issue will occur.

The options are to upgrade to a 5.5 or higher Cloudera Manager

or

revert to using JDK 1.7 (for instance, 1.7.0_67)

 

Note that the JDK change will need to be on the host where the Hive Export command is running, so that means reverting to JDK 1.7 on the source cluster.

It may be a better idea, then, to upgrade Cloudera Manager to 5.8.x

 

Regards,

 

Ben

Highlighted
Explorer
Posts: 14
Registered: ‎08-12-2014

Re: Hive Replication Failing with Invalid DFS Filename

Thank you for the quick response Ben.

We will be upgrading soon, so it is good to know that this will be fixed then.

For now, I will just keep using native mysql commands for syncing.

 

Thanks,

/* Joey */

Posts: 850
Topics: 1
Kudos: 197
Solutions: 105
Registered: ‎04-22-2014

Re: Hive Replication Failing with Invalid DFS Filename

Joey,

 

That sounds great.  I encourage you to consider going to Cloudera Manager 5.8.x so that you can leverage the new Hive Incremental Replication feature.  Check it out here:

 

http://www.cloudera.com/documentation/enterprise/latest/topics/cm_bdr_hive_replication.html#concept_...

 

Cheers,

 

Ben

Announcements