Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Hive Replication Failing with Invalid DFS Filename

avatar
Contributor

Could not find anything related to this or others that are experiencing the same issue.

 

java.lang.IllegalArgumentException: Pathname /user/hdfs/.cm/hive/2016-08-24-16-52-30+00:00-8590 from /user/hdfs/.cm/hive/2016-08-24-16-52-30+00:00-8590 is not a valid DFS filename.

 

I am using CDH 5.4.7.

I can't find a way to change the settings on how this task is generating the log name to remote the "+" from the name.

For now I am just going to use mysql function to backup and restore the Hive Metastore database on another cluster, to sync them.

 

Thanks,

/* Joey */

1 ACCEPTED SOLUTION

avatar
Master Guru

Hi Joey,

 

What you describe appears to be a known issue that is fixed in the latest versions of Cloudera Manager 5.5 and above.

If you are using Cloudera Manager 5.4.7 and JDK version 8, then this issue will occur.

The options are to upgrade to a 5.5 or higher Cloudera Manager

or

revert to using JDK 1.7 (for instance, 1.7.0_67)

 

Note that the JDK change will need to be on the host where the Hive Export command is running, so that means reverting to JDK 1.7 on the source cluster.

It may be a better idea, then, to upgrade Cloudera Manager to 5.8.x

 

Regards,

 

Ben

View solution in original post

3 REPLIES 3

avatar
Master Guru

Hi Joey,

 

What you describe appears to be a known issue that is fixed in the latest versions of Cloudera Manager 5.5 and above.

If you are using Cloudera Manager 5.4.7 and JDK version 8, then this issue will occur.

The options are to upgrade to a 5.5 or higher Cloudera Manager

or

revert to using JDK 1.7 (for instance, 1.7.0_67)

 

Note that the JDK change will need to be on the host where the Hive Export command is running, so that means reverting to JDK 1.7 on the source cluster.

It may be a better idea, then, to upgrade Cloudera Manager to 5.8.x

 

Regards,

 

Ben

avatar
Contributor

Thank you for the quick response Ben.

We will be upgrading soon, so it is good to know that this will be fixed then.

For now, I will just keep using native mysql commands for syncing.

 

Thanks,

/* Joey */

avatar
Master Guru

Joey,

 

That sounds great.  I encourage you to consider going to Cloudera Manager 5.8.x so that you can leverage the new Hive Incremental Replication feature.  Check it out here:

 

http://www.cloudera.com/documentation/enterprise/latest/topics/cm_bdr_hive_replication.html#concept_...

 

Cheers,

 

Ben