- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Hive Replication Failing with Invalid DFS Filename
Created on 08-24-2016 10:33 AM - edited 09-16-2022 03:36 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Could not find anything related to this or others that are experiencing the same issue.
java.lang.IllegalArgumentException: Pathname /user/hdfs/.cm/hive/2016-08-24-16-52-30+00:00-8590 from /user/hdfs/.cm/hive/2016-08-24-16-52-30+00:00-8590 is not a valid DFS filename.
I am using CDH 5.4.7.
I can't find a way to change the settings on how this task is generating the log name to remote the "+" from the name.
For now I am just going to use mysql function to backup and restore the Hive Metastore database on another cluster, to sync them.
Thanks,
/* Joey */
Created 08-24-2016 10:46 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Joey,
What you describe appears to be a known issue that is fixed in the latest versions of Cloudera Manager 5.5 and above.
If you are using Cloudera Manager 5.4.7 and JDK version 8, then this issue will occur.
The options are to upgrade to a 5.5 or higher Cloudera Manager
or
revert to using JDK 1.7 (for instance, 1.7.0_67)
Note that the JDK change will need to be on the host where the Hive Export command is running, so that means reverting to JDK 1.7 on the source cluster.
It may be a better idea, then, to upgrade Cloudera Manager to 5.8.x
Regards,
Ben
Created 08-24-2016 10:46 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Joey,
What you describe appears to be a known issue that is fixed in the latest versions of Cloudera Manager 5.5 and above.
If you are using Cloudera Manager 5.4.7 and JDK version 8, then this issue will occur.
The options are to upgrade to a 5.5 or higher Cloudera Manager
or
revert to using JDK 1.7 (for instance, 1.7.0_67)
Note that the JDK change will need to be on the host where the Hive Export command is running, so that means reverting to JDK 1.7 on the source cluster.
It may be a better idea, then, to upgrade Cloudera Manager to 5.8.x
Regards,
Ben
Created 08-24-2016 10:51 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thank you for the quick response Ben.
We will be upgrading soon, so it is good to know that this will be fixed then.
For now, I will just keep using native mysql commands for syncing.
Thanks,
/* Joey */
Created 08-24-2016 11:06 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Joey,
That sounds great. I encourage you to consider going to Cloudera Manager 5.8.x so that you can leverage the new Hive Incremental Replication feature. Check it out here:
Cheers,
Ben