Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

BDR - Hive Replication Log Directory

Solved Go to solution

BDR - Hive Replication Log Directory

Explorer

I'm having issues with running Hive replication jobs (these worked previously in the past), but due to some unknown system/configuration changes these jobs are now aborting almost immediately in the "Export Remote Hive Metastore" phase.  I've been hunting around on both the source and target clusters and I'm unable to find any trace of log files for this job.  Does anyone know where I should be looking for this information?

 

So far I've looked in:

  /var/log/hive on local filesystems where Hive Metastore and Hive Server are running

  /user/hdfs/.cm/hive on target cluster

  /var/run/cloudera-scm-agent/process/* on all nodes in both clusters

1 ACCEPTED SOLUTION

Accepted Solutions

Re: BDR - Hive Replication Log Directory

Explorer

I found out what I was doing wrong - I had a node on the source cluster with the Hive gateway role installed, but it wasn't configured 100% correctly.  For some reason when the BDR jobs were launched, they kept running on this node and immediately failing so I wasn't getting any errors.  The export metastore step of the Hive Replication job will run on a source cluster node that has either the HiveServer or Hive Gateway role installed

5 REPLIES 5

Re: BDR - Hive Replication Log Directory

Explorer

I found out what I was doing wrong - I had a node on the source cluster with the Hive gateway role installed, but it wasn't configured 100% correctly.  For some reason when the BDR jobs were launched, they kept running on this node and immediately failing so I wasn't getting any errors.  The export metastore step of the Hive Replication job will run on a source cluster node that has either the HiveServer or Hive Gateway role installed

Highlighted

Re: BDR - Hive Replication Log Directory

Community Manager

Congratulations on solving your issue TylerHale. Hopefully your solution will be able to help others who run into the same problem. Thanks for sharing! :)



Cy Jervis, Community Manager

Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community:
Community Guidelines
How to use the forum

Re: BDR - Hive Replication Log Directory

Explorer

Did you find the logs in the end ?  I cant find them myself

Re: BDR - Hive Replication Log Directory

Super Guru

Hi @Daggers ,

 

For which logs are you looking?

Hive Metadata Export generally happens on the *source* cluster, so you should be able to find the command and associated logs by logging into the source cluster's Cloudera Manager, navigating to the Hive service and clicking the "Commands" subtab.  You can then look at the results of the command for more information.  If the command contains stderr/stdout logs, the top few lines should include the host on which the command ran, too.

Re: BDR - Hive Replication Log Directory

Explorer

Thanks for that