Hive Replication to New Cluster Does Not Appear in Impala

[#1]

I am testing Hive replications to a new DR cluster, and the databases are not visible to Impala.  What must be done to make the data accessible via Impala?

Thank you,

Michael Reynolds

[#2]

As a follow-up, we have CDH 5.12.2 on both clusters.  Under the replication "Advanced" tab, instead of seeing "Replicate Impala Metadata" as an option, I have "Invalidate Impala Metadata on Destination".  Checking this option appears to have no effect in making the data visible to Impala on the destination cluster.

Michael Reynolds

[#3]

@Urantian,

I would verify on the destination cluster that the replication-initiated "invalidate metadata" worked.

Try running it manually on the destination cluster and see if that works.

Note:

https://www.cloudera.com/documentation/enterprise/5-12-x/topics/cm_bdr_impala_metadata_replication.h...

[#4]

@bgooley,

Thank you for your reply!  If I run "invalidate metadata" on the destination, the databases appear after doing a refresh.

I tested a new replication, ensuring that "Invalidate Impala Metadata on Destination" was selected, and the data does not appear on the destination until I manually run "invalidate metadata".

Is there a way to ensure the invalidate command will work automatically via the replication job?

Michael Reynolds

[#5]

@bgooley,

I found what appears to be the reason why the INVALIDATE METADATA command is not working.

ERROR: AuthorizationException: User 'impala/<server>@<domain>' does not have privileges to access: <DB>.<TABLE>
Could not execute command: invalidate metadata <DB>.<TABLE>

It is failing to run the command.  However, after manually issuing the INVALIDATE METADATA command, Impala has no trouble accessing the table.

Michael Reynolds

[#6]

@Urantian,

Make sure you run the exact cmomand listed in the error after running kinit as the impala user.  Sentry is not seeing that the "impala" user has permission to invalidate metadata on that database and table.

Make sure that the "impala" user is a member of a group that is allowed to invalidate metadata.

BDR-initiated metadata validation requires that the "impala" user be able to invalidate metadata.

Sentry is denying that action in your cluster.