Did this get sorted? I am still having to discuss with colleagues as to when you have to manually "invalidate metadata <tablename>" after a BDR run ... View more

Thanks for the information! ... View more

But with Informatica BDM you also have Blaze and Spark as alternative engines to Hive. The original post was quite vague and may not have been BDM related at all. ... View more

" UTC timezone conversion issue going on with only Parquet backed tables." > how do Cloudera Customers deal with this issue? I fear the first solution is to have all your servers use the same (UTC) timezone. We also have this flag setconvert_legacy_hive_parquet_utc_timestamps=true and hope to get rid of it once we move everything to UTC. ... View more

Hello bgooley, Thanks for your suggestion. I am re-reading the docs and I still think it tells me to add a CM peer any time I want to do a BDR replication, but I can accept that maybe my reading of the docs is wrong. I have been re-trying my tests without a peered CM but was not able to improve the situation. In the meantime we have taken a different track and started to use a new cluster as the target with a new Cloudera Manager and BDR seems to be working for that. ... View more

Are you using The BDR tool available with a cloudera Enterprise license?   If that is the case then you probably should be using that tool and two separate Hive Metastores. No DB copying required. (I am trying this now - I am no expert yet 🙂   If you don't then you might consider having a shared metastore. Does that work for you?   But finally if all you are doing is creating a Disaster Recovery type backup then I would assume you need all the tables in the Hive Metastore. But that is a guess. ... View more

Did you get this sorted?    I thought the idea was that they were supposed to be in the same realm.    Personally I am being told by the Cloudera Backup BDR tool that I have two different KDCs when I don't, AFAIK. You do! ... View more

> you are replicating from one Hive Service to another on the same cluster. No,, not the same cluster. Two different (but similarly named) clusters, one Cloudera Manager. ... View more

I can see records in the COMMANDS table with NAME HiveReplicationCommand appearing with STATE STARTED and then immediately STATE changes to FINISHED, but I cannot see why hive Replication sees this as not itself.   eg   COMMAND_ID bigint(20) NO PRI NULL NAME varchar(255) NO NULL STATE varchar(255) YES MUL NULL START_INSTANT bigint(20) YES MUL NULL END_INSTANT bigint(20) YES NULL ACTIVE int(11) YES MUL NULL RESULT_MESSAGE longtext YES NULL RESULT_DATA mediumblob YES NULL RESULT_DATA_MIME_TYPE varchar(255) YES NULL RESULT_DATA_FILENAME varchar(255) YES NULL SUCCESS bit(1) YES NULL SERVICE_ID bigint(20) YES MUL NULL ROLE_ID bigint(20) YES MUL NULL PARENT_ID bigint(20) YES MUL NULL HOST_ID bigint(20) YES MUL NULL RESULT_DATA_PATH varchar(255) YES NULL RESULT_DATA_REAPED bit(1) YES b'0' CLUSTER_ID bigint(20) YES MUL NULL OPTIMISTIC_LOCK_VERSION bigint(20) NO 0 SCHEDULE_ID bigint(20) YES MUL NULL ARGUMENTS longtext YES NULL AUDITED bit(1) NO b'0' FIRST_UPDATED_INSTANT bigint(20) YES NULL CREATION_INSTANT bigint(20) YES NULL 290498 HiveReplicationCommand STARTED 1532081148707 NULL 1 NULL NULL application/json summary.json \0 96 NULL NULL NULL /var/lib/cloudera-scm-server/commands/290498/summary9115221277369688254.json \0 NULL 9 1291 {"@class":"com.cloudera.cmf.service.hive.HiveReplicationCmdArgs","replicateData":true,"hdfsArguments":{"@class":"com.cloudera.cmf.service.hdfs.DistCpCommand$DistCpCommandArgs","alertConfig":null,"args":[],"atomic":false,"bandwidth":100,"copyListingOnSource":null,"delete":false,"destinationPath":"/bdr_test/hive_warehouse","diffRenameDeletePath":null,"dryRun":false,"exclusionFilters":[],"ignoreFailures":true,"ignoreSnapshotDiff":true,"log":null,"mapreduceServiceName":"CD-YARN-FKZFETrq","mrSchedulerPoolNameProperty":null,"numConcurrentMaps":20,"overwrite":false,"poolName":null,"preserve":"rbugpa","proxyUser":"tsk-xeu-cdl-smoke","rebase":false,"replaceNameservice":null,"scheduleId":1291,"scheduleName":"hive_test01","scheduledTime":null,"sequenceFilePath":null,"skipCrcCheck":true,"skipTrash":false,"snapshotPrefix":null,"sourceCluster":"EMILY_DEV_SQL","sourcePaths":null,"sourcePeer":"EMILY_DEV_SQL_SRC","sourceProxyUser":"tsk-xeu-cdl-smoke","sourceService":"CD-HDFS-YcXwCCHo","strategy":"DYNAMIC","summaryFile":null,"targetRoleIds":[],"update":true,"useSnapshots":null,"useSnapshotsDiff":null,"useWebHdfsForSource":null},"alertConfig":{"alertOnAbort":false,"alertOnFail":false,"alertOnStart":false,"alertOnSuccess":false},"allowColumnStats":true,"allowHiveFunctions":true,"args":[],"dryRun":false,"exportDir":null,"exportFile":"/user/hdfs/.cm/hive-staging/2018-07-20-10-05-48-290498/export.json","exportToHdfs":true,"lastSuccessfulEventId":null,"localExportOnly":false,"mappings":{},"overwrite":false,"replicateImpalaMetadata":false,"replicateImpalaMetadataUserOption":false,"runInvalidateMetadata":true,"scheduleId":1291,"scheduleName":"hive_test01","scheduledTime":null,"skipExportToTarget":true,"sourceCluster":"EMILY_DEV_SQL","sourcePeer":"EMILY_DEV_SQL_SRC","sourceService":"CD-HIVE-SqlQgVBT","tables":{"dbb_dst_scratch":[".*"]},"targetClientConfig":null,"targetRoleIds":[],"update":[]} 1532081149130 1532081148707   ... View more

>  If this does not help, then let us know...   cloudera Manager has been restarted without success.   > it is possible to mimic the database query that is responsible for detecting active commands.   That would be excellent if you could give me that query then I could see what is causing it to report incorrectly. Thanks!     ... View more

>   we need to execute INVALIDATE METADATA;  Oh - I thought that was done for you if you select the "   Invalidate Impala Metadata on Destination" option.  Are you invalidating all metadata across the board or just the tables that you know you update?   In any case *something* has to reload/recalculate that metadata. Ifyou don't want it to be the first real user then I would try running some sort of query on the table yourself. That way you get the hit of the initial latency issue.   I hope that helps. I am currently learning about how to use BDR so I am no expert. ... View more

>  " The remote command failed with error message " indicates that the Hive Export command failed on the  source  Cloudera Manager server.   yes.     OP said:  I happen to be copying betwee two clusters within the same Cloudera Manager   I can clearly see that there are no running Hive Replications. I am the only person who has tried BDR in the entire company. The source Cloudera Manager is the same as the target Cloudera Manager. Only the cluster is different.  There are no working/running Hive Replications.  ... View more

Let's go to the official docs   https://www.cloudera.com/documentation/enterprise/latest/topics/install_upgrade_to_cdh5x_parcels.html   " The minor version of Cloudera Manager you use to perform the upgrade must be equal to or greater than the CDH minor version. To upgrade Cloudera Manager, see  Overview of Upgrading Cloudera Manager   ."   That final link is https://www.cloudera.com/documentation/enterprise/latest/topics/cm_ag_upgrading_cm.html#concept_xkm_f4q_tw   Goodluck!   I hope that is your problem. I see from the community that one or two other people have had problems with finding the CDH 5.13 parcels but this is definitely something to try first.    I have to say that this upgrading of manager separately is confusing - but makes sense from a practical point of view. ... View more

I am sorry that this is not more helpful but you may want to know that there is a 5.13.3 which has a few more bug fixes than 5.13.2   https://community.cloudera.com/t5/Community-News-Release/ANNOUNCE-Cloudera-Enterprise-5-13-3-Released/m-p/66079   I think that you need to upgrade your Cloudera Manager first before you upgrade your cluster. I do not believe that works with parcels. Have you done upgraded Cloudera Manager? I don't beleive Cloudera Manager will let you install CDH versions newer than itself.   personally on a CDH 5.10.1 cluster Cloudera Manager I am seeing      CDH 5 5.13.3-1.cdh5.13.3.p0.2    Available Remotely   [Download] Error for parcel CDH-5.13.3-1.cdh5.13.3.p0.2-el7.parcel : Parcel version 5.13.3-1.cdh5.13.3.p0.2 is not supported by this Cloudera Manager. Upgrade Cloudera Manager to at least 5.13.0 before using this parcel version.   Is that what you are seeing?    PS Saying "It is urgent" to the community just winds people up. ... View more

Am I right in reading the 5.13.3 and 5.14.2 tarball components? They both use Hive 1.1.0 and not Hive 1.2.0   I have a very similar problem - I want to switch off hive using the trash for certain tables ... View more

Thanks for this info Bill. I was wondering about it and was about to post a similar question.    Does Cloudera have any guidelines as to using multiple AZ's (in one region)?   My current thinking is    1) Two (or more) separate clusters - one in each AZ 2) A single Cloudera Manager in one AZ which controls all. 3) A single Cloudera Director  in one AZ which creates all instances 4)  A single Cloudera Navigator  in one AZ which monitors/audits all.   If we can turn some or all of these (2-4) into multiple AZ HA setup that would be really cool.    I am a bit concerned about the cross region network traffic - but perhaps I should save that for my own thread. ... View more

Hello MSharma.    I hope you have solved your problem by now but here are some thoughts.   Is your data entirely hBase? I think that makes things more difficult and outside of my expertise.I think you need to look into procedures for backing up an hBase database. It is almost irrelevant that you are using S3. The problem you face would be the same no matter what the backup medium is.   Normally - for most files and Hive tables I would lift and shift: Read from HDFS and copy to S3. If you have "at rest" encryption then I would expect that the reading process would decrypt the encrypted HDFS blocks - and you could use server side encryption on the S3 bucket instead. (Test this out first so you are comfortable with it before doing so). You would keep the data files - but lose any HDFS block information. Restoring those files would mean writing them into your cluster again as if they were brand new.    If this data is not being updated though you might consider keeping it in S3 and reading it with fs.s3a   I hope that helps but I am sorry I don't know how to backup hBase. ... View more