I found what's going on.  The cluster for which I could not access the API is using V17, while the others are V19, even though all CDH versions are all 5.12.x.  After changing V19 to V17 in the URL, I can now access the API.  I wonder why the version is different.   Michael Reynolds   ... View more

@bgooley,   I found what appears to be the reason why the INVALIDATE METADATA command is not working.   ERROR: AuthorizationException: User 'impala/<server>@<domain>' does not have privileges to access: <DB>.<TABLE> Could not execute command: invalidate metadata <DB>.<TABLE>   It is failing to run the command.  However, after manually issuing the INVALIDATE METADATA command, Impala has no trouble accessing the table.   Michael Reynolds   ... View more

@bgooley,   Thank you for your reply!  If I run "invalidate metadata" on the destination, the databases appear after doing a refresh.   I tested a new replication, ensuring that " Invalidate Impala Metadata on Destination" was selected, and the data does not appear on the destination until I manually run "invalidate metadata".   Is there a way to ensure the invalidate command will work automatically via the replication job?   Michael Reynolds   ... View more

As a follow-up, we have CDH 5.12.2 on both clusters.  Under the replication "Advanced" tab, instead of seeing "Replicate Impala Metadata" as an option, I have " Invalidate Impala Metadata on Destination".  Checking this option appears to have no effect in making the data visible to Impala on the destination cluster.   Michael Reynolds   ... View more

Great!  Thank you very much for the detailed information.   Michael Reynolds   ... View more

Thank you for your reply!   Perhaps I am mistaken, but the informaiton in your link appears to be regarding how inactive sessions can be timed out, such as with the idle_session_timeout property.   I am seeking a method to disable a user account which has not been used for a period of time, such as 90 days.   Michael     ... View more

The cloudera-scm-server.log only shows the commands being run for the deployment, and no errors.  I also did not see errors in logs for the related services for which I am deploying configurations.   Using the "All Recent Commands" interface, and changing the filter in the command line to limit=10000, I can see that there were a few successful deployments, a couple of weeks ago, and no still-active deployment commands since then; only failures.   ... View more

Thank you for the reply.  Using the maximum of 250 past commands, the only "deploy client configuration" commands listed are the failed attempts.  This is a DR cluster, which is doing HDFS replications.  Therefore, the command list is filled with the replication commands.   Michael Reynolds   ... View more

This has apparently been resolved, because I am no longer having issues with closed connections during parcel downloads.   ... View more

I am experiencing a similar problem, which is preventing the parcels from downloading while installing CDH 5.4.5.  The percentage of the download process continues incrementing past 100%, and eventually starts over again at 0%, and loops like this indefinitely.   I checked /opt/cloudera/parcel-repo and found that the file "CDH-5.4.5-1.cdh5.4.5.p0.7-el6.parcel.part" grows in size and then starts over.   I tried downloading "CDH-5.4.5-1.cdh5.4.5.p0.7-el6.parcel" manually from http://archive.cloudera.com/cdh5/parcels/5/ using wget, and it finally succeeds after a few attempts:     --2015-09-20 15:37:14-- (try: 4) http://archive.cloudera.com/cdh5/parcels/5/CDH-5.4.5-1.cdh5.4.5.p0.7-el6.parcel Connecting to archive.cloudera.com|23.235.47.68|:80... connected. HTTP request sent, awaiting response... 206 Partial Content Length: 1259852705 (1.2G), 521097747 (497M) remaining Saving to: âCDH-5.4.5-1.cdh5.4.5.p0.7-el6.parcelâ 81% [++++++++++++++++++++++++++++++++++++++++++===============> ] 1,020,619,926 3.65M/s in 96s 2015-09-20 15:38:50 (2.79 MB/s) - Connection closed at byte 1020619926. Retrying. --2015-09-20 15:38:54-- (try: 5) http://archive.cloudera.com/cdh5/parcels/5/CDH-5.4.5-1.cdh5.4.5.p0.7-el6.parcel Connecting to archive.cloudera.com|23.235.47.68|:80... connected. HTTP request sent, awaiting response... 206 Partial Content Length: 1259852705 (1.2G), 239232779 (228M) remaining Saving to: âCDH-5.4.5-1.cdh5.4.5.p0.7-el6.parcelâ 100%[++++++++++++++++++++++++++++++++++++++++++++++++++++++++++=============>] 1,259,852,705 3.92M/s in 83s     Although it eventually downloads manually, the CDH installer does not recognize it, and continues trying to do the download.  Apparently, the CDH download process opens a new session on each attempt, instead of appending to the same, incomplete file.   ... View more

Hi Darren,   Thank you!!  I overlooked that.  I'm using parcels.   Michael Reynolds   ... View more

Hi Alex,   Thank you for your message.  I apologize for the delay.  I finally had a chance to try your suggestion.  The INSERT OVERWRITE with aggregations DOES work from the Impala shell.  The trouble appears to be caused by running it via Oozie.  I will need to do some experiments, because it needs to run in a workflow.   Michael   ... View more

Yes.  Sqoop appears to stall when an external process it's waiting for fails to complete. Thank you very much for the tip.   I plan to develop a procedure to automatically kill workflows that have exceeded a specified length of processing time.   Michael Reynolds   ... View more

Thank you for your reply!  Here is the command: =====================   import --connect jdbc:mysql://irbasedw-${shard}.db.xxxxxxxxxxx.net:3417/irbasedw_${shard}?dontTrackOpenResources=true&useCursorFetch=true --username iretl --password-file /irdw/${env}/lib/.passwordBaseDw --table base_action_fact2 --target-dir /irdw/${env}/legacy/fact/action_fact2/text/merge_delta/shard_${shard} --as-textfile --escaped-by \\ --hive-delims-replacement \\\\n --null-non-string \\N --null-string \\N --incremental lastmodified --check-column dlu --last-value ${wf:actionData('ObtainMaxDates')['nextDluText']} --where "doe_ts<'${wf:actionData('ObtainMaxDates')['nextDoeText']}'" --fetch-size -2147483648 --columns [ comma-separated column list ] -m 1   =====================   Michael Reynolds   ... View more