Member since
05-03-2017
29
Posts
0
Kudos Received
0
Solutions
08-17-2021
01:56 PM
Or actually how to register an avro schema with HIVE using anything....
... View more
08-17-2021
01:54 PM
I have the same question. Does anyone have any good ideas on how to do this ?
... View more
04-01-2020
02:24 PM
I am getting the same problem and I am on 5.16 My research shows that the redirect should happen automatically but that does not seem to be happening
... View more
12-18-2019
06:59 PM
I wonder if there is a way to ensure that all the files have finished being written to /tmp/log (The location at my site of yarn.nodemanager.remote-app-log-dir ) before I copy them ?
... View more
12-15-2019
07:08 PM
Hi,
If I try and get the logs for an application like this:
yarn logs -applicationId application_1575531060741_10424
The command fails because I am not running it as the application owner. I need to run it like this:
yarn logs -applicationId application_1575531060741_10424 -appOwner hive
The problem is I want to write out all the yarn logs to the os so I can ingest them into splunk. If I try and figure out the appOwner for each application then this is awkward and time consuming even in a script.
Is there a better way to dump all the yarn logs to the os ?
Thanks
... View more
Labels:
- Labels:
-
Apache YARN
08-28-2019
06:47 PM
How do I change the description/comment of an existing hive database ?
... View more
Labels:
- Labels:
-
Apache Hive
07-14-2019
03:01 PM
Do table statistics get replicated as part of hive BDR replication ?
... View more
Labels:
- Labels:
-
Cloudera Manager
-
Hive
02-20-2019
04:41 PM
Hi, We have HA enabled for the metastore and have 2 instances running. Is it possible to configre a load balancer in front of these ? and if so is this normal practice ? Thanks
... View more
Labels:
- Labels:
-
Hive
01-17-2019
04:30 PM
Hi, I can find this kind of message in the Impala daemon logs:
impalad.pxlbig04.nz.thenational.com.impala.log.INFO.20190117-104024.29754:I0117 10:42:17.900411 9206 admission-controller.cc:515] Stats: agg_num_running=7, agg_num_queued=0, agg_mem_reserved=525.00 GB, local_host(local_mem_admitted=0, num_admitted_running=0, num_queued=0, backend_mem_reserved=75.00 GB)
Which shows agg_num_running=7
Why is this same kind of metric not available in the CM graphs ?
I want to know how close to the max running queries of 25 that I set in the default queue on Impala Admission Control.
... View more
Labels:
- Labels:
-
Cloudera Manager
-
Impala
09-06-2018
01:27 PM
Thanks. Well yes I think it is what I need however.... it seems some of my tables have that date changing multiple times during a day. I looked at navigator to try and track down who may have been issuing 'COMPUTE STATS' commands but found nothing. To make the whole picture even stranger is the fact that the table is replicated at the database level using BDR. But the master metadata is different from the target metadata. The target table is the one I am looking at, this is the one getting update multiple times a day. Any suggestions on how to find what is making this LAST_ANALYZED timestamp to increment all the time ?
... View more
08-29-2018
03:51 PM
I want to generate 'compute statistics <table_name>' commands for all non-partitoned tables that have not been analyzed for at least 1 month. If that information was available I would imagine it would be in the hive metastore database which the rest of the table metadata... however I cant find anything. Thanks Steve.
... View more
Labels:
- Labels:
-
Impala
08-26-2018
08:02 PM
Hi, I have exactly the issue described in this post and I am running CDH 5.13.3 I have tried putting this in the hql script as mentioned above: set hive.variable.substitute=true; and that does not help either. Can someone advise what to try ? Thanks
... View more
05-03-2017
04:35 PM
Hi, My name is Steve I am an Oracle DBA who is getting to grips(well trying) with Hadoop. I work for a Bank and we have just installed 2x11 node clusters and I am looking to build a team to support it. I am also trying to understand data loading and ETL processes using the hadoop tools like sqoop
... View more