Re: how could i register an avro schema to hive me...

Daggers · ‎08-17-2021

Or actually how to register an avro schema with HIVE using anything....

Daggers · ‎08-17-2021

I have the same question. Does anyone have any good ideas on how to do this ?

Daggers · ‎11-12-2020

I have the same question. Can anyone from support answer ?

Daggers · ‎04-01-2020

I am getting the same problem and I am on 5.16 My research shows that the redirect should happen automatically but that does not seem to be happening

Daggers · ‎12-18-2019

I wonder if there is a way to ensure that all the files have finished being written to /tmp/log (The location at my site of yarn.nodemanager.remote-app-log-dir ) before I copy them ?

Daggers · ‎12-18-2019

Hey great suggestion ! I think this might work.

Daggers · ‎12-15-2019

Hi, If I try and get the logs for an application like this: yarn logs -applicationId application_1575531060741_10424 The command fails because I am not running it as the application owner. I need to run it like this: yarn logs -applicationId application_1575531060741_10424 -appOwner hive The problem is I want to write out all the yarn logs to the os so I can ingest them into splunk. If I try and figure out the appOwner for each application then this is awkward and time consuming even in a script. Is there a better way to dump all the yarn logs to the os ? Thanks

Daggers · ‎08-28-2019

How do I change the description/comment of an existing hive database ?

Daggers · ‎07-14-2019

Do table statistics get replicated as part of hive BDR replication ?

Daggers · ‎07-14-2019

Thanks for that

Daggers · ‎07-10-2019

Did you find the logs in the end ? I cant find them myself

Daggers · ‎02-20-2019

Hi, We have HA enabled for the metastore and have 2 instances running. Is it possible to configre a load balancer in front of these ? and if so is this normal practice ? Thanks

Daggers · ‎02-13-2019

Excellent ! these specific examples are what is needed.

Daggers · ‎01-17-2019

Hi, I can find this kind of message in the Impala daemon logs: impalad.pxlbig04.nz.thenational.com.impala.log.INFO.20190117-104024.29754:I0117 10:42:17.900411 9206 admission-controller.cc:515] Stats: agg_num_running=7, agg_num_queued=0, agg_mem_reserved=525.00 GB, local_host(local_mem_admitted=0, num_admitted_running=0, num_queued=0, backend_mem_reserved=75.00 GB) Which shows agg_num_running=7 Why is this same kind of metric not available in the CM graphs ? I want to know how close to the max running queries of 25 that I set in the default queue on Impala Admission Control.

Daggers · ‎09-06-2018

Thanks. Well yes I think it is what I need however.... it seems some of my tables have that date changing multiple times during a day. I looked at navigator to try and track down who may have been issuing 'COMPUTE STATS' commands but found nothing. To make the whole picture even stranger is the fact that the table is replicated at the database level using BDR. But the master metadata is different from the target metadata. The target table is the one I am looking at, this is the one getting update multiple times a day. Any suggestions on how to find what is making this LAST_ANALYZED timestamp to increment all the time ?

Daggers · ‎08-29-2018

I want to generate 'compute statistics <table_name>' commands for all non-partitoned tables that have not been analyzed for at least 1 month. If that information was available I would imagine it would be in the hive metastore database which the rest of the table metadata... however I cant find anything. Thanks Steve.

Daggers · ‎08-26-2018

Hi, I have exactly the issue described in this post and I am running CDH 5.13.3 I have tried putting this in the hql script as mentioned above: set hive.variable.substitute=true; and that does not help either. Can someone advise what to try ? Thanks

Daggers · ‎05-03-2017

Hi, My name is Steve I am an Oracle DBA who is getting to grips(well trying) with Hadoop. I work for a Bank and we have just installed 2x11 node clusters and I am looking to build a team to support it. I am also trying to understand data loading and ETL processes using the hadoop tools like sqoop

Online	Offline
Last Visited	‎09-25-2021 09:59 PM

Date Registered	‎05-03-2017 04:26 PM
Last Visited	‎09-25-2021 09:59 PM
Total Messages Posted	29

Re: how could i register an avro schema to hive me...

Re: how could i register an avro schema to hive me...

Re: Can we use hive tables over delta lake

Re: Resource manager API redirection doesn't show ...

Re: Using yarn logs command

Re: Using yarn logs command

Using yarn logs command

edit the description of a database

Do table statistics get replicated as part of hive...

Re: BDR - Hive Replication Log Directory

Re: BDR - Hive Replication Log Directory

Load balancer for metastore

Re: Number of impala queries running - graph

Number of impala queries running - graph

Re: What was the date that a HIVE/IMPALA table get...

What was the date that a HIVE/IMPALA table gets an...

Re: Compiling statement: FAILED: ParseException : ...

Re: INTRODUCE YOURSELF: Please Reply here to tell ...