HS2 & Hive Metastore logs on CDP PC

[#1]

I was trying to find some documents about where would the location for HS2 & HMS logs be located for CDP-PC on AWS, but could get none the nearest I could get is https://docs.cloudera.com/data-warehouse/cloud/troubleshooting/topics/impala-logs-s3.html. but this is not what I want. 

Appreciate any help in this.

[#2]

@wert_1311 Are we talking about a DataHub/Data Engineering cluster or DataWarehouse?

If you have a Cloudera Manager for the cluster, check under the configuration tab of respective services. You may search for "hive_log_dir". If you SSH to the specific AWS instance hosting Hive service, you should be able to find the service logs under that path.

[#3]

Hi @smruti 

I am looking in Data Warehouse> Virtual Warehouse (CDP- PC)and under configuration I am unable to get the path, unsure if I am looking at the right place

[#4 · Accepted solution]

@wert_1311 It should be in the following location in S3:

s3://<s3_bucket_name>/clusters/<environment_id>/<database_catalog_id>/warehouse/tablespace/external/hive/sys.db/logs/dt=YYYY-MM-DD/ns=<virtual_warehouse_id>/

Hiveserver2 log directory will be named app=hiveserver2/

You could also view hiveserver2 pod logs using kubectl:

e.g.

kubectl logs hiveserver2-0 -n compute-1584641610-f14f -c hiveserver2