HS2 & Hive Metastore logs on CDP PC

Question

I was trying to find some documents about where would the location for HS2 & HMS logs be located for CDP-PC on AWS, but could get none the nearest I could get is https://docs.cloudera.com/data-warehouse/cloud/troubleshooting/topics/impala-logs-s3.html. but this is not what I want.

Appreciate any help in this.

smruti · Accepted Answer

@wert_1311 It should be in the following location in S3:

s3://<s3_bucket_name>/clusters/<environment_id>/<database_catalog_id>/warehouse/tablespace/external/hive/sys.db/logs/dt=YYYY-MM-DD/ns=<virtual_warehouse_id>/

Hiveserver2 log directory will be named app=hiveserver2/

You could also view hiveserver2 pod logs using kubectl:

e.g.

kubectl logs hiveserver2-0 -n compute-1584641610-f14f -c hiveserver2

smruti · Answer

@wert_1311 Are we talking about a DataHub/Data Engineering cluster or DataWarehouse?

If you have a Cloudera Manager for the cluster, check under the configuration tab of respective services. You may search for "hive_log_dir". If you SSH to the specific AWS instance hosting Hive service, you should be able to find the service logs under that path.

Sign up

Please use your Cloudera account.

Login to the community

Please use your Cloudera account.