Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Can we get Load Stats for each query run via Metastore

Can we get Load Stats for each query run via Metastore

New Contributor

Hi,

We have a job which will load the data into partitioned table.

Each run data may be loaded into single partition or multiple partitions depending on type of job.

Is there a simpler way to get the number of records loaded to each of the partition.

Currently we have set parameter hiveconf hive.root.logger=INFO,console, this will have read Stats.

PFB

2018-11-26T05:07:08,402 INFO [main] fs.FSStatsAggregator: Read stats for : alb_supply_chain.tableName tm_dim_key=41966/ numRows 289002.

We are thinking of using script to read this log and to get the record count, Is this best way to get the load detail or are there any ways.