We have a job which will load the data into partitioned table.
Each run data may be loaded into single partition or multiple partitions depending on type of job.
Is there a simpler way to get the number of records loaded to each of the partition.
Currently we have set parameter hiveconf hive.root.logger=INFO,console, this will have read Stats.
2018-11-26T05:07:08,402 INFO [main] fs.FSStatsAggregator: Read stats for : alb_supply_chain.tableName tm_dim_key=41966/ numRows 289002.
We are thinking of using script to read this log and to get the record count, Is this best way to get the load detail or are there any ways.