Member since
01-24-2021
9
Posts
0
Kudos Received
0
Solutions
01-18-2024
10:04 PM
Much Thanks .
... View more
09-15-2023
02:06 AM
I use CDH 6.3.2 。 hive 2.1 hadoop 3.0 hive on spark 。yarn cluster 。 hive.merge.sparkfiles=true ; hive.merge.orcfile.stripe.level=true ; This configuration makes the 1099 reduce file result merge into one file when the result is small 。Then the merged file has about 1099 stripes in one file 。 Then the result is so slow when it is read. I tried hive.merge.orcfile.stripe.level=false ; The result is desirable 。One small file with one stripe and read fast 。 Can anyone tell the difference between true and false ? Why " hive.merge.orcfile.stripe.level=true " is the default one ?
... View more
02-18-2023
12:05 PM
Run hdfs fsck delete. And found that datanode config wrong. Less 2 directories datanode store direcory config. Is there any possible way to rebuild the lost corrupt block? Much thanks
... View more
11-28-2022
11:09 PM
Thanks a lot. This "yarn application -updatePriority 10 -appId application_xxxx_xx" seems a config of yarn. It does not work for spark 2.x in CDH 6.3.2 either. Does it the same reason which means the 'Application Priority' must match the yarn version with spark version?
... View more
01-24-2021
09:59 PM
hi, seems the same error . Excuse me , Where is the log of cm yarn usage aggretion logs ? I also set the pool rules and hive some error with user like admin which has no group matching. User admin can not submit job to yarn ,but the other normal user can do it .
... View more