04-17-2018 03:27 AM
We upgraded from CM 5.7.6 to CM 5.14.0 and we can no longer run HBase Export Snapshot to S3 bucket via s3a:// file system because it always fails in the VerifySnapshot step with OutOfMemoryError and a heap dump.
After tracing changes in source code we might have found HBase/Hadoop AWS issue related to metric reporting in S3AInstrumentation. More information could be found here:
As a work-around we increased client heap size extensively, but it's not a scalable solution. We were able to run backups with default 256 MiB heap size in CM 5.7.6.
Is there any option we can adjust in order to achieve permanent solution?