Hive query failed with java.io.IOException: Cannot find password option fs.s3a.bucket.sg-cdp.fs.s3a.server-side-encryption-algorithm
Hi, I am running CDH 7.1.7p0. I have a external ORC database on S3 bucket. I did not configure this fs.s3a.server-side-encryption-algorithm in hadoop core-site.xml. However, I will randomly get this error when running hive query. Rerun the same query may success or fail with same error message.
ERROR : Vertex failed, vertexName=Map 14, vertexId=vertex_1684599244994_0012_7_02, diagnostics=[Vertex vertex_1684599244994_0012_7_02 [Map 14] killed/failed due to:ROOT_INPUT_INIT_FAILURE, Vertex Input: ss1 initializer failed, vertex=vertex_1684599244994_0012_7_02 [Map 14], java.lang.RuntimeException: ORC split generation failed with exception: java.lang.RuntimeException: java.io.IOException: Cannot find password option fs.s3a.bucket.sg-cdp.fs.s3a.server-side-encryption-algorithm
The database and query was part of the hive benchmark tool from github:
https://github.com/hortonworks/hive-testbench
Can you help me to resolve this problem. Thank you.
Here are the s3a settings in core-site.xml, and you can see there is no encryption related setting.
<name>fs.s3a.endpoint</name>
<name>fs.s3a.access.key</name>
<name>fs.s3a.secret.key</name>
<name>fs.s3a.impl</name>
<value>org.apache.hadoop.fs.s3a.S3AFileSystem</value>
<name>fs.s3a.path.style.access</name>
<name>fs.s3a.connection.ssl.enabled</name>
<name>fs.s3a.ssl.channel.mode</name>
<name>fs.s3a.threads.max</name>
<name>fs.s3a.connection.maximum</name>
<name>fs.s3a.retry.limit</name>
<name>fs.s3a.retry.interval</name>
<name>fs.s3a.committer.name</name>
<name>fs.s3a.committer.magic.enabled</name>
<name>fs.s3a.committer.threads</name>
<name>fs.s3a.committer.staging.unique-filenames</name>
<name>fs.s3a.committer.staging.conflict-mode</name>
