Created 09-11-2022 02:17 PM
Hi All,
I am trying to run a Spark job using a python script in CDE cluster. The script is from a CDE tutorial.
The job fails due to the error message in log files (in Subject). Appreciate any inputs.
Thanks in advance ....
Sekhar
Created 09-11-2022 05:01 PM
@sekhar1 ,
The CDP user that you're using to execute your job needs an "IDBroker mapping" to a valid AWS role to be able to access the contents of the S3 bucket.
Please check this: https://docs.cloudera.com/cdf-datahub/7.2.10/nifi-hive-ingest/topics/cdf-datahub-hive-ingest-idbroke...
Cheers,
André
Created 09-11-2022 05:01 PM
@sekhar1 ,
The CDP user that you're using to execute your job needs an "IDBroker mapping" to a valid AWS role to be able to access the contents of the S3 bucket.
Please check this: https://docs.cloudera.com/cdf-datahub/7.2.10/nifi-hive-ingest/topics/cdf-datahub-hive-ingest-idbroke...
Cheers,
André
Created 10-14-2022 03:42 AM
Hello @sekhar1
We hope your Q was answered by André. As such, We are marking the Post as Resolved. If the Link shared by André didn't fix the issue, Feel free to Update the Post likewise.
Regards, Smarak