Member since
09-20-2023
2
Posts
1
Kudos Received
0
Solutions
09-28-2023
01:00 AM
@venkatsambath @DianaTorres Hi we are using 2 different kinit in same unix user account.the job is not parrallel. Our requirement is such that. Currently we are using Ambari 2.6.0 and HDFS/YARN version 2.7.3 to write the data in HDFS using spark version 2.x. We are using airflow as scheduler. Similarly our use case is to write the data in another HDFS cluster which has CDP in the same spark job we are writing to our HDFS data. Both the HDFS clusters are kerberised and we are able to connect. We do not want setup cross-realm trust between 2 cluster but should write data in same spark job to both HDFS. We tried combining krb5.conf and exported different keytab principal but its not working. Can you help us how can we achieve this. Writing data to 2 different kerberised cluster in one spark job(HDP and CDP).
... View more
09-20-2023
04:27 AM
1 Kudo
I want to transfer data from HDP 2.6 with kerberos authentication via spark to CDP with kerberos authentication. I am trying to run 2 kerberos ticket in a server but not able to run. One kerberos ticket will be used for reading data from HDP and second kerberos ticket for writing data in CDP cluster. Anyone has any better idea apart from spark please let me know
... View more
Labels: