Member since
10-28-2020
623
Posts
47
Kudos Received
40
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 3186 | 02-17-2025 06:54 AM | |
| 8079 | 07-23-2024 11:49 PM | |
| 1767 | 05-28-2024 11:06 AM | |
| 2397 | 05-05-2024 01:27 PM | |
| 1546 | 05-05-2024 01:09 PM |
01-21-2026
09:34 AM
@gurumoorthyk adding to Smruthi's update, Iceberg + Hive+ Spark2 is not supported in CDP 7.1.9, it's officially supported from CDP 7.3.1. We don't need to add any jars explicitly in 7.3.1. By default, service[Impala,spark3, Hive] classpaths will have the required iceberg jars. In CDP 7.1.9, you can use Iceberg with Impala and Spark 3, but with limited functionalities. It does not require any classpath manipulation. Doc: https://docs.cloudera.com/cdp-public-cloud/cloud/cdp-iceberg/topics/iceberg-in-cdp.html FYI: [You can check the commands below in your env] [root@host1 jars]# pwd [You can see impala & spark3 classpaths by default has the iceberg jars] /opt/cloudera/parcels/ [root@ccycloud-1.nightly-71x-kl.root.comops.site jars]# find ../ -name *iceberg-hive-runtime* ../lib/oozie/oozie-sharelib-yarn/lib/sqoop/iceberg-hive-runtime-1.3.0.7.1.9.1003-449.jar ../lib/hive/lib/atlas-hive-plugin-impl/iceberg-hive-runtime-1.3.0.7.1.9.1003-449.jar ../lib/impala/lib/iceberg-hive-runtime-1.3.0.7.1.9.1003-449.jar ====>impala classpath ../lib/atlas/hook/hive/atlas-hive-plugin-impl/iceberg-hive-runtime-1.3.0.7.1.9.1003-449.jar ../jars/iceberg-hive-runtime-1.3.0.7.1.9.1003-449.jar [root@host1 jars]# find ../ -name *iceberg-spark* find ../.. -name *iceberg-spark* ../../SPARK3-3.5.4.3.5.7191000.0-30-1.p0.68499982/lib/spark3/iceberg/iceberg-spark-runtime-3.5_2.12-1.3.0.3.5.7191000.0-30.jar. =====>SPARK3 Classpath ../../CDH-7.1.9-1.cdh7.1.9.p1003.74501001/lib/oozie/oozie-sharelib-yarn/lib/spark3/iceberg-spark-runtime-3.3_2.12-1.3.0.3.3.7191000.1-138.jar ../../CDH-7.1.9-1.cdh7.1.9.p1003.74501001/jars/iceberg-spark-runtime-3.3_2.12-1.3.0.3.3.7191000.1-138.jar
... View more
03-10-2025
09:49 PM
2 Kudos
Sorry for my late response. I did try with the JDK version as mentioned in the driver documentation. It didn't work. However, I am now using keytab method for connecting and I am fine with it. @asish thanks a ton for all the support.
... View more
02-17-2025
07:55 AM
Thanks a lot.. It worked like magic 😄
... View more
02-07-2025
11:50 AM
@zorrofrombrasil Did you get a chance to check the updates from @ggangadharan and @smruti? If so, did the information help to fix the issue? or Do you have any other concerns?
... View more
12-17-2024
10:14 AM
@Viki_Nodejs if you haven't resolved this issue could you try the below steps and revert. 1. Install the Required NPM Packages Use the hive-driver package for Node.js, which supports HiveServer2 over HTTP/HTTPS. npm install hive-driver 2. Prerequisites Ensure you have: HiveServer2 URL: Includes the hostname and port. SSL Configuration: Paths to your .jks trust store and its password. Hive httppath: Set to cliservice. Authentication details (if required): Username/password or Kerberos configuration. 3. Configure the Connection Here's an example of how to set up the connection using the hive-driver: const { HiveClient, TCLIServiceTypes } = require('hive-driver'); async function connectToHive() { const client = new HiveClient(TCLIServiceTypes); // Configure the Hive connection const connection = client.connect({ host: '<HIVE_SERVER_HOSTNAME>', // e.g., hive.example.com port: 10001, // HiveServer2 port, typically 10001 for HTTPS options: { path: '/cliservice', // HTTP path to HiveServer2 ssl: true, // Enable SSL sslOptions: { rejectUnauthorized: true, // Ensure certificates are verified ca: '<path/to/truststore.pem>' // Convert your JKS truststore to PEM format }, // Authentication username: '<YOUR_USERNAME>', password: '<YOUR_PASSWORD>', // You can add session configurations here } }); try { // Open the connection await connection.openSession(); console.log('Connected to Hive'); // Example query const result = await connection.executeStatement('SELECT * FROM your_table LIMIT 10'); console.log(result); // Close the session await connection.closeSession(); } catch (error) { console.error('Error connecting to Hive:', error); } finally { // Ensure the connection is closed await connection.close(); } } connectToHive(); 4. Key Point to Note !!!!!!!!! SSL Truststore [Very Important] Hive uses .jks files for its truststore, but hive-driver requires a .pem file for SSL. Convert your .jks file to .pem using the following commands: keytool -importkeystore -srckeystore truststore.jks -destkeystore truststore.p12 -deststoretype PKCS12 openssl pkcs12 -in truststore.p12 -out truststore.pem -nokeys I also saw an EAI_FAIL error in the screenshot this is related to not being able to resolve the DNS. Hope this helps
... View more
11-26-2024
12:06 AM
1 Kudo
I'm 3 years late, but in case anyone else facing the same issue, try using auth='NOSASL'. Maybe the issue does not occur with newer versions.
... View more
10-16-2024
08:00 AM
hi @NagendraKumar , You can use puthiveql as described in this article https://community.cloudera.com/t5/Support-Questions/how-to-use-puthiveql-in-NIFI/td-p/204542
... View more
10-02-2024
07:12 AM
@NagendraKumar were you able to truncate the data with the processor that my colleague Smruti mentioned?
... View more
09-18-2024
01:22 AM
1 Kudo
@zhuodongLi, Did the responses help resolve your query? If it did, kindly mark the relevant reply as the solution, as it will aid others in locating the answer more easily in the future.
... View more