Skip to main content
David_Ghazaleh
David_Ghazaleh
Silver ⭐️
October 22, 2025
Solved

HWC on CDP 7.3.1 with Spark 3.5

Hello.

I am on CDP CDH-7.3.1-1.cdh7.3.1.p400.67986116, using Spark 3.5.4.7.3.1.400-100 and no longer able to to find HWC classes under com.hortonworks.spark.sql.hive.llap.*. 

Is HWC deprecated and/or unsupported on CDP 7.3.1+?

Thank you.

David.

ggangadharan
ggangadharanAnswer
Gold ⭐️
October 22, 2025

HWC libraries will be present in the CDH parcels. 

Reference - 

ls -lrt /opt/cloudera/parcels/CDH/lib/hwc_for_spark3/
total 24
drwxr-xr-x 2 root root 4096 Jun 21 01:26 conf
drwxr-xr-x 6 root root 4096 Jun 21 01:26 SparklyrHWC
lrwxrwxrwx 1 root root   42 Jun 21 01:26 pyspark_hwc-spark3.zip -> pyspark_hwc-spark3-1.0.0.7.3.1.400-100.zip
-rw-r--r-- 1 root root 6116 Jun 21 01:26 pyspark_hwc-spark3-1.0.0.7.3.1.400-100.zip
lrwxrwxrwx 1 root root   64 Jun 21 01:26 hive-warehouse-connector-spark3-assembly.jar -> hive-warehouse-connector-spark3-assembly-1.0.0.7.3.1.400-100.jar
lrwxrwxrwx 1 root root   75 Jun 21 01:37 hive-warehouse-connector-spark3-assembly-1.0.0.7.3.1.400-100.jar -> ../../jars/hive-warehouse-connector-spark3-assembly-1.0.0.7.3.1.400-100.jar

 In 7.3.1 HWC is still supported. 

Reference - https://docs.cloudera.com/cdp-private-cloud-base/7.3.1/integrating-hive-and-bi/topics/hive_hivewarehouseconnector_for_handling_apache_spark_data.html

Still could see HWC related classes under the package  com.hortonworks.spark.sql.hive.llap. 

Reference 

jar -tvf /opt/cloudera/parcels/CDH/lib/hwc_for_spark3/hive-warehouse-connector-spark3-assembly.jar | grep -i "com.hortonworks.spark.sql.hive.llap" | wc -l
97



jar -tvf /opt/cloudera/parcels/CDH/lib/hwc_for_spark3/hive-warehouse-connector-spark3-assembly.jar | grep -i "com.hortonworks.spark.sql.hive.llap" | head
     0 Fri Jun 20 22:46:34 UTC 2025 com/hortonworks/spark/sql/hive/llap/
     0 Fri Jun 20 22:46:34 UTC 2025 com/hortonworks/spark/sql/hive/llap/catalog/
     0 Fri Jun 20 22:46:34 UTC 2025 com/hortonworks/spark/sql/hive/llap/common/
     0 Fri Jun 20 22:46:34 UTC 2025 com/hortonworks/spark/sql/hive/llap/examples/
     0 Fri Jun 20 22:46:34 UTC 2025 com/hortonworks/spark/sql/hive/llap/query/
     0 Fri Jun 20 22:46:34 UTC 2025 com/hortonworks/spark/sql/hive/llap/query/builder/
     0 Fri Jun 20 22:46:34 UTC 2025 com/hortonworks/spark/sql/hive/llap/streaming/
     0 Fri Jun 20 22:46:34 UTC 2025 com/hortonworks/spark/sql/hive/llap/streaming/examples/
     0 Fri Jun 20 22:46:34 UTC 2025 com/hortonworks/spark/sql/hive/llap/util/
     0 Fri Jun 20 22:46:34 UTC 2025 com/hortonworks/spark/sql/hive/llap/wrapper/

 

View in thread

4 replies

Deepan_N
Silver ⭐️⭐️⭐️
Deepan_N
Silver ⭐️⭐️⭐️
October 22, 2025

Hello David,

I believe HWC is still supported on CDP 7.3.1. We can see the documentation for HWC in CDP 7.3.1.
-> https://docs.cloudera.com/cdp-private-cloud-base/7.3.1/integrating-hive-and-bi/topics/hive_hivewarehouseconnector_for_handling_apache_spark_data.html

And the HWC libraries are also available in the CDH parcels.

 ls -ltr /opt/cloudera/parcels/CDH/lib/hwc_for_spark3/
total 8
drwxr-xr-x 6 root root   87 Dec  4  2024 SparklyrHWC
drwxr-xr-x 2 root root   31 Dec  4  2024 conf
lrwxrwxrwx 1 root root   40 Dec  4  2024 pyspark_hwc-spark3.zip -> pyspark_hwc-spark3-1.0.0.7.3.1.0-197.zip
-rw-r--r-- 1 root root 6116 Dec  4  2024 pyspark_hwc-spark3-1.0.0.7.3.1.0-197.zip
lrwxrwxrwx 1 root root   62 Dec  4  2024 hive-warehouse-connector-spark3-assembly.jar -> hive-warehouse-connector-spark3-assembly-1.0.0.7.3.1.0-197.jar
lrwxrwxrwx 1 root root   73 Dec  4  2024 hive-warehouse-connector-spark3-assembly-1.0.0.7.3.1.0-197.jar -> ../../jars/hive-warehouse-connector-spark3-assembly-1.0.0.7.3.1.0-197.jar

Please follow the below documentation to configure the HWC with CDP.
-> https://docs.cloudera.com/cdp-private-cloud-base/7.3.1/integrating-hive-and-bi/topics/hive-hwc-reader-mode.html

Translate
    David_Ghazaleh
    David_GhazalehAuthor
    Silver ⭐️
    October 22, 2025

    Thank you very much for pointing it out.

    Translate
    ggangadharan
    Gold ⭐️
    ggangadharanAnswer
    Gold ⭐️
    October 22, 2025

    HWC libraries will be present in the CDH parcels. 

    Reference - 

    ls -lrt /opt/cloudera/parcels/CDH/lib/hwc_for_spark3/
    total 24
    drwxr-xr-x 2 root root 4096 Jun 21 01:26 conf
    drwxr-xr-x 6 root root 4096 Jun 21 01:26 SparklyrHWC
    lrwxrwxrwx 1 root root   42 Jun 21 01:26 pyspark_hwc-spark3.zip -> pyspark_hwc-spark3-1.0.0.7.3.1.400-100.zip
    -rw-r--r-- 1 root root 6116 Jun 21 01:26 pyspark_hwc-spark3-1.0.0.7.3.1.400-100.zip
    lrwxrwxrwx 1 root root   64 Jun 21 01:26 hive-warehouse-connector-spark3-assembly.jar -> hive-warehouse-connector-spark3-assembly-1.0.0.7.3.1.400-100.jar
    lrwxrwxrwx 1 root root   75 Jun 21 01:37 hive-warehouse-connector-spark3-assembly-1.0.0.7.3.1.400-100.jar -> ../../jars/hive-warehouse-connector-spark3-assembly-1.0.0.7.3.1.400-100.jar

     In 7.3.1 HWC is still supported. 

    Reference - https://docs.cloudera.com/cdp-private-cloud-base/7.3.1/integrating-hive-and-bi/topics/hive_hivewarehouseconnector_for_handling_apache_spark_data.html

    Still could see HWC related classes under the package  com.hortonworks.spark.sql.hive.llap. 

    Reference 

    jar -tvf /opt/cloudera/parcels/CDH/lib/hwc_for_spark3/hive-warehouse-connector-spark3-assembly.jar | grep -i "com.hortonworks.spark.sql.hive.llap" | wc -l
    97



    jar -tvf /opt/cloudera/parcels/CDH/lib/hwc_for_spark3/hive-warehouse-connector-spark3-assembly.jar | grep -i "com.hortonworks.spark.sql.hive.llap" | head
         0 Fri Jun 20 22:46:34 UTC 2025 com/hortonworks/spark/sql/hive/llap/
         0 Fri Jun 20 22:46:34 UTC 2025 com/hortonworks/spark/sql/hive/llap/catalog/
         0 Fri Jun 20 22:46:34 UTC 2025 com/hortonworks/spark/sql/hive/llap/common/
         0 Fri Jun 20 22:46:34 UTC 2025 com/hortonworks/spark/sql/hive/llap/examples/
         0 Fri Jun 20 22:46:34 UTC 2025 com/hortonworks/spark/sql/hive/llap/query/
         0 Fri Jun 20 22:46:34 UTC 2025 com/hortonworks/spark/sql/hive/llap/query/builder/
         0 Fri Jun 20 22:46:34 UTC 2025 com/hortonworks/spark/sql/hive/llap/streaming/
         0 Fri Jun 20 22:46:34 UTC 2025 com/hortonworks/spark/sql/hive/llap/streaming/examples/
         0 Fri Jun 20 22:46:34 UTC 2025 com/hortonworks/spark/sql/hive/llap/util/
         0 Fri Jun 20 22:46:34 UTC 2025 com/hortonworks/spark/sql/hive/llap/wrapper/

     

    Translate
      VidyaSargur
      Community Manager
      VidyaSargur
      Community Manager
      October 23, 2025

      @David_Ghazaleh, Did the response assist in resolving your query? If it did, please mark the relevant reply as the solution, as it will help others locate the answer more easily in the future.

      Translate
      Regards, Vidya Sargur, Community Manager Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
      Terms & ConditionsAccessibility statement