Streamset Data Collector and HDP

[#1]

Hi,

We are trying to build a pipeline for reading data from JDBC (Source) and Hive Metastore (Destination)

in settings General Tab ==> Stage Library we choose Hive 2.1-HDP 2.6.2 1-1 (as it don't have VERSION Matching to our)

We have below configuration

1) HDP :: 3.0.1

2) Hive : 3.1

3) SDC : 3.8.0

Just have single record in JDBC table.

in preview mode or after running pipeline getting below error :

com.streamsets.pipeline.api.base.OnRecordErrorException: HIVE_17 - Information type missing or invalid in the metadata record: Record[headers='HeaderImpl[select * from shipping WHERE event_id >${offset} ORDER by event_id ;::rowCount:0:1]' data='Field[LIST_MAP:{event_id=Field[INTEGER:1], order_id=Field[INTEGER:123], event_type=Field[STRING:SHIPPED]}]']

at com.streamsets.pipeline.stage.destination.hive.HiveMetastoreTarget.write(HiveMetastoreTarget.java:200)

[#2]

Data Collector has a stage library for HDP 3.1, but not specifically 3.0 - see the documentation on additional stage libraries.