Reply
Highlighted
New Contributor
Posts: 1
Registered: ‎10-22-2018

some impala built-in function can't work in spark data source reader

I want to get subquery from impala table as one dataset.

Code like this:

String subQuery = "(select to_timestamp(unix_timestamp(now())) as ts from my_table) t"
Dataset<Row> ds = spark.read().jdbc(myImpalaUrl, subQuery, prop);

But result is error:

Caused by: java.sql.SQLDataException: [Cloudera][JDBC](10140) Error converting value to Timestamp.

to_timestmap() function failed ,but unix_timestmap() , now() work fine.

ps. I found another problem, when I use hive udf in the "jdbc" api still failed. 

Can anyone help me?

Announcements