question pentaho PDI (kettle) to phoenix in Archives of Support Questions (Read Only)

pentaho PDI (kettle) to phoenix

arunpoy — Wed, 03 Aug 2016 10:25:53 GMT

We are trying to ingest a csv file using pentaho kettle to a phoenix table and dont see any errors after the job execution. But the table is not showing the newly added data. Nothing on the logs too. Whereas if i commit a mistake in the upsert syntax, i do see the error.

Alternatively if we use squirrel client to upsert data, it is showing success and data is in table.

Any thoughts or clue. Do we need to look at any setting in the pentaho side or phoenix side.

Re: pentaho PDI (kettle) to phoenix

rchintaguntla — Wed, 03 Aug 2016 10:41:51 GMT

It seems that PDI is not supporting Apache Phoenix yet. While loading data PDI generating insert queries but Phoenix needs upsert queries.

http://jira.pentaho.com/browse/PDI-14038

https://mail-archives.apache.org/mod_mbox/phoenix-user/201509.mbox/%3CCAB3fahz1wacofQHDTyMMO-W_nCQ0gAcL0LpXdeMmZMxVXwwWOg@mail.gmail.com%3E

It would be better to contact Pentaho PDI community.

Re: pentaho PDI (kettle) to phoenix

arunpoy — Wed, 03 Aug 2016 11:03:05 GMT

@Rajeshbabu Chintaguntla, thanks for the response. we have used a custom sql that will do the upserts. the statement is executing, but the results dont reflecr in the table.

Re: pentaho PDI (kettle) to phoenix

rchintaguntla — Wed, 03 Aug 2016 11:54:02 GMT

I see. Can you see the regionserver UI whether any requests coming to the table or not? or Can you scan HBase table to check any data in it?

If you can provide any logs that I can take a look and try to help you out.

Re: pentaho PDI (kettle) to phoenix

asinghal — Wed, 03 Aug 2016 22:27:10 GMT

Are you doing commit after upserting rows? as it may be possible that pentaho PDI may set autocommit off before starting inserts.

Re: pentaho PDI (kettle) to phoenix

arunpoy — Thu, 04 Aug 2016 12:33:42 GMT

Have reached out to the pentaho PDI community