Reply
Highlighted
New Contributor
Posts: 2
Registered: ‎07-20-2016

Timeseries data insertion using Apache Phoenix

Hi, I'm trying to insert timeseries data to HBase using Phoenix 4.4.0, I would need a row per device per day. I'm capturing data from devices every minute. So in a row per day 1440 records (columns) has to be stored. I have designed a schema as follows: create table devicedata (deviceId integer not null, day date not null, ts timestamp, val double CONSTRAINT my_pk PRIMARY KEY (deviceId, day)); But when I use upsert, it updates the values and in a row, I always have only the last value of the day. Help me design a schema using phoenix, to store continuous device data. My data is going to be huge and performance is my first concern.

Expert Contributor
Posts: 101
Registered: ‎01-24-2014

Re: Timeseries data insertion using Apache Phoenix

Have you looked at OpenTSDB? That project sounds very similar to what you are describing as far as needs.

http://opentsdb.net/

 

New Contributor
Posts: 2
Registered: ‎07-20-2016

Re: Timeseries data insertion using Apache Phoenix

Yes @ben.hemphill. I have already looked at it. But I'm using a cloud distribution which has phoenix in it. If I want to use Opentsdb I have to spend some extra money on the infrastructure. Also is there any company provides professional support to Opentsdb?.

Expert Contributor
Posts: 101
Registered: ‎01-24-2014

Re: Timeseries data insertion using Apache Phoenix

Cloudera and a few others do HBase support, which OpenTSDB runs on. Support for OpenTSDB itself is community driven as far as I know. If you can run Phoenix, you can run opentsdb as both rely on HBase, you would just run the tsd process on your regionservers (assuming you have room)

Announcements