Reply
New Contributor
Posts: 2
Registered: ‎07-20-2016

Timeseries data insertion using Apache Phoenix

Hi, I'm trying to insert timeseries data to HBase using Phoenix 4.4.0, I would need a row per device per day. I'm capturing data from devices every minute. So in a row per day 1440 records (columns) has to be stored. I have designed a schema as follows: create table devicedata (deviceId integer not null, day date not null, ts timestamp, val double CONSTRAINT my_pk PRIMARY KEY (deviceId, day)); But when I use upsert, it updates the values and in a row, I always have only the last value of the day. Help me design a schema using phoenix, to store continuous device data. My data is going to be huge and performance is my first concern.

Expert Contributor
Posts: 101
Registered: ‎01-24-2014

Re: Timeseries data insertion using Apache Phoenix

Have you looked at OpenTSDB? That project sounds very similar to what you are describing as far as needs.

http://opentsdb.net/

 

Highlighted
New Contributor
Posts: 2
Registered: ‎07-20-2016

Re: Timeseries data insertion using Apache Phoenix

Yes @ben.hemphill. I have already looked at it. But I'm using a cloud distribution which has phoenix in it. If I want to use Opentsdb I have to spend some extra money on the infrastructure. Also is there any company provides professional support to Opentsdb?.

Expert Contributor
Posts: 101
Registered: ‎01-24-2014

Re: Timeseries data insertion using Apache Phoenix

Cloudera and a few others do HBase support, which OpenTSDB runs on. Support for OpenTSDB itself is community driven as far as I know. If you can run Phoenix, you can run opentsdb as both rely on HBase, you would just run the tsd process on your regionservers (assuming you have room)

Announcements

Our community is getting a little larger. And a lot better.


Learn More about the Cloudera and Hortonworks community merger planned for late July and early August.