Reply
New Contributor
Posts: 3
Registered: ‎06-28-2017

Basic Questions about Envelope

Hi, I am new to cloudera envelope.  I noticed that as output it does not support hbase.  I am under the impression that kafka  supports at least once delivery mechanism.  We were using hbase to handle duplicates.  Is the best idea to write to hbase via hive?

 

 Also is there any kafka offset management with this framework?

 

Thanks,

Neil

Cloudera Employee
Posts: 18
Registered: ‎08-26-2015

Re: Basic Questions about Envelope

Hi Neil,

 

The next version of Envelope that we will be releasing on Cloudera Labs will have both an HBase output and Kafka offset management built in.

 

In the mean time for HBase you could try writing to it in Envelope via an Hive output, although we haven't tested that so I can't say for sure that it will work.

 

Jeremy

New Contributor
Posts: 3
Registered: ‎06-28-2017

Re: Basic Questions about Envelope

[ Edited ]

Thanks Jeremy,

In the meantime, is there a way to manage offsets ourselves?  When would the next version be made available for usage in production systems? 

 

We are trying to decide whether it makes sense to leverage envelope for a production system as is or write something ourselves until envelope is ready and meets our use case. 

Cloudera Employee
Posts: 18
Registered: ‎08-26-2015

Re: Basic Questions about Envelope

You would need to wait for the next version to do offset management beyond just "read from the start" or "read from the end".

 

We are hoping to get the next version out over the next month or so but don't have a firm date. If you need it straight away then it would likely be easier to make your own enhancements to the Envelope code rather than start from scratch.

 

Jeremy

New Contributor
Posts: 3
Registered: ‎06-28-2017

Re: Basic Questions about Envelope

Thanks Jeremy

Just noticed that this is in a "lab" package.  Do you think we should run this in a production enviorment?

Cloudera Employee
Posts: 18
Registered: ‎08-26-2015

Re: Basic Questions about Envelope

Hi Neil,

 

Fair question but I think only you can decide that for yourself. Cloudera Labs projects are not supported but they can be a really good way to get a big head start on writing your own implementations from scratch. Generally because they're not supported you would need to do your own testing to make yourself comfortable that they meet your production quality standards.

 

Jeremy

Cloudera Employee
Posts: 18
Registered: ‎08-26-2015

Re: Basic Questions about Envelope

FYI, Envelope 0.4.0 has been released with support for offset management for Kafka, and for writing directly to HBase.
Announcements

Currently incubating in Cloudera Labs:

Envelope
HTrace
Ibis
Impyla
Livy
Oryx
Phoenix
Spark Runner for Beam SDK
Time Series for Spark
YCSB