Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Which tech for measures on Cloudera

Highlighted

Which tech for measures on Cloudera

Explorer
When using Cloudera techs to provide a star schema, which tech is the right one to use for traditional edw measures? I'm thinking calculated columns or measures in the MDX/DAX world of comparable Microsoft techs. I get that with impala if I've got a fact table with dimension keys I can do a typical group by on dimension attributes and do simple aggregation funcs (e.g. sum/avg etc.) But if I want to store standard business calculations that the business has agreed upon, what are some approaches for surfacing these so everyone who wants to consume the facts doesn't have to rewrite complex calculation queries?

Thanks in advance.
2 REPLIES 2

Re: Which tech for measures on Cloudera

Contributor

There are two common methods to add measurements and aggregations and expose them to other consumers:

 

1. Create aggregation tables with those measures - this means simply running the Impala query but doing "insert into" to update the new measures in the aggregation table.

 

2. Use HBase to store and expose the measurements. This requires more thought about the schema design - sometimes you store an entire report, keyed by date and the report ID in HBase and allow users to pull out complete pre-calculated reports.

Other times you need access to specific rows and their measures, so you need to decide on more specific keys.

 

The important thing to remember here is that HBase is fantastic to retrieve data that belongs to a single key, but not so great for scans.

HDFS/Hive/Impala tables are a good place to store measures that require scanning and joining with other tables.

 

Hope this helps.

 

Highlighted

Re: Which tech for measures on Cloudera

Explorer

That does help. Thank you very much for your reply!

Don't have an account?
Coming from Hortonworks? Activate your account here