Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Bloom filter on Managed and External table for Hive 3.1 version not showing up in Explain plan

Highlighted

Bloom filter on Managed and External table for Hive 3.1 version not showing up in Explain plan

New Contributor

Hi Team,

We are using Hive version (Apache Hive (version 3.1.0.3.1.0.0-78) and trying to implement Bloom filter in it. So basically I have created a managed table with table properties defined as:

'orc.bloom.filter.columns'='*******',  'orc.bloom.filter.fpp'='0.05',  'orc.stripe.size'='268435456',

and stored it as orc file. While checking the explain plan(running: explain select count(1) from the_table where <condition>) in the current Hive version, I couldn't see anything as "Bloom_Filter" in the Plan provided by the CBO. The table I'm querying data in has  records.

 

I have a few doubts:

  1. Is Hive 3.1 version not using Bloom filter? If so, I have queried a normal table with same query and condition have seen that it takes more time compared to a table having Bloom filter defined on the column that has condition.
  2. Is there any parameter that needs to be set to get the value/ Bloom filter in the table?
  3. I have come across three parameters, please let me know what does these signify : 
    hive.tez.max.bloom.filter.entries,hive.tez.min.bloom.filter.entries,hive.tez.bloom.filter.factor

I have tried the same process creating a Hive external table as well and tried Bloom filter in it, but in vain.

This experiment for both managed as well as external table is has failed for Composite Bloom filter(used 3 columns), hence used a single field as Bloom filter.

 

Please let me know if anyone has used Bloom filter. Let me know then the process(please check explain plan once Bloom filter is used).

1 REPLY 1
Highlighted

Re: Bloom filter on Managed and External table for Hive 3.1 version not showing up in Explain plan

New Contributor

I hava the same issue with yours, can anybody answer this? Thans a lot

Don't have an account?
Coming from Hortonworks? Activate your account here