Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

BLOOM_EXISTS(OBJECT_GET()) Cannot work in SENSOR ENRICHMENT CONFIG

Highlighted

BLOOM_EXISTS(OBJECT_GET()) Cannot work in SENSOR ENRICHMENT CONFIG

New Contributor

Hello,
Hi there,

I'm following this tutorial ( https://metron.apache.org/current-book/use-cases/typosquat_detection/index.html ). And in section Generate the Bloom Filter, i am very sure this command is working and return value true in Stellar command

BLOOM_EXISTS(OBJECT_GET('/tmp/reference/alexa10k_filter.ser'), 'gogle')


But, it does not work dan return an error when i put it in a Sensor at Apache Metron's Management -> RAW JSON -> SENSOR ENRICHMENT CONFIG. And i'm using ElasticSearch so i checked ELASTICSEARCH ENABLED.

How i get the error, i use ElasticSearch HEAD as Google Chrome Addons ( https://chrome.google.com/webstore/detail/elasticsearch-head/ffmkiejjmecolpfloofpjologoblkegm ). And select error_index_ that have been created after Apache Metron send index to ElasticSearch.


Software that i used

Operating System
- CentOS Linux release 7.6.1810 (Core)

Hortonworks Software Used
- Hortonworks Data Platform 2.6.5.0-292
- Hortonworks Data Flow 3.1.0.0
- Hortonworks CyberSecurity Platform 1.9.1.0

Software Used
- Apache Kafka 1.0.0
- Apache Storm 1.1.0
- Apache Nifi 1.5.0
- Apache Metron 0.7.1.1.9.1.0
- HDFS 2.7.3
- HBase 1.1.2

Hardware Used :
4 (four) node using VMWare Virtualization, each using 4 core, 15.51 GB RAM, Storage 17.98 GB



The Error :

{

  • "_index": "error_index_2019.06.19.10",
  • "_type": "error_doc",
  • "_id": "AWtt2DrFshKy3KgUuMHX",
  • "_version": 1,
  • "_score": 1,
  • "_source": {
    • "exception": "java.lang.IllegalStateException: ENRICHMENT error with stellar failed: Unable to parse: BLOOM_EXISTS(OBJECT_GET('/tmp/reference/alexa10k_filter.ser'), test_domain) due to: com.esotericsoftware.kryo.KryoException: Unable to find class: Serialization trace: bits (org.apache.metron.guava.enrichment.hash.BloomFilter) filter (org.apache.metron.stellar.common.utils.BloomFilter) with relevant variables test_domain=gogle",
    • "failed_sensor_type": "error",
    • "stack": "java.lang.IllegalStateException: ENRICHMENT error with stellar failed: Unable to parse: BLOOM_EXISTS(OBJECT_GET('/tmp/reference/alexa10k_filter.ser'), test_domain) due to: com.esotericsoftware.kryo.KryoException: Unable to find class: Serialization trace: bits (org.apache.metron.guava.enrichment.hash.BloomFilter) filter (org.apache.metron.stellar.common.utils.BloomFilter) with relevant variables test_domain=gogle at org.apache.metron.enrichment.parallel.ParallelEnricher.lambda$apply$0(ParallelEnricher.java:195) at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.metron.stellar.dsl.ParseException: Unable to parse: BLOOM_EXISTS(OBJECT_GET('/tmp/reference/alexa10k_filter.ser'), test_domain) due to: com.esotericsoftware.kryo.KryoException: Unable to find class: Serialization trace: bits (org.apache.metron.guava.enrichment.hash.BloomFilter) filter (org.apache.metron.stellar.common.utils.BloomFilter) with relevant variables test_domain=gogle at org.apache.metron.stellar.common.BaseStellarProcessor.createException(BaseStellarProcessor.java:173) at org.apache.metron.stellar.common.BaseStellarProcessor.parse(BaseStellarProcessor.java:154) at org.apache.metron.enrichment.adapters.stellar.StellarAdapter.process(StellarAdapter.java:118) at org.apache.metron.enrichment.adapters.stellar.StellarAdapter.enrich(StellarAdapter.java:190) at org.apache.metron.enrichment.adapters.stellar.StellarAdapter.enrich(StellarAdapter.java:43) at org.apache.metron.enrichment.parallel.EnrichmentCallable.apply(EnrichmentCallable.java:64) at org.apache.metron.enrichment.parallel.EnrichmentCallable.apply(EnrichmentCallable.java:30) at com.github.benmanes.caffeine.cache.LocalCache.lambda$statsAware$0(LocalCache.java:143) at com.github.benmanes.caffeine.cache.BoundedLocalCache.lambda$doComputeIfAbsent$14(BoundedLocalCache.java:2039) at java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853) at com.github.benmanes.caffeine.cache.BoundedLocalCache.doComputeIfAbsent(BoundedLocalCache.java:2037) at com.github.benmanes.caffeine.cache.BoundedLocalCache.computeIfAbsent(BoundedLocalCache.java:2020) at com.github.benmanes.caffeine.cache.LocalCache.computeIfAbsent(LocalCache.java:112) at com.github.benmanes.caffeine.cache.LocalManualCache.get(LocalManualCache.java:54) at org.apache.metron.enrichment.parallel.ParallelEnricher.lambda$apply$0(ParallelEnricher.java:182) ... 4 more Caused by: org.apache.metron.guava.enrichment.util.concurrent.UncheckedExecutionException: com.esotericsoftware.kryo.KryoException: Unable to find class: Serialization trace: bits (org.apache.metron.guava.enrichment.hash.BloomFilter) filter (org.apache.metron.stellar.common.utils.BloomFilter) at org.apache.metron.guava.enrichment.cache.LocalCache$Segment.get(LocalCache.java:2256) at org.apache.metron.guava.enrichment.cache.LocalCache.get(LocalCache.java:3980) at org.apache.metron.guava.enrichment.cache.LocalCache.getOrLoad(LocalCache.java:3984) at org.apache.metron.guava.enrichment.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4868) at org.apache.metron.enrichment.stellar.ObjectGet.apply(ObjectGet.java:110) at org.apache.metron.stellar.common.StellarCompiler.lambda$exitTransformationFunc$13(StellarCompiler.java:664) at org.apache.metron.stellar.common.StellarCompiler$Expression.apply(StellarCompiler.java:259) at org.apache.metron.stellar.common.BaseStellarProcessor.parse(BaseStellarProcessor.java:151) ... 17 more Caused by: com.esotericsoftware.kryo.KryoException: Unable to find class: Serialization trace: bits (org.apache.metron.guava.enrichment.hash.BloomFilter) filter (org.apache.metron.stellar.common.utils.BloomFilter) at com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:156) at com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:133) at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:670) at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:118) at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551) at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:759) at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:132) at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551) at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:790) at org.apache.metron.common.utils.SerDeUtils.fromBytes(SerDeUtils.java:262) at org.apache.metron.enrichment.stellar.ObjectGet$Loader.load(ObjectGet.java:87) at org.apache.metron.enrichment.stellar.ObjectGet$Loader.load(ObjectGet.java:72) at org.apache.metron.guava.enrichment.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3579) at org.apache.metron.guava.enrichment.cache.LocalCache$Segment.loadSync(LocalCache.java:2372) at org.apache.metron.guava.enrichment.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2335) at org.apache.metron.guava.enrichment.cache.LocalCache$Segment.get(LocalCache.java:2250) ... 24 more Caused by: java.lang.ClassNotFoundException: at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:154) ... 39 more ",
    • "hostname": "HCP1-Master1",
    • "source:type": "error",
    • "raw_message": "{"": {"test_domain":null,"url":"www.gogle.com"}

      ,"source.type":"typo"}",

    • "error_hash": "8219f37f7fd3f84b8dc6a2e04dd68b6139795b8e8fe4cc072ae9344ea559d1dc",
    • "error_type": "enrichments_error",
    • "guid": "6d295227-aa41-4a99-95e8-46dc5a938e1f",
    • "message": "ENRICHMENT error with stellar failed: Unable to parse: BLOOM_EXISTS(OBJECT_GET('/tmp/reference/alexa10k_filter.ser'), test_domain) due to: com.esotericsoftware.kryo.KryoException: Unable to find class: Serialization trace: bits (org.apache.metron.guava.enrichment.hash.BloomFilter) filter (org.apache.metron.stellar.common.utils.BloomFilter) with relevant variables test_domain=gogle",
    • "timestamp": 1560916000031}

}

Can you help me how to figure out what is going on. Thank in advance

109408-sensor-enrichment-config.png


109432-see-apache-metron-error-in-elasticsearch-using-goo.png