Support Questions
Find answers, ask questions, and share your expertise
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

Issues with Metron Profiler & Stellar


Using Metron 3.1 release, I am following the instructions here to create a profiler for my ingested data stream and perform statistical outlier analysis of it. I can confirm that the data is parsed, enriched, and indexed in elastic search correctly. The parsing, enrichment, indexing, and profiler topologies are all running without any errors and I can see that the profiler is writing into HBase. However, my "parser_score" as stored in elasticsearch is null, even when an outlier is pushed into the stream, and I'm trying to debug why.

I'm using the Stellar shell for that purpose. Once I get the message that all functions are loaded successfully, and after waiting sufficiently (20 minutes) for the data to be populated in HBase, I run the following command to get a profile:


which results in the following exception in Stellar shell:

[!] Unable to execute: Found interface org.objectweb.asm.MethodVisitor, but class was expected
org.apache.metron.common.dsl.ParseException: Unable to execute: Found interface org.objectweb.asm.MethodVisitor, but class was expected
at org.apache.metron.common.stellar.StellarCompiler.getResult(
at org.apache.metron.common.stellar.BaseStellarProcessor.parse(
at java.util.concurrent.ThreadPoolExecutor.runWorker(
at java.util.concurrent.ThreadPoolExecutor$
Caused by: java.lang.IncompatibleClassChangeError: Found interface org.objectweb.asm.MethodVisitor, but class was expected
at com.esotericsoftware.reflectasm.ConstructorAccess.insertConstructor(
at com.esotericsoftware.reflectasm.ConstructorAccess.get(
at org.apache.metron.common.utils.SerDeUtils$DefaultInstantiatorStrategy.newInstantiatorOf(
at com.esotericsoftware.kryo.Kryo.newInstantiator(
at com.esotericsoftware.kryo.Kryo.newInstance(
at com.esotericsoftware.kryo.serializers.FieldSerializer.create(
at com.esotericsoftware.kryo.Kryo.readClassAndObject(
at org.apache.metron.common.utils.SerDeUtils.fromBytes(
at org.apache.metron.profiler.client.HBaseProfilerClient.lambda$get$4(
at java.util.Spliterators$ArraySpliterator.forEachRemaining(
at org.apache.metron.profiler.client.HBaseProfilerClient.get(
at org.apache.metron.profiler.client.HBaseProfilerClient.fetch(
at org.apache.metron.profiler.client.stellar.GetProfile.apply(
at org.apache.metron.common.stellar.StellarCompiler.exitTransformationFunc(
at org.apache.metron.common.stellar.generated.StellarParser$TransformationFuncContext.exitRule(
at org.antlr.v4.runtime.Parser.triggerExitRuleEvent(
at org.antlr.v4.runtime.Parser.exitRule(
at org.apache.metron.common.stellar.generated.StellarParser.functions(
at org.apache.metron.common.stellar.generated.StellarParser.arithmetic_operands(
at org.apache.metron.common.stellar.generated.StellarParser.arithmetic_expr_mul(
at org.apache.metron.common.stellar.generated.StellarParser.arithmetic_expr(
at org.apache.metron.common.stellar.generated.StellarParser.transformation_expr(
at org.apache.metron.common.stellar.generated.StellarParser.transformation(
at org.apache.metron.common.stellar.BaseStellarProcessor.parse(
... 8 more

I have tried double-quotes instead of single-quotes in the command but the result is the same.

Any thought about what might be causing the issue?

My profiler config is very similar to what is described here , except that instead of capturing a 'global' statistical state for my 'value', I'm capturing that state per 'my_entity_id'.

Also, I'm setting the following in my without changing the other configuration parameters:


I'm not sure if the issue I'm facing in Stellar shell is related with why I'm not getting parser scores, but looks like something is not working correctly when calling "PROFILE_GET".

I appreciate your help.



So, I used 'global' instead of 'my_entity_id' with the same settings. After sometime, I can see parser_score values (though they don't seem to be computed correctly). So the issue with not getting scores seems to be related to how 'my_entity_id' values are parsed/interpreted or my potentially incorrect use of it. I still get the same exception in Stellar shell though.

As a side note, all my entities (as visible by elastic search head plugin) have the same value for 'my_entity_id' in my simple test. So there's no semantic difference between using 'global' or 'my_entity_id' as the entity id in the profile. Apart from the problem with the shell, I'm curious about what I'm not doing right when using 'my_entity_id'.

In other words, is there anything wrong with defining the profile below?

  "profiles": [
      "profile": "my_profile_name",
      "foreach": "my_entity_id",
      "onlyif": "true",
      "init" : {
        "s": "OUTLIER_MAD_STATE_MERGE(PROFILE_GET('my_profile_name','my_entity_id', PROFILE_FIXED(2, 'MINUTES')))"
      "update": {
        "s": "OUTLIER_MAD_ADD(s, my_double_value)"
      "result": "s"

and/or the following enrichment config:

  "enrichment": {
    "fieldMap": {
      "stellar" : {
        "config" : {
          "parser_score" : "OUTLIER_MAD_SCORE(OUTLIER_MAD_STATE_MERGE(PROFILE_GET( 'my_profile_name', 'my_entity_id', 
            PROFILE_FIXED(5, 'MINUTES')) ), my_double_value)"
         ,"is_alert" : "if parser_score > 3.5 then true else is_alert"
  ,"fieldToTypeMap": { }
  "threatIntel": {
    "fieldMap": { },
    "fieldToTypeMap": { },
    "triageConfig" : {
      "riskLevelRules" : [
          "rule" : "parser_score > 3.5",
          "score" : 10
      "aggregator" : "MAX"

Thanks in advance for your help!

New Contributor

You need to tell the profiler client that you are using 1 minute periods , otherwise it thinks you are using 15 minute default periods and so it cant generate the right row keys that match. you can either set that property for the profiler client


in your global properties at $METRON_HOME/config/zookeeper/global.json or pass it in directly to PROFILE_FIXED


PROFILE_GET('profile1','entity1', PROFILE_FIXED(30,'DAYS',{'profiler.client.period.duration':'1','profiler.client.period.duration.units':'MINUTES'}))