Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Metron Threat Triage not working with Profiler

Highlighted

Metron Threat Triage not working with Profiler

New Contributor

@asubramanian

I am trying to write a threat rule for AD sensor, which gives a score 5 if bruteforce attempt is detected, again a score 5 if malware attack is detected. If both attacks are present, score should be 10. Here, bruteforce is detected using a profiler and malware is detected using HBase threat intel.

The problem is if bruteforce attack is detected, I gets a score 5, if brute force and malware I get a score 10 but if malware alone, I cannot get a score.

According to the data present in HBase, 192.168.0.1 and 192.168.0.2 has malware attacks.

The sensor configuration is as given below:

{

"enrichment": {

"fieldMap": {

"stellar": {

"config": [

"distinct_attempts_profile := STATS_MERGE( PROFILE_GET( 'ditinct_login_ratio', ip_src_addr, PROFILE_FIXED( 2, 'MINUTES')))",

"login_mean := STATS_MEAN(distinct_attempts_profile)",

"distinct_attempts_profile := null"

]

}

},

"fieldToTypeMap": {},

"config": {}

},

"threatIntel": {

"fieldMap": {

"hbaseThreatIntel": [

"ip_src_addr"

],

"stellar": {

"config": [

"group_attempts_profile := STATS_MERGE( PROFILE_GET( 'group_login_ratio', 'global', PROFILE_FIXED( 2, 'MINUTES')))",

"group_mean := STATS_MEAN(group_attempts_profile)",

"group_sd := STATS_SD(group_attempts_profile)",

"login_outlier := ABS(group_mean + group_sd) < login_mean",

"malware_outlier := exists(threatintels.hbaseThreatIntel.ip_src_addr.malwareList)",

"is_alert := exists(is_alert) && is_alert",

"is_alert := is_alert || (login_outlier != null && login_outlier == true) || (malware_outlier != null && malware_outlier == true)",

"group_attempts_profile := null"

]

}

},

"fieldToTypeMap": {

"ip_src_addr": [

"malwareList"

]

},

"config": {},

"triageConfig": {

"riskLevelRules": [

{

"name": "Malware Outlier",

"comment": "Determine whether it is Malware Outlier.",

"rule": "exists(threatintels.hbaseThreatIntel.ip_src_addr.malwareList)",

"score": 5,

"reason": "FORMAT('IP %s has a malware effected', ip_src_addr)"

},

{

"name": "Bruteforce Outlier",

"comment": "Determine whether it is AD Login Outlier with malware attack.",

"rule": "login_outlier != null && login_outlier",

"score": 5,

"reason": "FORMAT('IP %s has a failure count (%f) compared to total failure %f', ip_src_addr,dist_mean,grp_dist_sd)"

}

],

"aggregator": "SUM",

"aggregationConfig": {}

}

},

"configuration": {}

}

85490-q6lec.png

Output I am getting is:

{"adapter.threatinteladapter.end.ts":"1533201827915","threat.triage.rules.1.comment":"Determine whether it is AD Login Outlier with malware attack.","enrichmentsplitterbolt.splitter.end.ts":"1533201827815","malware_outlier":false,"threat.triage.rules.0.comment":"Determine whether it is Malware Outlier.","adapter.stellaradapter.end.ts":"1533201827847","threat.triage.score":10.0,"enrichmentsplitterbolt.splitter.begin.ts":"1533201827815","login_mean":0.9889160831582192,"threat.triage.rules.0.score":5,"original_string":"user1,192.168.1.1,1532670161,4625","threatinteljoinbolt.joiner.ts":"1533201827965","threat.triage.rules.1.reason":"IP 192.168.1.1 has a failure count (null) compared to total failure null","enrichmentjoinbolt.joiner.ts":"1533201827864","threat.triage.rules.0.reason":"IP 192.168.1.1 has a malware effected","threatintelsplitterbolt.splitter.begin.ts":"1533201827887","login_outlier":true,"ip_src_addr":"192.168.1.1","adapter.stellaradapter.begin.ts":"1533201827838","timestamp":1532670161,"eventcode":"4625","threat.triage.rules.0.name":"Malware Outlier","threatintels.hbaseThreatIntel.ip_src_addr.malwareList":"alert","is_alert":"true","group_mean":0.943392389896573,"source.type":"adprofile","threatintelsplitterbolt.splitter.end.ts":"1533201827887","adapter.threatinteladapter.begin.ts":"1533201827915","threat.triage.rules.1.name":"Bruteforce Outlier","threat.triage.rules.1.score":5,"guid":"3376369c-682e-432d-9a18-e17eeb492d29","user":"user1","group_sd":0.013056093681356}

{"adapter.threatinteladapter.end.ts":"1533201827939","threat.triage.rules.1.comment":"Determine whether it is AD Login Outlier with malware attack.","enrichmentsplitterbolt.splitter.end.ts":"1533201827821","malware_outlier":false,"threat.triage.rules.0.comment":"Determine whether it is Malware Outlier.","adapter.stellaradapter.end.ts":"1533201827893","threat.triage.score":10.0,"enrichmentsplitterbolt.splitter.begin.ts":"1533201827821","login_mean":0.9889160831582192,"threat.triage.rules.0.score":5,"original_string":"user1,192.168.1.1,1532670163,4625","threatinteljoinbolt.joiner.ts":"1533201827982","threat.triage.rules.1.reason":"IP 192.168.1.1 has a failure count (null) compared to total failure null","enrichmentjoinbolt.joiner.ts":"1533201827908","threat.triage.rules.0.reason":"IP 192.168.1.1 has a malware effected","threatintelsplitterbolt.splitter.begin.ts":"1533201827933","login_outlier":true,"ip_src_addr":"192.168.1.1","adapter.stellaradapter.begin.ts":"1533201827872","timestamp":1532670163,"eventcode":"4625","threat.triage.rules.0.name":"Malware Outlier","threatintels.hbaseThreatIntel.ip_src_addr.malwareList":"alert","is_alert":"true","group_mean":0.943392389896573,"source.type":"adprofile","threatintelsplitterbolt.splitter.end.ts":"1533201827933","adapter.threatinteladapter.begin.ts":"1533201827939","threat.triage.rules.1.name":"Bruteforce Outlier","threat.triage.rules.1.score":5,"guid":"b4a96949-7676-4214-81a7-5584fb0ee7af","user":"user1","group_sd":0.013056093681356}

{"adapter.threatinteladapter.end.ts":"1533195520467","eventcode":"4624","enrichmentsplitterbolt.splitter.end.ts":"1533195520374","adapter.stellaradapter.end.ts":"1533195520400","enrichmentsplitterbolt.splitter.begin.ts":"1533195520374","login_mean":0.5843413436742739,"source.type":"adprofile","original_string":"user5,192.168.1.5,1533195519,4624","threatintelsplitterbolt.splitter.end.ts":"1533195520436","adapter.threatinteladapter.begin.ts":"1533195520467","threatinteljoinbolt.joiner.ts":"1533195520476","enrichmentjoinbolt.joiner.ts":"1533195520412","guid":"089759f1-c82b-4c96-af42-f91c3254e2b8","threatintelsplitterbolt.splitter.begin.ts":"1533195520436","user":"user5","ip_src_addr":"192.168.1.5","adapter.stellaradapter.begin.ts":"1533195520389","timestamp":1533195519}

{"adapter.threatinteladapter.end.ts":"1533201915992","eventcode":"4624","enrichmentsplitterbolt.splitter.end.ts":"1533201915872","malware_outlier":false,"adapter.stellaradapter.end.ts":"1533201915937","threatintels.hbaseThreatIntel.ip_src_addr.malwareList":"alert","enrichmentsplitterbolt.splitter.begin.ts":"1533201915872","login_mean":0.7349072438081725,"group_mean":0.9366117212056191,"source.type":"adprofile","original_string":"user2,192.168.1.2,1533201915,4624","threatintelsplitterbolt.splitter.end.ts":"1533201915973","adapter.threatinteladapter.begin.ts":"1533201915992","threatinteljoinbolt.joiner.ts":"1533201916009","enrichmentjoinbolt.joiner.ts":"1533201915955","guid":"d840afed-dcf2-4922-b408-bda02148fee3","threatintelsplitterbolt.splitter.begin.ts":"1533201915973","login_outlier":false,"user":"user2","ip_src_addr":"192.168.1.2","adapter.stellaradapter.begin.ts":"1533201915908","group_sd":0.016180687321585778,"timestamp":1533201915}

{"adapter.threatinteladapter.end.ts":"1533201961980","enrichmentsplitterbolt.splitter.end.ts":"1533201961867","malware_outlier":false,"threat.triage.rules.0.comment":"Determine whether it is AD Login Outlier with malware attack.","adapter.stellaradapter.end.ts":"1533201961892","threat.triage.score":5.0,"enrichmentsplitterbolt.splitter.begin.ts":"1533201961867","login_mean":0.9796404683384563,"threat.triage.rules.0.score":5,"original_string":"user4,192.168.1.4,1533201961,4625","threatinteljoinbolt.joiner.ts":"1533201962044","enrichmentjoinbolt.joiner.ts":"1533201961917","threat.triage.rules.0.reason":"IP 192.168.1.4 has a failure count (null) compared to total failure null","threatintelsplitterbolt.splitter.begin.ts":"1533201961939","login_outlier":true,"ip_src_addr":"192.168.1.4","adapter.stellaradapter.begin.ts":"1533201961882","timestamp":1533201961,"eventcode":"4625","threat.triage.rules.0.name":"Bruteforce Outlier","is_alert":"true","group_mean":0.9271187850382836,"source.type":"adprofile","threatintelsplitterbolt.splitter.end.ts":"1533201961939","adapter.threatinteladapter.begin.ts":"1533201961980","guid":"56f55a95-c67c-4ccf-a3e9-b2ef7f97bb03","user":"user4","group_sd":0.015343556745065257}

^X{"adapter.threatinteladapter.end.ts":"1533201962950","enrichmentsplitterbolt.splitter.end.ts":"1533201962861","malware_outlier":false,"threat.triage.rules.0.comment":"Determine whether it is AD Login Outlier with malware attack.","adapter.stellaradapter.end.ts":"1533201962890","threat.triage.score":5.0,"enrichmentsplitterbolt.splitter.begin.ts":"1533201962861","login_mean":0.9796404683384563,"threat.triage.rules.0.score":5,"original_string":"user4,192.168.1.4,1532670162,4625","threatinteljoinbolt.joiner.ts":"1533201962979","enrichmentjoinbolt.joiner.ts":"1533201962900","threat.triage.rules.0.reason":"IP 192.168.1.4 has a failure count (null) compared to total failure null","threatintelsplitterbolt.splitter.begin.ts":"1533201962928","login_outlier":true,"ip_src_addr":"192.168.1.4","adapter.stellaradapter.begin.ts":"1533201962875","timestamp":1532670162,"eventcode":"4625","threat.triage.rules.0.name":"Bruteforce Outlier","is_alert":"true","group_mean":0.9271187850382836,"source.type":"adprofile","threatintelsplitterbolt.splitter.end.ts":"1533201962928","adapter.threatinteladapter.begin.ts":"1533201962950","guid":"55f8e17f-f931-46f0-ade6-7b6a0487c8bb","user":"user4","group_sd":0.015343556745065257}

{"adapter.threatinteladapter.end.ts":"1533195536475","eventcode":"4624","enrichmentsplitterbolt.splitter.end.ts":"1533195536352","adapter.stellaradapter.end.ts":"1533195536390","enrichmentsplitterbolt.splitter.begin.ts":"1533195536352","login_mean":0.5843413436742739,"source.type":"adprofile","original_string":"user5,192.168.1.5,1533195535,4624","threatintelsplitterbolt.splitter.end.ts":"1533195536442","adapter.threatinteladapter.begin.ts":"1533195536475","threatinteljoinbolt.joiner.ts":"1533195536502","enrichmentjoinbolt.joiner.ts":"1533195536398","guid":"d69c6544-871e-42ee-a469-9555a758923b","threatintelsplitterbolt.splitter.begin.ts":"1533195536442","user":"user5","ip_src_addr":"192.168.1.5","adapter.stellaradapter.begin.ts":"1533195536383","timestamp":1533195535}