Hello, Using NiFi 1.5 and Vertica JDBC client 9.0.1-7 I have followed the Vertica Nifi integration guide over here https://www.vertica.com/kb/NifiCG/Content/Partner/NifiCG.htm I have setup a DBCP connection pool settings as shown below The Controller is enabled. However when I start the QueryDatabaseTable Processor literally nothing happens. No tasks are run, no flow files generated. My QueryDatabaseTable Processor settings are shown below There are no logs being generated for my processor as well. I have also tried the ExecuteSQL processor. It has the same problem. What could be the problem? Thanks for the help! Regards, Mustafa Akmal ... View more

I have partially implemented a parser chaining use case. I have one aggregate sensor which is receiving the actual log. There are two sequence numbers '123' and '456'. The aggregate parser is working fine. It sends data to topic '123' in case sequence number is '123' and topic '456' in case sequence number is '456'. The next two sub parsers are also parsing the 'messageField' correctly and they are indexed in elasticsearch. However, any metadata that is fields such as timestamp are not indexed. They are received on the specific topic but not indexed. I have specified the "rawMessageStrategy": "ENVELOPE" and readMetadata and mergeMetadata are set to true. ... View more

I have the following field in my logs 'severity' this field has integer values from 0-7. I have created a threat rule as follows severity == 6 and given it a score of 50. However when the logs get enriched and indexed. The rule does not assign a score to the record. It does get identified as a alert because the is_alert field is true. ... View more

Hi Using HCP 1.7.1 I have the following data loaded into HBASE 'threatintel' table https://zeustracker.abuse.ch/blocklist.php?download=domainblocklist Added ',abuse.ch' at end of each line. Then created the extractor config which looks like this { "config" : { "columns" : { "domain" : 0 ,"source" : 1 } , "indicator_column" : "domain" , "type" : "zeusList" , "separator" : "," } , "extractor" : "CSV" } However when the records get enriched and indexed they only contain this field in the elasticsearch document "threatintels:hbaseThreatIntel:domain_without_subdomains:zeusList": "alert" It does not contain the threatintel.source or threatintel.domain field. Is it supposed to be like that? Thanks! ... View more

Hello, I have a single node setup. I kerberized the cluster using the Hortonworks Docs. After kerberization the storm UI does not open and requires authentication. I have tried following the guides online. It asks to run the kinit command. I am guessing this needs to be run from the local computer. Can anyone guide me on how to do this? Do I need to install kerberos on my local linux system? Do I install the kerberos server + kerberos workstation? How do allow the local user to get the ticket for the storm service? Thanks ... View more

Hi, I am trying to filter out messages from the indexing topic which have a threat triage score > 50. For this, I have created a new JSONMap parser named 'notify_admin' and a Kafka topic named 'notify_admin'. I have set up the parser to get the messages from the indexing topic by specifying the 'sensorTopic' field. To send the message to the 'notify_admin' Kafka topic I have set the 'outputTopic'. However, when messages are sent to the indexing topic from other parsers I have configured for example squid they do get enriched and indexed but the JSON parser does not output any data to the Kafka topic. The JSON parser topology shows messages being emitted and acknowledged but no data is output to the 'notify_admin' Kafka topic What am I doing wrong? For now, I am just trying to build up the flow pipeline so haven't set up any filter using Stellar. Just trying to pass the messages from one topic to another. However, it doesn't work. Parser Config (notify_admin.json) { "parserClassName": "org.apache.metron.parsers.json.JSONMapParser", "filterClassName": null, "sensorTopic": "indexing", "outputTopic": "notify_admin", "errorTopic": null, "writerClassName": null, "errorWriterClassName": null, "readMetadata": false, "mergeMetadata": false, "numWorkers": null, "numAckers": null, "spoutParallelism": 1, "spoutNumTasks": 1, "parserParallelism": 1, "parserNumTasks": 1, "errorWriterParallelism": 1, "errorWriterNumTasks": 1, "spoutConfig": {}, "securityProtocol": null, "stormConfig": {}, "parserConfig": {}, "fieldTransformations": [], "cacheConfig": {}, "rawMessageStrategy": "DEFAULT", "rawMessageStrategyConfig": {} } Enrichment Config (notify_admin.json) { "enrichment": { "fieldMap": {}, "fieldToTypeMap": {}, "config": {} }, "threatIntel": { "fieldMap": {}, "fieldToTypeMap": {}, "config": {}, "triageConfig": { "riskLevelRules": [], "aggregator": "MAX", "aggregationConfig": {} } }, "configuration": {} } Indexing Config (notify_admin.json) { "hdfs": { "batchSize": 1, "enabled": true, "index": "notify_admin" }, "elasticsearch": { "batchSize": 1, "enabled": true, "index": "notify_admin" }, "solr": { "batchSize": 1, "enabled": true, "index": "notify_admin" } } A simple message in the indexing Kafka Topic { "full_hostname": "ahmedashid.com", "parallelenricher.splitter.end.ts": "1544721368377", "code": 503, "method": "GET", "threat.triage.rules.0.name": "Rule1", "threat.triage.rules.0.comment": "", "threat.triage.score": 50.0, "is_alert": "true", "parallelenricher.enrich.begin.ts": "1544721368377", "url": "http:\/\/ahmedashid.com\/", "source.type": "squid", "elapsed": 73, "parallelenricher.splitter.begin.ts": "1544721368377", "threat.triage.rules.0.score": 50, "original_string": "1544699853.257 73 ::1 TCP_MISS\/503 4147 GET http:\/\/ahmedashid.com\/ - HIER_NONE\/- text\/html", "bytes": 4147, "threatintels.hbaseThreatIntel.domain_without_subdomains.zeusList": "alert", "domain_without_subdomains": "ahmedashid.com", "threat.triage.rules.0.reason": null, "action": "TCP_MISS", "guid": "6e1f11be-2738-436a-85ed-b3e22bea8ab6", "ip_src_addr": "::1", "timestamp": 1544699853257 } Retrieving data from notify_admin topic (No records Processed) ${HDP_HOME}/kafka-broker/bin/kafka-console-consumer.sh --zookeeper $ZOOKEEPER --topic notify_admin --from-beginning Alerts UI (Shows alerts indexed with threat triage score) Storm showing messages being emitted and parsed (notify_admin) ... View more

Hi, I have a single node cluster with metron installed. I have integrated Squid, Bro and Snort sensors. The problem I am facing is that if I simply run the squid sensor without adding geo enrichments the alerts get logged in the alerts UI. However, once I add geo enrichments they stop showing. The alerts are getting indexed with geo data in elastic search and can be viewed in Kibana. But they don't show in the alerts UI. What could be the problem? Parser Config (Added 5 Transformations which work fine) { "parserClassName": "org.apache.metron.parsers.GrokParser", "filterClassName": null, "sensorTopic": "abc2", "writerClassName": null, "errorWriterClassName": null, "invalidWriterClassName": null, "readMetadata": false, "mergeMetadata": false, "numWorkers": null, "numAckers": null, "spoutParallelism": 1, "spoutNumTasks": 1, "parserParallelism": 1, "parserNumTasks": 1, "errorWriterParallelism": 1, "errorWriterNumTasks": 1, "spoutConfig": {}, "securityProtocol": null, "stormConfig": {}, "parserConfig": { "grokPath": "/apps/metron/patterns/abc2", "patternLabel": "SQUID_DELIMITED", "timestampField": "timestamp" }, "fieldTransformations": [ { "output": [ "full_hostname", "domain_name", "year", "month" ], "config": { "full_hostname": "URL_TO_HOST(url)", "domain_name": "DOMAIN_REMOVE_SUBDOMAINS(full_hostname)", "year": "YEAR(DAY_OF_MONTH(timestamp))", "month": "MONTH(timestamp)+1" }, "transformation": "STELLAR" } ] } Enrichment Config { "enrichment": { "fieldMap": { "geo": [ "ip_dst_addr" ] }, "fieldToTypeMap": {}, "config": {} }, "threatIntel": { "fieldMap": {}, "fieldToTypeMap": {}, "config": {}, "triageConfig": { "riskLevelRules": [], "aggregator": "MAX", "aggregationConfig": {} } }, "configuration": {} } Indexing Config { "hdfs": { "batchSize": 1, "enabled": true, "index": "abc2" }, "elasticsearch": { "batchSize": 1, "enabled": true, "index": "abc2" }, "solr": { "batchSize": 1, "enabled": true, "index": "abc2" } } Kibana Template "abc2": { "order": 0, "template": "abc2*", "settings": {}, "mappings": { "squid_doc": { "dynamic_templates": [ { "geo_location_point": { "match": "enrichments:geo:*:location_point", "match_mapping_type": "*", "mapping": { "type": "geo_point" } } }, { "geo_country": { "match": "enrichments:geo:*:country", "match_mapping_type": "*", "mapping": { "type": "keyword" } } }, { "geo_city": { "match": "enrichments:geo:*:city", "match_mapping_type": "*", "mapping": { "type": "keyword" } } }, { "geo_location_id": { "match": "enrichments:geo:*:locID", "match_mapping_type": "*", "mapping": { "type": "keyword" } } }, { "geo_dma_code": { "match": "enrichments:geo:*:dmaCode", "match_mapping_type": "*", "mapping": { "type": "keyword" } } }, { "geo_postal_code": { "match": "enrichments:geo:*:postalCode", "match_mapping_type": "*", "mapping": { "type": "keyword" } } }, { "geo_latitude": { "match": "enrichments:geo:*:latitude", "match_mapping_type": "*", "mapping": { "type": "float" } } }, { "geo_longitude": { "match": "enrichments:geo:*:longitude", "match_mapping_type": "*", "mapping": { "type": "float" } } }, { "timestamps": { "match": "*:ts", "match_mapping_type": "*", "mapping": { "type": "date", "format": "epoch_millis" } } }, { "threat_triage_score": { "mapping": { "type": "float" }, "match": "threat:triage:*score", "match_mapping_type": "*" } }, { "threat_triage_reason": { "mapping": { "type": "text", "fielddata": "true" }, "match": "threat:triage:rules:*:reason", "match_mapping_type": "*" } }, { "threat_triage_name": { "mapping": { "type": "text", "fielddata": "true" }, "match": "threat:triage:rules:*:name", "match_mapping_type": "*" } } ], "properties": { "timestamp": { "type": "date", "format": "epoch_millis" }, "source:type": { "type": "keyword" }, "ip_dst_addr": { "type": "ip" }, "ip_dst_port": { "type": "integer" }, "ip_src_addr": { "type": "ip" }, "ip_src_port": { "type": "integer" }, "alert": { "type": "nested" }, "guid": { "type": "keyword" } } } }, "aliases": {} } Index In Kibana { "took": 0, "timed_out": false, "_shards": { "total": 5, "successful": 5, "skipped": 0, "failed": 0 }, "hits": { "total": 13, "max_score": 1, "hits": [ { "_index": "abc2_index_2018.11.13.05", "_type": "abc2_doc", "_id": "5b958e39-aab9-4acd-9f08-7371a7023a91", "_score": 1, "_source": { "date": 12, "full_hostname": "www.help.1and1.co.uk", "code": 301, "enrichments:geo:ip_dst_addr:location_point": "51.2993,9.491", "threatinteljoinbolt:joiner:ts": "1542088567977", "year": 2018, "enrichmentsplitterbolt:splitter:begin:ts": "1542088567966", "enrichmentjoinbolt:joiner:ts": "1542088567973", "adapter:geoadapter:begin:ts": "1542088567971", "enrichments:geo:ip_dst_addr:latitude": "51.2993", "elapsed": 1700, "domain_name": "1and1.co.uk", "source:type": "abc2", "ip_dst_addr": "213.165.66.7", "original_string": "1542024963.640 1700 ::1 TCP_MISS/301 759 GET http://www.help.1and1.co.uk/domains-c40986/transfer-domains-c79878 - HIER_DIRECT/213.165.66.7 text/html", "action": "TCP_MISS", "adapter:geoadapter:end:ts": "1542088567971", "ip_src_addr": "::1", "threatintelsplitterbolt:splitter:end:ts": "1542088567974", "enrichments:geo:ip_dst_addr:longitude": "9.491", "timestamp": 1542024963640, "method": "GET", "enrichmentsplitterbolt:splitter:end:ts": "1542088567966", "url": "http://www.help.1and1.co.uk/domains-c40986/transfer-domains-c79878", "month": 11, "bytes": 759, "threatintelsplitterbolt:splitter:begin:ts": "1542088567974", "guid": "5b958e39-aab9-4acd-9f08-7371a7023a91", "enrichments:geo:ip_dst_addr:country": "DE" } }, { "_index": "abc2_index_2018.11.13.05", "_type": "abc2_doc", "_id": "04158570-affc-4a40-8003-0b9ca5bb1b32", "_score": 1, "_source": { "date": 12, "full_hostname": "www.microsoftstore.com", "code": 302, "enrichments:geo:ip_dst_addr:location_point": "37.751,-97.822", "threatinteljoinbolt:joiner:ts": "1542088567981", "year": 2018, "enrichmentsplitterbolt:splitter:begin:ts": "1542088567972", "enrichmentjoinbolt:joiner:ts": "1542088567977", "adapter:geoadapter:begin:ts": "1542088567975", "enrichments:geo:ip_dst_addr:latitude": "37.751", "elapsed": 464, "domain_name": "microsoftstore.com", "source:type": "abc2", "ip_dst_addr": "69.192.203.198", "original_string": "1542024965.339 464 ::1 TCP_MISS/302 1620 GET https://www.microsoftstore.com/store/msusa/en_US/pdp/Microsoft-Band-2-Charging-Stand/productID.329506400 - HIER_DIRECT/69.192.203.198 text/html", "action": "TCP_MISS", "adapter:geoadapter:end:ts": "1542088567975", "ip_src_addr": "::1", "threatintelsplitterbolt:splitter:end:ts": "1542088567979", "enrichments:geo:ip_dst_addr:longitude": "-97.822", "timestamp": 1542024965339, "method": "GET", "enrichmentsplitterbolt:splitter:end:ts": "1542088567972", "url": "https://www.microsoftstore.com/store/msusa/en_US/pdp/Microsoft-Band-2-Charging-Stand/productID.329506400", "month": 11, "bytes": 1620, "threatintelsplitterbolt:splitter:begin:ts": "1542088567979", "guid": "04158570-affc-4a40-8003-0b9ca5bb1b32", "enrichments:geo:ip_dst_addr:country": "US" } }, { "_index": "abc2_index_2018.11.13.05", "_type": "abc2_doc", "_id": "690143c7-534e-4e85-a5a4-0980aacd7eb6", "_score": 1, "_source": { "date": 12, "full_hostname": "www.ebay.com", "code": 301, "enrichments:geo:ip_dst_addr:location_point": "37.751,-97.822", "threatinteljoinbolt:joiner:ts": "1542088567981", "year": 2018, "enrichmentsplitterbolt:splitter:begin:ts": "1542088567972", "enrichmentjoinbolt:joiner:ts": "1542088567977", "adapter:geoadapter:begin:ts": "1542088567975", "enrichments:geo:ip_dst_addr:latitude": "37.751", "elapsed": 149, "domain_name": "ebay.com", "source:type": "abc2", "ip_dst_addr": "23.44.160.209", "original_string": "1542024977.842 149 ::1 TCP_MISS/301 536 GET http://www.ebay.com/itm/02-Infiniti-QX4-Rear-spoiler-Air-deflector-Nissan-Pathfinder-/172240020293? - HIER_DIRECT/23.44.160.209 -", "action": "TCP_MISS", "adapter:geoadapter:end:ts": "1542088567975", "ip_src_addr": "::1", "threatintelsplitterbolt:splitter:end:ts": "1542088567979", "enrichments:geo:ip_dst_addr:longitude": "-97.822", "timestamp": 1542024977842, "method": "GET", "enrichmentsplitterbolt:splitter:end:ts": "1542088567972", "url": "http://www.ebay.com/itm/02-Infiniti-QX4-Rear-spoiler-Air-deflector-Nissan-Pathfinder-/172240020293?", "month": 11, "bytes": 536, "threatintelsplitterbolt:splitter:begin:ts": "1542088567979", "guid": "690143c7-534e-4e85-a5a4-0980aacd7eb6", "enrichments:geo:ip_dst_addr:country": "US" } }, { "_index": "abc2_index_2018.11.13.05", "_type": "abc2_doc", "_id": "e132f06e-9e72-413d-b0e6-77df5dbc4828", "_score": 1, "_source": { "date": 12, "full_hostname": "www.aliexpress.com", "code": 200, "enrichments:geo:ip_dst_addr:location_point": "37.751,-97.822", "threatinteljoinbolt:joiner:ts": "1542088567981", "year": 2018, "enrichmentsplitterbolt:splitter:begin:ts": "1542088567972", "enrichmentjoinbolt:joiner:ts": "1542088567977", "adapter:geoadapter:begin:ts": "1542088567975", "enrichments:geo:ip_dst_addr:latitude": "37.751", "elapsed": 1164, "domain_name": "aliexpress.com", "source:type": "abc2", "ip_dst_addr": "23.60.73.45", "original_string": "1542024981.369 1164 ::1 TCP_MISS/200 478359 GET http://www.aliexpress.com/af/shoes.html? - HIER_DIRECT/23.60.73.45 text/html", "action": "TCP_MISS", "adapter:geoadapter:end:ts": "1542088567975", "ip_src_addr": "::1", "threatintelsplitterbolt:splitter:end:ts": "1542088567979", "enrichments:geo:ip_dst_addr:longitude": "-97.822", "timestamp": 1542024981369, "method": "GET", "enrichmentsplitterbolt:splitter:end:ts": "1542088567972", "url": "http://www.aliexpress.com/af/shoes.html?", "month": 11, "bytes": 478359, "threatintelsplitterbolt:splitter:begin:ts": "1542088567979", "guid": "e132f06e-9e72-413d-b0e6-77df5dbc4828", "enrichments:geo:ip_dst_addr:country": "US" } }, { "_index": "abc2_index_2018.11.13.05", "_type": "abc2_doc", "_id": "d919eb13-1685-40f8-a74b-161e8687145d", "_score": 1, "_source": { "date": 12, "full_hostname": "www.autonews.com", "code": 302, "enrichments:geo:ip_dst_addr:location_point": "37.751,-97.822", "threatinteljoinbolt:joiner:ts": "1542088567981", "year": 2018, "enrichmentsplitterbolt:splitter:begin:ts": "1542088567972", "enrichmentjoinbolt:joiner:ts": "1542088567977", "adapter:geoadapter:begin:ts": "1542088567975", "enrichments:geo:ip_dst_addr:latitude": "37.751", "elapsed": 282, "domain_name": "autonews.com", "source:type": "abc2", "ip_dst_addr": "192.230.67.240", "original_string": "1542024965.627 282 ::1 TCP_MISS/302 2444 GET http://www.autonews.com/article/20151115/RETAIL04/311169971/toyota-fj-cruiser-is-scarce-hot-and-high-priced - HIER_DIRECT/192.230.67.240 text/html", "action": "TCP_MISS", "adapter:geoadapter:end:ts": "1542088567975", "ip_src_addr": "::1", "threatintelsplitterbolt:splitter:end:ts": "1542088567979", "enrichments:geo:ip_dst_addr:longitude": "-97.822", "timestamp": 1542024965627, "method": "GET", "enrichmentsplitterbolt:splitter:end:ts": "1542088567972", "url": "http://www.autonews.com/article/20151115/RETAIL04/311169971/toyota-fj-cruiser-is-scarce-hot-and-high-priced", "month": 11, "bytes": 2444, "threatintelsplitterbolt:splitter:begin:ts": "1542088567979", "guid": "d919eb13-1685-40f8-a74b-161e8687145d", "enrichments:geo:ip_dst_addr:country": "US" } }, { "_index": "abc2_index_2018.11.13.05", "_type": "abc2_doc", "_id": "17de42c9-daa4-4991-b6f5-17c84e891678", "_score": 1, "_source": { "date": 12, "full_hostname": "www.help.1and1.co.uk", "code": 301, "enrichments:geo:ip_dst_addr:location_point": "51.2993,9.491", "threatinteljoinbolt:joiner:ts": "1542088567981", "year": 2018, "enrichmentsplitterbolt:splitter:begin:ts": "1542088567972", "enrichmentjoinbolt:joiner:ts": "1542088567977", "adapter:geoadapter:begin:ts": "1542088567975", "enrichments:geo:ip_dst_addr:latitude": "51.2993", "elapsed": 323, "domain_name": "1and1.co.uk", "source:type": "abc2", "ip_dst_addr": "213.165.66.7", "original_string": "1542024980.198 323 ::1 TCP_MISS/301 759 GET http://www.help.1and1.co.uk/domains-c40986/transfer-domains-c79878 - HIER_DIRECT/213.165.66.7 text/html", "action": "TCP_MISS", "adapter:geoadapter:end:ts": "1542088567975", "ip_src_addr": "::1", "threatintelsplitterbolt:splitter:end:ts": "1542088567979", "enrichments:geo:ip_dst_addr:longitude": "9.491", "timestamp": 1542024980198, "method": "GET", "enrichmentsplitterbolt:splitter:end:ts": "1542088567972", "url": "http://www.help.1and1.co.uk/domains-c40986/transfer-domains-c79878", "month": 11, "bytes": 759, "threatintelsplitterbolt:splitter:begin:ts": "1542088567979", "guid": "17de42c9-daa4-4991-b6f5-17c84e891678", "enrichments:geo:ip_dst_addr:country": "DE" } }, { "_index": "abc2_index_2018.11.13.05", "_type": "abc2_doc", "_id": "4aa30241-fa1d-401d-a868-bcb588f79436", "_score": 1, "_source": { "date": 12, "full_hostname": "www.facebook.com", "code": 200, "enrichments:geo:ip_dst_addr:location_point": "37.751,-97.822", "threatinteljoinbolt:joiner:ts": "1542088567981", "year": 2018, "enrichmentsplitterbolt:splitter:begin:ts": "1542088567972", "enrichmentjoinbolt:joiner:ts": "1542088567977", "adapter:geoadapter:begin:ts": "1542088567975", "enrichments:geo:ip_dst_addr:latitude": "37.751", "elapsed": 1671, "domain_name": "facebook.com", "source:type": "abc2", "ip_dst_addr": "157.240.11.35", "original_string": "1542024968.069 1671 ::1 TCP_MISS/200 2762560 GET https://www.facebook.com/Africa-Bike-Week-1550200608567001/ - HIER_DIRECT/157.240.11.35 text/html", "action": "TCP_MISS", "adapter:geoadapter:end:ts": "1542088567975", "ip_src_addr": "::1", "threatintelsplitterbolt:splitter:end:ts": "1542088567979", "enrichments:geo:ip_dst_addr:longitude": "-97.822", "timestamp": 1542024968069, "method": "GET", "enrichmentsplitterbolt:splitter:end:ts": "1542088567972", "url": "https://www.facebook.com/Africa-Bike-Week-1550200608567001/", "month": 11, "bytes": 2762560, "threatintelsplitterbolt:splitter:begin:ts": "1542088567979", "guid": "4aa30241-fa1d-401d-a868-bcb588f79436", "enrichments:geo:ip_dst_addr:country": "US" } }, { "_index": "abc2_index_2018.11.13.05", "_type": "abc2_doc", "_id": "e233ccaf-2493-4b81-ac83-6c41183266da", "_score": 1, "_source": { "date": 12, "enrichments:geo:ip_dst_addr:locID": "1850147", "full_hostname": "www.recruit.jp", "code": 301, "enrichments:geo:ip_dst_addr:location_point": "35.685,139.7514", "threatinteljoinbolt:joiner:ts": "1542088567981", "year": 2018, "enrichmentsplitterbolt:splitter:begin:ts": "1542088567972", "enrichmentjoinbolt:joiner:ts": "1542088567977", "adapter:geoadapter:begin:ts": "1542088567975", "enrichments:geo:ip_dst_addr:latitude": "35.685", "elapsed": 443, "domain_name": "recruit.jp", "source:type": "abc2", "ip_dst_addr": "52.194.52.214", "original_string": "1542024978.290 443 ::1 TCP_MISS/301 650 GET http://www.recruit.jp/corporate/english/company/index.html - HIER_DIRECT/52.194.52.214 text/html", "action": "TCP_MISS", "adapter:geoadapter:end:ts": "1542088567975", "ip_src_addr": "::1", "threatintelsplitterbolt:splitter:end:ts": "1542088567979", "enrichments:geo:ip_dst_addr:longitude": "139.7514", "timestamp": 1542024978290, "method": "GET", "enrichmentsplitterbolt:splitter:end:ts": "1542088567972", "enrichments:geo:ip_dst_addr:city": "Tokyo", "enrichments:geo:ip_dst_addr:postalCode": "190-0032", "url": "http://www.recruit.jp/corporate/english/company/index.html", "month": 11, "bytes": 650, "threatintelsplitterbolt:splitter:begin:ts": "1542088567979", "guid": "e233ccaf-2493-4b81-ac83-6c41183266da", "enrichments:geo:ip_dst_addr:country": "JP" } }, { "_index": "abc2_index_2018.11.13.05", "_type": "abc2_doc", "_id": "8a109c47-5a61-4fc3-87e6-07211ed761a0", "_score": 1, "_source": { "date": 12, "full_hostname": "www.aliexpress.com", "code": 200, "enrichments:geo:ip_dst_addr:location_point": "37.751,-97.822", "threatinteljoinbolt:joiner:ts": "1542088567977", "year": 2018, "enrichmentsplitterbolt:splitter:begin:ts": "1542088567966", "enrichmentjoinbolt:joiner:ts": "1542088567973", "adapter:geoadapter:begin:ts": "1542088567971", "enrichments:geo:ip_dst_addr:latitude": "37.751", "elapsed": 992, "domain_name": "aliexpress.com", "source:type": "abc2", "ip_dst_addr": "23.60.73.45", "original_string": "1542020614.967 992 ::1 TCP_MISS/200 480259 GET http://www.aliexpress.com/af/shoes.html? - HIER_DIRECT/23.60.73.45 text/html", "action": "TCP_MISS", "adapter:geoadapter:end:ts": "1542088567971", "ip_src_addr": "::1", "threatintelsplitterbolt:splitter:end:ts": "1542088567974", "enrichments:geo:ip_dst_addr:longitude": "-97.822", "timestamp": 1542020614967, "method": "GET", "enrichmentsplitterbolt:splitter:end:ts": "1542088567966", "url": "http://www.aliexpress.com/af/shoes.html?", "month": 11, "bytes": 480259, "threatintelsplitterbolt:splitter:begin:ts": "1542088567974", "guid": "8a109c47-5a61-4fc3-87e6-07211ed761a0", "enrichments:geo:ip_dst_addr:country": "US" } }, { "_index": "abc2_index_2018.11.13.05", "_type": "abc2_doc", "_id": "19b2a08e-6227-4152-8366-1af51e0c9e80", "_score": 1, "_source": { "date": 12, "full_hostname": "www.pravda.ru", "code": 301, "enrichments:geo:ip_dst_addr:location_point": "55.7386,37.6068", "threatinteljoinbolt:joiner:ts": "1542088567977", "year": 2018, "enrichmentsplitterbolt:splitter:begin:ts": "1542088567966", "enrichmentjoinbolt:joiner:ts": "1542088567973", "adapter:geoadapter:begin:ts": "1542088567971", "enrichments:geo:ip_dst_addr:latitude": "55.7386", "elapsed": 1180, "domain_name": "pravda.ru", "source:type": "abc2", "ip_dst_addr": "185.201.54.50", "original_string": "1542024964.825 1180 ::1 TCP_MISS/301 519 GET http://www.pravda.ru/science/ - HIER_DIRECT/185.201.54.50 text/html", "action": "TCP_MISS", "adapter:geoadapter:end:ts": "1542088567971", "ip_src_addr": "::1", "threatintelsplitterbolt:splitter:end:ts": "1542088567975", "enrichments:geo:ip_dst_addr:longitude": "37.6068", "timestamp": 1542024964825, "method": "GET", "enrichmentsplitterbolt:splitter:end:ts": "1542088567966", "url": "http://www.pravda.ru/science/", "month": 11, "bytes": 519, "threatintelsplitterbolt:splitter:begin:ts": "1542088567974", "guid": "19b2a08e-6227-4152-8366-1af51e0c9e80", "enrichments:geo:ip_dst_addr:country": "RU" } } ] } } ... View more

Both Hbase Master server and Hbase RegionServer are frequently going down. Please find below error I have extracted for both server. Can you please look at log and suggest what changes I need to make in configuration to fix it. Hbase Master /var/log/hbase/ gc.log-201810221025 [logfile] [centos@ip-172-31-46-233 hbase]$ cat gc.log-201810221025 Java HotSpot(TM) 64-Bit Server VM (25.181-b13) for linux-amd64 JRE (1.8.0_181-b13), built on Jul 7 2018 00:56:38 by "java_re" with gcc 4.3.0 20080428 (Red Hat 4.3.0-8) Memory: 4k page, physical 16265880k(227160k free), swap 0k(0k free) CommandLine flags: -XX:ErrorFile=/var/log/hbase/hs_err_pid%p.log -XX:InitialHeapSize=260254080 -XX:MaxHeapSize=1073741824 -XX:MaxNewSize=348966912 -XX:MaxTenuringThreshold=6 -XX:OldPLABSize=16 -XX:OnOutOfMemoryError=kill -9 %p -XX:+PrintGC -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC -XX:+UseParNewGC Heap par new generation total 76800K, used 10973K [0x00000000c0000000, 0x00000000c5350000, 0x00000000d4cc0000) eden space 68288K, 16% used [0x00000000c0000000, 0x00000000c0ab75a0, 0x00000000c42b0000) from space 8512K, 0% used [0x00000000c42b0000, 0x00000000c42b0000, 0x00000000c4b00000) to space 8512K, 0% used [0x00000000c4b00000, 0x00000000c4b00000, 0x00000000c5350000) concurrent mark-sweep generation total 170688K, used 0K [0x00000000d4cc0000, 0x00000000df370000, 0x0000000100000000) Metaspace used 2977K, capacity 4480K, committed 4480K, reserved 1056768K class space used 313K, capacity 384K, committed 384K, reserved 1048576K Hbase RegionServer /var/log/hbase/gc.log-201810221025 [logfile] [centos@ip-172-31-47-215 hbase]$ cat gc.log-201810221025 Java HotSpot(TM) 64-Bit Server VM (25.181-b13) for linux-amd64 JRE (1.8.0_181-b13), built on Jul 7 2018 00:56:38 by "java_re" with gcc 4.3.0 20080428 (Red Hat 4.3.0-8) Memory: 4k page, physical 16265880k(380008k free), swap 0k(0k free) CommandLine flags: -XX:CMSInitiatingOccupancyFraction=50 -XX:ErrorFile=/var/log/hbase/hs_err_pid%p.log -XX:InitialHeapSize=2147483648 -XX:MaxHeapSize=2147483648 -XX:MaxNewSize=427819008 -XX:MaxTenuringThreshold=6 -XX:NewSize=427819008 -XX:OldPLABSize=16 -XX:OnOutOfMemoryError=kill -9 %p -XX:+PrintGC -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+UseCMSInitiatingOccupancyOnly -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC -XX:+UseParNewGC Heap par new generation total 376064K, used 20060K [0x0000000080000000, 0x0000000099800000, 0x0000000099800000) eden space 334336K, 6% used [0x0000000080000000, 0x00000000813970c8, 0x0000000094680000) from space 41728K, 0% used [0x0000000094680000, 0x0000000094680000, 0x0000000096f40000) to space 41728K, 0% used [0x0000000096f40000, 0x0000000096f40000, 0x0000000099800000) concurrent mark-sweep generation total 1679360K, used 0K [0x0000000099800000, 0x0000000100000000, 0x0000000100000000) Metaspace used 2981K, capacity 4480K, committed 4480K, reserved 1056768K class space used 313K, capacity 384K, committed 384K, reserved 1048576K [centos@ip-172-31-47-215 hbase]$ Also I am getting below output while submitting hadoop ls command [centos@ip-172-31-46-233 hbase]$ hadoop fs -ls /user % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1344 100 1344 0 0 3468 0 --:--:-- --:--:-- --:--:-- 3463 Cron exists Regards, Mustafa Akmal ... View more