Support Questions
Find answers, ask questions, and share your expertise

Metron squid demo not indexing

Metron squid demo not indexing

Explorer

Hey,

im new in the metron world and i was trying the squid demo for this link.

https://cwiki.apache.org/confluence/display/METRON/2016/04/25/Metron+Tutorial+-+Fundamentals+Part+1%...

everything worked as expected except for the index in the elasticsearch it doesn't show using the below link.

http://localhost:9200/_cat/indices?v

health status index   uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   twitter IlyhgBkVS3ejRrMdeeNdzw   5   1          0            0       810b           810b
yellow open   .kibana w44VwW9_RyC2QvVJ9RU3GQ   1   1         46            4    126.7kb        126.7kb

i have 5 node cluster installed using Ambari.

currently the storm UI log shows no errors as follows below:

Capture.JPG

 

2020-01-29 15:33:35.231 o.a.m.p.GrokParser Thread-13-parserBolt-executor[5 5] [DEBUG] Grok parser formatting timestamp 1580109305.258
2020-01-29 15:33:35.231 o.a.m.p.GrokParser Thread-13-parserBolt-executor[5 5] [DEBUG] Grok parser parsed message: {elapsed=355, code=301, ip_dst_addr=104.108.35.126, original_string=1580109305.258    355 127.0.0.1 TCP_MISS/301 2179 GET http://www.aliexpress.com/af/shoes.html? - HIER_DIRECT/104.108.35.126 -, method=GET, bytes=2179, action=TCP_MISS, ip_src_addr=127.0.0.1, url=http://www.aliexpress.com/af/shoes.html?, timestamp=1580109305258}
2020-01-29 15:33:35.231 o.a.m.s.c.CachingStellarProcessor Thread-13-parserBolt-executor[5 5] [DEBUG] Created cache key; org.apache.metron.stellar.common.CachingStellarProcessor$Key@500373cc[expression=URL_TO_HOST(url),input={url=http://www.aliexpress.com/af/shoes.html?}]
2020-01-29 15:33:35.231 o.a.m.s.c.CachingStellarProcessor Thread-13-parserBolt-executor[5 5] [DEBUG] Created cache key; org.apache.metron.stellar.common.CachingStellarProcessor$Key@19fc94d6[expression=DOMAIN_REMOVE_SUBDOMAINS(full_hostname),input={full_hostname=www.aliexpress.com}]
2020-01-29 15:33:35.232 o.a.m.p.GrokParser Thread-13-parserBolt-executor[5 5] [DEBUG] Grok parser validating message: {full_hostname=www.aliexpress.com, code=301, method=GET, url=http://www.aliexpress.com/af/shoes.html?, source.type=squid, elapsed=355, ip_dst_addr=104.108.35.126, original_string=1580109305.258    355 127.0.0.1 TCP_MISS/301 2179 GET http://www.aliexpress.com/af/shoes.html? - HIER_DIRECT/104.108.35.126 -, bytes=2179, domain_without_subdomains=aliexpress.com, action=TCP_MISS, guid=cb522852-5d23-46a5-a39f-f2ded6dfc8a0, ip_src_addr=127.0.0.1, timestamp=1580109305258}
2020-01-29 15:33:35.232 o.a.m.p.GrokParser Thread-13-parserBolt-executor[5 5] [DEBUG] Grok parser validated message: {full_hostname=www.aliexpress.com, code=301, method=GET, url=http://www.aliexpress.com/af/shoes.html?, source.type=squid, elapsed=355, ip_dst_addr=104.108.35.126, original_string=1580109305.258    355 127.0.0.1 TCP_MISS/301 2179 GET http://www.aliexpress.com/af/shoes.html? - HIER_DIRECT/104.108.35.126 -, bytes=2179, domain_without_subdomains=aliexpress.com, action=TCP_MISS, guid=cb522852-5d23-46a5-a39f-f2ded6dfc8a0, ip_src_addr=127.0.0.1, timestamp=1580109305258}
2020-01-29 15:33:35.232 o.a.m.w.AckTuplesPolicy Thread-13-parserBolt-executor[5 5] [DEBUG] Adding tuple with messages ids: cb522852-5d23-46a5-a39f-f2ded6dfc8a0
2020-01-29 15:33:35.291 o.a.s.u.LocalState heartbeat-timer [DEBUG] New Local State for /hadoop/storm/workers/5829d9a2-da96-49fc-b594-9d5d50fe2af6/heartbeats
2020-01-29 15:33:35.410 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:35.411 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:35.612 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:35.613 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:35.813 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:35.814 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:36.015 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:36.016 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:36.217 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:36.218 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:36.293 o.a.s.u.LocalState heartbeat-timer [DEBUG] New Local State for /hadoop/storm/workers/5829d9a2-da96-49fc-b594-9d5d50fe2af6/heartbeats
2020-01-29 15:33:36.418 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:36.419 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:36.620 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:36.621 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:36.821 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:36.831 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:37.031 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:37.033 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:37.215 o.a.k.c.c.i.AbstractCoordinator Thread-15-kafkaSpout-executor[4 4] [DEBUG] Received successful heartbeat response for group squid_parser
2020-01-29 15:33:37.233 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:37.234 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:37.295 o.a.s.u.LocalState heartbeat-timer [DEBUG] New Local State for /hadoop/storm/workers/5829d9a2-da96-49fc-b594-9d5d50fe2af6/heartbeats
2020-01-29 15:33:37.324 o.a.s.s.o.a.z.ClientCnxn main-SendThread(mdr-3:2181) [DEBUG] Reading reply sessionid:0x36fc314e93528ce, packet:: clientPath:null serverPath:null finished:false header:: 353,3  replyHeader:: 353,4298258571,0  request:: '/storm/workerbeats/squid-73-1580304550/ab01252f-a649-4317-9e74-21acf9289615-6701,F  response:: s{4298257363,4298258548,1580304569310,1580304824885,85,0,0,0,550,0,4298257363} 
2020-01-29 15:33:37.327 o.a.s.s.o.a.z.ClientCnxn main-SendThread(mdr-3:2181) [DEBUG] Reading reply sessionid:0x36fc314e93528ce, packet:: clientPath:null serverPath:null finished:false header:: 354,5  replyHeader:: 354,4298258572,0  request:: '/storm/workerbeats/squid-73-1580304550/ab01252f-a649-4317-9e74-21acf9289615-6701,#1fffffff8b80000000ffffffed574b6effffffc23010ffffff9dfffffffc5a20ffffffc9ffffffa2ffffffa2ffffff87ffffffe8629ffffff884f5137ffffffad7affffff85ffffffee6b59105048ffffff9affffff9845ffffff8fffffffd0ffffffe3fffffff466ffffffbd411bfffffff2ffffff81ffffffd436ffffffce1022554544ffffff8a6211ffffffbf673cfffffff3fffffffc6662ffffff8360ffffffedffffffe86d134c3afffffff7ffffffbd4e7730fffffff27a5e7f30fffffff05cffffffd01d277e77ffffffd548ffffffa6ffffffe8dffffffd0ffffffb7f7041ffffffb3ffffffdd7814ffffffff474ffffffe0ffffff80ffffffe1ffffff80ffffff96ffffffff6effffffc6ffffffb7ffffffd5fffffff546ffffff9effffffe7ffffffb4ffffffe2ffffffa16627fffffff0621ffffffd17bffffffc4fffffffcffffffa5ffffff9dffffffd0ffffffb4959fffffffa2ccffffffc611ffffffa1ffffffb359ffffffe8ffffffcf28fffffff321ffffffbb6effffffe3ffffffdb18ffffff9e4ffffffb746ffffffc3fffffffe494ffffffcd7ffffffba587458ffffffb0fffffff4ffffffabffffff92ffffffecffffffa2ffffffe3ffffff82ffffffc1ffffffc7ffffffc5ffffffac449795cffffffaaffffffc3ffffffb3ffffffb8542728ffffffc4ffffffa522ffffff89bffffffe6452645ffffff9958177514494c30ffffff9ffffffffaffffffd938751e23751effffffa3ffffffd4794038733bffffffb5411e67ffffffe9a5affffffbaffffff8256ffffffd1ffffffdb5a6b1a467efffffff8ffffffbc5affffffb074ffffff8b4d42ffffffe878ffffffbeffffffbd556245ffffffc244ffffff9122ffffff81727162ffffffc048ffffffef42501ffffff88ffffffaa44ffffffc2443522ffffff81721562ffffffc06a6f3affffffc334ffffffabffffffbcffffffe7ffffffecffffffb22bffffff9cfffffffcffffffeffffffff8ffffff8a4ffffffffefffffff61ffffffe29ffffffb7ffffff9368ffffff82ffffffd2ffffffb0ffffffeeffffffb2060ffffffe7ffffffa6ffffffbbffffffc6ffffffceffffffde6f15ffffff85ffffffc84fffffffed3fffffffde2ffffffdaffffffcf51ffffffddffffffc3ffffffcd7effffffa5ffffff88ffffffd1ffffff906dffffffd6ffffffb91010ffffffbdffffff831a5cffffffda39ffffffa8ffffffe1ffffffa8ffffffbe41417154ffffffd770ffffff987ffffffa0ffffffb46750ffffff834b3b6351cffffffd52f2829ffffffc45a6effffffa5ffffffb5ffffffdcffffffaaffffffe73bffffffe5ffffff8a1016ffffffa8fffffffdffffffbd1c512a1696ffffffa5101e57ffffffedffffff9b236779fffffffcffffffc2ffffff8a42ffffff822855ffffff824ffffff834a3fffffff8f3bffffffe007a567ffffffe6743affffffa72fffffffebffffffd526ffffffab3ffffffd7137f4a37bffffffb6ffffff8bffffffd1ffffffd169ffffffaeffffff9dffffffb920ffffff87ffffffdaffffffb939ffffffd9ffffffd4ffffffcb7fffffffc0754a17ffffffc953ffffff8759ffffffe4ffffff97fffffffa6a672effffffa8ffffffb4766e4effffffcd75fffffff3ffffff8b4e67ffffffa64e67ffffffaa7a172effffffd142ffffffb1ffffff97ffffffbcffffffe7affffffbc64ffffff86ffffffa4ffffffa8fffffff3ffffffb314ffffffbdffffffc85ffffffffdffffffa5ffffffc4fffffffd75456fffffffc1ffffffc7ffffffd878ffffffed7e7c36ffffffb65bffffffd04cfffffff81ffffffbaffffffcffffffffe59281300,-1  response:: s{4298257363,4298258572,1580304569310,1580304827890,86,0,0,0,551,0,4298257363} 
2020-01-29 15:33:37.434 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:37.436 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:37.636 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:37.638 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:37.838 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:37.840 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:38.040 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:38.041 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:38.243 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:38.244 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:38.298 o.a.s.u.LocalState heartbeat-timer [DEBUG] New Local State for /hadoop/storm/workers/5829d9a2-da96-49fc-b594-9d5d50fe2af6/heartbeats
2020-01-29 15:33:38.444 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:38.446 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:38.646 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:38.648 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:38.766 o.a.s.s.o.a.z.ClientCnxn main-SendThread(mdr-3:2181) [DEBUG] Reading reply sessionid:0x36fc314e93528ce, packet:: clientPath:null serverPath:null finished:false header:: 355,3  replyHeader:: 355,4298258581,0  request:: '/storm/assignments/squid-73-1580304550,T  response:: s{4298257313,4298257313,1580304561404,1580304561404,0,0,0,0,204,0,4298257313} 
2020-01-29 15:33:38.769 o.a.s.s.o.a.z.ClientCnxn main-SendThread(mdr-3:2181) [DEBUG] Reading reply sessionid:0x36fc314e93528ce, packet:: clientPath:null serverPath:null finished:false header:: 356,3  replyHeader:: 356,4298258581,0  request:: '/storm/storms/squid-73-1580304550,T  response:: s{4298257312,4298257312,1580304561076,1580304561076,0,0,0,0,194,0,4298257312} 
2020-01-29 15:33:38.769 o.a.s.s.o.a.z.ClientCnxn main-SendThread(mdr-3:2181) [DEBUG] Reading reply sessionid:0x36fc314e93528ce, packet:: clientPath:null serverPath:null finished:false header:: 357,4  replyHeader:: 357,4298258581,0  request:: '/storm/storms/squid-73-1580304550,T  response:: #1fffffff8b8000000035ffffff8e3deffffffc230cffffff855dffffffa0101affffffba70ffffff904870416ffffffa64e456223ffffff8a5a53ffffffa2ffffffd2ffffffa6ffffffd8ffffffe9ffffffc0d389ffffffe724fffffffdffffffc1ffffffd2ffffffb3ffffff9ffffffff42cfffffffbffffff9310140ffffffccffffffafffffffdeffffff96216ffffffc1472ffffff96ffffffc34861254530ffffffebffffffa0ffffffa436fffffff7ffffffdaffffffe4ffffff9dffffffebfffffffd105ffffffedffffff91ffffffc851ffffff86ffffffccffffffa6ffffffc22b59ffffff8f3427426b7effffffb3ffffffc76ffffffa63a6bffffffddffffffa0275bffffffb0ffffffa34affffff99ffffffce14f54f533affffffd7ffffffa939382affffffb66dffffffadffffffd83b6a543efffffff4ffffff8b6dfffffff0695bffffffccffffffa6ffffff853cffffffc4fffffff3fffffff9ffffffa433ffffffc44827fffffff7fffffffcffffffa36cffffffb436453dfffffffd1710ffffffdfeffffff9fffffffaf1cffffffa9ffffffe3fffffff160a5bffffffb91b40242423ffffffd0f0ffffffa7ffffffa7ffffffa6fffffff3000,s{4298257312,4298257312,1580304561076,1580304561076,0,0,0,0,194,0,4298257312} 
2020-01-29 15:33:38.770 o.a.s.d.worker refresh-active-timer [DEBUG] Event debug options {}
2020-01-29 15:33:38.849 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:38.850 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:39.050 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:39.052 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:39.253 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:39.254 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:39.300 o.a.s.u.LocalState heartbeat-timer [DEBUG] New Local State for /hadoop/storm/workers/5829d9a2-da96-49fc-b594-9d5d50fe2af6/heartbeats
2020-01-29 15:33:39.436 o.a.z.ClientCnxn Thread-13-parserBolt-executor[5 5]-SendThread(mdr-3:2181) [DEBUG] Got ping response for sessionid: 0x36fc314e93528cf after 0ms
2020-01-29 15:33:39.454 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:39.456 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:39.656 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:39.658 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:39.859 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:39.860 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:40.060 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:40.062 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:40.215 o.a.k.c.c.i.AbstractCoordinator Thread-15-kafkaSpout-executor[4 4] [DEBUG] Received successful heartbeat response for group squid_parser
2020-01-29 15:33:40.262 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:40.264 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:40.303 o.a.s.u.LocalState heartbeat-timer [DEBUG] New Local State for /hadoop/storm/workers/5829d9a2-da96-49fc-b594-9d5d50fe2af6/heartbeats
2020-01-29 15:33:40.329 o.a.s.s.o.a.z.ClientCnxn main-SendThread(mdr-3:2181) [DEBUG] Reading reply sessionid:0x36fc314e93528ce, packet:: clientPath:null serverPath:null finished:false header:: 358,3  replyHeader:: 358,4298258584,0  request:: '/storm/workerbeats/squid-73-1580304550/ab01252f-a649-4317-9e74-21acf9289615-6701,F  response:: s{4298257363,4298258572,1580304569310,1580304827890,86,0,0,0,551,0,4298257363} 
2020-01-29 15:33:40.332 o.a.s.s.o.a.z.ClientCnxn main-SendThread(mdr-3:2181) [DEBUG] Reading reply sessionid:0x36fc314e93528ce, packet:: clientPath:null serverPath:null finished:false header:: 359,5  replyHeader:: 359,4298258585,0  request:: '/storm/workerbeats/squid-73-1580304550/ab01252f-a649-4317-9e74-21acf9289615-6701,#1fffffff8b80000000ffffffed574b6effffffc23010ffffff9dfffffffcfffffff8255954fffffff410ffffffdd205fffffff129ffffffeaffffffa655ffffffafffffffd07d2db242ffffff80ffffffa0ffffff8959fffffff48ffffffbd4ffffffffd61bffffffb4211f486dffffffe3c21525544ffffffa42816fffffff17bffffffc633ffffffcf6f2636680ffffffd0effffffdfffffffb6fffffff349ffffffe7ffffffbeffffffd7ffffffe9e465effffffcfffffffebf6ffffff9ebffffffbaffffffe344ffffffef6affffff8d78ffffff8affffffde07dfffffff701734ffffffdbffffff8d46ffffffd1404f6e18e68ffffffd9ffffffef66745b5d6fffffffe4794e2b1a6a76c6f1012ffffffbeffffff87ffffffcc5fffffffd9314dffffff9bffffff90ffffff95ffffffcfffffff82fffffff938247436bfffffffc19653effffffa4ffffffd76d741bffffffc3ffffffb3ffffffe0ffffffd668ffffffd83fffffff8bffffffa0fffffff940ffffff97ffffffcbeffffff9bffffffaffffffffcffffffb224fffffffbffffffe8ffffffb860fffffff071314b514216ffffff97fffffff2fffffff0342effffffe5972712949ffffffe2ffffff8279ffffff95495e26ffffffd6551d791213ffffffccffffffa77e3a4effffff9cffffffc748ffffff9cffffffc728741e10ffffffceffffffdc5e6dffffff90ffffffc559ffffffbaffffff82ffffff96ffffffacffffffa0ffffff95fffffff4ffffffb6ffffffd6ffffff866ffffffa11f3cffffffafffffff972cffffffd962ffffff93103a5effffffec6effffff9558ffffff913051ffffffa448ffffffa05cffffff9c1830ffffffd2ffffffbb10540ffffffa22affffff913051ffffff8d48ffffffa05cffffff8518ffffffb0ffffffdaffffff9b2e30ffffffcd2affffffefffffffb9ffffffb8ffffffeca27ffffffff3bffffffbaffffffa2ffffff93ffffffbf7bffffff8078ffffffc2ffffffed38ffffff9affffffa034ffffffacffffffbb340ffffffd8ffffffb9ffffffc9ffffffaeffffffb1ffffffb3f5b4521ffffffb253fffffffbffffff8fffffffb7ffffff80fffffff67354fffffff770735829643460ffffffdb4d26444ffffffefffffffa061776e6a38ffffffaa6f5050ffffff9cffffffd4351cffffffe7128ffffffec19ffffffd4ffffffe0ffffffc2ffffff8e41d47fffffff5b4aaffffffb1ffffff965b492dffffffb7ffffffaafffffff94effffffa911ffffffc2ffffffe66a7f2f4614ffffff8a45ffffff824129ffffff84ffffffc7ffffff95fffffffbffffffe6ffffffc8581effffffbfffffffb0ffffffa2ffffff9020affffff9520ffffffc1ffffffa0ffffffd2ffffffcfffffffe3ffffff8e38ffffff80ffffff9effffffd6ffffff815ffffff9d2effffffe8ffffffcb66ffffffbd4dffffffeb407dffffffe24fffffffe976ffffffc9fffffff6313a39ffffffcdffffff9533ffffffe7ffffffe450393727ffffff9b6afffffff9ffffff8fffffffb84effffffe12259ffffffea30ffffff8bfffffffc525fffffffe5ffffffcc39ffffff9556ffffffceffffffcdffffffa9ffffffb96a7effffffd1ffffffe9ffffffccffffffc4ffffffe94c55ffffffefffffffc2255a28fffffff6ffffff92fffffff75cffffff81ffffff97ffffffccffffff9014757effffff96ffffffa217fffffff9ffffffabffffffbf14ffffffbbffffffbfffffffaeffffffe82dfffffff8181bffffffafffffffddffffff8fffffffcfffffffc66eb5a1d7e036ffffffd94438281300,-1  response:: s{4298257363,4298258585,1580304569310,1580304830896,87,0,0,0,551,0,4298257363} 
2020-01-29 15:33:40.464 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:40.465 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:40.666 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:40.667 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:40.867 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:40.869 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:41.069 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:41.070 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:41.270 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:41.272 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:41.305 o.a.s.u.LocalState heartbeat-timer [DEBUG] New Local State for /hadoop/storm/workers/5829d9a2-da96-49fc-b594-9d5d50fe2af6/heartbeats
2020-01-29 15:33:41.472 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:41.473 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:41.674 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:41.675 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:41.875 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:41.877 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:42.077 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:42.078 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:42.279 o.a.s.k.s.KafkaSpout Thread-15-kafkaSpout-executor[4 4] [DEBUG] Polled [0] records from Kafka
2020-01-29 15:33:42.280 o.a.s.k.s.KafkaSpoutRetryExponentialBackoff Thread-15-kafkaSpout-executor[4 4] [DEBUG] Topic partitions with entries ready to be retried [{}] 
2020-01-29 15:33:42.307 o.a.s.u.LocalState heartbeat-timer [DEBUG] New Local State for /hadoop/storm/workers/5829d9a2-da96-49fc-b594-9d5d50fe2af6/heartbeats

following file contents as follows ${METRON_HOME}/config/zookeeper/indexing/squid.json


{
"elasticsearch": {
"index": "squid",
"batchSize": 5,
"enabled" : true
},
"hdfs":{
"index": "squid",
"batchSize": 5,
"enabled" : true
}
}

 

i really appreciate all the help 

many thanks