Member since
10-15-2016
1
Post
0
Kudos Received
0
Solutions
10-15-2016
12:28 PM
connector.json and extractor.json created as per this article. ansible pushed opentaxii as per this article. the service is started and i was able to pull several thousand elements from a few of the taxii servers. i can connect to localhost:9000 and there is a service waiting for input. when i enter this: 0.2.1BETA/bin/threatintel_taxii_load.sh -b "2016-10-13 00:00:00" -c taxii_connector.json -e taxii_extractor.json -p 10000 i get this (bold for emphasis): WARNING: Use "yarn jar" to launch YARN applications. 16/10/15 12:14:55 INFO taxii.TaxiiHandler: Loading configuration: TaxiiConnectionConfig{endpoint=http://localhost:9000/services/discovery, port=443, proxy=null, username='guest', password='******', type=DISCOVER, allowedIndicatorTypes=domainname:FQDN,address:IPV_4_ADDR, collection='guest.MalwareDomainList_Hostlist', subscriptionId='null', beginTime=Thu Oct 13 00:00:00 UTC 2016, table=threatintel:t} 16/10/15 12:14:55 INFO taxii.TaxiiHandler: Initializing client.. 16/10/15 12:14:55 INFO taxii.TaxiiHandler: Discovering endpoint 16/10/15 12:14:57 INFO taxii.TaxiiHandler: Request made : org.mitre.taxii.messages.xml11.DiscoveryRequest => org.mitre.taxii.messages.xml11.DiscoveryResponse (expected org.mitre.taxii.messages.xml11.DiscoveryResponse) 16/10/15 12:14:57 INFO taxii.TaxiiHandler: Discovered endpoint as http://localhost:9000/services/poll 16/10/15 12:14:57 INFO taxii.TaxiiHandler: Configured, starting polling http://localhost:9000/services/poll for guest.MalwareDomainList_Hostlist 16/10/15 12:14:57 INFO taxii.TaxiiHandler: Polling...10/15/16 12:14 PM 16/10/15 12:14:57 INFO taxii.TaxiiHandler: Begin Time: 2016-10-13T00:00:00Z 16/10/15 12:14:57 INFO taxii.TaxiiHandler: Request made : org.mitre.taxii.messages.xml11.PollRequest => org.mitre.taxii.messages.xml11.PollResponse (expected org.mitre.taxii.messages.xml11.PollResponse) 16/10/15 12:14:57 INFO taxii.TaxiiHandler: Got Poll Response with 0 blocks 16/10/15 12:15:07 INFO taxii.TaxiiHandler: Polling...10/15/16 12:15 PM 16/10/15 12:15:07 INFO taxii.TaxiiHandler: Begin Time: 2016-10-15T12:14:57Z 16/10/15 12:15:07 ERROR taxii.TaxiiHandler: Connection pool shut down java.lang.IllegalStateException: Connection pool shut down at org.apache.metron.httpcore.dataload.util.Asserts.check(Asserts.java:34)
at org.apache.metron.httpcore.dataload.pool.AbstractConnPool.lease(AbstractConnPool.java:169)
at org.apache.metron.httpcore.dataload.impl.conn.PoolingHttpClientConnectionManager.requestConnection(PoolingHttpClientConnectionManager.java:217)
at org.apache.metron.httpcore.dataload.impl.execchain.MainClientExec.execute(MainClientExec.java:158)
at org.apache.metron.httpcore.dataload.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195)
at org.apache.metron.httpcore.dataload.impl.execchain.RetryExec.execute(RetryExec.java:85)
at org.apache.metron.httpcore.dataload.impl.execchain.RedirectExec.execute(RedirectExec.java:108)
at org.apache.metron.httpcore.dataload.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:186)
at org.apache.metron.httpcore.dataload.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.mitre.taxii.client.HttpClient.callTaxiiService(HttpClient.java:297)
at org.apache.metron.dataloads.nonbulk.taxii.TaxiiHandler.call(TaxiiHandler.java:336)
at org.apache.metron.dataloads.nonbulk.taxii.TaxiiHandler.call(TaxiiHandler.java:242)
at org.apache.metron.dataloads.nonbulk.taxii.TaxiiHandler.run(TaxiiHandler.java:171)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505) Exception in thread "Timer-0" java.lang.RuntimeException: Unable to make request at org.apache.metron.dataloads.nonbulk.taxii.TaxiiHandler.run(TaxiiHandler.java:214)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505) Caused by: java.lang.IllegalStateException: Connection pool shut down at org.apache.metron.httpcore.dataload.util.Asserts.check(Asserts.java:34)
at org.apache.metron.httpcore.dataload.pool.AbstractConnPool.lease(AbstractConnPool.java:169)
at org.apache.metron.httpcore.dataload.impl.conn.PoolingHttpClientConnectionManager.requestConnection(PoolingHttpClientConnectionManager.java:217)
at org.apache.metron.httpcore.dataload.impl.execchain.MainClientExec.execute(MainClientExec.java:158)
at org.apache.metron.httpcore.dataload.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195)
at org.apache.metron.httpcore.dataload.impl.execchain.RetryExec.execute(RetryExec.java:85)
at org.apache.metron.httpcore.dataload.impl.execchain.RedirectExec.execute(RedirectExec.java:108)
at org.apache.metron.httpcore.dataload.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:186)
at org.apache.metron.httpcore.dataload.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.mitre.taxii.client.HttpClient.callTaxiiService(HttpClient.java:297)
at org.apache.metron.dataloads.nonbulk.taxii.TaxiiHandler.call(TaxiiHandler.java:336)
at org.apache.metron.dataloads.nonbulk.taxii.TaxiiHandler.call(TaxiiHandler.java:242)
at org.apache.metron.dataloads.nonbulk.taxii.TaxiiHandler.run(TaxiiHandler.java:171)
... 2 more am i missing somthing? i am not behind a proxy, this is a cloud based server, there is no real firewall between me and the internet, there is no local firewall, there is no SSL interception, i can access other sites with no issues, and metron as a whole is functional. thanks for any insight.
... View more