Hi all, I'm trying out the new Hortonworks Data Cloud but am encountering an error "Infrastructure creation failed. Reason: Operation timed out. Could not reach ssh connection in time". Any advice on what is causing this issue? Also, is there a means to check / track the progress of the install? The information on the UI is quite limited and it took nearly 4 hours before the job failed. Thanks! ... View more

I'm running a sample topology from storm-starter in local mode but an encountering an error 14:16:24.038 [Thread-16] ERROR o.a.s.e.EventManagerImp - {} Error when processing event java.lang.RuntimeException: java.io.IOException: Unable to delete file: C:\Users\<UserId>\AppData\Local\Temp\ef9408c7-6ad6-432f-928c-e01ded7f4c33\supervisor\tmp\b44d290f-ff15-46cf-b39e-ab02cf93a451\stormconf.ser at org.apache.storm.daemon.supervisor.SyncSupervisorEvent.run(SyncSupervisorEvent.java:173) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT] at org.apache.storm.event.EventManagerImp$1.run(EventManagerImp.java:54) [storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT] Caused by: java.io.IOException: Unable to delete file: C:\Users\<UserId>\AppData\Local\Temp\ef9408c7-6ad6-432f-928c-e01ded7f4c33\supervisor\tmp\b44d290f-ff15-46cf-b39e-ab02cf93a451\stormconf.ser at org.apache.storm.shade.org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2381) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT] at org.apache.storm.shade.org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1679) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT] at org.apache.storm.shade.org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1575) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT] at org.apache.storm.shade.org.apache.commons.io.FileUtils.moveDirectory(FileUtils.java:2916) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT] at org.apache.storm.daemon.supervisor.SyncSupervisorEvent.downloadLocalStormCode(SyncSupervisorEvent.java:354) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT] at org.apache.storm.daemon.supervisor.SyncSupervisorEvent.downloadStormCode(SyncSupervisorEvent.java:326) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT] at org.apache.storm.daemon.supervisor.SyncSupervisorEvent.run(SyncSupervisorEvent.java:122) ~[storm-core-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT] ... 1 more Would appreciate any assistance to solve this. Thanks! ... View more

I have Metron deployed on a single node on AWS. Recently had to update the hostname to use AWS private DNS instead of the public DNS (which changes with each reboot). I think I have got most of the services working after the update but I have still have some issues with Storm. A sample of the storm worker logs are copied below. In particular, the o.a.h.i.Client seems to be still referring to the old public EC2 domain name but I have been unable to figure out where that config is specified. Could someone assist in pointing me to where that particular variable is stored? 2016-07-27 06:41:21.625 s.k.ZkCoordinator [INFO] Task [1/1] Deleted partition managers: [] 2016-07-27 06:41:21.625 s.k.ZkCoordinator [INFO] Task [1/1] New partition managers: [] 2016-07-27 06:41:21.625 s.k.ZkCoordinator [INFO] Task [1/1] Finished refreshing 2016-07-27 06:41:22.253 b.s.m.n.Server [INFO] Getting metrics for server on port 6704 2016-07-27 06:41:24.037 o.a.h.i.Client [INFO] Retrying connect to server: ec2-54-213-184-142.us-west-2.compute.amazonaws.com/54.213.184.142:8020. Already tried 32 time(s); maxRetries=45 2016-07-27 06:41:44.058 o.a.h.i.Client [INFO] Retrying connect to server: ec2-54-213-184-142.us-west-2.compute.amazonaws.com/54.213.184.142:8020. Already tried 33 time(s); maxRetries=45 2016-07-27 06:42:04.078 o.a.h.i.Client [INFO] Retrying connect to server: ec2-54-213-184-142.us-west-2.compute.amazonaws.com/54.213.184.142:8020. Already tried 34 time(s); maxRetries=45 2016-07-27 06:42:21.626 s.k.ZkCoordinator [INFO] Task [1/1] Refreshing partition manager connections 2016-07-27 06:42:21.627 s.k.DynamicBrokersReader [INFO] Read partition info from zookeeper: GlobalPartitionInformation{partitionMap={0=ip-10-0-0-21.us-west-2.compute.internal:6667}} 2016-07-27 06:42:21.627 s.k.KafkaUtils [INFO] Task [1/1] assigned [Partition{host=ip-10-0-0-21.us-west-2.compute.internal:6667, partition=0}] 2016-07-27 06:42:21.628 s.k.ZkCoordinator [INFO] Task [1/1] Deleted partition managers: [] 2016-07-27 06:42:21.628 s.k.ZkCoordinator [INFO] Task [1/1] New partition managers: [] 2016-07-27 06:42:21.628 s.k.ZkCoordinator [INFO] Task [1/1] Finished refreshing 2016-07-27 06:42:22.254 b.s.m.n.Server [INFO] Getting metrics for server on port 6704 2016-07-27 06:42:24.104 o.a.h.i.Client [INFO] Retrying connect to server: ec2-54-213-184-142.us-west-2.compute.amazonaws.com/54.213.184.142:8020. Already tried 35 time(s); maxRetries=45 2016-07-27 06:42:44.121 o.a.h.i.Client [INFO] Retrying connect to server: ec2-54-213-184-142.us-west-2.compute.amazonaws.com/54.213.184.142:8020. Already tried 36 time(s); maxRetries=45 2016-07-27 06:43:04.139 o.a.h.i.Client [INFO] Retrying connect to server: ec2-54-213-184-142.us-west-2.compute.amazonaws.com/54.213.184.142:8020. Already tried 37 time(s); maxRetries=45 2016-07-27 06:43:21.629 s.k.ZkCoordinator [INFO] Task [1/1] Refreshing partition manager connections 2016-07-27 06:43:21.630 s.k.DynamicBrokersReader [INFO] Read partition info from zookeeper: GlobalPartitionInformation{partitionMap={0=ip-10-0-0-21.us-west-2.compute.internal:6667}} 2016-07-27 06:43:21.631 s.k.KafkaUtils [INFO] Task [1/1] assigned [Partition{host=ip-10-0-0-21.us-west-2.compute.internal:6667, partition=0}] ... View more

Hi, is it possible to deploy Metron on a single node (or maybe 3 node cluster at most) on AWS? We managed to follow the instructions to set up the 10 node AWS cluster but for exploration purposes we don't need such a large setup. Simply changing the number of hosts / roles in the playbook.yml file doesn't seem to work either as it breaks the installation at various points. ... View more

Hi, I'm experimenting with using Zeppelin / Spark to perform geo-location on IP addresses using the Maxmind GeoIP library. I am encountering a NoSuchMethodError which from reading the forums appears to be a dependency issue with the method not being in certain versions of the jackson lib. How can I go about identifying and resolving this dependency issue in Zeppelin? I load geoip2 via %dep and have removed the older versions of the jackson lib from zeppelin/lib/lib to no avail. Thanks! %dep z.addRepo("geoip2").url("http://mvnrepository.com/artifact/com.maxmind.geoip2/geoip2/2.7.0") z.load("com.maxmind.geoip2:geoip2:2.7.0") java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.node.ArrayNode.<init>(Lcom/fasterxml/jackson/databind/node/JsonNodeFactory;Ljava/util/List;)V ... View more

I'm using the NiFi ConvertCSVToAvro processor, with an InferAvroSchema processor upstream to obtain the schema. I'm getting an error Failed to Convert 1031/6058 records so most of the records are being converted successfully but I'm unable to figure out why the remainder rows failed. How do I go about debugging / identifying reasons for the failed rows and are there some typical reasons for the failure? ... View more