Hey @Nor Liana Kamaruzzaman, this definitely looks to me like an issue with the Docker version. This error message especially: "Sending build context to Docker daemon ", "[ERROR] Command execution failed.", "org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)", "\tat org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404)", "\tat Would you be able to bring up a CentOS 7 environment afresh? It looks like the Docker pre-requisites asks for CentOS7. Here is an excerpt from their docs: https://docs.docker.com/install/linux/docker-ce/centos/#prerequisites OS requirements To install Docker CE, you need a maintained version of CentOS 7. Archived versions aren’t supported or tested. ... View more

Hm.. that is strange. I see from your output of platform-info.sh that you have the right version of the vagrant hostmanager plugin (ver 1.8.9). The one difference I see is that your machine is named as ' default ', while it should show as ' node1 '. See sample output from my box. Observe the logs.. ➜ ~ vagrant up Running with ansible-skip-tags: ["sensors"] Bringing machine 'node1' up with 'virtualbox' provider... ==> node1: Importing base box 'centos/6'... ==> node1: Matching MAC address for NAT networking... ==> node1: Checking if box 'centos/6' is up to date... ==> node1: Setting the name of the VM: centos6_node1_1537451480618_72587 ==> node1: Clearing any previously set network interfaces... ==> node1: Preparing network interfaces based on configuration... node1: Adapter 1: nat node1: Adapter 2: hostonly ==> node1: Forwarding ports... node1: 22 (guest) => 2222 (host) (adapter 1) ==> node1: Running 'pre-boot' VM customizations... ==> node1: Booting VM... ==> node1: Waiting for machine to boot. This may take a few minutes... node1: SSH address: 127.0.0.1:2222 node1: SSH username: vagrant node1: SSH auth method: private key node1: node1: Vagrant insecure key detected. Vagrant will automatically replace node1: this with a newly generated keypair for better security. node1: node1: Inserting generated public key within guest... node1: Removing insecure key from the guest if it's present... node1: Key inserted! Disconnecting and reconnecting using new SSH key... ==> node1: Machine booted and ready! ==> node1: Checking for guest additions in VM... node1: No guest additions were detected on the base box for this VM! Guest node1: additions are required for forwarded ports, shared folders, host only node1: networking, and more. If SSH fails on this machine, please install node1: the guest additions and repackage the box to continue. node1: node1: This is not an error message; everything may continue to work properly, node1: in which case you may ignore this message. ==> node1: Setting hostname... ==> node1: Configuring and enabling network interfaces... node1: SSH address: 127.0.0.1:2222 node1: SSH username: vagrant node1: SSH auth method: private key ==> node1: Rsyncing folder: /Users/asubramanian/Desktop/Metron/anand-metron-fork/shane-cypress-test/metron/metron-deployment/development/centos6/ => /vagrant ==> node1: [vagrant-hostmanager:guests] Updating hosts file on active guest virtual machines... ==> node1: [vagrant-hostmanager:host] Updating hosts file on your workstation (password may be required)... Password: ==> node1: Running provisioner: ansible... node1: Running ansible-playbook... PLAY [all] ********************************************************************* <br> Can you confirm that you are running the vagrant up command from the following folder location: metron/metron-deployment/development/centos6 Also, as suggested by @Otto Fowler, can you try to compile outside of the full-dev deployment and see if that works? You can do this like follows and look for any failures in the output.: cd <root-level-metron-folder> mvn clean package I would also like to look at your Vagrantfile at the following location. Can you attach it? metron/metron-deployment/development/centos6/Vagrantfile ... View more

Hi @Nor Liana Kamaruzzaman , the vagrant folder not being present seems to be okay. Did the vagrant up command proceed further after upgrading the node and npm? ... View more

Looks like the node and npm versions are still old. You can grab the ver 9.11.2 version of nodejs from here - https://nodejs.org/download/release/v9.11.2/ ... View more

Yes, it means Docker is running. However I see that it is an old version. Can you try with a more recent version of Docker and node ? ... View more

Hello @Rudy Hartono, You can disable writing into HDFS per sensor by changing the 'enabled' indexing configuration under $METRON_HOME/config/zookeeper/indexing/<your-sensor>.json For e.g., here's a sample for the bro sensor: [metron@metron-1 ~]$ cat $METRON_HOME/config/zookeeper/indexing/bro.json { "hdfs" : { "index": "bro", "batchSize": 5, "enabled" : true }, "elasticsearch" : { "index": "bro", "batchSize": 5, "enabled" : true }, "solr" : { "index": "bro", "batchSize": 5, "enabled" : true } } Note: Once you change the configuration, you need to use the 'zk_load_configs.sh' command to push the config changes into ZK. Refer here for more details. As to your other question.. for metron, HDFS serves as an archival store for an analyst to refer to at a later stage. As of now, the UI interface does not make use of the archived data for fetching events. ... View more

Hi @Nor Liana Kamaruzzaman , The docker and npm not getting listed should not happen. Also, I notice that you are using an older version of Metron (0.3.1). Can you try with a more recent version? i.e. Metron 0.5.0 (which is released) or 0.5.1 (master). ... View more

Hi @Nor Liana Kamaruzzaman , Yes, docker and nodejs are requisites. Here is a more recent version of the README for Ubuntu: https://github.com/apache/metron/tree/master/metron-deployment/development/ubuntu14 Let me know how it goes. ... View more

I'm afraid you will have to take your IT team's help in resolving the connection timeout issue since it is environment specific. ... View more

Looks like the settings are still not correct per this message: Connect to repo.hortonworks.com:80 [repo.hortonworks.com/54.225.131.199] failed: Connection timed out (Connection timed out) ... View more

@Rakesh S - refer to http://dev.hortonworks.com.s3.amazonaws.com/HDPDocuments/Ambari-Trunk/bk_ambari-installation/content/ch_using-local-repos.html ... View more

Hello @Nor Liana Kamaruzzaman , welcome to Metron! Can you tell more about which version of metron you are trying to use? And about your setup where are trying to do this? The simplest way to get Metron up and running is to follow these steps: https://github.com/apache/metron/tree/master/metron-deployment/development/centos6 Get the pre-requisites installed and deploy metron. If you happen to run into any issues, please post the error output and also the output of the script metron-deployment/scripts/platform-info.sh ... View more

If you're trying on a baremetal CentOS 7, try these instructions: https://cwiki.apache.org/confluence/display/METRON/Metron+0.4.1+with+HDP+2.5+bare-metal+install+on+Centos+7+with+MariaDB+for+Metron+REST ... View more

Hey @Gowdaman G, here's a link that shows how to setup Metron on a single node setup: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=68718548 Can you give this a try? ... View more

I noticed that your node and NPM versions are pretty old. Can you upgrade to latest? Here are my versions: node v10.6.0 -- npm 6.1.0 -- ... View more

Okay, from the below message it looks like maven is not able to reach the mirror. I think you need to review your maven settings.xml to see if it is in order. Failed to read artifact descriptor for org.apache.maven.plugins:maven-site-plugin:jar:3.7: Could not transfer artifact org.apache.maven.plugins:maven-site-plugin:pom:3.7 from/to central (https://repo.maven.apache.org/maven2): Connect to repo.maven.apache.org:443 [repo.maven.apache.org/151.101.12.215] failed: Connection timed out (Connection timed out) ... View more

Quick tip: You can use the 'CODE' in the editing toolbar to make command output look more readable. ... View more

@Gowdaman G Can you run the script at metron-deployment/scripts/platform-info.sh and paste the output here? ... View more

Hello @Lija Mohan, The Metron full dev deployment is tightly knit only to work on linux or mac OS'. Windows is unsupported. Please try on one of the *nix platform. As an additional note, you can run the metron-deployment/scripts/platform-info.sh script to ascertain if you have met the install pre-requisites. ... View more

Did you restart the Indexing topology after the change? Note that you might have to restart from command line, since Ambari will not allow stopping of an (already) stopped service. Can you post the version of python-requests you are running? ... View more

@Anil Reddy, @Maxim Dashenko Could you guys give this a try with from the master? It seems to work fine for me. I recall that a similar issue was addressed in the recent past (I am unable to find the PR number, though). Regarding #1 in the question, it is an expected behavior. The Alerts UI is expected to fetch all indices created in the Elasticsearch, so long as the event source has the nested alert field definition present in the event template. Here's an example: $ curl -XGET 'http://node1:9200/_template/squid_index' -d <snip> "alert": { "type": "nested" }, <snip> So, in essence, the Alerts UI is more like an investigator UI. The name "Alerts" is a bit misleading. ... View more

Here's the issue... you need to install docker ./platform-info.sh: line 64: docker: command not found ... View more

@Gaurav Bapat I am assuming you have provided the right switch for the command (is that a typo in your comment when you said '-PbuildRmps'?) Can you check if Docker service is up and running ? Please paste the output of metron-deployment/scripts/platform-info.sh as well. ... View more

Hm.. this error seems different than the earlier one. Can you list the version of urllib3 ? (pip list | grep url) ... View more

ok.. can you install the following version of requests and check if things work? pip install requests==2.6.1 ... View more

Also, can you describe your test environment more... * full dev or bare metal * version of Metron ... View more