I see your Docker version seems old. Can you upgrade to a more recent and try again? I am running ... Docker version 17.12.0-ce, build c97c6d6 ... View more

@Habtamu Wubneh the install seems broken looking at this error: Applying File['/usr/hcp/1.5.1.0-16/metron/config/enrichment-splitjoin.properties'] failed, parent directory /usr/hcp/1.5.1.0-16/metron/config doesn't exist Can you check if this is a fresh install and that the folder is present? Also, you seem to be using an older version of HCP. Can you try with latest? ... View more

Hi @Akmal Latip, any way you can give this a try on another system? I think this has to do with the node and npm versions since I see that 'npm ci' is failing. ... View more

You will need to address this issue then. Can you post the error you are seeing upon running mvn clean package -DskipTests ... View more

Yup, that's the top level directory ... View more

Yes, those are the directories. You can spin up full dev from either of them. Also, before attempting vagrant up , can you try to run mvn clean package -DskipTests from the top level metron directory? This should pass without issues for the full dev deployment to work properly. ... View more

Can you confirm that you are following the below steps to deploy: a) cd metron-deployment/development/ubuntu14 b) vagrant up ... View more

Hi @Akmal Latip, I am having the following versions of node and npm on my box and it works fine. Can you try with the below versions and see if it resolves your issue? node v9.11.1 -- npm 5.6.0 ... View more

Hello @Tony Jabbour can you post the output of metron/metron-deployment/scripts/platform-info.sh ... View more

This is not supported yet. ... View more

Hello @Moka, HCP does not support Ambari 2.7 and HDP 3.1. Please refer to https://supportmatrix.hortonworks.com for the list of supported versions. ... View more

Looks like this topology is already running per the log: Exception in thread "main" java.lang.RuntimeException: Topology with name `batch_indexing` already exists on cluster You can run storm list to check if the topology is running and if it is, then run storm kill batch_indexing to terminate the existing one and restart indexing topology ... View more

Hi @Hemamalini Nithyanandam , can you tell me the steps that fixed the problem for you? ... View more

It looks like you are trying this with HDP 3.x -- which is not supported as yet by HCP. Can you try with HDP 2.6.5.0 ? ... View more

It looks like you're running into METRON-1958 There's a pull request open to fix this issue (and should be merged soon enough) - https://github.com/apache/metron/pull/1317. You might want to give a try once this change is merged. ... View more

Have you checked this issue with python-requests package not installed: https://issues.apache.org/jira/browse/METRON-1451 ? ... View more

From your platform-info output, I see that you are running a more recent version of Docker (than what I have; I have pasted my platform info output below for your reference.). I recall someone from the community talk about issues with Docker. You can join the #metron slack channel in ASF and search for it in the archives. I think they mention having to reset Docker to factory default which resolved the issue. Also, let me know if the `buildrpms` command worked for you. My platform-info output: ➜ metron git:(master) metron-deployment/scripts/platform-info.sh Metron 0.6.1 -- * master -- commit 42fc699ce1df04fd065d131293a8322a095c1e9c (HEAD -> master, origin/master, origin/HEAD) Author: nickwallen <nick@nickallen.org> Date: Tue Nov 27 12:44:58 2018 -0500 METRON-1888 Default Topology Settings in MPack Cause Profiler to Stall (nickwallen) closes apache/metron#1276 -- -- ansible 2.5.0 config file = None configured module search path = [u'/Users/asubramanian/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules'] ansible python module location = /usr/local/lib/python2.7/site-packages/ansible executable location = /usr/local/bin/ansible python version = 2.7.15 (default, Aug 17 2018, 22:39:05) [GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.2)] -- Vagrant 2.0.2 -- vagrant-hostmanager (1.8.9) -- Python 2.7.15 -- Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-18T00:03:14+05:30) Maven home: /usr/local/Cellar/maven/3.5.4/libexec Java version: 1.8.0_181, vendor: Oracle Corporation, runtime: /Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre Default locale: en_US, platform encoding: UTF-8 OS name: "mac os x", version: "10.13.6", arch: "x86_64", family: "mac" -- Docker version 17.12.0-ce, build c97c6d6 -- node v9.11.1 -- npm 5.6.0 -- Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1 Apple LLVM version 9.1.0 (clang-902.0.39.2) Target: x86_64-apple-darwin17.7.0 Thread model: posix InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin -- Compiler is C++11 compliant -- Darwin HW10098-2.local 17.7.0 Darwin Kernel Version 17.7.0: Thu Jun 21 22:53:14 PDT 2018; root:xnu-4570.71.2~1/RELEASE_X86_64 x86_64 -- Total System Memory = 16384 MB Processor Model: Intel(R) Core(TM) i7-4770HQ CPU Processor Speed: 2.20GHz Total Physical Processors: 4 Total cores: 4 Disk information: /dev/disk1s1 466Gi 251Gi 201Gi 56% 2175192 9223372036852600615 0% / /dev/disk1s4 466Gi 13Gi 201Gi 7% 13 9223372036854775794 0% /private/var/vm /dev/disk2s1 118Mi 82Mi 36Mi 70% 26 4294967253 0% /Volumes/next-macos-webkit /Volumes/next-macos-webkit/Next.app 118Mi 82Mi 36Mi 70% 26 4294967253 0% /private/var/folders/1f/071y_b_n4fv95v1xvl8qd82w0000gp/T/AppTranslocation/C93448EF-8135-4526-BC2F-0ACBF977787C This CPU appears to support virtualization ... View more

Hm..okay. Lets take a step back to see if everything else is ok. Can you paste the output of the following command: cd metron/metron-deployment/scripts ./platform-info.sh Also, just to isolate if this is a Deb-only issue, can you try the following as well: cd metron/metron-deployment mvn clean package -Pbuild-rpms ... View more

@Amirul, can you describe your environment more? Metron version? how did you deploy? Typically the 'user_settings' table is automatically created during deploy time. ... View more

Hi @suhans dev , can you try to run the following command separately? cd metron/metron-deployment mvn clean package -Pbuild-debs ... View more

Hi @Vinoth Krish and @Michael Hidalgo, Question: Do you see indices being written into ES (or Solr) for your parser? Here are few things you can check on your setup. a) Check the offsets on the kafka topics for your topic using the command: source /etc/default/metron /usr/hdp/current/kafka-broker/bin/kafka-consumer-groups.sh --describe --group <the-topic-group> --bootstrap-server $BROKERLIST --new-consumer You can repeat the above by substituting the group with the 'enrichment', 'indexing-ra' and so on. This will tell you the consumed offsets, current offset, lag and so on. From your error, I am guessing there could be an issue with the current offset and the log-end-offset values. b) Check your Kafka Enrichment offset strategy settings in Ambari UI, under Services -> Metron -> Configs -> Enrichment tab. One thing you can try for troubleshooting is to set this to 'EARLIEST' which allows all the tuples in the topic to be processed from the beginning . ... View more

Hi @David Auclair, can you check the contents of /etc/default/metron and the value for 'METRON_HOME' in that file? So, in the /etc/rc.d/init.d/metron-alerts-ui file, the above file is sourced (see this link) which sets the proper METRON_HOME variable. ... View more

Hi @Nor Liana Kamaruzzaman , Can you tell me which process you stopped (using Ctrl+C)? Did you do this inside the full-dev node or on your local host? I usually do the following in order to stop a running full dev setup and resume cd metron/metron-deployment/centos6 vagrant halt # -> This will shutdown the VM instance gracefully # Then when you want to resume, run: vagrant up Once the VM has come up, ambari-server process should be up and running (in case it hasn't, you can SSH to the node and issue ambari-server restart command) Login to the ambari server (http://node1:8080) and start all the services using Actions -> 'Start All' ... View more

Hey @Nor Liana Kamaruzzaman, this is great news! I am glad you were able to resolve the issues. Would you mind accepting the answer that helped you the most? Cheers, Anand ... View more

Hi @Nor Liana Kamaruzzaman, the single node uses Ambari to deploy the cluster. So what you are seeing is correct. I am suspecting if there is resource starvation because of which you are seeing the CPU locked out issues. Can you try to find a server with higher resources? Or upgrade the present server? ... View more

Hi @Nor Liana Kamaruzzaman, Can you try this solution here for the timeout issue with the vagrant VM? Also, from the platform-info script, I can see that node JS installation is missing. Can you fix that too and try again? node scripts/platform-info.sh: line 77: node: command not found -- npm scripts/platform-info.sh: line 82: npm: command not found ... View more

Hey @Nor Liana Kamaruzzaman , I am hoping you have got a chance to set up a different system with CentOS 7 and able to proceed further. In the context of this thread, would you mind marking the best answer if you feel the original question was answered? Thanks! ... View more

Hey @Nor Liana Kamaruzzaman , cool! I would suggest that you try with the Dev VM install for CentOS link, and with the latest 'master' branch. It should be pretty straight forward once you have the pre-requisites setup. Note that the latest master requires ansible version to be 2.4.0+ (see latest instructions here). ... View more

@Rudy Hartono, I cannot think of an immediate demerit at this point. Metron is actively having a lot of features added, at a quick pace in that. If any feature comes up that makes use of the HDFS store, then there is a chance you will miss the functionality if you have disabled writing in the first place. If you think my response helped, would you mind marking the answer as accepted? Thank you. ... View more