Member since
08-11-2016
10
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 816 | 06-20-2017 12:24 PM |
06-20-2017
12:24 PM
Just got this working. As the issue says, table not found. In these cases just create a table in hbase create 'threatintel','t'
... View more
06-20-2017
05:40 AM
I am running a Metron cluster (v. 0.4.0.1.1.0.0) installed via Ambari. While trying to load the threat intel feeds into Hbase using flatfile_loader.sh I am seeing the following error This is what I ran /usr/hcp/current/metron/bin/flatfile_loader.sh -n enrichment_config.json -i block_list.csv -t threatintel -c t -e extractor_config.json Error: Processing block_list.csv
Processed 3 - \17/06/20 05:59:37 ERROR client.AsyncProcess: Failed to get region location
org.apache.hadoop.hbase.TableNotFoundException: Table 'threatintel' was not found, got: profiler.
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(ConnectionManager.java:1284)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1165)
at org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:380)
at org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:331)
at org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:206)
at org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:183)
at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1422)
at org.apache.hadoop.hbase.client.HTable.put(HTable.java:1025)
at org.apache.metron.dataloads.nonbulk.flatfile.importer.LocalImporter.lambda$null$2(LocalImporter.java:111)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:291)
at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
17/06/20 05:59:37 ERROR client.AsyncProcess: Failed to get region location
org.apache.hadoop.hbase.TableNotFoundException: Table 'threatintel' was not found, got: profiler.
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(ConnectionManager.java:1284)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1165)
at org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:380)
at org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:331)
at org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:206)
at org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:183)
at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1422)
at org.apache.hadoop.hbase.client.HTable.put(HTable.java:1025)
at org.apache.metron.dataloads.nonbulk.flatfile.importer.LocalImporter.lambda$null$2(LocalImporter.java:111)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:291)
at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
17/06/20 05:59:37 ERROR client.AsyncProcess: Failed to get region location
org.apache.hadoop.hbase.TableNotFoundException: Table 'threatintel' was not found, got: profiler.
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(ConnectionManager.java:1284)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1165)
at org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:380)
at org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:331)
at org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:206)
at org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:183)
at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1422)
at org.apache.hadoop.hbase.client.HTable.put(HTable.java:1025)
at org.apache.metron.dataloads.nonbulk.flatfile.importer.LocalImporter.lambda$null$2(LocalImporter.java:111)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:291)
at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinTask.doInvoke(ForkJoinTask.java:401)
at java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:734)
at java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:160)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:583)
at org.apache.metron.dataloads.nonbulk.flatfile.importer.LocalImporter.lambda$null$3(LocalImporter.java:108)
at java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1386)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
17/06/20 05:59:37 ERROR client.AsyncProcess: Failed to get region location
org.apache.hadoop.hbase.TableNotFoundException: Table 'threatintel' was not found, got: profiler.
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(ConnectionManager.java:1284)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1165)
at org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:380)
at org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:331)
at org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:206)
at org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:183)
at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1422)
at org.apache.hadoop.hbase.client.HTable.put(HTable.java:1025)
at org.apache.metron.dataloads.nonbulk.flatfile.importer.LocalImporter.lambda$null$2(LocalImporter.java:111)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:291)
at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
From the above does it mean, a table name "threatintel" was supposed to be created when installing Metron but seems like this is missing in my case. If I have to create the table manually what is the schema for "threatintel"? Any help is greatly appreciated. Thanks Deep
... View more
Labels:
- Labels:
-
Apache Metron
08-12-2016
06:38 PM
Hi @Jobin George it is working now. I am able to authenticate via AD. I had made a small mistake and corrected it. Thanks for your quick response and also for the great tutorial. Thanks Deep
... View more
08-12-2016
04:06 PM
@Jobin George I configured for ldap as above but only thing is I connected it with AD. SSL works great but I don't see any trace of ldap authentication happening in logs. Certificate based authentication is working but not ldap. I have used ldap-provider and configured the same. Is there any property or anything that needs to be enabled for ldap authentication to be triggered other than using ldap-provider ?
... View more