Support Questions
Find answers, ask questions, and share your expertise

nutch web crawling using hbase in hortonworks

Contributor

i want crawl the web urls information using nutch and store the data in hbase db. any one can suggest for how to do this with some example. bcoz i am new one for nutch.

1 ACCEPTED SOLUTION

Accepted Solutions

4 REPLIES 4

Contributor

i got this error message

[root@sandbox ~]# bin/nutch fetch 1456727546-2019589981

Exception in thread "main" java.lang.RuntimeException: job failed: name=apache-nutch-2.3.1.jar, jobid=job_local522155708_0001 at org.apache.nutch.util.NutchJob.waitForCompletion(NutchJob.java:120) at org.apache.nutch.fetcher.FetcherJob.run(FetcherJob.java:205) at org.apache.nutch.fetcher.FetcherJob.fetch(FetcherJob.java:251) at org.apache.nutch.fetcher.FetcherJob.run(FetcherJob.java:314) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.nutch.fetcher.FetcherJob.main(FetcherJob.java:322)

@sivasaravanakumar k FYI: Nutch is not part of HDP stack