Code Repositories

Find and share code repositories
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Advanced Search

Sparkler Nutch Crawler in Spark

Labels (1)
Labels:
TimothySpann
Super Guru

Created on ‎12-18-2016 03:00 AM

Repo Description

A web crawler bot written in Spark with Kafka and Tika to replace Nutch. It renders Javascript and processes files with Tika.

https://github.com/uscdataScience/sparkler/wiki/sparkler-0.1

Repo Info
Github Repo URL https://github.com/USCDataScience/sparkler
Github account name USCDataScience
Repo name sparkler
1,517 Views
0 Kudos
Take a Tour of the Community
Don't have an account?
Register
Your experience may be limited. Sign in to explore more.
Announcements
Product Announcements
[ANNOUNCE] CDP Private Cloud Base 7.1.7 Service Pack 2 Released
Product Announcements
[ANNOUNCE] CDP Private Cloud Data Services 1.5.0 Released
Community Announcements
January 2023 Community Highlights
Product Announcements
[ANNOUNCE] New Cloudera JDBC Connector 2.6.30 for Impala is Released
What's New @ Cloudera
Cloudera Operational Database (COD) provides a CLI option to enable HBase region canaries
View More Announcements
Version history
Last update:
‎12-18-2016 03:00 AM
Updated by:
Super Guru Super Guru
Contributors
Top Kudoed Authors
View Full Leaderboard