Code Repositories
Find and share code repositories
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Advanced Search

Sparkler Nutch Crawler in Spark

Labels (1)
Labels:
TimothySpann
Super Guru

Created on ‎12-18-2016 03:00 AM

Repo Description

A web crawler bot written in Spark with Kafka and Tika to replace Nutch. It renders Javascript and processes files with Tika.

https://github.com/uscdataScience/sparkler/wiki/sparkler-0.1

Repo Info
Github Repo URL https://github.com/USCDataScience/sparkler
Github account name USCDataScience
Repo name sparkler
1,414 Views
0 Kudos
Don't have an account?
Register
Announcements
What's New @ Cloudera
CDP Operational Database supports listing all HBase snapshots using various filters
What's New @ Cloudera
CDP Operational Database supports disabling the ephemeral storage in CLI
What's New @ Cloudera
CDP Operational Database supports DataHub/OpDB deployment across Multi-AZ
What's New @ Cloudera
CDP Operational Database supports a built-in coprocessor ‘AggregateImplementation’
Support Announcements
How validations can help enable your CDP upgrade success
View More Announcements
Version history
Last update:
‎12-18-2016 03:00 AM
Updated by:
Super Guru Super Guru
Contributors