Options
- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
CoreNLP Server on Spark
Labels:
- Labels:
-
Apache Spark
New Contributor
Created ‎07-12-2020 05:25 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I am new to Cloudera & even Spark. I tested my code with the CoreNLP server. Suppose I deploy this code on Spark & expect it to run in parallel on multiple documents, won't the CoreNLP server become a bottleneck & even a single point of failure.
When I searched for "CoreNLP Server on Spark", the results take me to Databricks - https://github.com/databricks/spark-corenlp
How do I make the NLP tasks running on Spark independent of a singleton service? Or am I missing something.
Stanford CoreNLP wrapper for Apache Spark. Contribute to databricks/spark-corenlp development by creating an account on GitHub.
1 REPLY 1
Rising Star
Created ‎07-14-2020 06:11 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
AFAIK, the Stanford CoreNLP wrapper for Apache Spark should not be a bottleneck in terms of parallel processing. Spark would take care of running it parallelly on multiple documents. Regardless of the number of documents, the number of API requests to the CoreNLP server would remain the same.
AFAIK, the Stanford CoreNLP wrapper for Apache Spark should not be a bottleneck in terms of parallel processing. Spark would take care of running it parallelly on multiple documents. Regardless of the number of documents, the number of API requests to the CoreNLP server would remain the same.
