Created on 08-28-202009:44 AM - edited 10-26-202009:45 AM
The all new Cloudera Data Engineering Experience
I recently had the opportunity to work with Cloudera Data Engineering to stream data from Kafka. It's quite interesting how I was able to deploy code without much worry about how to configure the back end components.
This demo will pull from the Twitter API using NiFi, write to payload to a Kafka topic named "twitter". Spark Streaming on Cloudera Data Engineering Experience CDE will pull from the twitter topic, extract the text field from the payload (which is the tweet itself) and write back to another Kafka topic named "tweet"
The following is an example of a twitter payload. The objective is to extract only the text field:
What is Cloudera Data Engineering?
Cloudera Data Engineering (CDE) is a serverless service for Cloudera Data Platform that allows you to submit Spark jobs to an auto-scaling cluster. CDE enables you to spend more time on your applications, and less time on infrastructure.
How do I begin with Cloudera Data Engineering (CDE)?