Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Labels (1)
Super Guru

Automating Social Media: Sending Tweets with Apache NiFi: Building A Smart Bot


72659-tweetflow1.png

72660-tweetflow2.png

72661-tweetflow3.png

This is the new processor, PutTwitterProcessor, it will use Twitter4J to send tweets from a parameter you set.

72662-tweetautomatedtwitter.png

This is an example tweet and my reply to it. It's a regular tweet.

72663-puttweetresults.png

This is the result of a run. I put the tweet id and datetime stamp in the result attribute.

72664-puttweetatmos.png

Another example:

72668-puttweetparameters.png

You need to create a Twitter application and obtain the Consumer Key, Consumer Secret, Access Token and Access secret. If you have a lat and long in your data you can send that. It's optional. The message takes expression language and lets you build a message.

JUnit Test of my Processor

72665-tweetcode1.png

JUnit Run Results

72666-tweetcodejunit.png

Most of the code is using the very easy to use Twitter4J library.

72667-tweetcodeviatwitter4j.png


So what can I do with this? In my flow I am ingesting Tweets and with a streaming SQL query I can retweet ones where there's no existing retweet and the followers of the tweet are more than 1,000. I can also feed all this information to a Hive table and then run some machine learning on it to figure out other parameters to filter on for my bot.

This is a start. I put a scheduler on my PutTweet to only tweet every 10 minutes (600 seconds). Let's see how this goes.

SQL for QueryRecord

SELECT * FROM FLOWFILE
WHERE CAST(retweet_count AS DOUBLE) <= 0
AND CAST(followers_count AS DOUBLE) > 1000

Example Message Building

${'user_name':append( ' '):append( ${'location'} ):append(' '):append( ${'hashtags'} ):append(' ' ):append(${'msg'})}

Source:

https://github.com/tspannhw/nifi-puttwitter-processor

Download NAR to install in Apache NiFi lib directory and then restart:

https://github.com/tspannhw/nifi-puttwitter-processor/releases/tag/1.0

499 Views
Comments
Super Guru

Quick tip: When you create a free Twitter app at apps.twitter.com, they will let you read and write but not a lot. Don't tweet every 5 minutes for hours or you will get that app's write ability blocked. Also be sure you are no reposting SPAM.

Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
2 of 2
Last update:
‎08-17-2019 07:33 AM
Updated by:
 
Contributors
Top Kudoed Authors