How can I force the getTwitter processor to not truncate tweets?

[#1]

Hey all,

I am currently ingesting some tweets from the filter endpoint of the getTwitter processor. I've been noticing that many of the tweets (I believe over 140 characters) are truncated and simply provide a link to the full tweet. After looking around, it seems that many other twitter REST APIs have the ability to throw a flag on their stream, tweet_mode=extended, that can force the full text of a tweet to be sent.

Is there any way to implement this with the existing getTwitter processor, or will I have to come up with some REST calls of my own?

Any implementation ideas or suggestions are appreciated!

[#2 · Accepted solution]

@Nick Lewis

I don't think there are any configuration options within the current GetTwitter processor. I took a look at the source code and I didn't see anything that would expose that option or anything that limits the length of the text. Here is the code: https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-social-media-bundle/nifi-twitter-pr...

You can probably do what you want using other processors to query the Twitter API directly. I'm not sure if that will provide the "real time" feed you are looking for. Take a look at InvokeHTTP to see if that might work for you.

[#3]

I was thinking that would be the case. I'm getting all my data from other social media platforms with invokeHttp at the moment. Thanks for looking into things!

[#4]

The underlying Hosebird library that GetTwitter uses to interface with the Twitter REST API does not yet support the extended tweets. Unfortunately, it also looks like the project is not very active so I'm not sure we'd see that capability added anytime soon.