Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How can I force the getTwitter processor to not truncate tweets?

Solved Go to solution

How can I force the getTwitter processor to not truncate tweets?

New Contributor

Hey all,

I am currently ingesting some tweets from the filter endpoint of the getTwitter processor. I've been noticing that many of the tweets (I believe over 140 characters) are truncated and simply provide a link to the full tweet. After looking around, it seems that many other twitter REST APIs have the ability to throw a flag on their stream, tweet_mode=extended, that can force the full text of a tweet to be sent.

Is there any way to implement this with the existing getTwitter processor, or will I have to come up with some REST calls of my own?

Any implementation ideas or suggestions are appreciated!

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: How can I force the getTwitter processor to not truncate tweets?

@Nick Lewis

I don't think there are any configuration options within the current GetTwitter processor. I took a look at the source code and I didn't see anything that would expose that option or anything that limits the length of the text. Here is the code: https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-social-media-bundle/nifi-twitter-pr...

You can probably do what you want using other processors to query the Twitter API directly. I'm not sure if that will provide the "real time" feed you are looking for. Take a look at InvokeHTTP to see if that might work for you.

3 REPLIES 3
Highlighted

Re: How can I force the getTwitter processor to not truncate tweets?

@Nick Lewis

I don't think there are any configuration options within the current GetTwitter processor. I took a look at the source code and I didn't see anything that would expose that option or anything that limits the length of the text. Here is the code: https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-social-media-bundle/nifi-twitter-pr...

You can probably do what you want using other processors to query the Twitter API directly. I'm not sure if that will provide the "real time" feed you are looking for. Take a look at InvokeHTTP to see if that might work for you.

Re: How can I force the getTwitter processor to not truncate tweets?

New Contributor

I was thinking that would be the case. I'm getting all my data from other social media platforms with invokeHttp at the moment. Thanks for looking into things!

Re: How can I force the getTwitter processor to not truncate tweets?

The underlying Hosebird library that GetTwitter uses to interface with the Twitter REST API does not yet support the extended tweets. Unfortunately, it also looks like the project is not very active so I'm not sure we'd see that capability added anytime soon.