Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Issue while trying to implement spark streaming to get twitter data to process

Highlighted

Issue while trying to implement spark streaming to get twitter data to process

Explorer
6 REPLIES 6
Highlighted

Re: Issue while trying to implement spark streaming to get twitter data to process

Mentor

you probably missing some twitter dependencies. See where twitter conf is coming from

Highlighted

Re: Issue while trying to implement spark streaming to get twitter data to process

Explorer

can you please suggst me how to add twitter dependency as I am using hdp 2.4 and created jar and trying to submit.

I guess while using spark-submit I need to add some packages.

Highlighted

Re: Issue while trying to implement spark streaming to get twitter data to process

Mentor

Edit your pom.xml file to add the proper dependencies

Highlighted

Re: Issue while trying to implement spark streaming to get twitter data to process

Explorer

Hey Artem, I have done this in sbt project and I have uploaded my build.sbt file as well can you please check once.

Highlighted

Re: Issue while trying to implement spark streaming to get twitter data to process

Explorer

I am able to get now. One thing I had to import all twitter streaming related jars during spark submit. Do we have any other option for this?

Highlighted

Re: Issue while trying to implement spark streaming to get twitter data to process

Mentor

Instead of pulling dependencies during spark-submit you can bundle the with your jar by adding it's dependency in maven pom.xml

Don't have an account?
Coming from Hortonworks? Activate your account here