Support Questions

Find answers, ask questions, and share your expertise

Unable to fetch twitter data using flume getting java heap outofmemory error

avatar

when i execute below command from terminal with sudo user getting below error

 

command:sudo flume-ng agent -n TwitterAgent -c /opt/cloudera/parcels/CDH-5.2.1-1.cdh5.2.1.p0.12/lib/flume-ng/conf -f /opt/cloudera/parcels/CDH-5.2.1-1.cdh5.2.1.p0.12/lib/flume-ng/conf/flumetwitter.conf

 

Error:

at twitter4j.AbstractStreamImplementation$1.run(AbstractStreamImplementation.java:100)
at twitter4j.internal.async.ExecuteThread.run(DispatcherImpl.java:116)
Exception in thread "Twitter Stream consumer-1[Receiving stream]" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2367)
at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130)

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "Twitter Stream consumer-1[Receiving stream]"

 

requesting any one guide me in resolving above issue

 thanks

 

1 ACCEPTED SOLUTION

avatar

yes!

my self i found the answer for this issue

 

solution:

increse the java heap size using the command

export JAVA_OPTS="-Xms1000m -Xmx3000m -Dcom.sun.management.jmxremote"

or else modify the same in flume-env.sh which you can find in conf directory under flume instalation

View solution in original post

1 REPLY 1

avatar

yes!

my self i found the answer for this issue

 

solution:

increse the java heap size using the command

export JAVA_OPTS="-Xms1000m -Xmx3000m -Dcom.sun.management.jmxremote"

or else modify the same in flume-env.sh which you can find in conf directory under flume instalation