Unable to fetch twitter data using flume getting java heap outofmemory error

vamsibalineni — Fri, 16 Sep 2022 10:56:18 GMT

when i execute below command from terminal with sudo user getting below error

command:sudo flume-ng agent -n TwitterAgent -c /opt/cloudera/parcels/CDH-5.2.1-1.cdh5.2.1.p0.12/lib/flume-ng/conf -f /opt/cloudera/parcels/CDH-5.2.1-1.cdh5.2.1.p0.12/lib/flume-ng/conf/flumetwitter.conf

Error:

at twitter4j.AbstractStreamImplementation$1.run(AbstractStreamImplementation.java:100)
at twitter4j.internal.async.ExecuteThread.run(DispatcherImpl.java:116)
Exception in thread "Twitter Stream consumer-1[Receiving stream]" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2367)
at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130)

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "Twitter Stream consumer-1[Receiving stream]"

requesting any one guide me in resolving above issue

thanks

Re: Unable to fetch twitter data using flume getting java heap outofmemory error

vamsibalineni — Tue, 24 Jan 2017 13:31:00 GMT

yes!

my self i found the answer for this issue

solution:

increse the java heap size using the command

export JAVA_OPTS="-Xms1000m -Xmx3000m -Dcom.sun.management.jmxremote"

or else modify the same in flume-env.sh which you can find in conf directory under flume instalation

question Unable to fetch twitter data using flume getting java heap outofmemory error in Archives of Support Questions (Read Only)

Unable to fetch twitter data using flume getting java heap outofmemory error

Re: Unable to fetch twitter data using flume getting java heap outofmemory error