Member since
11-26-2022
1
Post
0
Kudos Received
0
Solutions
11-26-2022
07:11 PM
0 I download Apache Flume 1.11.0, Hadoop 3.3.1 and Java JDK 8u341. I configured everything correctly and tried to retrieve data from Twitter. My Hadoop and java does work well. Also created a directory in localhost: The following is the conf file: TwitterAgent.sources = Twitter TwitterAgent.channels = MemChannel TwitterAgent.sinks = HDFS TwitterAgent.sources.Twitter.type = org.apache.flume.source.twitter.TwitterSource TwitterAgent.sources.Twitter.consumerKey = TwitterAgent.sources.Twitter.consumerSecret = TwitterAgent.sources.Twitter.accessToken = TwitterAgent.sources.Twitter.accessTokenSecret = TwitterAgent.sources.Twitter.keywords = tutorials point,java, bigdata, mapreduce, mahout, hbase, nosql TwitterAgent.sinks.HDFS.type = hdfs TwitterAgent.sinks.HDFS.hdfs.path = TwitterAgent.sinks.HDFS.hdfs.fileType = DataStream TwitterAgent.sinks.HDFS.hdfs.writeFormat = Text TwitterAgent.sinks.HDFS.hdfs.batchSize = 1000 TwitterAgent.sinks.HDFS.hdfs.rollSize = 0 TwitterAgent.sinks.HDFS.hdfs.rollCount = 10000 TwitterAgent.channels.MemChannel.type = memory TwitterAgent.channels.MemChannel.capacity = 10000 TwitterAgent.channels.MemChannel.transactionCapacity = 100 TwitterAgent.sources.Twitter.channels = MemChannel TwitterAgent.sinks.HDFS.channel = MemChannel But while using Flume to retrieve data, it gives the following message: Even after a hour, this stays like this. It doesn't show errors and no data is collected... Should I provide any other info? What should be done?
... View more
Labels:
- Labels:
-
Apache Flume
-
Apache Hadoop