Member since
12-21-2016
83
Posts
5
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
41274 | 02-08-2017 05:56 AM | |
5729 | 01-02-2017 11:05 PM |
01-02-2017
11:01 PM
Is there any spelling mistake in it?
... View more
01-02-2017
10:41 PM
I have logged into the twitter app and registered my app and through that i got the Oauth keys and i have placed them in the config file. below is the conf file and i have removed my keys and replaced it as 'XXX' (masking it). TwitterAgent.sources = Twitter TwitterAgent.sinks = HDFS TwitterAgent.channels = MemChannel TwitterAgent.sources.Twitter.type = com.cloudera.flume.source.TwitterSource TwitterAgent.sources.Twitter.channels = MemChannel TwitterAgent.sources.Twitter.consumerkey = XXX TwitterAgent.sources.Twitter.consumerSecret = XXX TwitterAgent.sources.Twitter.accessToken = XXX TwitterAgent.sources.Twitter.accessTokenSecret = XXX TwitterAgent.sources.Twitter.keywords = seattle TwitterAgent.sinks.HDFS.channel = MemChannel TwitterAgent.sinks.HDFS.type = hdfs TwitterAgent.sinks.HDFS.hdfs.path = hdfs://localhost:8020/user/cloudera/tweets/ TwitterAgent.sinks.HDFS.hdfs.filetype = DataStream TwitterAgent.sinks.HDFS.hdfs.writeFormat = Text TwitterAgent.sinks.HDFS.hdfs.batchsize = 1000 TwitterAgent.sinks.HDFS.hdfs.rollSize = 0 TwitterAgent.sinks.HDFS.hdfs.rollCount = 10000 TwitterAgent.channels.MemChannel.type = memory TwitterAgent.channels.MemChannel.capacity = 10000 TwitterAgent.channels.MemChannel.transactionalCapacity = 100
... View more
01-02-2017
10:06 PM
I am trying to fetch twitter data into Hdfs and i have set all the required credentials and attached is the error log, which i am getting while running flume-agent and attached is the is my conf file. Appreciate if someone could be help me. ERROR lifecycle.LifecycleSupervisor: Unable to start EventDrivenSourceRunner: { source:com.cloudera.flume.source.TwitterSource{name:Twitter,state:IDLE} } - Exception follows. java.lang.IllegalStateException: Authentication credentials are missing. See http://twitter4j.org/en/configuration.html for details at twitter4j.TwitterBaseImpl.ensureAuthorizationEnabled(TwitterBaseImpl.java:215) at twitter4j.TwitterStreamImpl.filter(TwitterStreamImpl.java:317) at com.cloudera.flume.source.TwitterSource.start(TwitterSource.java:150) at org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44) at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Flume.conf
... View more
Labels:
- Labels:
-
Apache Flume
12-29-2016
11:29 PM
OK .. thanks for the update. you mean to say that the hive version which i am using is not supported to the i am looking for ?
... View more
12-29-2016
07:22 PM
capture.pngI am using Hadoop 2.7X version and hive i am using 1.2 x version and below is the screen shot.
... View more
12-29-2016
06:51 PM
Thanks and when i am trying to give command in hive. I am getting SemanticException [Error 10011]: Line 1:7 Invalid function 'md5/' select md5(location) as md_location from testing; Basically i am trying to get the checksum on the location column.
... View more
12-29-2016
06:16 PM
1 Kudo
In Hive, I want to compare the data between two tables and i want to generate a checksum for each column and then compare each checksum for each column. Appreciate if you could let me know if there is any checksum function in hive.
... View more
Labels:
- Labels:
-
Apache Hive
12-27-2016
07:34 AM
what is the difference between both of the configuration which i posted and the one which you gave me?
... View more
12-25-2016
07:19 AM
Any suggestions?
... View more