Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Twitter Analysis via FLUME - Why can't I view the log or see any tweets?

Highlighted

Twitter Analysis via FLUME - Why can't I view the log or see any tweets?

Explorer

Hi

I'm trying to view the twitter agent log, however I can't as per below:

[root@sandbox ~]# hdfs dfs -chmod 777 /user/flume/tweets

chmod: changing permissions of '/user/flume/tweets': Permission denied. user=root is not the owner of inode=tweets

[root@sandbox ~]# /var/log/flume/flume-TwitterAgent.log

-bash: /var/log/flume/flume-TwitterAgent.log: Text file busy

I'm trying to use flume to retrieve tweets based on some given keywords, but not having any luck, please can someone advise?

Thanks

Raf

3 REPLIES 3

Re: Twitter Analysis via FLUME - Why can't I view the log or see any tweets?

Unsure what's your problem - looking at the log

[root@sandbox ~]# hdfs dfs -chmod 777 /user/flume/tweets
chmod: changing permissions of '/user/flume/tweets': Permission denied. user=root is not the owner of inode=tweets

You need to log as hdfs - the root user on Linux is nothing on Hadoop - the hdfs user is the FS owner.

[root@sandbox ~]# su - hdfs
[hdfs@sandbox ~]# hdfs dfs -chmod 777 /user/flume/tweets

On the second part, you are trying to execute a log file

[root@sandbox ~]# /var/log/flume/flume-TwitterAgent.log
-bash: /var/log/flume/flume-TwitterAgent.log: Text file busy
Highlighted

Re: Twitter Analysis via FLUME - Why can't I view the log or see any tweets?

If you are trying to read a log file. You may want to use the following command - tail, less, vim... When i want to read a log files and see the changes, I tend to use the following command.

[root@sandbox ~]# tail -f /var/log/flume/flume-TwitterAgent.log

Also, you seem to have posted twice the same question. Could you please close the other one?

Highlighted

Re: Twitter Analysis via FLUME - Why can't I view the log or see any tweets?

Explorer

@Olivier Renault Thank's for this. Yeah I realised i needed to use hdfs as opposed to root as hdfs is equivalent to a 'superuser'. somehow I've managed to 'Lock' the log file and as such I can't seem to view it. I will report back with further info and hopefully a resolution for my own issue :-)

Don't have an account?
Coming from Hortonworks? Activate your account here