I tried with my desktop path also but it didn't worked ... View more

@Artem Ervits i added the dataset in attachment. PFB the script i have used: CASE1 = LOAD '/home/vaibhav/Desktop/mydata.csv' USING PigStorage(',') as (cmte_id:chararray,cand_id:chararray) STORE CASE1 INTO 'media/vaibhav' USING PigStorage(',') ... View more

@Artem Ervits Yes the path is correct . I can see in the logs that it is not even able to read the records from the files While using Store operation in localmode ... View more

@Joy can you please have a look. ... View more

@Joy I edited my command ,it's as follows bin/flume-ng agent --conf conf --conf-file flume.conf --name TwitterAgent -Dflume.root.logger=INFO,console It worked but at the end it shows some twitter exceptions. I searched for the issue on internet it showed there is some problem with flume-sources-1.0-SNAPSHOT.jar .So i downloaded it again from a different source but still its throwing the same error.I havn't pasted the full logs here . vaibhav@vaibhav-Lenovo-G570:/usr/local/flume$ bin/flume-ng agent --conf conf --conf-file flume.conf --name TwitterAgent -Dflume.root.logger=INFO,console Info ... Exception in thread "Twitter4J Async Dispatcher[0]" java.lang.NoSuchMethodError: twitter4j.json.JSONObjectType.determine(Ltwitter4j/internal/org/json/JSONObject;)Ltwitter4j/json/JSONObjectType; at twitter4j.AbstractStreamImplementation$1.run(AbstractStreamImplementation.java:100) at twitter4j.internal.async.ExecuteThread.run(DispatcherImpl.java:116) ... View more

@Joy Yes i had altered the log4j but i reverted it back now and cannot see any warning or errors, but no data is getting streamed to my hdfs directory.Below are the logs : vaibhav@vaibhav-Lenovo-G570:/usr/local/flume$ bin/flume-ng agent -n TwitterAgent -c conf -f /usr/local/flume /conf/flume.conf Info: Sourcing environment configuration script /usr/local/flume/conf/flume-env.sh + exec /usr/local/java/bin/java -Xms500m -Xmx1000m -Dcom.sun.management.jmxremote -cp '/usr/local/flume/conf:/usr/local/flume/lib/*:usr/local/flume/conf' -Djava.library.path= org.apache.flume.node.Application -n TwitterAgent -f /usr/local/flume /conf/flume.conf ... View more

@Joy I'm getting the below logs after running the above command. it is not able to instantiate the folowing class com.webmethods.sc.logging.log4j.WmDailyFileAppender vaibhav@vaibhav-Lenovo-G570:/usr/local/flume$ ./bin/flume-ng agent -n TwitterAgent -c conf -f /usr/local/flume /conf/flume.conf Info: Sourcing environment configuration script /usr/local/flume/conf/flume-env.sh + exec /usr/local/java/bin/java -Xms500m -Xmx1000m -Dcom.sun.management.jmxremote -cp '/usr/local/flume/conf:/usr/local/flume/lib/*:usr/local/flume/conf' -Djava.library.path= org.apache.flume.node.Application -n TwitterAgent -f /usr/local/flume /conf/flume.conf log4j:ERROR Could not instantiate class [com.webmethods.sc.logging.log4j.WmDailyFileAppender]. java.lang.ClassNotFoundException: com.webmethods.sc.logging.log4j.WmDailyFileAppender at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.log4j.helpers.Loader.loadClass(Loader.java:198) at org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:326) at org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:123) at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:752) at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:735) at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:615) at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:502) at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:547) at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:483) at org.apache.log4j.LogManager.<clinit>(LogManager.java:127) at org.slf4j.impl.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:73) at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:242) at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:254) at org.apache.flume.node.Application.<clinit>(Application.java:59) log4j:ERROR Could not instantiate appender named "testAppender". log4j:WARN No appenders could be found for logger (org.apache.flume.node.PollingPropertiesFileConfigurationProvider). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. ... View more

@Kuldeep Kulkarni i have made the changes it just showing me the logs mentioned above .No data is going to the created directory. ... View more

@Joy this is my output : vaibhav@vaibhav-Lenovo-G570:/usr/local/flume/bin$ flume-ng agent -n TwitterAgent -c conf -f /usr/local/flume /conf/flume.conf + exec /usr/local/java/bin/java -Xmx20m -cp 'conf:/usr/local/flume/lib/*' -Djava.library.path= org.apache.flume.node.Application -n TwitterAgent -f /usr/local/flume/conf/flume.conf log4j:WARN No appenders could be found for logger (org.apache.flume.lifecycle.LifecycleSupervisor). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. ... View more