I'm trying to stream data from twitter to hdfs with Flume, i'm using Cloudera Quickstart VM 5.13, i don't have any error but the destination directory is empty. This is my flume.conf file: TwitterAgent.channels = MemChannel TwitterAgent.sinks = HDFS TwitterAgent.sources.Twitter.type = org.apache.flume.source.twitter.TwitterSource TwitterAgent.sources.Twitter.channels = MemChannel TwitterAgent.sources.Twitter.consumerKey = Sp0ti7peTvFPDJSWMGk2ChMZM TwitterAgent.sources.Twitter.consumerSecret = Cncmq5b6rKxWPb6qNSPkqpzIR7L3EcQ8WUCeG0gX4L9sPIzflN TwitterAgent.sources.Twitter.accessToken = 1370386818609377287-IsLuhCt54wK4T2Ua9Cb0TC14rrs1c5 TwitterAgent.sources.Twitter.accessTokenSecret = AL7oYsVUQXz5KXtQSj0tu36R85MyvAsBjcgktdZD63Ou6 TwitterAgent.sources.Twitter.keywords = hadoop, big data, analytics, bigdata, cloudera, data science, data scientist, business intelligence, mapreduce, data warehouse, data warehousing, mahout, hbase, nosql, newsql, businessintelligence, cloudcomputing TwitterAgent.sinks.HDFS.channel = MemChannel TwitterAgent.sinks.HDFS.type = hdfs TwitterAgent.sinks.HDFS.hdfs.path = hdfs://quickstart.cloudera:8020/user/flume/tweets/ TwitterAgent.sinks.HDFS.hdfs.fileType = DataStream TwitterAgent.sinks.HDFS.hdfs.writeFormat = text TwitterAgent.sinks.HDFS.hdfs.batchSize = 1000 TwitterAgent.sinks.HDFS.hdfs.rollSize = 0 TwitterAgent.sinks.HDFS.hdfs.rollCount = 10000 TwitterAgent.sinks.HDFS.hdfs.rollInterval = 600 TwitterAgent.channels.MemChannel.type = memory TwitterAgent.channels.MemChannel.capacity = 10000 TwitterAgent.channels.MemChannel.transitionCapacity = 100 I'm invoking this command to stream: flume-ng agent --conf ./conf/ -f /home/cloudera/flume.conf -n TwitterAgent​ Please i want to know on which part i'm doing it wrong. Any valuable suggestion is much appreciated. Thanks in advance. ... View more

I'm working on Cloudera Quickstart VM 5.13 and I'm trying to start hadoop; But i'm getting these errors:  Failed to start Hadoop proxyserver. Return value: 1 [FAILED] Failed to start Hadoop secondarynamenode. Return value: 1 [FAILED]   In the log file for the secondarynamenode i have this exception:   HttpServer.start() threw a non Bind IOException java.net.BindException: Port in use: 0.0.0.0:50090 at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:959) at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:895) at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.startInfoServer(SecondaryNameNode.java:488) at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.main(SecondaryNameNode.java:691) Caused by: java.net.BindException: Address already in use at sun.nio.ch.Net.bind0(Native Method) at sun.nio.ch.Net.bind(Net.java:444) at sun.nio.ch.Net.bind(Net.java:436) at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) at org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216) at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:954) ... 3 more Failed to start secondary namenode java.net.BindException: Port in use: 0.0.0.0:50090 at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:959) at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:895) at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.startInfoServer(SecondaryNameNode.java:488) at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.main(SecondaryNameNode.java:691) Caused by: java.net.BindException: Address already in use at sun.nio.ch.Net.bind0(Native Method) at sun.nio.ch.Net.bind(Net.java:444) at sun.nio.ch.Net.bind(Net.java:436) at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) at org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216) at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:954) ... 3 more Exiting with status 1   And this for the proxyserver: Service org.apache.hadoop.yarn.server.webproxy.WebAppProxy failed in state INITED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: yarn.web-proxy.address is not set so the proxy will not run. org.apache.hadoop.yarn.exceptions.YarnRuntimeException: yarn.web-proxy.address is not set so the proxy will not run. at org.apache.hadoop.yarn.server.webproxy.WebAppProxy.serviceInit(WebAppProxy.java:76) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) at org.apache.hadoop.yarn.server.webproxy.WebAppProxyServer.serviceInit(WebAppProxyServer.java:73) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.webproxy.WebAppProxyServer.startServer(WebAppProxyServer.java:132) at org.apache.hadoop.yarn.server.webproxy.WebAppProxyServer.main(WebAppProxyServer.java:115) Service org.apache.hadoop.yarn.server.webproxy.WebAppProxyServer failed in state INITED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: yarn.web-proxy.address is not set so the proxy will not run. org.apache.hadoop.yarn.exceptions.YarnRuntimeException: yarn.web-proxy.address is not set so the proxy will not run. at org.apache.hadoop.yarn.server.webproxy.WebAppProxy.serviceInit(WebAppProxy.java:76) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) at org.apache.hadoop.yarn.server.webproxy.WebAppProxyServer.serviceInit(WebAppProxyServer.java:73) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.webproxy.WebAppProxyServer.startServer(WebAppProxyServer.java:132) at org.apache.hadoop.yarn.server.webproxy.WebAppProxyServer.main(WebAppProxyServer.java:115) Stopping WebAppProxyServer metrics system... WebAppProxyServer metrics system stopped. WebAppProxyServer metrics system shutdown complete. Exiting with status -1 ... View more

I installed Hortonworks Sandbox via Virtualbox. And when i started ambari every service is stopped like you can see in this screenshot . I tried to start manually each of the services but nothing happens when i click the start button. And plus, i have many erros in my notifications section as you can see in this folder. Also this is waht my ambari agent logs looks like. Any idea on how i can resolve this? ... View more

I installed Hortonworks Sandbox via Virtualbox. And when i started ambari every service is stopped like you can see in this screenshot . I tried to start manually each of the service but nothing happens when i click the start button. And plus, i have many erros in my alert section as you can see in this folder. Any idea on how i can resolve this? ... View more