I can write to the flume http source using requests from python. Now i am trying to do similar thing with JAVA. When i try to write to the http source using a dataoutputstream. Nothing is written to the directory designated in flume agent conf. Though i can write to the same hdfs directory when using the python requests. Below is the method i use to do the work.
public static void sendGet(String ar) throws Exception {
//r=requests.get('http://192.xxx.xx.21:44444',data=f3.read())
//I get the string and then i try to write to the dataoutputstream of the urlconnection url2(URL of the flume httpsource agent).
// I am able to write using python requests lib to the flume Now i have been trying with Java.
// I dont see any file written to the path defined in the flume agent conf.
//I fear now this may be issue with the data format?
String url2="http://192.xxx.xx.21:44444";
URL obj2 = new URL(url2);
HttpURLConnection con2 = (HttpURLConnection) obj2.openConnection();
con2.setRequestMethod("POST");
//The below line gets printed
System.out.println("\nSending 'post' request to URL : " + url2);
con2.setDoOutput(true);
DataOutputStream wr =null;
try {
//This line is also printed.
System.out.println("Create the stream");
wr = new DataOutputStream(con2.getOutputStream());
}finally{
wr.writeChars(ar);
//This line is also printed.
System.out.println("Written");
wr.close();
con2.disconnect();
}
try{
System.out.println(ar);
System.out.println("printed on console");
}
catch(Exception e)
{
System.out.println(e);
}
}