Support Questions
Find answers, ask questions, and share your expertise

is my url of http-source of flume agent correct?

Explorer

After setting the flume agent with http-source type and sink as hdfs. I am doing some get and post requests using the url of my flume source ( http-source).

>>> import json
>>> b={'a':'b'}
>>> a=json.dumps(b)
>>> a
'{"a": "b"}'
>>> res=requests.post('http://hdp.localdomain:41414',data=a)
>>> res.status_code
400
>>> res=requests.get('http://hdp.localdomain:41414')
>>> res.status_code
500

But i am not getting ok status_code for my requests. I have two question. One is the url of the flume source correct().

http://hdp.localdomain:41414

hostname:portnumber. The port is defined in the agent configuration. The agent log shows its running oki.

2nd question is how should i know what type of data i can pass. In the above i passed a as a json. Due to these issue i cannot see anything on the sink.

I am really stuck here. I dont know where is the deadlock

1 ACCEPTED SOLUTION

Explorer

You have to send an array of JSONEvents otherwise the handler will fail to deserialize the events. An event must have at least a body and the body must be a string. You can also add optional headers. See the event specification in the user guide.

import requests
import json

a = [{'body': 'my 1st event data'}, {'body': 'my 2nd event data'}]
requests.post('http://localhost:44444', data=json.dumps(a))

You can also use GET method but still have to specify data to send.

View solution in original post

1 REPLY 1

Explorer

You have to send an array of JSONEvents otherwise the handler will fail to deserialize the events. An event must have at least a body and the body must be a string. You can also add optional headers. See the event specification in the user guide.

import requests
import json

a = [{'body': 'my 1st event data'}, {'body': 'my 2nd event data'}]
requests.post('http://localhost:44444', data=json.dumps(a))

You can also use GET method but still have to specify data to send.

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.