Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

is my url of http-source of flume agent correct?

avatar
Rising Star

After setting the flume agent with http-source type and sink as hdfs. I am doing some get and post requests using the url of my flume source ( http-source).

>>> import json
>>> b={'a':'b'}
>>> a=json.dumps(b)
>>> a
'{"a": "b"}'
>>> res=requests.post('http://hdp.localdomain:41414',data=a)
>>> res.status_code
400
>>> res=requests.get('http://hdp.localdomain:41414')
>>> res.status_code
500

But i am not getting ok status_code for my requests. I have two question. One is the url of the flume source correct().

http://hdp.localdomain:41414

hostname:portnumber. The port is defined in the agent configuration. The agent log shows its running oki.

2nd question is how should i know what type of data i can pass. In the above i passed a as a json. Due to these issue i cannot see anything on the sink.

I am really stuck here. I dont know where is the deadlock

1 ACCEPTED SOLUTION

avatar
New Member

You have to send an array of JSONEvents otherwise the handler will fail to deserialize the events. An event must have at least a body and the body must be a string. You can also add optional headers. See the event specification in the user guide.

import requests
import json

a = [{'body': 'my 1st event data'}, {'body': 'my 2nd event data'}]
requests.post('http://localhost:44444', data=json.dumps(a))

You can also use GET method but still have to specify data to send.

View solution in original post

1 REPLY 1

avatar
New Member

You have to send an array of JSONEvents otherwise the handler will fail to deserialize the events. An event must have at least a body and the body must be a string. You can also add optional headers. See the event specification in the user guide.

import requests
import json

a = [{'body': 'my 1st event data'}, {'body': 'my 2nd event data'}]
requests.post('http://localhost:44444', data=json.dumps(a))

You can also use GET method but still have to specify data to send.