Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

is my url of http-source of flume agent correct?

Solved Go to solution
Highlighted

is my url of http-source of flume agent correct?

Explorer

After setting the flume agent with http-source type and sink as hdfs. I am doing some get and post requests using the url of my flume source ( http-source).

>>> import json
>>> b={'a':'b'}
>>> a=json.dumps(b)
>>> a
'{"a": "b"}'
>>> res=requests.post('http://hdp.localdomain:41414',data=a)
>>> res.status_code
400
>>> res=requests.get('http://hdp.localdomain:41414')
>>> res.status_code
500

But i am not getting ok status_code for my requests. I have two question. One is the url of the flume source correct().

http://hdp.localdomain:41414

hostname:portnumber. The port is defined in the agent configuration. The agent log shows its running oki.

2nd question is how should i know what type of data i can pass. In the above i passed a as a json. Due to these issue i cannot see anything on the sink.

I am really stuck here. I dont know where is the deadlock

1 ACCEPTED SOLUTION

Accepted Solutions

Re: is my url of http-source of flume agent correct?

New Contributor

You have to send an array of JSONEvents otherwise the handler will fail to deserialize the events. An event must have at least a body and the body must be a string. You can also add optional headers. See the event specification in the user guide.

import requests
import json

a = [{'body': 'my 1st event data'}, {'body': 'my 2nd event data'}]
requests.post('http://localhost:44444', data=json.dumps(a))

You can also use GET method but still have to specify data to send.

1 REPLY 1

Re: is my url of http-source of flume agent correct?

New Contributor

You have to send an array of JSONEvents otherwise the handler will fail to deserialize the events. An event must have at least a body and the body must be a string. You can also add optional headers. See the event specification in the user guide.

import requests
import json

a = [{'body': 'my 1st event data'}, {'body': 'my 2nd event data'}]
requests.post('http://localhost:44444', data=json.dumps(a))

You can also use GET method but still have to specify data to send.

Don't have an account?
Coming from Hortonworks? Activate your account here