Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How can i dynamically pass the url from file to invoke HTTP?

Solved Go to solution
Highlighted

How can i dynamically pass the url from file to invoke HTTP?

New Contributor

I have an set of urls in a file.

sample file:

localhost:1234/aaaa/uyt

localhost:1234/bbbb/grt

localhost:1234/cccc/hgd

etc..

how can I pass this URL dynamically from that file to invoke HTTP processor.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: How can i dynamically pass the url from file to invoke HTTP?

Super Guru

@Rajesh AJ

Use Get File (or) List/FetchFile processors to fetch the file.

then use

1.Split Text Processor(if you are having each url for a line) with line split count as 1

45464-splittext.png

If your file having 4 lines then after split text processor will give seperate flowfiles for each line.

Example:-

Input 1 file(having 4 lines) and output will be 4 flow files(each line as seperate flowfile)

2.Extract Text processor to extract the url's to attributes for the flowfile. According to your file content size you need to change the Maximum Buffer size property and i'm extracting all the contents of the flowfile to url attribute by using regex .*

url

(.*) //extract the whole content of the flowfile add the content to the flow file url attribute

45465-extracttext.png

then use

3.invoke http processor with ${url}

45466-invokehttp.png

We are going to use the extracted url attribute from extract text processor in invoke http processor.

The extracted url attribute will be changed dynamically according to the flowfile content.

Flow:-

1.Get File
2.Split Text
3.Extract Text
4.Invoke HTTP

If the Answer helped to resolve your issue, Click on Accept button below to accept the answer, That would be great help to Community users to find solution quickly for these kind of errors.

1 REPLY 1

Re: How can i dynamically pass the url from file to invoke HTTP?

Super Guru

@Rajesh AJ

Use Get File (or) List/FetchFile processors to fetch the file.

then use

1.Split Text Processor(if you are having each url for a line) with line split count as 1

45464-splittext.png

If your file having 4 lines then after split text processor will give seperate flowfiles for each line.

Example:-

Input 1 file(having 4 lines) and output will be 4 flow files(each line as seperate flowfile)

2.Extract Text processor to extract the url's to attributes for the flowfile. According to your file content size you need to change the Maximum Buffer size property and i'm extracting all the contents of the flowfile to url attribute by using regex .*

url

(.*) //extract the whole content of the flowfile add the content to the flow file url attribute

45465-extracttext.png

then use

3.invoke http processor with ${url}

45466-invokehttp.png

We are going to use the extracted url attribute from extract text processor in invoke http processor.

The extracted url attribute will be changed dynamically according to the flowfile content.

Flow:-

1.Get File
2.Split Text
3.Extract Text
4.Invoke HTTP

If the Answer helped to resolve your issue, Click on Accept button below to accept the answer, That would be great help to Community users to find solution quickly for these kind of errors.

Don't have an account?
Coming from Hortonworks? Activate your account here