Support Questions

Find answers, ask questions, and share your expertise

Use REST API as source in Flume

avatar
Contributor

I am trying to read the data from SQL Server using REST API as a Flume source with JSON format.

Not sure which Flume source I can use here (may be HTTP) ?

 

If it is HTTP then where I should mention the API URL(http://api.xxxx.com.xxxx) in Flume source property ?

Please advice.

1 ACCEPTED SOLUTION

avatar
The flume http source is for creating a REST API that you can post data to from the upstream sender. If you are looking for a source that will consume from SQL server via accessing the SQL server API, you'll need to write a custom source for that, or possibly try this: https://github.com/keedio/flume-ng-sql-source

Additionally, if you don't need real time processing, you may want to consider using sqoop to import data via batch processing, and it can handle incremental updates.

-pd

View solution in original post

1 REPLY 1

avatar
The flume http source is for creating a REST API that you can post data to from the upstream sender. If you are looking for a source that will consume from SQL server via accessing the SQL server API, you'll need to write a custom source for that, or possibly try this: https://github.com/keedio/flume-ng-sql-source

Additionally, if you don't need real time processing, you may want to consider using sqoop to import data via batch processing, and it can handle incremental updates.

-pd