Support Questions
Find answers, ask questions, and share your expertise

​Need parsing and import data from ftp to sql database with nifi.

Solved Go to solution

​Need parsing and import data from ftp to sql database with nifi.

Explorer

Hi. It's my first days with nifi, so I need a lot of help!

Firstly, I've got ftp from which I need to get data. After it need to parsing and processing data. And download them on sql database.

P.S. As I understand, need to use ListFTP and FetchFTP to take data from ftp and transform them into FlowFile. But what after it? Which processors should I use?

P.S.S. Hope clearly explained.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: ​Need parsing and import data from ftp to sql database with nifi.

Super Guru

@Vladislav Shcherbakov

You need to use PutDataBaseRecord processor to push data to Sql database from NiFi.

Configure/enable Record Reader controller service(avro schema that matches incoming data) that can read the incoming data and and database connection then give all the details about your target table in sql server.

Refer to this link to get more details regarding configs/usage of PutDatabaseRecord processor and to this link how to Configure Record Reader controller service.

View solution in original post

4 REPLIES 4

Re: ​Need parsing and import data from ftp to sql database with nifi.

Super Guru

@Vladislav Shcherbakov

You need to use PutDataBaseRecord processor to push data to Sql database from NiFi.

Configure/enable Record Reader controller service(avro schema that matches incoming data) that can read the incoming data and and database connection then give all the details about your target table in sql server.

Refer to this link to get more details regarding configs/usage of PutDatabaseRecord processor and to this link how to Configure Record Reader controller service.

View solution in original post

Re: ​Need parsing and import data from ftp to sql database with nifi.

Explorer

Thx a lot!
But I still got a few questions. If you can help - it will be perfect!
At the output FetchFTP, we get a file of the CSV format?

Re: ​Need parsing and import data from ftp to sql database with nifi.

Super Guru
@Vladislav Shcherbakov

It depends on how the file is been stored on FTP server side if the file is in CSV format then FetchFTP processor pulls the file in CSV format. Fetching of files won't change the format of the file in NiFi.

If the file is in CSV format on source server side then in PutDatabaseRecord processor you need to configure/enable CSV Reader processor so that processor reads the incoming data and executes sql statements based on the SqlType property value.

Re: ​Need parsing and import data from ftp to sql database with nifi.

Explorer

And thanks again!
I hope this will be enough for the first time.