Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Basic CSV to MySQL

avatar
New Contributor

Hello everyone,

 

I am new to Apache Nifi and would like to create a basic flow that takes a very simple CSV file (2 columns, 2 rows) as input and loads it into a MySQL database.

 

Here is the CSV file : 

juliendata92_0-1689754339118.jpeg

 

I have already created the "DBCPConnectionPool" service controller, which has been validated.

juliendata92_1-1689754367803.jpeg

 

 

 

I use a GetFile processor to retrieve the file and a PutDatabaseRecord processor to try to write it to the database.

 

The PutDatabaseRecord processor is configured to use the CSVReader.

juliendata92_2-1689754395834.jpeg

 

Here is the CSVreader controller service : 

juliendata92_4-1689754476655.jpeg

 

 

 

I have also created the target table : 

juliendata92_3-1689754426791.jpeg

 

That's the (very basic) flow :

juliendata92_5-1689754502193.jpeg

 


However, I am encountering this error:

juliendata92_6-1689754521360.jpeg

 


Could you please indicate what might be causing the problem?

 

Thank you very much in advance.
Julien

 

2 REPLIES 2

avatar
New Contributor

I just understood why it wasn't working: the field separator was set to ",", while my file uses ";" as the field separator. 

juliendata92_0-1689771900677.jpeg

 

 

After modifying it, everything works perfectly!

 

 

I have another question: would it be possible to load data in CSV format into a MySQL database without having to create the structure of the different fields beforehand?

 

Because when there are only a few fields, it's simple. But if the CSV file has several dozens of fields, it can quickly become tedious and cumbersome to create all the fields.

 

 

So, I'm wondering how to proceed to automatically create the schema within the target table.

 

Thank you very much in advance for your help.

 

Julien

avatar
Community Manager

@juliendata92, Welcome to our community! To help you get the best possible answer, I have tagged our NiFi experts @cotopaul @SAMSAL @MattWho @hegdemahendra  who may be able to assist you further.

Please feel free to provide any additional information or details about your query, and we hope that you will find a satisfactory solution to your question.

 



Regards,

Vidya Sargur,
Community Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community: