Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]

Cloudera Community

Support Questions

Find answers, ask questions, and share your expertise
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Advanced Search
Announcements
Share your experience with Cloudera on G2 and get a $25 Amazon Gift card.
Hi, I'm CLEO! Something exciting is coming to the Community. Stay Tuned!

Who agreed with this solution

avatar
author-rank steven-matison author-rank
Guru

Created on ‎07-17-2023 08:38 AM - edited ‎07-17-2023 08:41 AM

@Madhav_VD This is a job that nifi can do itself.  You would just tail the log file w/ nifi TailFile. 

 

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.20.0/org.apach...

 

 

 Once you have the log file going into flowfiles in nifi, you can build your flow to filter lines you want to versus lines you do not want.  Then when ready use any flavor of database processor (ExecuteSQLRecord, ExecuteSQL, PutSQL, PutDatabase, etc) to translate the log line into a sql insert statement.

View solution in original post

4,820
0 Kudos
1
Who agreed with this solution