I have seen similar question on stack overflow
but I am not really sure
1. How to use the code in actual working example. I have written some code but it is not working for the outputting the number of rows inputting rows works. The output metrics are always none.
Code writing to db.
2. the best or preferred way of doing this.
Basically it seems like I can get the row count from the spark ui but how can I get it from within the spark code.
I know that before I write the database I can do a count on a dataframe but how do it after I write to get the count.
I have posted a lot of info but I just want to know how can I see programmatically the number of rows written by a dataframe to a database.
I'm facing same issue. Can you please let me know , if you found the solution?