we are running pyspark code with NIFI ExecuteSparkInteractive, but in YARN see a lot of generated livy-session about 103024
Why this happen ?
103024 sessions could translate to a session every 90 seconds for the last few months. How many sessions did you expect to have (and what is the context)?
Did you get a solution on this issue ? We are facing a similar problem with the sessions created by the nifi-livy integration not cleaning up/closing up.
Even I am facing the same issue. As soon as I enable the controller service its creating livy sessions. I havent started any processors.