We are using scala with maven to build spark applications along with git as code repository and jenkins integrated with git to build the jar.
I am not sure how to use jenkins to deploy our apps on cluster.
Can anyone explain what could be the next step?
Is jenkins supporting deployment of spark apps like it does for other apps.
Dear @Fabricio Carboni:
Can you please share some document on how we can implement CI/CD for pyspark based applications. Also, is it possbile to do it without using containers (like we do development in Java/Scala (first locally on windows and then build it on Linux dev/tst/Prod))