Created on 06-16-2016 02:11 PM - edited 09-16-2022 01:34 AM
First of all, you need a Hadoop user with correct permission with Ranger (I will use mine : mlanciau), some HDFS quota (if configured), and some YARN queue (I hope it is configured)
For this example, I am using default Jenkins configuration with SSH plugin and the TeraSort benchmark suite.
1. Configure SSH remote hosts in Jenkins by entering your credential (of course you can use different config)
2. Create a new item (on the main portal), I called it Hadoop performance, then click on it and configure
3. Click on "This project is parameterised" and add these 2 parameters (you may want to change the parameter ;-))
4. Go to build and click on add a step to the build and choose Execute shell script on remote host using ssh
5. Then use below information to configure the different scripts (you may want to configure your own user)
6. Click on save
7. Click on Build with Parameters, check the parameter values and then Build
8. Follow the progress by clicking on the step in progress and Console Output !
If you want you can also check execution time, success / failure thanks to the Jenkins UI
Cheers 😉
Created on 06-16-2016 02:40 PM
Cool stuff
Created on 06-16-2016 05:16 PM
That is amazing!