[FIXED] How can I incorporate a long delay into a Jenkins build process?

Issue

I am using Jenkins to deploy changes to a system which manages and runs lots of different jobs which are scheduled daily. We have a staging setup which does not write to the real database, and a production setup which does.

The Jenkins flow I would like to have a happen when a change is pushed is this

1. Run checks.
2. Deploy to the staging system.
3. Wait 24 hours.
4. Check logs to make sure that the staging system has not had any errors in the last 24 hours.
5. Deploy to the production system.

There could be more than one of these builds running concurrently at any time - eg. I push changes at 11 am, they are deployed to staging. At 5 pm I push more changes and they are also deployed to staging. The next day at 11 am the first set of changes only are deployed to prod. At 5pm that day the second set of changes are deployed.

Now, I have managed to build a system which does this, by using the Build Flow Plugin, and creating a job called wait_one_day which runs sleep $((24 * 60 * 60)) in a bash shell.

This doesn't seem like the most elegant solution, and has the disadvantage that I am tying up two Build Executors for 24 hours (one for the build flow job, and one for wait_one_day), each time we make a change.

Is there any better way of doing this, or any plugin which is designed to help with this process? Can a Jenkins job schedule another Jenkins job to run as a one-off?

I would equally be happy to hear about an alternative approach to solve the same problem if anyone has any suggestions or constructive criticism of my design.

Solution

There was similar SO question recently that I answered, although I'm not sure that my answer there exactly fits your scenario.

You could potentially dynamically create a job that does steps 4 & 5 which would run periodically every 24 hours. The catch here is that you would actually only run this job once, and have a build step in that job that deletes itself (groovy code or shell script). It would be easy enough to create a deactivated template job that you could just clone and then modify for the particular task. An intermediary job would be necessary which would trigger upon completion of any job that runs steps 1 and 2. The intermediary job would then create the temporary job from the template.

Alternatively, you could create some sort of handler, either within jenkins or external that would run off of some properties file or database containing the scheduling for when jobs need to be fired off. Granted, if you are going to go the route of writing a handler, you might consider putting in a little extra effort and writing a jenkins plugin...

Answered By - TheEllis

Answer Checked By - Candace Johnson (PHPFixing Volunteer)