Hello Eric,
That sounds like a pretty good idea. If there was a pre-built image available for whatever release I wanted to test against I could just cache it and (hopefully) get my tests running a bit faster. I'm not sure if anyone else is already doing this?
Also, I remember there being mentioned pre-building docker images for each release of Galaxy, which would accomplish something similar, but I'm not really sure how that's being handled. I think Björn's Docker image is kept up to date with Galaxy stable each time it's built
https://github.com/bgruening/docker-recipes/blob/master/galaxy/Dockerfile#L51. So, this could be handled by modifying his Dockerfile to build Galaxy at whatever tagged release you want to test against.
One downside to docker is that you need to get it installed on your CI server, which may or may not be possible (needs a very recent kernel for example).