Who is not aware of the following problem. It’s Sunday afternoon, you’re bored. You want to do something useful, you want to contribute to the community – you want to review a pull request!
This might be a more or less known scenario to you. How would somebody start this, how do you identify a suitable pull request that you can review. I spend most of my time working on the Puppet community Vox Pupuli. Vox Pupuli has currently 518 open pull requests. Not all of them are in a state where you can or should review code. A few examples:
- The PR is already reviewed and the author needs to add tests, but the code is fine
- The PR is already reviewed and the author needs to document their new feature, the code is fine
- The PR is already reviewed and the author needs to update the code
- The PR is already reviewed and the author needs to update the code because it breaks existing tests
- The author needs to update the code because it breaks existing tests
- The PR has merge conflicts that must be resolved
It’s possible that you spend an hour just checking existing PRs and you notice that all of them already are reviewed and/or waiting for the author to work on it. This is frustrating and kills the rare time open source collaborators have. Nothing of the above is Puppet- or Vox Pupuli specific. This happens in every bigger community or company. Besides that, there are also some Puppet specific tasks that maintainers need to do on a regular basis (yak shaving):
- Drop EOL puppet versions from metadata.json
- Add new Puppet versions to metadata.json
- Drop EOL linux versions + their tests + their code
- Implement new linux versions
- Ensure dependencies are up2date and resolve cleanly
This list is far away from being complete, but it provides a good overview of all the regular tasks. They all consume time that could be spend on reviewing new codes.
My friend Robert Müller and I identified those painpoints because we suffer them every day. Together we want to automate as much as possible of those jobs so the Vox Pupuli collaborators can be more productive and efficient – we started Vox Pupuli Tasks.
All the Vox Pupuli modules are stored at GitHub. And GitHub has this nice concept of Git Hub Apps. Those are webservices that get notifications for every action that happens in a specific GitHub namespace. Also the Apps can respond to it. We (mostly Robert, because he’s awesome) created a Ruby on Rails application that receives GitHub events. Based on that we currently solve two and a half use cases:
Broken test suite
GitHub does not notify anybody if a check fails, also Travis-CI does not notify the pull request author. Many people are capable and interested in fixing their PR, they just don’t know it’s broken. So normally a collaborator checks a PR, sees it has a failed CI job and writes a comment so the PR author gets notified. The community has a guideline to add the tests-fail label to it. That’s partly helpful. Based on that we can filter PRs out in the GitHub search UI. But! the labels don’t disappear after the CI is working again so this approach is pretty much useless. Our Rais App gets notified about all pull requests and changes related to that PR.
- If a PR has a successful CI run, we check if the label
tests-failis attached. If so, we remove it. - If the PR has a running CI job, we schedule a job that checks the CI status in a minute
- If Ci failed, we attach the
tests-failcomment. This makes it easy to filter the PR. But it does not notify the PR author about it. Therefore we also write a comment once into the PR
Merge conflicts
Pretty much as above. PR authors do not get notified if their pull request has a merge conflict. We normally attach the has-conflicts label, but that’s pointless if they don’t get removed automatically. Our Rails application again adds or removes a label and adds a comment. If one pull request gets merged we also trigger a job for all open pull requests in the same module. A merge can trigger conflicts in them.
Yak Shaving
Every puppet module provides parsable information. We’ve got the metadata.json that lists supported OS and Puppet versions. Our .msync.yml tracks the version of modulesync_config that got applied last. travis.yml lists the complete test suite, with two acceptance test jobs for each supported mainstream OS. We can automatically parse the data and at least create statistics about it and inform people when they need to act. In the past we had a very ugly Ruby script for this, which I wrote some time ago. Now with the Rails application, we can act upon changes in the code. We already render some nice statistics so people know when and where to act, but it’s not yet as much automated as it could be. But all checks are currently organized in a plugin system and can be enabled/disabled. This doesn’t hardcode the application into the Puppet ecosystem and keeps it flexible
VPT Status Quo
So what’s the current status of VPT (Vox Pupuli Tasks)? The above features are currently live for all Vox Pupuli puppet modules (not all Vox Pupuli git repositories). This already helps alot while reviewing pull requests. We’ve a customized GitHub search query that filters all PR’s out where a collaborators is waiting for the PR author. This reduces the number of 518 open PRs down to 175!
What’s next?
We want that people use this application. This requires two things:
- Working documentation about the featureset and installation
- Application needs to be portable and not tied to Vox Pupuli
Robert and I are currently working on both parts. New users also means new collaborators, and we’re always happy if people jump in and develop with us together. Our goal is to provide a generic Rails application that can act on events from version control systems. We’re currently working together with two companies that might use this. We want to make the application modular so everybody can use this, no matter if you maintain Puppet modules on github.com or python projects on your internal GitLab or Salt modules on yout GitHub enteprise instance.
Roadmap??
We currently don’t have a fixed roadmap, but we want to! There are already a bunch of open issues with features and enhancements that we want to implement. We’re currently in contact with potential users and identify their hard and soft requirements. Based on that we want to create a loose roadmap on GitHub. Do you miss a specific feature? Create an issue!
Do you have feedback? Write @voxpupuliorg, Robert or directly to me on Twitter or respond here with a comment. And most important: Don’t forget to checkout https://voxpupu.li/ ! (The Rails app I wrote about for an hour).