Hydraulic's first product is a packaging tool for desktop apps and we're mostly a JetBrains shop. The gist is:

- A large dedicated machine in a cheap colo provider (Hetzner), with

- Gitolite with some custom configs

- YouTrack for tickets

- TeamCity master, some agents and a Windows VM for testing.

- Dedicated Mac hardware in the office for Mac CI testing, also running TeamCity agents.

The workflow is a homegrown one that we call "git oriented review". I'll briefly describe it and then discuss why we use it:

1. (Almost) Every git repository is owned by someone specific. There are no shared repositories. All code flows upwards to my repository via merges, kernel style, and that's the one that's used for releases.

2. Gitolite is configured to allow users to push into each others repositories but only under a special branch namespace (rr/$user/whatever). Other branches are protected and yours alone. You also have a personal set of build configs in TeamCity, so if you want to create a customized CI setup you can, and so branches you push to your personal repo don't interfere with the greenness of anyone else's builds.

3. To submit code for review, you push a "review request" branch in the rr/$user namespace of the reviewer's repository. The commit message(s) have a command in them that's interpreted by YouTrack once the build goes green to update the ticket, which in turn then notifies the reviewer that new code exists and is green. The notification can be via email, or IDE notifications, or Slack etc. Lots of options, up to you.

4. The reviewer then makes a code review by adding commits to the rr branch. For small changes, the reviewer just makes the change directly. For larger changes they add a //FIXME comment to the code. FIXMEs may not be merged into non-rr branches, so they are a request for the original author to remove them by e.g. fixing the issue, or adding a comment to explain why in reality it's not meant to be fixed. Thus code and commits are used as a type of discussion forum. You can of course also just jump into a CodeWithMe session to do a bit of pair programming on it for more complex discussions (everyone is currently remote at Hydraulic so everything is done via tools).

5. When satisfied the reviewer merges the rr branch to their own master or dev branches and deletes it. The merge commit contains another command to mark the ticket as fixed, again, it's only applied if the build goes green. At this point they "own" the result because it's in their personal repository. Finger pointing isn't allowed.

I designed this workflow due to lack of satisfaction with the GitHub PR based workflows used at my previous firm, which was a fairly typical centralized one involving a single shared repository, a protected master branch with people pushing ad-hoc branches into it and then opening PRs for reviews. The problems I wanted to fix were:

• Reviewers would only respond with comments because that's what the GitHub workflow promotes. Often many comments were small and it would have been both much faster and also more collaborative for the reviewer to make the change directly, but lack of clear ownership over branches made conflicts likely, and people were reluctant to do this.

• Sometimes what the reviewer wanted wasn't obvious. Again if they could have made the change directly it would have been better.

• Reviewers would often get tired after enough comments, or more than a few rounds of review, especially if the dev wasn't actually applying all the requested fixes. So they'd end up waving code through that wasn't really fully fixed.

• PRs notified reviewers before CI had tested the code. This would often lead to races in which the review was completed before CI pointed out that the code was broken, wasting a review cycle (unfortunately CI was quite slow at the old company due to it being a database engine with lots of IO heavy regression tests). This problem has led GitHub to create "Draft PRs" which don't make conceptual sense.

• We ended up with many branches where it wasn't entirely clear if they were abandoned or not. People became reluctant to delete branches in case they were being used to back up important but unfinished work, and again, there was no clear ownership of who was supposed to do this (I didn't get to pick the management approach and would have fixed this stuff if sufficiently empowered).

• Relatedly we lost clear ownership of the codebase. At first ownership was mine because I approved all reviews, but as that stopped scaling the firm transitioned to a system in which coders could pick their own reviewers and ownership effectively became collectivized. The codebase wasn't really laid out with CODEOWNERS files in mind, so devs just had to get a review from someone and then they could commit. This led to a lot of externalization of costs and juniors reviewing each other's code, often letting serious problems through without realizing.

Git oriented review solves these problems. Code ownership is always concrete and well defined by repository, which avoids needing to mangle the codebase itself to try and reflect shifting reporting lines in the directory hierarchy. Reviewers become collaborators on a branch, relying on git's merging features to avoid conflicts. Discussions use commit messages, or whatever is more appropriate when that's insufficient, instead of being tied to a relatively poor and low-featured ad-hoc "discussion forum" like a GitHub PR is. People can organize their own branch namespaces. Reviewers are informed there's work to do only when a build goes green, and they can control how those notifications work. CI and ticketing are closely integrated so tickets have work logs. Finally, the history of the code review is backed up in a portable and vendor-neutral git repository.

Downsides? Not many found so far. It's unfamiliar to new devs and requires a bit of training especially if their git skills aren't fluent. Gitolite is powerful but has a few awkward limitations and was a bit of a bear to configure. The JetBrains tools are great and cheap/free for small operations like ours, but do require payment later. To view logs without the review history polluting things you have to know about the `git log --first-parent` flag which many people don't realize exists, and which doesn't have any equivalent in the IntelliJ git view. Overall these things are pretty easy to fix. IntelliJ git is open source so we could even add that feature ourselves if necessary, but so far it wasn't.

There's also clarity over ownership that way - the `rr` branch is owned by the owner of the repository and they can push to it, to make it into whatever form they want.

Devs may also explicitly send a message indicating there's work to be reviewed, but they don't have to.