Graham Christensen here, cofounder of DetSys. Happy to answer any questions! The Magic Nix Cache has been a huge boon to us internally, and we're really excited to share it with the world today.
As someone who has led a ~2yr adoption of Nix in an enterprise scenario, I'm really glad for more projects that help with bridging the gap between Nix and containers/k8s.
You can get fast startups if you're willing to define your containers upfront (dockerTools, nix2container) and/or adopt a dynamic container-server (nixery, flakehub). And you can get reasonably fast substitutions if your binary cache is on MinIO in the same cluster as the workers.
But I feel like there's still room for a "magic" /nix/store that skips the copying and decompression stage altogether— something that works using standard nix invocations (like Magic Nix Cache), but presents itself as a Kubernetes Volume, so that in cases where a path already exists on-node, the existing files (in the cache pod) are simply mounted/served directly into whatever container ran a nix command.
I don't feel like I really know enough about either k8s or nix to assess the practicality of such a thing, but the thought of lightning-fast substitutions for arbitrary Nix workflows is massively appealing.
Several years ago I looked at implementing a custom component for k8s which would exchange nix store paths instead of containers, substitute, and bind mount them in at run time. It was an interesting experiment, but was Quite Difficult to pull off for someone who wasn't already familiar with k8s.
I've seen some projects similar to what you're describing though: the lightning-fast substitutions. It was incredible! They had the benefit of a fabulously fat network connection, though, and I'm not sure the experience translates very well. We will see!
Thanks! The domain is robotics, and we gave a brief conference talk last fall about our experiences so far, if you're interested— although it's mostly directed at a Nix-unaware audience, you can get a glimpse of our workflow that's heavily oriented around hourly builds delivered as Nix flake tags: https://vimeo.com/767139940
Definitely there's an obviousness to the concept of "magic" Nix stores in various spaces, and I know the tvix project seeks to realize some of this as well— reusing OCI tools to supply the build sandbox, leveraging existing container orchestrators for job management and queuing, all those goodies. So I'm excited to keep watching the space.
> But I feel like there's still room for a "magic" /nix/store that skips the copying and decompression stage altogether— something that works using standard nix invocations (like Magic Nix Cache), but presents itself as a Kubernetes Volume, so that in cases where a path already exists on-node, the existing files (in the cache pod) are simply mounted/served directly into whatever container ran a nix command.
Mostly a digression since it's not Nix, but I've wondered a bit about this sort of thing when building AUR packages on Arch Linux. The very last step of the builder is to compress the package, which in the vast majority of my uses is followed immediately by installing it, which of course decompresses the package. I've wondered why there isn't some (non-default) option to say "I don't need to keep the package itself around; just install it as soon as its built". I'm sure for my specific use case there's a simpler solution, but I've always wondered if there's a hacky way to get around things more generally by making a tool that can mimic the expected compression API but then creates a "fake" compressed artifact that no-ops (or maybe puts in a valid header followed by non-compressed data) and then injects that implementation into the PATH. You'd be able to invoke it with something like `fakecompress --zstd makepkg -si`, and it would invoke `makepkg -si` with the no-op zstd implementation.
Tar would also be the answer to how to do fake compression (keeping files together and in order in one file) IF you can choose the "compression" library/tool.
Ironically, Python has moved in the other direction, with now requiring a wheel be built before installing it, when wheel itself was originally an optional bolt-on to setuptools.
Yes! The Action isn't directly aware of what Nix itself is doing or which commands are being run; it's only aware of the Nix store. So whether you're using flakes or channels it works the same.
