Installing each package into it's own directory, and maintaining symlinks into that directory structure, was an approach explored for large system installations in the 90s and 00s.
NIST depot, Xheir, CMU, and Nix all did this in varying ways. The advantage of this approach was that packages could be installed once on an NFS, scoped by version/configuration, and mounted into the filesystem of every work station on a campus. The workstations then just needed to maintain references to the files on the NFS for their particular software configurations. There is also some interesting exploration of composable configuration templates that allow workstations to inherit and extend a particular configuration (i.e. the physics department might have a base template).
Nix is easily the most successful in this space. It even uses something similar to the NFS approach! It can maintain a remote cache of observed configurations to pull from instead of redoing work.
Reading these LISA papers is a lot of fun. If you work in system administration (i.e. maintaining an Artifactory instance for your company) and have a day to spare - I'd very much recommend reading them, starting with the Nix paper!
Nix is great! But GoboLinux is actually a few years older than Nix, isn't it? First release in 2003. :)
Interestingly, GoboLinux Runners are basically the same concept as NixOS' FHSUserEnvs, where user namespaces are used to mount a subset of the existing shared libraries and other dependencies to make them visible (and the only ones visible) to a running program in a compatible way. This spares them some of the filesystem overhead of running a full OS container with its own package manager. So there's a bit of further convergence there, too.
I love the cleanliness and legibility of the GoboLinux file hierarchy, and I think the simple, Mac-ish app bundling approach it takes is something many users might like and prefer over the rough edges of Nix or the complexity of Flatpak.
A GoboLinux-like presentation could be great for a stable, impure NixOS downstream some day.
Thanks for the link to the Depot Lite paper! I'd never heard of it.
Very much welcome! I can geek out about package management all day.
If you do want to dive into this stuff, I’d recommend reading both the Nix and Depot Lite papers of course, but their references section is also a gold mine of fascinating systems.
I spent some time with the Depot Lite paper, and it's wild how much much there are little hints (less strict) of many of the exact principles the are used in Nix. Turning software installations read-only after a few weeks was one that stuck out the most, along with the basic FHS adjustments.
I've read through a few if the Nix papers before, but they're probably worth revisiting by now after using Nix for a few years and experimenting with Guix a bit.
> I can geek out about package management all day.
Same! I've been fascinated with package management probably since I was about 12 or 13, when I first tried desktop Linux. Package managers are incredibly powerful systems tackling an equally incredibly tough problem space. Multi-language package managers (i.e., ports systems and Linux distro package managers) have always especially impressed me. (Imo even the 'bad' Linux distro package managers are pretty good, as much as I can be harsh on them in certain aspects when comparing them.)
Getting into Nix has made me think more deeply about the designs of the many language-specific package managers, since each one needs/gets a different treatment in Nixpkgs according to its unique properties. It's similarly provided an illuminating alternative to the container approach to traditional deployment troubles— one that conserves more of what is good about the traditional package management paradigms on Linux and elsewhere.
Luet also looks interesting, but I've not really played with it yet. Its approach seems to be to add more package-like granularity to existing container systems: https://github.com/mudler/luet
That’s kind of funny, because Microsoft do something similar with WinSxS. Only worse.
What they do is store packages in the winsxs folder. They then do an NTFS hard link to where the component is installed, usually system32.
If you ever used Vista and Windows 7 and started to run out of disk space over time, that’s because Microsoft never used to remove the old components after they were updated. In Windows 7, the initial plan after SP1 was to allow people to use a special switch on DISM.exe to remove old service pack files.
Microsoft removed the idea of service packs and instead did continuous patching with rollups. It seems to taken them quite some time before they twigged to the fact that the old updates were using up a lot of disk space, because a long time later (almost at the end of the Windows 7 lifecycle) they added a Windows update to the disk cleanup wizard. Even worse, it would only remove the components after you rebooted but before you could login to Windows. If you ran it, it could sometimes take literally hours to get the computer to finish removing files from the component store.
At least in my previous jobs, maintaining the artifactory instance included maintaining package integrity in the repositories. Answering questions about shadowing in Java, unpublishing in Node, upstream timelines for patched debs, and tracking deprecation was part of that the job - and the teams worked to make machines do as much of that job as possible.
Packages bootstrap filesystems for installations. We’ve parted out system administration at scale - multiple paths you can take.
This is actually a neat idea, and I am saying that as the archetypical, grumpy "I like my sys the way it is, thank you very much!" - kinda guy.
A lot of the directory structure of POSIX systems is completely obsolete in the 21st century. The difference between sbin and bin? Historical, for the most part, and irrelevant in the age of rescue systems.
One question that is nagging at my mind with this: What if two programs require the same lib, how is this resolved?
Because, when I install progA, the lib is symlinked into the index, so when I now install progB, the lib is installed again if I understood correctly. Storage is irrelevant, the pkg manager sees there is a symlink, so it stays in place, all is well.
But now I remove progA, so what happens? progB is still present, still requires the lib, it is also still installed, but now the symlink, which refered to /Programs/progA/libs/thelib.so is broken. How is this situation handled?
I think libs are installed in /Programs too, and everything is versioned, and all libraries are linked to a central /System/Index/lib directory, which is referenced by ld by default: ( See https://gobolinux.org/at_a_glance.html )
I guess it's up to the package manager or application to reference the version of the library at the correct specificity. And then it's up to the package manager to GC libs that aren't referenced anymore.
Personally I think it would have been interesting to version the system index itself, like /System/Index/Current|1.0|myappenv|etc/lib|bin|etc, and each app is chrooted into their own system index by default.
The question is, when 2 packages, pkgA & pkgB bring in the same library, call it "thelib.so" into the system, logically, only 1 of them can be refered to by the symlink in System/Index/lib.
What happens now if I remove pkgA? If nothing changes, the symlink would be dangling, and pkgB would no longer function because the libraries are looked up in the index.
My question is: how is the pkg management system resolving this?
I explained it because your example makes me think you don't understand how and where packages are installed. Libraries are installed independently, but your example looks like the library thelib.so is installed inside pkgA which is only related to thelib.so because pkgA is the first package that depends on thelib.so, but this is incorrect, instead you'd have:
The same way any existing package management system handles this - by only allowing you to install one of the two packages in the first place.
But Gobolinux can actually handle this scenario better than most distros since package-specific libraries don't need to go into the global index in the first place. So neither pkgA nor pkgB's 'thelib.so' would end up in the index, it'd just be local to the package.
Except Nix. Each package has a unique hash, which is reproducible. So if there is one minor difference, you get a different path. This works incredibly well, but I admit the paths of Gobolinux are more user-friendly in their naming scheme (though one might argue against a capital letter at start of each entry due to increased finger movement requirement).
Sbin and bin separation actually causes problems with portability. A certain bluetooth-related utility wants to run iptables command [1] which can usually be found in `/sbin`. But `/sbin` is not in the PATH so they have hardcoded `/sbin/iptables` into the source. Now distributions that have iptables in `/usr/sbin` have to patch the program.
So it would be better if either there were no `/sbin` or if it was always in the PATH.
Modern distros just have a single bin directory and symlinks for the rest to resolve this.
My favourite one is how people try to come up with reasons for the /usr dir and why /bin and /usr/bin both exist. The original reason was that one of the original Unix devs ran out of space on their hard drive and needed a second bin folder for the new drive.
Back in the days, what often happened was /usr was mounted over NFS or a second harddisk like you mentioned (space and maintainability being possible reasons). Because NFS locking was an issue, you'd use lockable files (such as /var and /tmp) on different partitions.
Hardcoding two locations is only marginally better than hardcoding one. (By probe I assume you mean probing a hardcoded list rather than something even more fragile and confusing.)
Until it suddenly isn't in either of these paths, maybe because its required to run in some quirky setup.
Hrdcoded paths for things a program doesn't bring with itself, should be avoided in general. There is a reason $PATH exists and is easily extended. It is the users/admins responsibility to make sure envvars are set correctly, and it's the programs responsibility to give a useful error message when it cannot locate the programs it expects in $PATH
Well in a lot of new distributions /bin, /sbin and /usr/sbin are symlinks to /usr/bin.
The historical difference is that in /sbin there are binaries meant to be executed by root, and thus they were not be put in the path of non root users (Debian still do that), so you don't get binaries that you can't run in your path (these days it's stupid since you may want to use sudo and thus have them in path only for the shell autocomplete to work properly).
By the way I prefer how things are nowadays, otherwise you get the same confusion that I have every time I use Windows or macOS that you either have an infinite PATH variable or have to specify the full path every time.
Yes, the package manager can make symlinks, but at that point what is the difference?
Also having everything in /usr/bin has the good side effect of avoiding that by mistakes someone creates a program that has the same name of one of the thousands of programs that are in the repository of a distribution, to avoid confusion to the user that can launch a program or another depending on who comes first in the path.
I think for shared libraries, the convention is to have the major version number of the API in the name (e.g. foo.so, foo2.so) and then to put any ABI changes as a trailing extension (e.g. foo.so.2, foo.so.3, foo2.so.2, foo2.so.3). It's kind of a hack, but it makes sure that you can link unambiguously to one if you need it.
EDIT: didn't realize you were talking about Gobo specifically; not sure how they deal with this, but theoretically a similar strategy seems like it could work
This is very similar to how npm works. Each installed package has its own dependencies installed that are kept separate from other packages even if another package has the same dependency.
For all the hate of npm and node_modules it’s wonderful to have this isolation
Python also has virtualenv, so what though? Like Docker, its a dependency nightmare, and a waste of space. Nix also wastes space, but it allows you to run separate versions of libraries and programs universally on the OS, and it allows you to rollback, plus it can be easily removed with a command (man nix-collect-garbage).
As far as I can tell, the package manager handles dependencies, like most other package managers. Packages can declare dependencies on other packages, and the package manager installs each dependency only once globally.
I think GP was asking what happens when you remove a package that has thelib.so as a dependency. Is thelib.so removed? Does the previous existing thelib.so used as a replacement?
With any normal package manager, no. You don't remove a package that has any remaining 'reverse dependencies' (a.k.a 'dependents'). Removing only one of many dependent packages just has no effect on the dependency.
But if BOTH packA & packB come with the same libs, one of them HAS TO get symlinked in the system index. And they do not depend on one another, as each has to bring the same lib to the system, so logically, they can both be removed without breaking a dependency.
But since there can be only ne symlink to the library (because the lib is searched by name in the index) the question remains: How is the package manager handling this for Gobolinux?
and then you'd have to use SymlinkProgram to enable another provider of the shared libraries.
In a way this is inside-out from what Nix and Guix do, where their equivalent of the System Index (the Nix/Guix store) is the build target, and everything all symlinks point to that. That makes this kind of thing a little easier to deal with, at some other costs.
Why would two different packages have the same library in them? Surely for every library there's a single package for that library, and if multiple applications depend on that library then their packages would depend on that package.
I think they just intend to duplicate everything, but I'm not sure.
One of the biggest issues with 'improved' systems is that even when they are actually a whole lot better, it makes it nearly impossible to reason about the system or the state of the system using filesystem primitives or other first principles.
Take journals in systemd for example, they are better than plaintext log files in some cases, but now you are stuck with a format that needs to be interpreted and parsed before it is useful.
That doesn't mean improvements shouldn't be done, but every step that is a big change often also makes it impossible to do some basic things that used to be possible for decades. This is similar to analog radio (i.e. only requiring a diode, tunable capacitor and a coil to receive audio! - no power supply needed!) vs. DAB. DAB has a lot of improvements over the base feature, as well as a whole lot of new features. But it is no longer possible to simply receive the radio waves and get audio.
In a way you lose simplicity to gain sophistication. Sometimes we get sophisticated simplicity where a complex problem was solved in a very neat and elegant way that can be reasoned about with ease, but most times it isn't the case and that makes me (and a whole lot of other people it seems) wonder how we can get the improvement without losing the simplicity.
For the very simple "the filesystem is the database" concept, this might actually be possible, all we really get is a well-known structure and versioning with symlinks, similar to the alternatives system used in Debian. But then we get /Programs which is capitalised which looks nice but mixed capitalisation seems too much of a "they are doing it, thus so are we" thing, while on a filesystem level there isn't any gain. Same with spaces in names or dashes. For visual representation that is nice, but everything else it sucks.
I would take out the capitalisation and add some sort of unionfs/aufs/overlayfs hooking (symlink into a bundled filesystem would be one way) so from the system's perspective it's still a single filesystem tree but you add the option to get all those newfangled distribution methods in there as well. The problem we still haven't solved is versioning, but since it is arbitrary at this point /programs (or just call it /apps) could just contain <shortname>/<versionstring> for the application name and then version of the application. If you then dump in a version that comes from a docker image, flatpak, some weird weine-dxvk-steam construction or something else you symlink it in as a <versionstring> and then it's no longer the problem of the distro.
There is still the issue of partitioning off parts of the system you don't want to be touched or want to have safe; a RO-snapshot plus user-overrides could work, but you might end up with a huge RO-snapshot that needs to be entirely replaced, and then a billion override mounts on top of that...
As simple as Gobolinux presents it, this only seems to be useful for a single distro with a single distribution point, which is no longer where we are. Even FreeBSD is past that and that's not even Linux.
The problem with case-sensitive filesystems is - which locale's case-sensitivity rules does the filesystem use?
Remember that some systems may have multiple users (some may even have multiple simultaneous users) who use different locales. Or a single user might run different programs using different locales, e.g. if a program has a bad translation in their native language, they may prefer to run it with a different language that they also speak.
A lot of computer systems only got rid of the problem of storing filenames in locale-specific character encodings (e.g. ISO-8859-*) relatively recently, by switching to a locale-neutral encoding (UTF-*/UCS-*). It would be nice if they could not repeat some of those problems with locale-specific filename matching rules.
You could remove the problem of "requiring you to type the name with exact case" by having shells' autocomplete feature fix case issues. So you could type "vi foo<TAB>" and the autocomplete change that to "vi Foo.txt". That could have the advantage of also fixing combining character normalisation issues. And the question of "which locale's rules get used?" is easy - the locale of the shell doing the autocomplete.
Because case insensitive filesystem doesn't require you to press Shift key when typing a command. Typing `ls /Program` requires one keypress more than `ls /program`. Also, it is easier to remember just a name rather than remember how exactly it is spelled.
I think the utility of a real case-sensitive file system cannot be nullified with an "I need to press shift!" argument.
Apple's APFS will create the folder as "Programs" if you use "mkdir Programs", and will give you an error when you try to create "programs". You can navigate with any case combination, but it just plays along like the capitalization you entered is present. It feels like a hack tbh.
Another thing I noticed that I never notice that I'm pressing shift while capitalizing. It's not a deal-breaker issue for me (while this might sound like "it works for me", I'm sure that I'm not alone in that).
Case insensitive systems created a lot of problems down the road. Esp. due to popularity of Windows. Program porting is a pain, using macOS with real case-sensitive mode is again a pain for some scenarios. It also encourages sloppy programming practices and "I don't case as long as it works" ism.
What is this? It's not capitalised so it can't be an acronym or initialism. And your username doesn't start with a capital like a Proper Noun, so it's unreadable. This is my problem with case-sensitivity, it's not how humans use writing, think about writing, design fonts, etc. 'a' and 'A' are the same letter, they're the same letter in the alphabet, they are pronounced the same, they make the same word "apple" or "Apple" which is also pronounced the same and means the same, they could be rendered the same in a given font without making words unreadable or confusing (e.g. small caps fonts).
If the computer isn't letting me use "apple" or "Apple" interchangably as I (the user, supposed to be in control here) wish, that isn't helping me, it's nitpickingly hindering me and making me implicitly care about character encoding values I can't even see. This is dreadful UX.
Domain names aren't case sensitive, email addresses could be but generally aren't, usernames could be but generally aren't, phone numbers aren't, Twitter handles aren't, my bank cards have my name in ALL CAPS but it's still me, you wouldn't join an IRC channel and think it okay to use a nick which matches someone else's but for capitalisation differences, you know which book I mean when I say "the martian" and so does Google search and my email search, PostgreSQL table and column names aren't case sensitive.
It's nice, convenient, helpful when identifiers aren't case sensitive.
> What is this? It's not capitalised so it can't be an acronym or initialism.
It's a case insensitive "to be honest". Non capitalizing it wasn't intentional, but it has shown to be a prime example how case also conveys meaning.
> And your username doesn't start with a capital like a Proper Noun, so it's unreadable.
No, it's a combination of my name and surname, and a play on C/C++ header convention, more specifically "bayindir.h", which is based on a joke one of my university friends. See? Again case conveys meaning. In this case, my username is case sensitive and has a meaning when it's all lowercase. Bayindirh won't do in this case.
> it's not how humans use writing, think...
Case carries information, conveys meaning. "apple" and Apple are not the same word, doesn't point to the same "thing". Computer is the ultimate data storage and manipulation device we have, and I don't want ambiguity in any parts of it.
If humans didn't use cases, and this is not how we use writing, i can just write will all lowercase letters and make distinctions between apple the fruit and apple the company, or turkey the animal and turkey the country some other, more convoluted way. Cases matter. They carry information. Try to tell a mathematician or any scientist that their formulas are moot because X and x are the same variable.
> If the computer isn't letting me use "apple" or "Apple" interchangably as I (the user, supposed to be in control here) wish, that isn't helping me, it's nitpickingly hindering me and making me implicitly care about character encoding values I can't even see. This is dreadful UX.
I'm the opposite. I consider apple being equal to Apple "Dreadful UX", so it's subjective. IMHO, every user have preferences and a computer can adapt to that. Given the choice I'll prefer case sensitive, you'll prefer case insensitive. That's OK. What's not OK is pointing a finger to a property and calling "This is useless!"
On the technicalities side, there are some capitalization rules which make case insensitive systems shiver and break. For you probably i and I are same. For me it's not. We have i, İ, ı, I. How to make this case insensitive? A lot of programs break because of this. This is fun. This is called Turkish.
So, no. Neither a and A, nor i and I are the same letters. They don't have to sound the same either.
> Domain names aren't case sensitive, email addresses could be but generally aren't, usernames...
Case insensitive passwords and usernames are big security nightmares, including the technicalities I've described above. Names on credit cards are built upon the assumption that capitalization works the same everywhere (it isn't). Just because Google wants to match something case-insensitive doesn't make it right.
Some things are designed to be case insensitive against the norms. In normal human language case conveys meaning. Some conventions decide to ditch that meaning. Many programming languages are case sensitive. Are they wrong? No they're not. SQL statements use cases to denote keywords, even if they're case insensitive.
Case conveys meaning. Ditching is a choice.
> It's nice, convenient, helpful when identifiers aren't case sensitive.
In some cases yes, in most cases no. Try to lower() this comment, and try to read it. It may not be nice, convenient and helpful.
> "If humans didn't use cases, and this is not how we use writing, i can just write will all lowercase letters and make distinctions between apple the fruit and apple the company, or turkey the animal and turkey the country some other, more convoluted way. Cases matter. They carry information. Try to tell a mathematician or any scientist that their formulas are moot because X and x are the same variable."
I have occasionally complained that I find math notation frustrating to read because of the use of the same letter with a tiny variation, maybe an unpronouncable one. Mathematicians are not particularly interested in what outsiders find difficult (why would they be?).
I have occasionally enthused about PowerShell, a shell/scripting language where string comparison is case insensitive (by default, sensitive if you want it), where variable names, method names, object properties, function names, are all case insensitive, and how wonderfully convenient, liberating, easing and brilliant I think that is for an interactive shell experience. And about other simplified systems such as APL and Dr Iverson's work to simplify math precedence rules and how its builtins try to work the same over scalars, vectors, and multidimensional arrays. About Simplified Music Notation and their work to reduce cognitive load by in-lining sharp and flat information into the shape of the note instead of having to remember it from the beginning of the bar; the Esperanto language and the 'clockwork-morphology' where all the patterns about past/present/future apply to every verb with no exceptions for strong verbs or loan-words, and how much easier that makes it to learn.
Humans use case flexibly, it adds meaning but it can be overridden. News headline "Apple Announces New Computer" means something different from "Fruit Grower Announces New Apple". Humans now have to deal with more and more different systems, the fewer nitpicking rules humans have to learn for each one to be able to do that, the more the systems adapt to humans, the better.
> "Computer is the ultimate data storage and manipulation device we have, and I don't want ambiguity in any parts of it."
I don't want ambiguity, mapping "a = A" for information retrieval isn't ambiguous, it's completely specified. I don't know what it would do in Turkish, but your own example tells me you can see those four things as variants of 'i' differing in capitalisation and you don't confuse any of them with K or Z. If you can, computers should be able to. I would expect datetime and number formatting to be different in Turkey, and capitalisation too. If i, İ, ı, I are different, then they are different and need to be carefully specified each time, that Turkish case shouldn't mean English a-z and A-Z must be considered different.
> "I'm the opposite. I consider apple being equal to Apple "Dreadful UX", so it's subjective."
You consider it "a dreadful experience" that someone searching Google for 'apple computer' gets results instead of nothing? That "December 2021" and "december 2021" are considered the same month? Someone who doesn't remember the correct capitalisation for XF86Config, what should they search for to find it in your world? All possible case variants? One way they could find out is to list a directory which contains it and read the listing, and the only reason that works is because their brain can recognise 'the same thing' without knowing the case(!). Humans do it, machines should do it (unless explicitly overridden).
> "Case insensitive passwords and usernames are big security nightmares"
Passwords yes. Usernames, no. Case sensitive names are a security nightmare, they lead to problems like phishing. It's bad enough getting an email from @bankofamerlca.com but if the official one was bankofamerica.com and BankofAmerica.com was a scam domain, what chance would anyone have?
Like, apple and APPLE are the same company name in the UK! Search for 'apple' on this government site https://find-and-update.company-information.service.gov.uk/c... - and it says "The company name "APPLE" is considered to be the 'same as' the existing names shown below: APPLE.COM LTD". If that's ambiguous, a computer system should represent the ambiguity of real life.
> "In normal human language case conveys meaning."
But the case difference is not pronouncable, not memorable. I don't have to learn apple and Apple as two different words, I change the case on-the-fly as I need it. Same way I would not want to condiser apple and apple as different words where I have to get the italics correct before the computer will find them, or underlines, or bolding, or colour, I don't want to have to specify the way the news headline was written before I can find it - not font size or emphasis or capitalization. "README.TXT" is still at the core saying "read me" not anything fundamentally different.
> "Try to lower() this comment, and try to read it. It may not be nice, convenient and helpful."
Try to transcribe a talk and write the case they are speaking. They aren't speaking cases, you have to guess from your knowledge of language and work out from context which things are names and where sentences begin. Case isn't present in speech but it still carries meaning.
If you are manually typing a command, then the shell could do this work instead of the file system, right? That's what the shell is for, to be the bridge between humans and the computer right.
In a perfect system, you would type ls /p, and one of the suggestions of the tab completion would be /Program.
You are right. Imagine that you had a fancy speech-recognition interface to your system. When you pronounce the name of a command, there's no difference whether you pronounce it uppercase or lowercase.
Also imagine having two files in the same directory which differ only by case. What would be the benefit of being able to have two or more such files?
Are there any use-cases for which case-sensitive file-system is better than case-insensitive one?
However, the pronunciation of the actual body text will trigger some words to be upper case, so there's definitely a meaning under that capitalized letters.
Actually, not having the exact file may not provide a utility for you, or most of us, but searching for a file in a case sensitive manner, and getting ENOTFOUND just because it doesn't present in the way I searched is valuable from my point of view.
Also, having a case-sensitive system is simpler from technical point of view, and prone to less bugs. This is especially true after UTF-8 become ubiquitous. Searching for an exact byte stream is much more simpler than a country dependent, exceptions filled search/match algorithm.
Bash normally doesn't do that, but there are a lot of options available for both Bash and ZSH, which can do much more than just "case insensitive matching".
Specifically: For bash, `set completion-ignore-case on` in ~/.inputrc, for zsh ... I have `zstyle ':completion:*' matcher-list 'm:{a-zA-Z}={A-Za-z}'`, which seems too complicated but that's what's in my zshrc.
A case sensitive filesystem doesn't require you to hit shift either if you just call the directory "programs". Also, there are many ways to save your pinky from moving the huge distance to the shift-key other than an arbitrary restriction on all file names in the file system..
What are the advantages of using a case-sensitive file system? I cannot imagine a situation when you need to create two files that differ only in letter case. This also would make program that create such files not portable.
The primary advantage for me is simplicity. Case insensitive file systems are complicated when you start dealing with non-ASCII characters and different software implementing the case rules with subtle differences. With case sensitive file systems you know exactly what you get.
If you're going to move to non-ASCII characters, what about Unicode combining characters, do you have to care if your file named é is a single accented e or a pair of e-and-combining-accent before you can move it?
If you shouldn't have to care, if there should be a layer of Unicode normalization happening, why is that okay but case normalisation is not okay? If you do have to care, then you no longer "know exactly what you get".
> If you do have to care, then you no longer "know exactly what you get".
You do know you get exactly what you put in, whether that is é (U+00E9) or é (U+0065 U+0301). When you reference files created by yourself, this is not a problem as realistically speaking, your input method will only have a convenient way of forming one of these, and it will consistently generate the same one every time. When you reference files created by someone else, this may be a problem but no more than e.g. the distinction between file.txt (lowercase L) and fiIe.txt (uppercase i): from the user's perspective, the problem is pretty much avoided by selecting the file using tab completion, TUIs, GUIs, whatever you use.
> e.g. the distinction between file.txt (lowercase L) and fiIe.txt (uppercase i)...
Just want to remind a little but extremely important thing: There's absolutely no guarantees that upper("i") is I. There's at least one language breaks that convention, and you won't believe how a big headache is that.
https://prnt.sc/257kxxa lowercase l and uppercase I are quite distinct. If you're going to select all your filenames with autocomplete, then why argue restrict me to case sensitive filenames?
Windows NTFS is sometimes described as case-preserving, case-insensitive; i.e. if you name a file "Test" it will stay in that case, but if you ask for "test" it will find the file "Test". I don't know whether that happens in Win32 or NTFS, but it seems like best of both worlds; I don't want "case insensitive" where it could show me the name in a different case than I entered.
I don't override the font, either HN or FireFox or Windows is picking a serif font. I assume you didn't explicitly choose a font where different things look the same, but if it rendered 'a' and 'Z' the same glyph, you wouldn't say that was a problem with the English alphabet or with case sensitivity, or anything other than bad font design, right?
You showed a screenshot earlier, I know what they look like for you. :) I don't override the font either. HN specifies a CSS font family "Verdana, Geneva, sans-serif", where Verdana is installed by default on Windows but is not a full sans-serif font, so it's a somewhat odd choice by HN. As for the English alphabet, I didn't say there was a problem with it, just that you cannot reliably tell just from looking at letters which letters they are. In practice it is not an issue.
Frankly, the use case where I make a file to send to a customer "Example Ltd DNS details" and want to open it from a command line happens to me infinitely more often than the use case where I have a file called "fiIe.txt" and want to open it from a command line.
Perhaps a new concept of a "symlink with backups" could be introduced.
You install ProgA and the lib is symlinked into the index
You install ProgB and the index's symlink to ProgA's lib gets some information added to it along the lines of "if the symlink to ProgA/lib is broken, try ProgB/lib"
Then when you uninstall ProbA the fallback symlink is activated.
At some point you would want to clean up the dangling primary link but maybe the best time to do it is immedately. i.e. the first time the primary fails you make the backup the new primary and now there's no backup, i.e. it looks the way it would look if you had only ever installed ProgB.
Why not use hard links instead? Both ProgA and ProgB will have their own links to the same inode. Either one can delete their own link, but the library will still exist unless all programs using it delete their links.
GoboLinux uses patchelf just like Nix but you don't have to learn the convoluted undebuggable mess of Nixlang (I run NixOS on all my machines, it's a love-hate relationship)
Who are you to define the purpose of Nix? Many people use Nix as an imperative package manager too. So no, the purpose of Nix is not reliable declarative configuration, it's more than that.
The GP's stated benefits are listed directly on the main page of the NixOS website, and was in reply to a comment about Nix in NixOS. You seem to be talking about using Nix standalone; if you are, the GP's comment does not necessarily apply to you.
I've only tried Nix once or twice, and I'm left wondering: why don't they use a scripting language? Is part of the appeal that the Nix language is a functional language?
As a longtime Nix user with very little prior FP experience (1/4 of one course in college doing a few exercises in Haskell), I think the Nix language is actually very well-suited to task.
The biggest difficulty comes from the fact that it is a dynamic, weakly-typed language, which can lead error messages in configuration to point you toward library code rather than your config file. This is a pitfall shared by many of the languages that are commonly suggested as replacements or alternatives.
There's been work to solve this by adding gradual typing (like in Typescript) to Nix, whose most promising iteration atm is (imo) a Nix-like language called Nickel, which is almost ready for preview by the community.
The other, more minor, issue is tooling, which is being addressed by emerging LSP work for Nix.
(Scripting is done in Nixpkgs, mostly via Bash.)
The beauty of Nixlang, imo, is that for simple use cases it really feels like a dead simple configuration language, which is only possible with a declarative language.
The functional character and Nix's laziness were important in the early design of Nix. The latter may still give Nix some nice performance characteristics. But since the initial Nix design, Nix has become eager in some places, and Guix is implemented as a lazy DSL embedded in an eager language, so I guess we could implement something like Nix on top of an eager scripting language.
Imperative scripting languages might not be as well suited for the kinds of overrides that Nixpkgs and NixOS use, afaict. But imo aside from its novelty, the Nix language is nice to use because it's really simple and totally declarative, which is what you expect from a configuration language.
I don't know what people consider "eager" in Guix. In the `package` DSL all input fields are thunked, so they are not eagerly resolved. You don't need all values of your language to have lazy semantics --- delaying evaluation of some values is enough. Just because Guile Scheme is no lazily evaluated by default does not mean that "Guix is eager" is a useful statement.
> You don't need all values of your language to have lazy semantics
Once upon a time (no longer, I think), Nix was 'maximally lazy', so I think historically at least, there has been a question of how essential laziness is to the design of package managers in the functional paradigm. Since then I guess we've seen what you describe (some targeted laziness) is all that's required in both DSLs.
I've edited my comment above to better reflect Guix's design and stop spreading the 'Guix is eager' meme. :)
I’m a fan of the NixOS module style typing. It feels like the right compromise for something like Nix. Domain specific type system is also helpful for documentation. E.g. in NixOS, the types and the docs live together and benefit each other. Giving that better support in the language and building editor tooling around it seems like the way to go.
There’s an open RFC/PR to bring NixOS style modules to Nixpkgs for package management (with derivations being the primitive).
Reproducibility. You need a pure (side effect-free) language to ensure that a package expression always evaluates to the same value [1]. If you have a non-pure language, external factors (e.g. environment variables or a server returning a different response) can influence/change what a package expression evaluates to.
[1] That said, Nix was not completely free of side-effects either. Though this is one of the issues that Nix Flakes attempt to solve.
You can use pretty much any Turing complete language to achieve this constraint, as long as its inputs and execution are deterministic. The language itself doesn't have to be purely functional.
It’s mostly about using the right tool for the right job. The Nix language is used for large scale configuration, not scripting. Anything that requires scripting is offloaded to Bash. As a result of that, Nix resembles JSON but with variables and lambdas to make writing complex configurations easier. The competitor of the Nix language is not Bash or Python, it’s JSON and YAML.
A friend uses YAML in that way. Each entry is valid JS code. The result is you have a mixed declarative-imperative configuration language. It's very powerful and I have found a few uses too. The best part is that the evaluator is at most 30 lines of code.
Haskell is a general-purpose language, and Nix is made specifically for package management and system configuration. So Nix is significantly easier to learn than Haskell.
Most programming languages are already generally deterministic.
Pick any programming language. Now, remove access to I/O (except standard input and output), networking, time, random number generators, threads, OS syscalls, etc.
Given an input, the program will always generate the same output, i.e. it is deterministic. However, the program is still allowed to have global mutable state; the state is just encapsulated to the program's memory and execution time.
A trivial example is Brainfuck. It is most certainly not a functional programming language, but it is deterministic.
> Pick any programming language. Now, remove access to I/O (except standard input and output), networking, time, random number generators, threads, OS syscalls, etc.
The devil is in the details, and if you miss even one case then the problems are just as bad as if you hadn't bothered at all. For example, comparing two URLs in Java is not deterministic. For another example, iterating through a set in Python will give you the elements in a different order on 32- versus 64-bit systems.
In practice it's just not practical to retrofit deterministic behaviour onto a language that was not designed for it. Even your "trivial example" isn't; Brainfuck does not standardize overflow behaviour and so the same program may behave differently on different systems.
> For example, comparing two URLs in Java is not deterministic.
Well, it does network access, so it would excluded.
> For another example, iterating through a set in Python will give you the elements in a different order on 32- versus 64-bit systems.
This is a good counter-point.
> Brainfuck does not standardize overflow behaviour and so the same program may behave differently on different systems.
Different implementations of Nix might have the same issue in some places. In this case, whether the language is fully defined is orthogonal to whether its behavior is deterministic; here, a particular implementation will continue to be as such.
As far as I know, Nix doesn't have a formal specification; it is defined by its reference implementation.
Deterministic execution simply requires you remove sources of non-determinism. This is not to be confused with defined vs undefined behaviour, like a program behaving different on another machine. As long as it's behaviour remains constant over all executions in the same input environment (where architecture, for example, is part of the input), it is entirely fine.
After all you need to be non-constant over architecture, since you need to download the correct binaries for 32-bit and 64-bit architectures.
The Nix language is just JSON + lambda functions + variables. And nothing more. If you know modern JavaScript then you already know the Nix language.
The problem is Nixpkgs, the largest Nix program. Nix-the-language doesn't lend itself too well to programming in the large. It's a bit like writing a large JavaScript SPA without frameworks or strict coding standards.
So everything has to be pure and side-effect free, I'm not into the history and when you're not doing too much Voodoo it looks like what you'd want a configuration file to look like.
But the downside is that debugging is impossble.
Guix uses guile, which is a Lisp so we could accomplish the same with other languages.
me too. I broke my system a bunch of times, and eventually just gave up. I remembered the other day that I have a lot of programs installed with it that I don't even want to think about updating. rofi changed it's configuration file format recently, and that was tied up in nix. I am too lazy to fight with it for now, so I've just disabled my config file. It bothers me every time I use it.
I'm positive it works. My problem was that figuring out why something went wrong was too time consuming, or difficult. I'm sure I could figure it out, but the problem again is time.
TL;DR NixOS uses hashes based on all dependencies that go into a package build and GoboLinux uses version numbers. Which makes Gobo easier to use, but less secure.
GoboLinux is more impure and more compatible with processes that people are used to. Imo it will take an impure, downstream distro of NixOS to really make GoboLinux 'obsolete'. Something where, in a pinch, you can just
./configure && make
without grokking Nix, and have it work. Or where you can install Linuxbrew and Pkgsrc and have them just work, as additional, non-sandboxed escape hatches.
And such a downstream system could benefit from mimicking GoboLinux in a few ways, imo.
You could make a CLI that does the templating for you, but there's nothing fundamental here that would require a fork of Nix (or Nixpkgs/NixOS) itself.
Envfs looks interesting, but I don't think compromising on Nix(OS)'s core to be more familiar is a sustainable approach, and the unreliability of the produced frankenbuilds seem more like something that'd scare newcomers away once things start breaking.
I'd rather look into how to make the Nix-native workflow be as appealing and welcoming as the impure one, than compromise and end up with a system ultimately just that has the downsides of both.
> I don't think compromising on Nix(OS)'s core to be more familiar is a sustainable approach
That's why I say this would be a downstream distro, because then
> and the unreliability of the produced frankenbuilds [can] scare newcomers away
to NixOS proper, which can remain pure, principled, and a little difficult. :)
The idea is a workstation/desktop-centric downstream where people can
* use NixOS modules to configure the base system (which is super convenient)
* learn the Nix language ‘by immersion’ and tooling without *immediately* having to choose between packaging and giving up any proprietary/oddball software they might already be using that isn't already in Nixpkgs
* try Nix in its most attractive form (fully declarative OOTB and in control of a full operating system, unlike Nix on macOS or foreign Linux distros)
* start out with all of the commonsense community goodies (flakes, flake-utils-plus, Home-Manager, NUR, and extra binary caches enabled when relevant) pre-enabled and configured in the simplest possible way
* comes with a reset button (like a slightly gentler NIXOS_LUSTRATE) that users can easily use to remove any impurely installed software
While many users have had success diving into NixOS head-first, many others have reported that they found it easier and more productive to first learn the Nix language and Nix tools on macOS. Those users report that they have an easier time because it allows them to learn more gradually or in smaller chunks according to the availability of their free time, with the macOS base system and impure tooling built on top of it providing an escape hatch for when they get stuck or frustrated, or want to quickly try something that's not packaged in Nixpkgs.
There's no reason that Nix itself couldn't be used to create the same kind of stable, impurely accessible, compatible (with traditional tools), kind of base system. I think it'll happen some time in the next few years. There's much wider interest in NixOS than their used to be, and with that has come increased interest in making it even more accessible to new users.
I thought Gobo came first. I remember installing it ~2003. Didn't hear about Nix until years later and when I did it reminded of that old Gobo install.
Docker (in my opinion) is a poor solution for distributing apps. It resembles a quick hack rather than a solid, scalable, future-proof solution. The problem is: you need to deploy an application with all required libraries with specific versions and build options. Obviously, a proper solution would be a package manager which would read the declarative, non-executable configuration file and install or build necessary dependencies.
Another problem is that you want to run an application on differen platforms, not only Linux. Again the proper solution would be to write a cross-paltform application and install it using a cross-platform package manager.
Docker doesn't offer a proper solution; instead it suggests that you create a virtual machine, install a whole operating system (without a kernel) there and then write a bash script that will download and build dependencies. As docker image configuration file is an executable script, it means that it cannot be processed with automated tools; it can only be read by a human.
Also, docker requires a daemon running as a root. This increases potential attack surface.
Docker is not good for desktop applications because it doesn't have portals which can restrict access to DBus, audio daemons, file system etc.
Why is it that AppImages, snaps, flatpaks, Nix and every distro package manager is replaced by docker?
Could it be that docker is missing something, maybe the Dockerfile isn't easily reproducible? Maybe the ephemeral status of all containers is quite some effort to work with?
Maybe it's just not the one true answer to the question of software packaging.
Identifying your dependencies is hard. The easy solution is not to care: develop on a machine, and ship the entire machine! That way, software with the Works On My Machine seal of approval can be pushed to production without trouble!
I love Docker for the simplicity it offers, but the gigabytes of storage wasted on yet another slightly different version of Debian every time I pull a container irks me to no end. Sadly, it's the only way some software is distributed.
> Why is it that AppImages, snaps, flatpaks, Nix and every distro package manager is replaced by docker?
Where are you seeing distros using docker for containerization? If anything, I'd say Flatpak is the one you're most likely to find installed in a 2021 workstation linux.
Docker's killing features are the same as they always were: usability and ecosystem. 'docker run thingamajig' is simple and DockerHub is very likely to have thingamajig in its repos.
Of the ones you listed, I'd say flatpak is the only one that compares favorably to Docker on both aspects ('flatpak install thingamajig'). And since flatpak is more designed for desktop applications, it's actually the one gaining ground. Docker or other containerd-based solutions should remain the preferred choice for servers, since they're designed for that.
Nix is vastly less user friendly. Snaps, last I checked, still don't have an open-source repository implementation, meaning that Canonical has absolute control over them. I don't know much about AppImage, but AFAICT it's just a file format and not a full package manager.
In my mind the principal reason for the current hierarchy is read/write usage.
With /etc and /usr it is possible after installation and configuration to set one or both as Read Only. With /var and /home you have the directories that should be Read/Write on a running system. This allows you to mount each of those directories on a different partition with different settings or even different filesystems depending on how you want to optimize/secure your application.
Now admittedly actually doing this is pretty rare in these days, but I still like having the option. I believe he does address this talking about "union mounts" and "overlay filesystems". I'm really not too familiar with either or how production ready they are, but it may address my concerns.
> Now admittedly actually doing this is pretty rare in these day
It’s pretty frequent! When Atlassian launched their Cloud offerings in 2013, they installed Confluence and Jira on their own servers (1.5GB each), one per customer, and set /bin and /etc as read-only.
Then they mounted /etc and /bin from the network. 1.5GB saved per instance!
So it makes sense not because you can set them as read-only, but because you can mount them separately.
I always change my home folders to a lowercase shorthened version and modify ~/.config/user-dirs.dirs. Of course, when you do an accidental rm -rf you're at disk to erase ~/work, but that's what regular backups are for. Expanding capitalisation to the whole filesystem hierarchy is a no go for me, it would only waste my time and further degrade my productivity. I also never liked Pascal, Java and C# specifically because of the weird capitalisation conventions which in my view makes the code look messy. PowerShell? Just call the exorcist.
FWIW I tend to put directories first character as Uppercase and files as lowercase. this is useful when sorting and also identifying what type of file something is (so long as its a file I created).
A file or folder can have a name with mixed uppercase and lowercase characters. That name would show in the directory listing. But it should be addressable case-insensitively.
And it shouldn't be possible to create two or more separate files in the same directory which differ only by case.
Honestly, now that Apple has proven you can run a POSIX OS on a case-insensitive filesystem, I'd think any "let's clean up the directory structure" project should be leveraging that work and going full case-insensitive. Case sensitivity should be avoided for anything outside of passwords and base64 and the like
Imho, yes. Well, except for the 8.3 thing. And the total path length limit. And the primitive security model. And the limits on having files named com and the like.
Cool! I thought it was no longer maintained. Although it influenced the distro landscape much less than it could, things have changed enough so that I have mostly a "don't care" feeling about gobolinux.
Most end users rarely interact directly with the filesystem, so a different hierarchy won't impact many people. Installing up-to-date packages that do not threaten the stability of your system is easy with flatpaks/snaps; it is even possible to install GNOME Software add-ons to handle them. If you want to install "system-software" that is not possible with snaps or flatpaks, there's always Nix, Guix or homebrew. All that without considering AppImages, of course.
So... gobolinux is a great idea. But it fixes problems that are much smaller today.
It's going to need a lot more "Recipes" for packages in "Compile" (1) for GoboLinux to get mainstream. As it stands now, a lot of users are going to have to manually build/install many of their required programs to make them work. Is there a dedicated team out there making those recipes for different programs or do they just hope maintainers will take notice and do it themselves?
This is a really excellent way to manage packages in a system, I ran lots of systems a long time ago this way because of Dan Bernstein https://cr.yp.to/slashpackage/management.html
It hurts me how close we could be to a single, cross-platform module system for Nix, because I know how awesome it would be.
It's theoretically closer than ever, thanks to stuff like nix-processmgmt and Initware for the BSDs, but it feels like something that keeps coming close only to fade away :'(
But Nix can absolutely do it, and it could be so extremely good!
(There's lots of progress in the Nix world to enjoy and celebrate, as usual, but it's just that type of project that makes you think of all kinds of crazy new things it could do)
I'm not sure what course source means, but I'm down to work on this. I don't really know much about other Unix-likes but I'm willing to stumble through it!
What do you want to start with? FreeBSD + Initware (so that we can reuse the existing NixOS systemd service file generation stuff?)
I think I'll try messing around with them a little on their own to get a feel for what they need in order to run
oh oops, I think I was intending to type "crowd source" but I really met "crowd fund".
I do want to finish my FreeBSD cross PR, but is is hard. We already have NetBSD working, including building the kernel, so I think it is better to start there.
I opened https://github.com/cleverca22/not-os/issues/16 because I think that would be the absolute easiest first step, with the fewest moving parts. But I don't know how kernels, even Linux, are packaged into bootable thingies at all.
NixOS is organized into modules, as are a few community projects modeled after it, like Nix-Darwin and Home Manager. A more unified module system could mean
* NixOS supporting kernels other than Linux
* NixOS supporting init systems other than systemd
* NixOS supporting usermode services (i.e., code sharing between Home Manager modulea and NixOS modules)
* NixOS supporting a 'guest mode' for configuring system services (i.e., NixOS foldinf in Nix-Darwin and supporting similar systemwide service configuration on other foreign operating systems).
In this case, I'm talking mainly about the first, as a starting point, and hinting at the others.
Long ago (maybe 15 years) I tried Gobolinux and thought it was a better approach for the typical Linux convert from Windows. I believe it was the lack of software, relative to Debian, Ubuntu, or Fedora, that kept me from running it.
Some people are asking how it compares to some cross-distro package managers.
The answer is, it doesn't – but it does seem to me that it would be very cool to see Gobo contain support for Flatpak, Snap and AppImage.
This would alleviate its shortage of apps somewhat. Keep the distro simple, and just don't package apps: outsource that to the cross-distro package formats.
Secondly, ISTM that Gobo would be a natural fit for a desktop that already treats certain specially-structured directories _as_ apps. I only know of 2 of these: GNUstep, which uses essentially the same `.app` bundles as macOS, and the ROX Desktop, which has !AppDirs that are closely modelled on Acorn's RISC OS. This, incidentally, is the internal format of AppImage.
The above paths have the format of "/Programs/<PackageName>/<VersionNumber>/<TraditionalDirectory>/<FileName>". GoboLinux presumably scans every package directory and adds symbolic links to "/System/Index/<TraditionalDirectory>/<FileName>", like the following example (respectively):
The primary file locations make it easy to handle one package at a time, cleanly adding and removing entire packages. The indexed view makes it easy to find files (binaries, libraries, includes, man pages, etc.) by name without worrying about what packages they belong to.
Generally speaking, tagging can be half-simulated using hierarchies and (hard/soft) links. But a tag-based system would be cleaner and not prioritize one retrieval method over another. Here is an illustration of how I might rephrase GoboLinux's package storage and retrieval system in terms of tags:
Blob("... binary data of bash executable ..."), with hash = 9e19e455.
Blob("... binary data of ping executable ..."), with hash = 28224f5b.
Blob("... binary data of libpng.so library ..."), with hash = 6243c115.
Blob("... binary data of stdio.h C code ..."), with hash = d0335126.
Tag(packageName="Bash", version="4.4", target=9e19e455).
Tag(packageName="Netkit-Base", Version="0.17", target=28224f5b).
Tag(packageName="LibPNG", version="1.2.5", target=6243c115).
Tag(packageName="Glibc", version="2.24", target=d0335126).
Tag(traditionalDirectory="bin", fileName="bash", target=9e19e455).
Tag(traditionalDirectory="bin", fileName="ping", target=28224f5b).
Tag(traditionalDirectory="lib", fileName="libpng.so.3", target=6243c115).
Tag(traditionalDirectory="include", fileName="stdio.h", target=d0335126).
Starting directory names with capital letter is not a good idea. While that might look nice, it requires more keypresses when typing it in the terminal. So /apps or /soft would be better than /Programs.
It is convenient in case-insensitive filesystems like those used in Windows or Mac, but not in Linux. By the way it would be better if Linux also switched to case-insensitive filesystem, or even a filesystem where similar looking characters from different alphabets are considered the same. So that A is always A no matter what alphabet is used.
Also I don't like the way they hide classic directories like /bin. Instead, they should just not create these directories; well-written programs should not require their presence.
The response by one of the devs to the first problem is "In fact, the concerns on typing-friendliness always comes up in discussions about the GoboLinux tree. To that, I can only respond that, in a properly configured shell like the one that comes by default with GoboLinux, typing /Programs takes the exact same number of keystrokes as typing /usr: slash, lowercase p, Tab." [0]
The response to your second problem is that not all programs are well written, and they cannot afford to patch every poorly-written program.
I hear you, but I also think tab completion is one of the basic amenities we should expect shell users to avail themselves of in 2021. And distro-makers can even pre-configure them for their users. I can understand why a distro maker might not be interested in accommodating users who don't want to use decent tab completion with smart casing.
Yes I agree, just playing devils advocate that you shouldn’t have to change off defaults for ergonomics when there isn’t a real _benefit_ to pascal case.
I agree that PascalCase makes more sense on systems like Windows, where everything is case-insensitive, or in shells like PowerShell, where virtually all commands are shell functions rather than separate binaries, and the shell itself is case-insensitive. Then, I suppose you get some legibility benefit, without any extra hassle.
I'm personally still on the fence about the more verbose style. I think I like it for directory names but not really for executable names.
From the faq it sounds like the purpose is to differentiate the gobolinux top level folders from the hidden compatibility folders, which makes sense to me.
But I don't think that it is an unsolvable task. If you simply use Unicode-aware version of "to lowercase" function, it will be better than what we have today. But of course there is a place for improvement (dealing with accented letters in European languages, dealing with combining characters, different alphabets like Hiragana/Katakana etc).
Hum... I think simpler is better, and having each letter be what they are only seems much simpler than having to guess if the file system is going to decide my new file name is equivalent to another existing file name and overide it rather than creating a new one (for example).
It is "simple" for a programmer who doesn't want to learn Unicode and prefers working with bytes instead of letters, but inconvenient for a user. More keypresses, easier to make a mistake when typing.
I cannot imagine a situation where you need two files whose names differ only in case.
Since it's relevant and funny, I'll quote the famous rant of Linus Torvalds on the topic (it was because of issues it caused with Git, but I assume it still means it's unlikely Linux will ever follow this path ^^)
"The true horrors of HFS+ are not in how it’s not a great filesystem, but in how it’s actively designed to be a bad filesystem by people who thought they had good ideas.
The case insensitivity is just a horribly bad idea, and Apple could have pushed fixing it. They didn’t. Instead, they doubled down on a bad idea, and actively extended it – very very badly – to unicode. And it’s not even UTF-8, it’s UCS2 I think.
Ok, so NTFS did some of the same. But apple really took it to the next level with HFS+.
There’s some excuse for case insensitivity in a legacy model (“We didn’t know better”). But people who think unicode equivalency comparisons are a good idea in a filesystem shouldn’t be allowed to play in that space. Give them some paste, and let them sit in a corner eating it. They’ll be happy, and they won’t be messing up your system.
And then picking NFD normalization – and making it visible, and actively converting correct unicode into that absolutely horrible format, that’s just inexcusable. Even the people who think normalization is a good thing admit that NFD is a bad format, and certainly not for data exchange. It’s not even “paste-eater” quality thinking. It’s actually actively corrupting user data. By design. Christ.
And Apple let these monkeys work on their filesystem? Seriously?"
I can think of one. When we write (say "ES6") modules it makes sense that the name of the module-file without extension should be the same as the value exported by that module.
Inside the module you'd prefer to follow conventions like name classes title-case and name functions camel-case.
Thus you might have one module exporting a class named "Camel", and another module exporting a function named "camel". You could then have two files, Camel.js, and camel.js in the same directory, containing two separate programs.
I think one criterion could be that you should be able to identify a name by its pronunciation. The same utterance should always refer to the same thing on the computer. When you speak you don't speak line-breaks. By the same logic commas should not be allowed in names. And when you speak we don't typically hear whitespaces either.
By the same logic the whitespace sensitivity of Python is a bad idea. :-)
My criterion is that paths should not be painful to type, parse, or copy-paste. So no keyboard layout switching, and escaping should not be necessary. Lower-case Latin letters, digits, periods, and dashes (of the unique variety that's available on keyboards, and not in the initial position) are simple and enough.
I wouldn't mind but there are a bunch of people in this world who have no idea what those ASCII symbols even mean, and who might prefer using symbols they understand ^^
Also, my native language does not use the Latin alphabet. Attempts by American companies to be more inclusive and internationalized are often misguided. MS is especially nasty with its translated formula functions in Excel, but having differently named filesystem paths in different alphabets depending on the phase of the Moon or something is also very inconvenient.
Oh yeah Excel (or libreoffice) is really awful to use in another language, I often have to force it to English because of the formula mess. But I guess that's a developer-who-can-speak English point of view.
Though I do have to admit that I do not know what, e.g., ASCII code 8 would mean in a filename. Or 27. I’d very much prefer using symbols I understand.
Filenames with terminal control escape sequences are lots of fun.
Limiting possibilities to just lowercase Latin, numbers, periods and dashes (in non-initial positions) decreases chances people see symbols they do not understand by a lot.
I think this approach is very similar to how Android is handling the different apps where each gets installed into its own dedicated path directory. (I may be wrong though, cant remember since the last time I‘ve looked into the AOSP)
I have a Gobo VM set up, but I have to warn you, in the latest ISO the package manager is broken, it requires a loopback device, which isnt present in the install (why?)
Seeing those files with capitalized filenames, please tell me they're using a case-insensitive filesystem. I hate capitals on case-sensitive filesystems.
Flatpak is an app packaging system, distro-neutral.
GoboLinux is a distro and a new, different way of organizing the Unix filesystem.
Flatpak contains conventional (or at least similar) filesystem layouts inside app packages. Gobo replaces the contents of apps as well as the whole filesystem.
Flatpak requires runtime infrastructure on the distro you wish to use. Gobo is a distro.
Flatpak is distro-neutral; Gobo replaces other distros.
Flatpack puts all apps in its own hierarchy deep inside /var and manages it itself. Gobo does not have a /var, or a /bin or /usr or anything else.
Flatpak could be run on Gobo. Gobo could not be run on Flatpak, because Flatpak is for apps.
I do not know how familiar you are with Flatpak, what it is and what it is for, and Gobo, what it is and what it is for.
So this is a very broad request and could take tens of pages.
Flatpak installs apps on any distro which has or has had Flatpak support installed on it. It does this by unpacking apps into its own directory trees. Apps go into:
`/var/lib/flatpak/exports/share/applications`
Inside there are more complex trees, so for instance, to reply, I just installed Sublime Text, and it is in:
Inside that is a `files` folder. Inside that we see an encapsulated version of a traditional xNix filesystem:
total 40
drwxr-xr-x 8 root root 4096 Jan 5 12:30 .
drwxr-xr-x 4 root root 4096 Jan 5 12:30 ..
drwxr-xr-x 2 root root 4096 Jan 1 1970 bin
drwxr-xr-x 4 root root 4096 Jan 5 12:30 extra
drwxr-xr-x 2 root root 4096 Jan 1 1970 lib
-rw-r--r-- 2 root root 4817 Jan 5 12:30 manifest.json
-rw-r--r-- 1 root root 0 Jan 5 12:30 .ref
drwxr-xr-x 6 root root 4096 Jan 1 1970 share
drwxr-xr-x 2 root root 4096 Jan 1 1970 sublime_merge
drwxr-xr-x 2 root root 4096 Jan 1 1970 utils
In here are traditional xNix dirs called `/bin` and `/lib` and so on.
Flatpak is a wrapper around traditional xNix directory layout. It encapsulates them inside a layout it manages, allowing apps to be transported between different distros by having their own layout inside a private tree, but it's a fairly conventional xNix tree.
With me so far?
The conventional xNix layout has globally-shared folders, segregated by functional roles: this subtree is for sysadmin stuff, this subtree is for all users; this subtree is on the main disk and is needed for boot, this one is not essential for boot and is only used in multiuser mode, this subtree is auxiliary additions later, this one is local to this network node only, and so on.
This was needed in the 1970s with very small disks that could not hold the entire OS. `/usr` was originally for home directories but some of the people using the OS were also simulatenously writing the OS. Stuff in their home directory became essential to the system. So it got fossilised into place meaning that a new `/home` tree was added later.
The Mac and Windows conceptual model is different.
It says, each app lives in a separate tree of its own, and we will put only files expected to be shared with other apps into system-global locations.
So the OS itself is in `\OS\` and all the components of the app Bar by Foo Corp get put into `\apps\foo\bar`. Any shared libraries get copied into `\OS\libs\` so that other apps could use them. Later this was found to be a mistake and so `\app\Foo\bar\libs` was added for a known good copy.
Gobo discards the traditional xNix model of global shared folders containing binaries from multiple sources, and segregates them in a Mac and DOS-like model. There is no real `/bin` or `/lib` folders. Instead, there are new trees: Programs, Users, System, Data, Mount etc.
Inside Programs is a tree for each app, and inside that, for each version of that app.
xNix: the files containing the components of each apps are split up and put in the system-global dirs for that type of component. Apps do not have their own dir. Config is in the config tree, libraries are in the libs tree, binaries are in the binaries tree, etc.
Flatpak manages its own version of the global OS layout, one per app, inside the Flatpak tree. Apps are not put into the OS global dirs but each has its own private version of the OS-global tree inside its own dir tree inside the Flatpak tree.
DOS, Windows: the OS lives in its own tree. Each app has its own subtree in the Apps tree and copies only shared components into OS-managed shared folders.
Mac OS X: the OS lives in simplified xNix tree. Apps live entirely in their own private shared tree, one per app, with an OS-mandated layout.
Gobo: the Mac OS X concept is extended to the OS itself. The OS' components are in private, versioned folders inside an OS-mandated tree, with the traditional xNix layout hidden. For legacy apps that expect trad xNix dirs, they are there but populated only with symlinks and the OS hides them from the user.
Flatpak adapts the xNix model and encapsulates private versions of it.
OS X keeps some of the xNix model for the OS only and enforces a new private-dir-tree model upon apps.
Gobo takes the app model of OS X and applies it to the entire OS, and only fakes the trad xNix model to make it easier to port legacy code.
I appreciate the really in depth reply; however, they truly do sound similar if Flatpak was the only package manager installed in the OS. The PopOS! distro is like this and is working on relying more on Flatpak than Apt, for example.
It's not "apps are packaged with all their dependencies inside a private directory hierarchy". That is true of both, yes. But it is also true of Canonical's Snap format, and 0install, and of AppImage, and of NeXTstep & Mac OS X's .app bundles. In other words, it is far from unique.
The real difference with Gobo is twofold:
• Most of the others preserve significant elements of the traditional xNix directory layout. macOS in the main OS if not inside the bundles; Flatpak and Snap inside the apps; and so on. Gobo gets rid of it everywhere, top to bottom, from kernel to text editor.
• Flatpak, Snap etc. are just app-packaging systems. Even on OSes which use Flatpak exclusively – and PopOS is a poor example here; better ones are Fedora SilverBlue or EndlessOS, with read-only root filesystems, no package manager at all, and OS updates using `ostree` – the directory layout is conventional. Like macOS, the app directory structure is entirely separate from the OS directory structure. In Gobo they are one and the same.
I'm struggling to figure out exactly which directory structure Windows has changed every major release -- are you referring to when "Documents and Settings" was renamed to "Users" with the release of Windows Vista in 2006? Or perhaps you are talking about when they added "Program Files (x86)" with the introduction of 64-bit support in Windows XP, dating back to 2001...
I believe it used to default to installing Windows NT to C:\WINNT... but that was almost certainly longer ago than you're suggesting. To be fair, the Desktop also used to live at C:\WINDOWS\Desktop, but there are some pretty good reasons why it doesn't any longer.
In Win9x, it was c:\windows\desktop etc., before windows became multi-user, when it moved to c:\windows\profiles\xxx\desktop and later to C:\Users\xxx\Desktop. But only on US installations, in international versions it was c:\benutzer\xxx\desktop or whatever the translation of "users" was. That is until windows 7 (I think) when they moved to the English name for all systems, and had the UI (Explorer) translate it, which led to strange issues that Explorer shows "C:\Benutzer\x\Desktop" while the command prompt shows "C:\Users\x\Desktop"
On some versions it was also "C:\Documents and Settings\...", it is so convoluted that even I don't remember all the specifics, but yes, it was a total mess, especially on non-english installations
When did ProgramData and AppData become a thing? Is there a guide somewhere about what should be placed in each folder? Local, LocalLow, Roaming? Seems like there are several competing conventions.
ProgramData is the global application storage. AppData has been there since at least XP. Local is intended to be specific to this computer whereas roaming is something that is user specific that can be available across machines.
Sometime before 2008 (and, since the change isn't the kind MS tends to do during a major OS version lifecycle, probably with or prior to the release of Vista in 2007) though the documentation on it (not just when it happened, but how to use them best) seems to be well-obscured. But here's a post originally from 2008 covering Windows Vista which is the least-bad coverage of ProgramData/AppData I can find:
Installing each package into it's own directory, and maintaining symlinks into that directory structure, was an approach explored for large system installations in the 90s and 00s.
NIST depot, Xheir, CMU, and Nix all did this in varying ways. The advantage of this approach was that packages could be installed once on an NFS, scoped by version/configuration, and mounted into the filesystem of every work station on a campus. The workstations then just needed to maintain references to the files on the NFS for their particular software configurations. There is also some interesting exploration of composable configuration templates that allow workstations to inherit and extend a particular configuration (i.e. the physics department might have a base template).
Nix is easily the most successful in this space. It even uses something similar to the NFS approach! It can maintain a remote cache of observed configurations to pull from instead of redoing work.
Reading these LISA papers is a lot of fun. If you work in system administration (i.e. maintaining an Artifactory instance for your company) and have a day to spare - I'd very much recommend reading them, starting with the Nix paper!
To get you started:
Nix - https://www.usenix.org/conference/lisa-04/nix-safe-and-polic...
Depot Lite - https://www.usenix.org/conference/lisa-viii/depot-lite-mecha...