This is great! I’ve been remotely logging into systems for all of my adult life and last week I went on a bit of a bender because I realized didn’t really grasp how these things actually work.
I am not sure I do now (specifically how SSH ties in to this).
But at least I now know more about terminals, shells, rawmode and ptms/ptx and more.
This link right here fits right in with the following ones:
Thanks for the links. So many vital information there especially with regards TTY. I had a bit of an issue with it while working with docker sometime thus year. It will be nice to learn more about it.
For those interested in learning more about terminal emulators and character graphics, checkout Nick Black's book Hacking the Planet (with Notcurses): A Guide to TUIs and Character Graphics[1].
While the overall focus of the book is on programming with Notcurses[2], the author shares a wealth of related info and history throughout its pages. Note also that Notcurses has evolved quite a bit, but the book is still a nice resource.
I bought that book a while ago, and it reminds me of the old days when a lot of the leading lights in the open source community were a bit more of a personality. I enjoyed it, he's obviously incredibly bright and knowledgeable and he makes the subject understandable and fascinating. Until you commented I had never run into another soul that knew about him much less had read the book.
(I know it sounds like I could be Nick with all this praise but I'm just a guy that appreciates people with interesting obsessions that may benefit me some day).
Still the main question remains regarding this kind of TUI stuff: Why?
I see that it's a impressive demo and great hack, sure. But why should we use a glorified virtual line-printer to display all those things? It's just layers of craziness (like I started to call such stuff).
In the end our hardware will render pixels! (In big contrast to the original hardware for which this stuff was invented in the first place.)
Misusing our glorified virtual line-printers should be finally replaced with something that acknowledges our current reality (which is: HW based on pixel framebuffers) instead of fighting this reality. It's a complete joke that I need a multi-GHz computer and an expensive GPU with many TFLOPs only to be able to scroll somehow "smooth" in an UI form the 60's…
> Misusing our glorified virtual line-printers should be finally replaced with something that acknowledges our current reality (which is: HW based on pixel framebuffers)
A local PC or phone may be your only experience but it is far from the current reality in computers. most computers these days are embedded devices or remote servers, neither of which have pixel framebuffers, or even "video displays". Heck, even inside your PC are computers that only have serial connections: I once had a flaky consumer-grade hard drive that required me to hook up a serial line to its control console from time to time and issue text commands to clear its error buffer (not buying a Seagate ever again).
In the end our hardware gets its input from sensors, operates actuators, and performs arithmetic transformations from its input to its output. A very few of those actuators result in coloured blobs on a large LED matrix. Don't get confused with that as a single universal use of computers.
Even the interaction with your HDD through its serial interface ended up on, no big surprise, your display.
I don't question TUIs as such!
I question how we implement them.
The whole "terminal emulator dance" is imho strictly unnecessary.
You could also talk to your HDD through its line interface by, for example, some proper RPC mechanism. (With the help of some adequate client software, which could be a CLI application, but also a GUI tool). The HDD doesn't have to provide a framebuffer based user interface, of course. Your client does.
Someone else pointed even already to something baking my assumption that getting rid of all the layers of emulation of more than ancient hardware would be not only possible but is even desirable:
Good bit of history, and definitely a lot about terminals that I didn’t know, or things that filled in the “why” of some things I did know (^Q/^S for example).
Some things that poked me while reading:
* varying use of “encrypt” and “encode” for transmitting data; suggest sticking with encode/decode
* mention of Intel x87 when I think you probably meant x86
Yeah, my bad. I'm not a native English speaker, and somehow to me the word "encoding" was only used for encoding information on a computer. I was confused when I wanted to say the same thing for telegrams; I began to use "codify", and then "encrypt", which is obviously wrong.
the subject matter is good but the text is riddled with these minor errors that make me distrust the accuracy of any parts that drift out of the knowledge I already have
Yeah, I saw it mentioned encryption and I was like... are you sure? Encryption wasn't commonplace until this past decade with HTTPS finally being widely used.
I'm wondering a lot how things would look if we would stop emulating 100 year old tech and build a CLI environment from scratch today.
Do some projects in this direction even exist? Or are we going to stick around with abstractions of things even most people's grant great parents haven't ever seen because this tech was already long obsolete at their time?
I wonder what computers would be capable off if we would develop a new stack from the ground up (bits and bytes) with the silicon in mind which we have available today…
We should get rid of all the historic accidents. Really.
Imho it's completely crazy how computers "work" today.
Modern computers still need to emulate "a PDP-7" because that's the only kind of machine C runs on fast.
But the sequential command stream machine (which is the abstract machine of C) is extremely inefficient. It's a mayor pain to extract parallelism from an inherently sequential command stream! We built crazy shit like "hardware JIT compilers", that are embedded nowadays in all CPUs, only so the CPU can maintain the PDP-7 illusion to the outside world (which is, in the end, still C, and nothing else) even it's a proper data-flow machine internally.
But because all the SW world is build in C (or languages that work the same) nobody invests in alternative architectures. Because C would not run fast on such machines, and they would fail in the market if you would need to rebuild the whole SW world first to get any advantage.
C (and, tragicomicaly its portability) was a trap!
And we will die a painful death even before anybody would start to think to finally remove the von Neumann bottleneck (which is inherent to sequential command stream machines with RAM).
I for my part would love if someone would invest in something like a modern take on the Connection Machine, but build form the ground up with a HW-SW co-design approach. (Of course this would be some work; you would need to build the hardware, which is comparably easy, but also some adequate programming language and ecosystem, up to a completely new designed operating system, and convince people to "rewrite the world" for your new system — which is the hard part).
> from the ground up (bits and bytes)
In this context it should read "(trits and 'trytes')" (only that we would maybe need a new world as trytes are historically six trits, but would be better 9 trits; so we could directly use a septemvigesimal system, which maps excellently to the English alphabet + "point"), and it should be of course balanced.
Time to finally switch to the most efficient information encoding, when we're at it! (Binary was also just an historic accident, chosen because it was most easy implemented on the ancient integrated circuits).
Enough with this 'emulating a PDP-11' cliche (never seen it quoted as the PDP-7 before). Explicit instruction-level parallelism has its own issues that make it difficult to use effectively, precisely because it must do statically what out-of-order processors do dynamically.
The use of C as a scapegoat is odd. What alternatives would have led to a different world of processors? Pascal? PL/I? Experiments in parallelism at the time like Erlang are about process-level parallelism, not the kind of instruction-level parallelism this line of argument calls for.
You absolutely can have simpler systems than we use today, but let's not fool ourselves about how we got here.
> Enough with this 'emulating a PDP-11' cliche (never seen it quoted as the PDP-7 before).
That's not a "cliche". It's the reality how computers work today.
Internally they're data-flow machines since quite some time. But they need to emulate a command stream machine (which did not change fundamentally since the time of the PDP-7!) to the outside world.
> Explicit instruction-level parallelism has its own issues that make it difficult to use effectively, precisely because it must do statically what out-of-order processors do dynamically.
That's why nobody here ever talked about "explicit instruction-level parallelism"…
> The use of C as a scapegoat is odd.
No it isn't, as the success of C is the root of all evil in this case.
> What alternatives would have led to a different world of processors? Pascal? PL/I?
No, of course not. Because Pascal or PL/I are of course also "just C" (with some minor, and regarding this consideration here, completely irrelevant differences).
> Experiments in parallelism at the time […]
Are irrelevant.
The topic is: How would things look like if we would start over with all what we know now, and with our current technical capabilities.
> the kind of instruction-level parallelism this line of argument calls for.
At least I have talked explicitly about data-flow (and nothing else)!
Imho the whole command stream based approach is a dead end.
Dynamically reconfigurable data-flow hardware (with local scratch memory instead of RAM) is imho the answer.
But of course it's almost impossible to compile sequential command streams (aka. "imperative programs", which is synonym to C and all languages that work the same, so actually almost all languages in existence) into anything that could be efficiently executed on such data-flow hardware. That's why you would need to "rewrite the world" form scratch to get any benefits from finally sane hardware (instead of the expected degradation in case you would try to map our current sequential command streams into this new world).
But that's just another "layer of craziness", imho. (Even it's quite cool).
I think the whole terminal should be replaced. Not even with something funky. Just a proper RPC + streaming protocol based on some sane modern tech. (No, no HTTP! Simple and lightweight. Tailor made for the task. So it works also in an embedded setting; and lasts for the next 50 years :-)). Of course it should also have separate control and data channels. (That's a mayor flaw of the "classical" terminal tech. The in-band transport of control is a big PITA, and can be even an security risk).
I love these types of guides, especially when they cover pieces under the hood like TTYs.
I find that most people who don't use terminals don't do it out of pure preference for GUIs over CLIs. Instead, they use GUIs because they simply don't want to spend time learning how to use CLIs.
I started out as a sysadmin on VME, moving to SunOS/Solaris in the mid 90s. I worked largely on DEC VT100 and subsequently a I had a Sun Ultra 5 workstation. I can genuinely argue that I have more real world CLI time under my belt that your average HNer. I have extremely fond memories of that time, but I don’t miss it. I’m much happier with a GUI to click around. I still drop into the terminal to complete some tasks, more out of habit than anything else but the idea of going back to using a terminal 90% of the time just doesn’t fill me with joy. There is a lot of fetishisation of tools, especially of terminals and the CLI. Admittedly things are a better now in someways, in others, not so much.
Why should I have to take a computer science course and memorize a whole language to install a program, manipulate files, change a configuration, etc… Not all of us who use computers for a living are programmers. I manage several computers that perform different tasks, but thats only part of my job. Dropping into CLI is something I only need to do a few times a year only because some of the software I use has no other UI. I’ll posit that people like myself use GUIs much easier because they are self learning. I don’t need a CS degree just to use it, all the options are laid before me to see at once. It’s the fastest way for me to do what I want with the software and move on with my life.
I'm not a programmer either. I just started running Linux out of curiosity, followed a few "copy-paste this into your terminal" guides, eventually got the hang of it and now I do almost everything in it.
It both is and isn't "a whole language". Most commands just execute a single program with options and/or arguments, except you don't have the overhead of opening a GUI, finding the right dropdown menu, etc. You quickly start to pick it up like you would a spoken language, remembering the "words" for actions, and it becomes a very natural way to think.
IMO the biggest stumbling block is just learning how to type efficiently. This includes knowing about tab completion, history search, putting stuff in your .bashrc and so on.
I use them all the time, in their original meaning. What's important about those bytes is that they are part of the POSIX non-portable ASCII character set, and thus can safely separate objects whose contents come from the portable character set.
My shell library uses them to pass portable data around in groups of records, to parse and modify portable filenames, to construct shell commands and scripts automatically and to prepare plaintext data chunks for binary encoding and processing.
I don't know of plaintext editors making use of the separators by default, because they are not considered plain-text in the POSIX sense (although this leads to some tedious semantic debates that I am profoundly disinterested in).
I’ve always wondered: does the carriage return key on an actual teletype machine do CR+LF in one button press? I know these were separate levers on old typewriters, but it seems like the “return” key has always done both.
The Teletype Model 33 ASR keyboard has two seperate keys, one for carriage return, one for line feed [1], [2].
These correspond to the it primarily being a printer with control codes, with keyboards, tape punches and tape readers being additional (in various incarnations of the larger Teletype device family).
The codings are distinct to enable overstrike and faster "next line w/out return" mechanical performance.
If the tty is a file in /dev, is there a way to monitor a user that logged in by watching their tty? I remember in freebsd it's possible but not sure if possible in Linux...
Grow up with an asr-33 connected to an EMAS system in the basement at home. It was noisy but much less noisy than the card punch and remote entry station it replaced (the joys of a professorial parent who got a 200 baud line to work installed) It had cylindrical keys -then at uni used the same with decwriters and vt52s and subsequently vt100s and adm5. In memory, the adm5 was wonderful. The one in my basement looks at me and says:
I whine (flyback transformer) and I shimmer (unstable tube needs a degauss) and I am slow (-really not reliable at 115200 if even capable, which I doubt. I last ran it 9600 attached to a sun workstation serial console) and best of all my keyboard is basically the same as that asr33 and really, really tiresome to use.
They're mostly better in memory than in real life. The vt100 and subsequent weren't bad, good enough for people
like systime in leeds to clone them (they got done for cloning Vax boards, and selling them behind the iron curtain. The factory is a secret spy headquarters in the original "edge of darkness" TV series) and sell cheaper. The vt240 was pretty good. The blit and subsequent were lovely, but had a brief life as real graphics workstations came up. Really, the depraz mouse made the blit/gnot/jerq worthwhile. The first time I met a terminal device which was not 80x24 was a blit class terminal in ucl-cs.
On an ICL (ok. 3 rivers. ICL made it under licence in musselburgh south of edinburgh) Perq I used before then you used a terminal(ish) entry which may also have been not 80x24. The vertically mounted hard disk had a translucent cover and you could see static type sparks coming off the drive hub. really bizarre. It used display memory to compile so your text input window corrupted as you compiled. stty reset/clear fixed it.
Having multiple vty on the various BSD on Intel was nice. Vty4 was like steering, behind the scenes or something during the installer, and they launched a debug shell on one.
You learned to use stty a lot on old lines. Some were half duplex and some worked like bad antenna at high baudrate and picked up phantom presses in the 50 pair bundle.
ASR-33 was upper-case only hence Getty on BSD detecting upper-case only at login and v7 working UC and vi having an UC mode. It got removed from FreeBSD in the 90s.
My memory is the xterm implements the ANSI terminal escape codes which are pretty much vt100 because DEC either informed the standards committee decision with a pre implementation or implemented the standard, and MIT had lots of vtxxx series and so used the same to write the escape codes on X so Emacs and vi came along for the ride easily in termcap.
I didn't try to be smart, I just got super confused what verb to use for telegrams (weirdly enough, to me "encoding" was only used in the context of computer... which is wrong; I tried to use "codify" also, but I'm pretty sure it's also wrong... from there the confusion spread in the whole article).
I'm not a native speaker. It doesn't excuse my mistake though.
Article starts of with the incorrect term "virtual shell". If it's supposed to be the character in the story using an incorrect term, I don't think it fits the rest of the article.
I feel like this article has been written better before, e.g. in the article's own sources.
So the article begins with a story where a character, who doesn't know what he's talking about, try to mentor one of his colleague. It might be awkward though, and difficult to understand; sorry for that.
For sure this article was written better by others, but to me they were also lacking some stuff. I never found an article summarizing the whole story; most of the other sources were brushing away too much information for me to understand really why the TTY device had this weird structure.
But I guess not everybody will agree... and that's fine :)
It was a very entertaining read, but you may have wanted to include Windows Terminal and xterm. WT is seriously good, and xterm is unrivaled when it comes to compatibility (but WT is progressing fast!)
For your next issue, I would suggest talking about pictures in terminals: from the historical sixel to the custom formats like kitty, and maybe connecting them with sixel-tmux
Actually, terminal session managers like screen and tmux would be worth another
Just started reading the article, it looks nice! In this sentence:
It’s a 7 bits codes, allowing to send bigger messages than with the Baudot (or the Murray) code, even including the luxe of lowercase and uppercase letters.
Yeah, I'm a bit confused by the concept of "controlling terminal". As I understand it, in the physical video terminal days, the console connected to the computer was the "controlling terminal", but I'm not sure how this concept translate in the terminal emulator world.
So, do you know what's the difference between the "controlling termina" and the "terminal connected to stdin"? I though the terminal connected to stdin was the controlling terminal...
As in, the screen is the part that makes it a video terminal, as opposed to the ancient printer terminals.
(Also, what do you mean "casually states"? There's a long section detailing the history of the terminal right in the article, am I misunderstanding your question?)
I am not sure I do now (specifically how SSH ties in to this).
But at least I now know more about terminals, shells, rawmode and ptms/ptx and more.
This link right here fits right in with the following ones:
[0] https://biriukov.dev/docs/fd-pipe-session-terminal/4-termina...
[1] https://www.sobyte.net/post/2022-05/tty/
[2] http://www.linusakesson.net/programming/tty/index.php