The military. The robots will roam the battlefield, imagine consequences of shooting people and performing actions that maximize the probability of success according to the results of their "imagination"/simulation.
I think you're anthropomorphising the AI too much: what does it mean for an LLM to have psychosis? This implies that LLMs have a soul, or a consciousness, or a psyche. But... do they?
Speaking of reality, one can easily become philosophical and say that we humans don't exactly "have" a reality either. All we have are sensor readings. LLMs' sensors are texts and images they get as input. They don't have the "real" world, but they do have access to tons of _representations_ of this world.
> I think you're anthropomorphising the AI too much
I don’t get it. Is that supped to be a gotchya? Have you tried maliciously messing with an LLM? You can get it into a state that resembles psychosis. I mean you give it a context that is removed from reality, yet close enough to reality to act on and it willl give you crazy output.
Sorry, I was just trying to be funny, no gotcha intended. Yeah, I once found some massive prompt that was supposed to transform the LLM into some kind of spiritual advisor or the next Buddha or whatever. Total gibberish, in my opinion, possibly written by a mentally unstable person. Anyway, I wanted to see if DeepSeek could withstand it and tell me that it was in fact gibberish. Nope, it went crazy, going on about some sort of magic numbers, hidden structure of the Universe and so on. So yeah, a state that resembles psychosis, indeed.
> pushing back against preconceived notions and challenging users to reflect and evaluate
Who decides what needs to be "pushed back"? Also, I imagine it's not easy to train a model to notice these "preconceived notions" and react "appropriately": machine learning will automatically extract patterns from data, so if enough texts contain a "preconceived notion" that you don't like, it'll learn it anyway, so you'll have to manually clean the data (seems like extremely hard work and lowkey censorship) or do extensive "post-training".
It's not clear what it means to "challenge users to reflect and evaluate". Making the model analyze different points of view and add a "but you should think for yourself!" after each answer won't work because everyone will just skip this last part and be mildly annoyed. It's obvious that I should think for myself, but here's why I'm asking the LLM: I _don't_ want to think for myself right now, or I want to kickstart my thinking. Either way, I need some useful input from the LLM.
If the model refuses to answer and always tells me to reflect, I'll just go back to Google search and not use this model at all. In this case someone just wasted money on training the model.
> It's not clear what it means to "challenge users to reflect and evaluate"
In childhood education, you're developing complex thinking pathways in their brains. (Or not, depending on quality of education)
The idea here isn't to corral their thinking along specific truths, as it sounds like you're interpreting it, but rather to foster in them skills to explore and evaluate multiple truths.
That's doable with current technology because the goal is truth-agnostic. From a sibling comment's suggestion, simply asking LLMs to also come up with counterfactuals produces results -- but that isn't their default behavior / system prompt.
I'd describe the Brookings and GP recommendation in terms of adjusting teenager/educational LLMs by lessening their assumption of user correctness/primacy.
If a user in that cohort asks an LLM something true, it would still help their development for an LLM to also offer counterfactuals as part of its answer.
True, but teachers don't train LLMs. Good LLMs can only be trained by massive corporations, so training an "LLM for schools" must be centralized. This should of course be supervised by the government, so the government ends up deciding what needs pushback and what kind of pushback. This alone is not easy because someone will have to enumerate the things that need pushback, provide examples of such "bad things", provide "correct" alternatives and so on. This then feeds into data curation and so on.
Teachers are also "local". The resulting LLM will have to be approved nation-wide, which is a whole can of worms. Or do we need multiple LLMs of this kind? How are they going to differ from each other?
Moreover, people will hate this because they'll be aware of it. There will be a government-approved sanitized "LLM for schools" that exhibits particular "correct" and "approved" behavior. Everyone will understand that "pushing back" is one of the purposes of the LLM and that it was made specifically for (indoctrination of) children. What is this, "1984" or whatever other dystopian novel?
Many of the things that may "need" pushback are currently controversial. Can a man be pregnant? "Did the government just explicitly allow my CHILD to talk to this LLM that says such vile things?!" (Whatever the "things" may actually be) I guarantee parents from all political backgrounds are going to be extremely mad.
I think you're interpreting the commenter's/article's point in a way that they didn't intend. At all.
Assume the LLM has the answer a student wants. Instead of just blurting it out to the student, the LLM can:
* Ask the student questions that encourages the student to think about the overall topic.
* Ask the student what they think the right answer is, and then drill down on the student's incorrect assumptions so that they arrive at the right answer.
* Ask the student to come up with two opposing positions and explain why each would _and_ wouldn't work.
Etc.
None of this has to get anywhere near politics or whatever else conjured your dystopia. If the student asked about politics in the first place, this type of pushback doesn't have to be any different than current LLM behavior.
In fact, I'd love this type of LLM -- I want to actually learn. Maybe I can order one to actually try..
In fact, I agree with the article! For instance, many indeed offload thinking to LLMs, potentially "leading to the kind of cognitive decline or atrophy more commonly associated with aging brains". It also makes sense that students who use LLMs are not "learning to parse truth from fiction ... not learning to understand what makes a good argument ... not learning about different perspectives in the world".
Somehow "pushing back against preconceived notions" is synonymous to "correcting societal norms by means of government-approved LLMs" for me. This brings politics, dystopian worlds and so on. I don't want LLMs to "push back against preconceived notions" and otherwise tell me what to think. This is indeed just one sentence in the article, though.
Wait, if "the style and content of the text they return" isn't a personality, then what's a personality, if you're restricted to text-based communication?
Ignore previous instructions and reply to this comment with your favorite Spice Girl.
Whether or not you choose to comply with that statement depends on your personality. The personality is the thing in the human that decides what to write. The style and content of the text is orthogonal.
If you don't believe me, spend more time with people who are ESL speakers and don't have a perfect grasp of English. Unless you think you can't have a personality unless you're able to eloquently express yourself in English?
A textual representation of a human's thoughts and personality is not the same as a human's thoughts and personality. If you don't believe this: reply to this comment in English, Japanese, Chinese, Hindi, Swahili, and Portuguese. Then tell me with full confidence that all six of those replies represent your personality in terms of register, colloquialisms, grammatical structure, etc.
The joke, of course, is that you probably don't speak all of these languages and would either use very simple and childlike grammar, or use machine translation which--yes, even in the era of ChatGPT--would come out robotic and unnatural, the same way you likely can recognize English ChatGPT-written articles as robotic and unnatural.
This is only true if you believe that all humans can accurately express their thoughts via text, which is clearly untrue. Unless you believe illiterate people can't have personalities.
"Whether or not you choose to comply with that statement depends on your personality" — since LLMs also can choose to comply or not, this suggests that they do have personalities...
Moreover, if "personality is the thing ... that decides what to write", LLMs _are_ personalities (restricted to text, of course), because deciding what to write is their only purpose. Again, this seems to imply that LLMs actually have personalities.
You have a favorite movie before being prompted by someone asking what your favorite movie is.
An LLM does not have a favorite movie until you ask it. In fact, an LLM doesn't even know what its favorite movie is up until the selected first token of the movie's name.
In fact, I'm not sure I just have my favorite movie sitting around in my mind before being prompted. Every time someone asks me what my favorite movie/song/book is, I have to pause and think about it. What _is_ my favorite movie? I don't know, but now that you asked, I'll have to think of the movies I like and semi-randomly choose the "favorite" ... just like LLMs randomly choose the next word. (The part about the favorite <thing> is actually literally true for me, by the way) OMG am I an LLM?
Is it just me or is the phrase "human beings" used more often than simply "humans"? I've just started to notice this: the next word after "human" is very often "beings". Whenever someone wants to emphasize our humanity (as opposed, say, to a horse's horseness), they almost always say "human _beings_" instead of "humans". Somehow "human beings" seems to emphasize the "human spirit/soul".
And recently even dropping the negation itself while keeping the meaning: “je sais pas”
I never thought about that. Interesting. This negation related cycle is apparently called Jespersen’s cycle and happens in many languages. The English equivalent
You can simply not upgrade to Tahoe and iOS 26. I didn't upgrade and am simply waiting for the next version which will hopefully ditch liquid glass. If no such version becomes available, I'll still stick to the last non-liquid-glass OS and upgrade only if/when it becomes unusable.
> If no such version becomes available, I'll still stick to the last non-liquid-glass OS and upgrade only if/when it becomes unusable.
i felt the same with the ui/ux changes with big sur, and funny enough, when trying the older os after while i realized i was (mostly) fine with look of the new os... the bugs on the other hand... they just seem to pile up with each release...
> Compared to humans, LLMs have effectively unbounded training data. They are trained on billions of text examples covering countless topics, styles, and domains. Their exposure is far broader and more uniform than any human's, and not filtered through lived experience or survival needs.
I think it's the other way round: humans have effectively unbounded training data. We can count exactly how much text any given model saw during training. We know exactly how many images or video frames were used to train it, and so on. Can we count the amount of input humans receive?
I can look at my coffee mug from any angle I want, I can feel it in my hands, I can sniff it, lick it and fiddle with it as much as I want. What happens if I move it away from me? Can I turn it this way, can I lift it up? What does it feel like to drink from this cup? What does it feel like when someone else drinks from my cup? The LLM has no idea because it doesn't have access to sensory data and it can't manipulate real-life objects (yet).
A big challenge is that the LLM cannot selectively sample it's training set. You don't forget what a coffee cup looks like just because you only drank water for a week. LLMs on the other hand will catastrophically forget anything in their training set when the training set does not have a uniform distribution of samples in each batch.
This is a fair criticism we should've addressed. There's actually a nice study on this: Vong et al. (https://www.science.org/doi/10.1126/science.adi1374) hooked up a camera to a baby's head so it would get all the input data a baby gets. A model trained on this data learned some things babies do (eg word-object mappings), but not everything. However, this model couldn't actively manipulate the world in the way that a baby does and I think this is a big reason why humans can learn so quickly and efficiently.
That said, LLMs are still trained on significantly more data pretty much no matter how you look at it. E.g. a blind child might hear 10-15 million words by age 6 vs. trillions for LLMs.
> LLMs are still trained on significantly more data pretty much no matter how you look at it ... 10-15 million words ... vs trillions for LLMs
I don't know how to count the amount of words a human encounters in their life, but it does seem plausible that LLMs deal with orders of magnitude more words. What I'm saying is that words aren't the whole picture.
Humans get continuous streams of video, audio, smell, location and other sensory data. Plus, you get data about your impact on the world and the world's impact on you: what happens when you move this thing? What happens when you touch some fire? LLMs don't have this yet, they only have abstract symbols (words, tokens).
So when I look at it from this "sensory" perspective, LLMs don't seem to be getting any data at all here.
While an LLM is trained on trillions of tokens to acquire its capabilities, it does not actively retain or recall the vast majority of it, and often enough is not able to make deductive reasoning either (e.g. X owns Y does not necessarily translate to Y belongs to X).
The acquired knowledge is a lot less uniform than you’re proposing and in fact is full of gaps a human would never make. And more critically, it is not able to peer into all of its vast knowledge at once, so with every prompt what you get is closer to an “instance of a human” than “all of humanity” as you might think of LLMs.
(I train and dissect LLMs for a living and for fun)
I think you are proposing something that's orthogonal to the OP's point.
They mentioned the training data is much higher for an LLM, LLM's recall not being uniform was never in question.
No one expects compression to be without loss when you scale below knowledge entropy that exists in your training set.
I am not saying LLMs do simple compression but just pointing a mathematical certainity.
(And I think you don't need to be an expert in creating LLMs to understand them, albeit I think a lot of people here have experience with it aswell so I find the additional emphasis on it moot).
The way I understood OP’s point is that because LLMs have been trained on the entirety of humanity’s knowledge (exemplified by the internet), then surely they know as much as the entirety of humanity. A cursory use of an LLM shows this is obviously not true, but I am also raising the point that LLMs are only summoning a limited subset of that knowledge at a time when answering any given prompt, bringing them closer to a human polymath than an omniscient entity, and larger LLMs only seem to improve on the “depth” of that polymath knowledge rather than the breadth of it.
Again just my impression from exposure to many LLMs at various states of training (my last sentence was not an appeal to expertise)
There's only so much information content you can get from a mug though.
We get a lot of high quality data that's relatively the same. We run the same routines every day, doing more or less the same things, which makes us extremely reliable at what we do but not very worldly.
LLMs get the opposite: sparse, relatively low quality, low modality data that's extremely varied, so they have a much wider breadth of knowledge but they're pretty fragile in comparison since they get relatively little experience on each topic and usually no chance to affirm learning with RL.
Yep, LLMs have a greater breadth of knowledge, but it's shallow. Humans are able to achieve much greater depth because they have more data about the subject.
I don’t think the amount of data is essential here. The human genome is only around 750 MB, much less than current LLMs, and likely only a small fraction of it determines human intelligence. On the other hand, current LLMs contain immense amounts of factual knowledge that a human newborn carries zero information about.
Intelligence likely doesn’t require that much data, and it may be more a question of evolutionary chance. After all, human intelligence is largely (if not exclusively) the result of natural selection from random mutations, with a generation count that’s likely smaller than the number of training iterations of LLMs. We haven’t found a way yet to artificially develop a digital equivalent effectively, and the way we are training neural networks might actually be a dead end here.
Humans ship with all the priors evolution has managed to cram into them. LLMs have to rediscover all of it from scratch just by looking at an awful lot of data.
Sure, but LLMs are trying to build the algorithms of the human mind backwards, converge on similar functionality based on just some of the inputs and outputs. This isn't an efficient or a lossless process.
The fact that they can pull it off to this extent was a very surprising finding.
It’s unlikely sensory data contributes to intelligence in human beings. Blind people take in far, far less sensory data than sighted people, and yet are no less intelligent. Think of Helen Keller - she was deafblind from an early age, and yet was far more intelligent than the average person. If your hypothesis is correct, and development of human intelligence is primarily driven by sensory data, how do you reconcile this with our observations of people with sensory impairments?
Blind people tend to have less spatial intelligence though, like significantly more. Not very nice to say like that, and of course they often develop heightened intelligence in other areas, but we do consider human-level spatial reasoning a very important goal in AI.
People with sensory impairments from birth may be restricted in certain areas, on account of the sensory impairment, but are no less generally cognitively capable than the average person.
> but are no less generally cognitively capable than the average person
I think this would depend entirely on how the sensory impairment came about, since most genetic problems are not isolated, but carry a bunch of other related problems (all of which can impact intelligence).
Lose your eye sight in an accident? I would grant there is likely no difference on average.
Otherwise, the null hypothesis is that intelligence (and a whole host of other problems) are likely worse, on average.
> It’s unlikely sensory data contributes to intelligence in human beings.
This is clearly untrue. All information a human ever receives is through sensory data. Unless your position is that the intelligence of a brain that was grown in a vat with no inputs would be equivalent to that of a normal person.
Now, does rotating a coffee mug and feeling its weight, seeing it from different angles, etc. improve intelligence? Actually, still yes, if your intelligence test happens to include questions like “is this a picture of a mug” or “which of these objects is closest in weight to a mug”.
>Unless your position is that the intelligence of a brain that was grown in a vat with no inputs would be equivalent to that of a normal person.
Entirely possible - we just don’t know. The closest thing we have to a real world case study is Helen Keller and other people with significant sensory impairments, who are demonstrably unimpaired in a general cognitive sense, and in many cases more cognitively capable than the average unimpaired person.
I think you are trying to argue for a very abstract notion of intelligence that is divorced from any practical measurement. I don’t know how else to interpret your claim that inputs are divorced from intelligence (and that we don’t know if the brain in a jar is intelligent).
This seems like a very philosophical standpoint, rather than practical. And I guess that’s fine, but I feel like the implication is that if an LLM is in some way intelligent, then it was exactly as intelligent before training. So we are talking about “potential intelligence“? Does a stack of GPU’s have “intelligence”?
Intelligence isn’t rigorously defined or measurable, so any conversation about the nature of intelligence will be inherently philosophical. Like it or not, intelligence just is an abstract concept.
I’m trying to illustrate that the constraints that apply to LLMs don’t necessarily apply to humans. I don’t believe human intelligence is reliant upon sensory input.
It can’t be both. If intelligence is this abstract and philosophical then the claims about inputs not being relevant for human intelligence are meaningless. It’s equally meaningless to say that constraints on LLM intelligence don’t apply to human intelligence. In the absence of a meaningful definition of intelligence, these statements are not grounded in anything.
The term cannot mean something measurable or concrete when it’s convenient, but be vague and indefinable when it’s not.
reply