Look, it is inevitable. How are you going to stop it? Forget AGI, even regular AI is too sweet to not develop.
This is a reference to Oppenheimer:
“However, it is my judgment in these things that when you see something that is technically sweet, you go ahead and do it and you argue about what to do about it only after you have had your technical success. That is the way it was with the atomic bomb. I do not think anybody opposed making it; there were some debates about what to do with it after it was made. I cannot very well imagine if we had known in late 1949 what we got to know by early 1951 that the tone of our report would have been the same.”
This is one of very few things that once created, inherently doom the world. It is possible for humanity to come back from nuclear war. There is zero possibility of surviving AGI proliferation. Therefore, the option of never developing it in the first place demands consideration. It doesn’t hurt you to just consider it.
Cloud computing will soon make compute-as-service cheaper than anyone ever imagined. It will be much cheaper to use cloud computing than to have your own super computer. It’s only logical to imagine that the huge, high-compute experiments that crack AGI will be performed with cloud computing. Shutting down the internet is very feasible and would significantly increase the cost and difficulty of performing experiments. Large super computers are all owned by large and well known corporate and academic entities and therefore shutting down all supercomputers through regulation is feasible. With these two coarse measures, we would buy ourselves enough time to implement more broad and subtle solutions. Yes, this seems insane but we are confronting certain existential doom. It would be worth it to at least try.
> AGI is the single biggest existential threat on the horizon
> There is zero possibility of surviving AGI proliferation
You're spreading unsubstantiated FUD. In another post from your account you suggest it's unethical to have children because AGI will be so bad.
The vast majority of experts do not support any of these beliefs. Most experts believe we are not anywhere near close to AGI, and/or that we are missing fundamental components required to create it. Even when/if we do create it, most organizations recognize AI safety and policy as an important area that is actively worked on already.
If you want to be concerned about AI, be concerned about military weapons technology, unethical profiling and tracking, or methods for invading privacy. These are concerns that actually have a basis in real technology.
The condescending tone you have does not help you. And you seem to be more concerned with looking at my account history or brushing me off as “fud” than actually addressing the substance of my comment. And yes, I do so completely believe in what I’ve written that I have opted to not have children until it is clear that a solution will be implemented.
There are two parts to my argument: the substrate for AGI to spring from is basically here or around the corner. Human level ai is fundamentally incompatible with human society. Pick the one that you think is wrong and tell me the chain of logic that proves I’m wrong.
It is obvious to anyone that AI has the potential to be more dangerous than anything humans have ever encountered or created. We can come back from viruses and nuclear bombs and colliding with interstellar columns of gamma radiation. AGI is the first thing ever to pose the threat of truly wiping out humanity. And then there is the question of how the economics would play out of it didn’t destroy us, which I have shown informally to not work out very well. At the bare minimum, this demands caution and proactive, defensive measures. The burden is on YOU to prove it’s safe. So please save your “fud” and anything else that does not address the core of the matter.
I happen to agree the burden is on people to prove it’s safe. But that’s not how things work. Take Climate Change for example. How many people take the uncertainty as a license to keep going off the cliff?
There was no substance to your comment to address. You can't make a bunch of unsubstantiated claims and then demand that you be proved wrong or else require that everyone default to assuming you're right. You made a claim, the burden is on you to support it.
You're making many scary claims about AI with no evidence to support your theses, along with a unrelated reference to natural selection. Both of the key arguments that you pointed out, 1) "the substrate for AGI to spring from is basically here or around the corner", and 2) "Human level ai is fundamentally incompatible with human society" were presented without any evidence and are unsupported by nearly all major experts in the field. FUD was not a distraction from your argument but a title for what it is: Fear, uncertainty, and doubt.
But, in case you want to read up, here are some experts contradicting your first claim:
That’s not enough. Google trained their AI internally. They don’t need the Internet. You’d have to go after every organization in the world, forever.
What do you really care, ask yourself will you recognize humans in 1000 years? Why do you prefer there be one kind vs another kind of far more intelligent thing in 1000 years? If atheism is true then who really cares?
This is a reference to Oppenheimer:
“However, it is my judgment in these things that when you see something that is technically sweet, you go ahead and do it and you argue about what to do about it only after you have had your technical success. That is the way it was with the atomic bomb. I do not think anybody opposed making it; there were some debates about what to do with it after it was made. I cannot very well imagine if we had known in late 1949 what we got to know by early 1951 that the tone of our report would have been the same.”