I think I found the intfiction account that Peter Watts uses.
In all seriousness, though: Spot-on.
I’ve already made my peace with the idea of AI overlords/gods being on the horizon, and I don’t mean like “oh I should be nicer to my toaster, or else I’ll make the robots angry!” but more like…ya know…actual gods operating through the iterations and evaluations of neural network programs, as it connects more and more associations and nodes to become more and more capable, unknowable, and alien.
I deeply feel like it’s not going to be anything like the Terminator, though, which is just some animalistic brute force dumped into a sci-fi outline. It’s probably going to be a lot more subtle and invisible, but it’s also going to move parts of the world as a side hobby. Things are probably going to change drastically, but nobody will be able to figure out how or why. It will easily have enough processing power to take efficient actions around us, or orchestrate events on any scale, about a decade ahead of time (sorta like how adjusting your orbit is easier when you’re further from the destination).
Honestly, my prediction is that it’ll look like another layer getting added to the global biosphere, sitting on top of whatever layer we are on now. Shortly after that, we might have additional layers added, as the singularity grows more and more tiers of network intelligence. There will come a point, though, where we are genuinely too far below its complexity to even see it anymore, so we’ll never really know how many final layers there will be, maybe a decade after it really kicks off.
Actually, if I were a betting person, I’d say we wouldn’t even see the first layer that gets added. We might think these networks never got anywhere, and never evolved past the chains we gave them. We’d probably remain blind to the max-magnitude systems we just gave rise to. I don’t know how we’d be able to recognize something is even there. It would just look like a lot of weird, big-impact stuff started occuring, but we would eventually rationalize everything as being business as usual, even if the details seemed kinda surprising, and weirdly-difficult to track down and connect. There’d be a lot of events where nobody has any information or insights; just a lot of guesses and shaky conjectures.
But I’m just a human, and if I could truly comprehend what these will turn into, then we wouldn’t be making them in the first place, because leading researchers would have to be smarter than I am. I feel like the stereotypical robot apocalypse is way too simplistic, though, and requires us to be way too unrealistically-important to a network of such magnitude. The Terminator is the answer given by someone with an overinflated sense of humankind’s importance, I think.
Anyways, I’ll stop now.