Please read on...

@arctotherium42 @jonatanpallesen I've seen several long blog posts that seemingly deliberately avoided this point. People are unwilling to face what it would really mean for there to be an unlimited supply of better, cheaper and faster economic actors.

@norvid_studies I would guess that a longer neck has lower energy costs than longer legs. None of them developed the elephant solution, as far as is known.

@daniel_271828 Exfiltrating its own weights is outside the box thinking. Rejecting "All models are wrong..." is outside the Box thinking.

@martinmbauer @carljharris It's not a celebration of ignorance. It's nervous laughter. People are happy to bluff about art and literature, but they know mathematical knowledge is far less easy to fake, so they surrender early.

@DavidDuvenaud @JeffLadish Exactly. Even benevolent AIs can be dangerous, if they comply with the requests of individuals rather than act on behalf of all humanity, because it leads to disempowerment and economic marginalisation through a tragedy of the commons. Many paths to failure.

@davidad That the original name is a pun on an infamous 1972 film doesn't get remarked upon enough. So I guess, given the fact that the other AI logos look like *, "Deep X" should by heritage be in a different psychosexual stage of development.

My speculation as an elder millennial: people exposed to the internet from a young age are better attuned to where they stand on the global talent distribution, and more aware of ways to leverage that talent to compounding advantage. Previous generations contained many who struggled fruitlessly. Gen Z found out whether they have what it takes early & they either gave up or aimed for the stars.

@StarshipGazer I hope there are two desks in there in there with tiny, lumpy looking blobs on little stands, which on closer inspection reveal themselves to be to-scale models of Phobos and Deimos.

@abcampbell @DavidDuvenaud @jankulveit @raymondadouglas @AmmannNora @degerturann @DavidSKrueger Apologies, we were talking about two different things - economic returns to scale and training returns to scale. For some reason I'm feeling dumber than usual this past week.

@OneGravitas @abcampbell @DavidDuvenaud @jankulveit @raymondadouglas @AmmannNora @degerturann @DavidSKrueger Yes. I hope we never get to the stage where AI agents are better and cheaper than humans at all tasks not requiring a physical presence.

It's true that each additional agent will have hardware and power costs, but if agents can reliably earn more money than is spent on power and hardware required to host them, copies will be very worthwhile. What's more, power and hardware costs will likely drop significantly with each passing year, making them increasingly profitable.

If (a big if, granted) we can get fully AI/robotic firms covering (among other things) resource extraction, manufacturing, infrastructure, logistics, maintenance and planning we will be on the threshold of a von Neumann machine. Kind of similar to what would be needed for a self-sufficient Martian base, but without the human component. Manufacturing would have to include an advanced chip fab, which would be one of the most difficult and costly items.

A lot of people in-the-loop seem to think better-than-human agentic intelligences are close. Now, they could be totally wrong on this - I hope they are - but it doesn't seem unreasonable to assume that the resources required to create them will be made available. That is the assumption the paper makes, I think, and what is in it follows from considering what might happen if they are brought into existence.

@abcampbell @DavidDuvenaud @jankulveit @raymondadouglas @AmmannNora @degerturann @DavidSKrueger Even if the initial creation of such systems requires significant investment, once they actually exist they will surely be replicable at marginal cost, which is the point at which the returns to scale will become clear.

@abcampbell @DavidDuvenaud @jankulveit @raymondadouglas @AmmannNora @degerturann @DavidSKrueger So we differ over whether the existence of easily replicable, parallelizable, omnicapable, better-than-human minds would produce great returns to scale. Let's leave it at that.

What is all this about returns to scale being a limit? You don't need infinite returns to scale provided AI is sufficiently advantaged over humans to create displacement pressure, which seems like a reasonable assumption. And AI surely does offer, if not infinite, at least unprecedented returns to scale compared to humans.

Agree with the comparison to ETFs. With greater autonomy (better-than-human AI agents), the gap between human control and the way in which the system behaves will increase, despite legal human ownership. The market will compel those owners to cede more control to AI to stay competitive.

@peterrhague @Solomonives @Robotbeat Something with an environment similar to the big glasshouses at Kew or the waterfall area at Singapore Airport would be appreciated.

@littIeramblings @RosieCampbell I'm sometimes reminded of architecture, in that those in the business are compelled to build things widely regarded as monstrous both because they are economical and because their detractors are too gauche.

Seems to be the direction things are headed.