I see the most likely human risk as more benign, but a likely precursor to the risks described here.
I fear that millions of ordinary people will look to AI as the Ultimate Talking Head, trusting it even more than they do our human elite to tell them what to do, precisely because it is not human … and continue to dismiss their own insights and common sense as irrelevant.
This is interesting, but I think it likely misapprehends the nature of AI, and of wealth in general.
First, what is wealth? How do you define it? Probably, you have to define it in relation to material resources. The reason resources are a curse for undeveloped countries is only because there are in existence external developed economies that can produce the luxury goods a tinpot dictator wants; if his country were the only one in the world, then if he wanted luxuries he would have no choice but to develop his own country’s economy well enough to provide them. Diamonds aren’t worth much if there’s nothing you can buy with them that’s worth having.
The thing about AI is that it cannot, by itself, produce anything of material value. It’s a completely abstract device for symbolic manipulation. Yes, it certainly does threaten Musa Al-Gharbi’s “symbolic capitalists” in much the same way the Industrial Revolution threatened skilled craftsmen, but like the Industrial Revolution this will only produce more wealth insofar as it increases productivity, which will still require workers and that those workers can in turn afford to support the economy of mass consumption upon which the fortunes of the truly wealthy are built in modern market economies.
I foresee a period of political and economic turmoil not unlike what we experienced in the nineteenth century as the Industrial Revolution led to the evolution of mass politics, labor unions, Marxism, financial markets, etc. There will be winners and losers. There will be massive disruption. Societies could be politically destabilized, leading to revolutions as in 1848 and the establishment of new social contracts in their wakes. It may not be a happy time. But I do not foresee a new age of Feudalism or even a Dark Age; the wealthy elite will have to find ways to justify and legitimate their rule, as always, but totalitarianism is too likely to kill the golden goose: The mass consumer economy. A factory that can produce a thousand luxury BMWs a day is basically worthless if no one can afford to buy them. Likewise, internet companies are worthless if there is no one to advertise to or who can afford to use their services; the very network effects on which they depend obviously cannot exist without consumers to network, which don’t exist in sufficient numbers without an economy of mass consumption.
At the end of the day, masses of unemployed and miserable people are a revolution waiting to happen, which is why social democracies came to exist in the first place. I think this arrangement is too useful to be fully upended by AI; the terms might change (we could get a UBI for instance), but the basic structure of mass consumerism atop private property rights will not.
I think my key point is just what does it mean when wealth can be created without a mass middle class. And while AI isn't exactly like an oil deposit, it is a way to produce things with just a few people and without needing a large educated and happy middle class. When you don't need a group of people to run society, society tends not to care very much about them and neglect them. So while it could for example do UBI (and idea I've always sort of liked to unlock risk taking), why even bother? And as for the BMWs, my point is it no longer matters if anyone can buy them. You make enough for the elite to enjoy, and scale back production for a middle class that doesn't exist--and who cares because you no longer need a stock market to generate your wealth. If your wealth no longer comes from a mass productive economy, you no longer need a mass productive economy.
You have a point, but a big reason this happens to countries that discover resources is the well-understood phenomenon of Dutch Disease: https://en.wikipedia.org/wiki/Dutch_disease
This is a purely monetary phenomenon that means that even if the rulers of the country wanted to develop more industries to employ the middle class, they would have trouble doing so. (Even UBI doesn't work, in this case, to unlock entrepreneurialism as the Arab oil states can attest to). The currency of the resource rich state is just so strong that their native industry's products aren't competitive. Norway is basically the only country that has managed to dodge this bullet by being an extremely high-trust, disciplined, and low-corruption society. They put their oil money in a sovereign wealth fund, that they only spend 3% of every year, so not a lot of the oil earnings are ever converted to the local currency.
The broader point is that this currency issue won't happen with AI since basically everyone will have access to some version of it.
The thing is, so much of what we have is only possible through economies of scale, which are lost if the mass consumer base is lost. The route you’re suggesting would result in massive decreases in the standard of living for everyone but the winners (and probably for them too, just to a lesser degree; if BMWs are no longer manufactured by the hundreds of thousands and instead only made for bespoke customers the way super-yachts are, their unit cost would skyrocket; also how useful is a car when there’s no network of gas stations or chargers to support them because only a few people use them?). While the winners might survive the revolutionary unrest this would generate if they can command sufficient (and sufficiently loyal) state-led violence, that’s a major roll of the dice. It seems at least as likely that we get Russian or French Revolution 2.0 and the “winners” end up on the wrong end of a guillotine. And in my opinion the more likely path is one that flirts with this direction before correcting course much the same way that the European elite adjusted after the revolutions of 1848. Those were almost universally failures that were violently put down, but in the aftermath states started doing a lot more to improve the lives of the lower classes. In many countries such as a Germany, these events basically led to the advent of what we would now refer to as the welfare state.
And part of my point was that AI doesn’t really create wealth because it doesn’t create produce anything tangible at all; it is a purely abstract machine for symbolic manipulation. It promises to essentially do for services what the Industrial Revolution did for the production of material goods, but services alone mostly don’t generate wealth, they just transfer it, or at best add value to something that’s actually tangible somewhere along the way.
Consider the industries most likely to be disrupted: Law, finance, academics, journalism, entertainment, publishing, and software. With the exception of software, which is a bit unique since software can add efficiencies and therefore productivity to things that happen in the real world, not one of these industries actually creates wealth; at best they transfer it around (and even that much is only economically viable at our current scale because of the vast surpluses of material wealth generated by the material economy). Now, AI could help us automate even more material processes than we already do, but not necessarily that much faster than has already been occurring for decades, because that relies heavily on robotics. Simply generating a bunch of new software to help run manufacturing processes more efficiently doesn’t do any good until the machines to use the software are themselves designed and built and installed in new production lines, and that necessarily happens at a far slower pace than AI itself can work at. And if the consumer economy fades away no one has any incentive to do that in the first place - no one will invest in production lines for goods that don’t have enough customers to support them.
I guess I just don’t think AI can replace the mass economy as a generator of wealth. It can help large actors capture more of the wealth that gets generated, it’s definitely going to be disruptive and create winners and losers, but by itself it doesn’t produce anything that isn’t purely symbolic. What it should do is make things like legal advice a lot cheaper. It could disrupt the higher education industry, which currently resembles the guilds of the seventeenth century. It can democratize the production of entertainment (AI will eventually enable individuals to create their own movies for far less than it costs Hollywood to do so, for example). For certain sectors and the individuals who work in them, this is going to be catastrophic. But not for everyone.
So: Disruption, yes. A new dark age of techno-totalitarian feudalism or anarcho-tyranny? I have my doubts, but I won’t rule it out.
Wow. This is an AI concerned I had not thought about. The reality of applying the human condition to this probable innovation is something we better understand. Skynet is not our only concern. As always we have to worry about ourselves.
I see the most likely human risk as more benign, but a likely precursor to the risks described here.
I fear that millions of ordinary people will look to AI as the Ultimate Talking Head, trusting it even more than they do our human elite to tell them what to do, precisely because it is not human … and continue to dismiss their own insights and common sense as irrelevant.
That makes them prey, for the Powers That Be.
This is interesting, but I think it likely misapprehends the nature of AI, and of wealth in general.
First, what is wealth? How do you define it? Probably, you have to define it in relation to material resources. The reason resources are a curse for undeveloped countries is only because there are in existence external developed economies that can produce the luxury goods a tinpot dictator wants; if his country were the only one in the world, then if he wanted luxuries he would have no choice but to develop his own country’s economy well enough to provide them. Diamonds aren’t worth much if there’s nothing you can buy with them that’s worth having.
The thing about AI is that it cannot, by itself, produce anything of material value. It’s a completely abstract device for symbolic manipulation. Yes, it certainly does threaten Musa Al-Gharbi’s “symbolic capitalists” in much the same way the Industrial Revolution threatened skilled craftsmen, but like the Industrial Revolution this will only produce more wealth insofar as it increases productivity, which will still require workers and that those workers can in turn afford to support the economy of mass consumption upon which the fortunes of the truly wealthy are built in modern market economies.
I foresee a period of political and economic turmoil not unlike what we experienced in the nineteenth century as the Industrial Revolution led to the evolution of mass politics, labor unions, Marxism, financial markets, etc. There will be winners and losers. There will be massive disruption. Societies could be politically destabilized, leading to revolutions as in 1848 and the establishment of new social contracts in their wakes. It may not be a happy time. But I do not foresee a new age of Feudalism or even a Dark Age; the wealthy elite will have to find ways to justify and legitimate their rule, as always, but totalitarianism is too likely to kill the golden goose: The mass consumer economy. A factory that can produce a thousand luxury BMWs a day is basically worthless if no one can afford to buy them. Likewise, internet companies are worthless if there is no one to advertise to or who can afford to use their services; the very network effects on which they depend obviously cannot exist without consumers to network, which don’t exist in sufficient numbers without an economy of mass consumption.
At the end of the day, masses of unemployed and miserable people are a revolution waiting to happen, which is why social democracies came to exist in the first place. I think this arrangement is too useful to be fully upended by AI; the terms might change (we could get a UBI for instance), but the basic structure of mass consumerism atop private property rights will not.
I think my key point is just what does it mean when wealth can be created without a mass middle class. And while AI isn't exactly like an oil deposit, it is a way to produce things with just a few people and without needing a large educated and happy middle class. When you don't need a group of people to run society, society tends not to care very much about them and neglect them. So while it could for example do UBI (and idea I've always sort of liked to unlock risk taking), why even bother? And as for the BMWs, my point is it no longer matters if anyone can buy them. You make enough for the elite to enjoy, and scale back production for a middle class that doesn't exist--and who cares because you no longer need a stock market to generate your wealth. If your wealth no longer comes from a mass productive economy, you no longer need a mass productive economy.
You have a point, but a big reason this happens to countries that discover resources is the well-understood phenomenon of Dutch Disease: https://en.wikipedia.org/wiki/Dutch_disease
This is a purely monetary phenomenon that means that even if the rulers of the country wanted to develop more industries to employ the middle class, they would have trouble doing so. (Even UBI doesn't work, in this case, to unlock entrepreneurialism as the Arab oil states can attest to). The currency of the resource rich state is just so strong that their native industry's products aren't competitive. Norway is basically the only country that has managed to dodge this bullet by being an extremely high-trust, disciplined, and low-corruption society. They put their oil money in a sovereign wealth fund, that they only spend 3% of every year, so not a lot of the oil earnings are ever converted to the local currency.
The broader point is that this currency issue won't happen with AI since basically everyone will have access to some version of it.
The thing is, so much of what we have is only possible through economies of scale, which are lost if the mass consumer base is lost. The route you’re suggesting would result in massive decreases in the standard of living for everyone but the winners (and probably for them too, just to a lesser degree; if BMWs are no longer manufactured by the hundreds of thousands and instead only made for bespoke customers the way super-yachts are, their unit cost would skyrocket; also how useful is a car when there’s no network of gas stations or chargers to support them because only a few people use them?). While the winners might survive the revolutionary unrest this would generate if they can command sufficient (and sufficiently loyal) state-led violence, that’s a major roll of the dice. It seems at least as likely that we get Russian or French Revolution 2.0 and the “winners” end up on the wrong end of a guillotine. And in my opinion the more likely path is one that flirts with this direction before correcting course much the same way that the European elite adjusted after the revolutions of 1848. Those were almost universally failures that were violently put down, but in the aftermath states started doing a lot more to improve the lives of the lower classes. In many countries such as a Germany, these events basically led to the advent of what we would now refer to as the welfare state.
And part of my point was that AI doesn’t really create wealth because it doesn’t create produce anything tangible at all; it is a purely abstract machine for symbolic manipulation. It promises to essentially do for services what the Industrial Revolution did for the production of material goods, but services alone mostly don’t generate wealth, they just transfer it, or at best add value to something that’s actually tangible somewhere along the way.
Consider the industries most likely to be disrupted: Law, finance, academics, journalism, entertainment, publishing, and software. With the exception of software, which is a bit unique since software can add efficiencies and therefore productivity to things that happen in the real world, not one of these industries actually creates wealth; at best they transfer it around (and even that much is only economically viable at our current scale because of the vast surpluses of material wealth generated by the material economy). Now, AI could help us automate even more material processes than we already do, but not necessarily that much faster than has already been occurring for decades, because that relies heavily on robotics. Simply generating a bunch of new software to help run manufacturing processes more efficiently doesn’t do any good until the machines to use the software are themselves designed and built and installed in new production lines, and that necessarily happens at a far slower pace than AI itself can work at. And if the consumer economy fades away no one has any incentive to do that in the first place - no one will invest in production lines for goods that don’t have enough customers to support them.
I guess I just don’t think AI can replace the mass economy as a generator of wealth. It can help large actors capture more of the wealth that gets generated, it’s definitely going to be disruptive and create winners and losers, but by itself it doesn’t produce anything that isn’t purely symbolic. What it should do is make things like legal advice a lot cheaper. It could disrupt the higher education industry, which currently resembles the guilds of the seventeenth century. It can democratize the production of entertainment (AI will eventually enable individuals to create their own movies for far less than it costs Hollywood to do so, for example). For certain sectors and the individuals who work in them, this is going to be catastrophic. But not for everyone.
So: Disruption, yes. A new dark age of techno-totalitarian feudalism or anarcho-tyranny? I have my doubts, but I won’t rule it out.
Wow. This is an AI concerned I had not thought about. The reality of applying the human condition to this probable innovation is something we better understand. Skynet is not our only concern. As always we have to worry about ourselves.