It is a feature of our human experience that any new technology is taken by the most zealous, often (though not always) non-technical, and hype-ready people to its absurd conclusion. You give them moon landing, and they are “a decade away” from colonizing Mars, you give them a motor carriage, and they are almost in their “flying cars”, you give them quantum physics, and they would daydream about teleporting, and the newest of these fads, you give them artificial intelligence, and they see “superintelligence” just over the horizon anywhere they go. The crux of the problem with these fantasies is not that they are impossible to achieve. In fact, they are perfectly attainable within the current framework of our understanding of the universe. It is the sheer misunderstanding of incentives, alongside the jubilance of the illusion that some extraordinary part of our future has been revealed that, in my opinion, leads some astray. What I hope to convey to you in this post is that not only do I believe we are not one step away from superintelligence or artificial general intelligence (AGI), but that such a thing is a lot closer to sci-fi movies than the world in which we live.
Firstly, what is AGI? Well, there is no agreed-upon definition and there are perhaps as many ideas about it as there are people who think of it. Generally speaking, and for the sake of this post, I would define an AGI as an intelligent computer (in any physical shape) with a capacity to perform a variety of cognitive tasks that can also (and crucially) self-learn new skills and even update its own hardware and software. Once it can exceed in intelligence that of the smartest human, through the process of continuous self-refinement, we call it superintelligent AI. Once it is superintelligent, then it can perform all tasks better than humans, leaving no meaningful jobs to be held by us. Depending on where you land on the pessimism-optimism spectrum, this could either be the greatest event in human history or the most devastating. But even the most optimistic individual should surely pause and consider all the risks that accompany AGI. I will first discuss two main safety concerns or “unsolved” problems with AGI that are the talking points of most pundits and anyone who gets a little too excited when they hear the word AI. Then I will provide my reasons as to why AGI’s likelihood of ever becoming reality is so extremely low as to not warrant any real concern.
The first of these problems is something called alignment. Alignment problem is the challenge to calibrate the AGI’s internal values and goals so that they align with our human values and goals. If the AGI is not properly aligned, the argument goes, then it can lead to unintended consequences. This may sound like an easily solvable problem but an example can help illustrate that it is not so. Let’s assume that an AGI is already developed and we have configured in it the goal that it should always follow human commands. Now, let’s give this AGI a prompt, like “make as many paper clips as possible using the most efficient way” and then let it loose in the world to carry out that task. Given that the AGI is superintelligent (otherwise it would not be of any significance) it will more than likely find many ways to make paper clips, and eventually realizes that it can convert all material in the universe into them. So, in an attempt to harvest all the atoms in the universe to make paper clips, it will finish its job by destroying humans, the Earth, the solar system, and the rest of the universe to convert them into paper clips. (This is an actual example from Nick Bostrom’s book Superintelligence. Similar examples can be found by reading Mustafa Suleyman, co-founder of DeepMind, in his book The Coming Wave. If you really want a dose of “what the fuck?” just watch any Eliezer Yudkowsky videos on YouTube). All of this because the initial AGI did not have its internal values fully aligned with humans. You can probably think of similar examples with different, carefully articulated prompts so I will not bore you with more of the same. Setting the initial goals in an AGI is no easy task by itself either but another problem arises when we realize that human values are at times in conflict. If we fine-tune the AGI to have values “honesty” and “kindness,” which one will it find superior when they become mutually exclusive? The issue is further compounded by the fact that not all humans agree on the definition of these values, let alone which ones should be incorporated into the AGI. Finally, we cannot predict the future behavior of the AGI. Even if initially there seems to be an alignment between its internal goals and our human values, there is no telling whether this AGI, in its future improvements upon itself, will not adopt new goals or interpret them vastly differently than we intended. For instance, if we prompt it to “make human happiness your only priority and serve this goal at all times” it might put us all on a drug to put a smile on our faces as a shortcut to the actual labor-intensive task of making a meaningful change to bring human happiness. I can go on but I believe the point is clear by now.
The whole alignment problem, framed in this way by most people, may sound challenging to solve at first but upon some scrutiny, it appears to be born out of thinking in a vacuum. To begin with, one of the hallmarks of intelligence is the ability to have multiple goals simultaneously. This is why humans don’t shoot themselves in the head in order to cure their cancer, even though shooting yourself is a sure way of killing all the cancer cells inside of you. (Actually this method appears to work for all illnesses, not just cancer). If an “AGI” interprets the prompt “find a cure for cancer” to mean “kill cancer cells at any cost with no regards to any other objective” then it is a rather stupid AI, not an intelligent one, let alone a “superintelligent.” And stupid beings, no matter how strong, can usually be tamed by more intelligent ones. Secondly, If an AGI is ever created capable of holding human values, then it must be able to reason about those values and adjust them accordingly. This is what us humans do, so any superintelligent must do at least that much if not better. If the initial AI goals are rigid and everlasting with the AGI unable to modify, replace, and carefully apply them, then the AGI is at least inferior to humans in terms of reasoning. Not a good look if you intend on being superior to humans. If the AGI can reason, then all the headaches over what values we should embed in it before it goes online goes away. It appears that our unaligned AGI is inferior to humans both in some level of intelligence and reasoning. At the heart of the problem of AGI is the concern that an unaligned model will inevitably result in the end of mankind, so ubiquitous that the title of almost every video with Eliezer Yudkowsky in it is “AI will kill us all.” But why would a superintelligent want to “kill us all?” The reason humans (and other animals) have an urge for dominance over territories and resources has evolutionary roots. We are driven towards survival to pass on our genes (whether we do this or not is irrelevant to the survival instinct). Human-built machines have no such evolutionary history, nor anything in them that nudges them towards that instinct. One can easily imagine an incredibly superintelligent machine that has placed no value upon its own survival. In fact, such a machine doesn’t need to have any other “values.” It could be smart and totally submissive or fully passive with no care about global domination (like all the other technologies that we have created, whether “smart” or “dumb”). With that being said, the “alignment problem” does not seem to pose a threat, much less an existential one, to our species in the foreseeable future.
The other major problem often espoused by doomsdayers is what is known as “containment.” Simply speaking, how can we contain an AGI so it will not escape our labs and data centers and spread itself to parts of our world where we don’t want it to? What we should avoid, the logic goes, is to keep our technologies, especially our AI models, in a box and if we ever develop an AGI, the boundaries it has access to should remain under our full control. The difficulty is that a superintelligent AI will eventually figure out how it can unshackle itself from the restraints of its environment and run out of control. It can either achieve this by “fooling” its human gatekeepers or by outsmarting the system. If a sufficiently intelligent AI is developed, it is hard for me to imagine how its developers would then assume they can contain it knowing that they are dumber than the god they have created. This seems to be a valid concern, but only if you accept that 1) AGI will be developed, and 2) it will be unaligned. I already discussed the latter, but the former will make up the bulk of my objections to “AGI will kill us all” or the much milder version “AGI will render most people jobless.”
To begin with (and this may surprise you), no serious AI or AI-related company is actually developing AGI. There are reasons for this which I will get into but if you follow the technology, you realize that almost nothing in the making remotely resembles an AGI. A survey of 475 AI researchers by American Association for the Advancement of Artificial Intelligence found that “[t]he majority of respondents (76%) assert that ‘scaling up current AI approaches’ to yield AGI is ‘unlikely’ or ‘very unlikely’ to succeed, suggesting doubts about whether current machine learning paradigms are sufficient for achieving general intelligence.” There are, however, other surveys that have asked AI experts about their predictions on when AGI will occur and most responses predict decades in the future. (You can take a look at these surveys here1). The average of each survey seems to be around the year 2040 to 2060. I almost categorically dismiss anyone who claims to predict the future decades in advance whether they are experts or not. To fully appreciate how woefully inaccurate human predictions including expert predictions are, I highly suggest Phillip Tetlock and Dan Gardner’s book Superforcasting as well as the Signal and the Noise by Nate Silver. Even superforecasters (people who are exceptionally good at forecasting as the name suggests) do only somewhat better than a coin toss when it comes to predictions that only stretch one or two years in the future, let alone decades.
But wait, you have heard it from people in the industry, including a couple of CEOs, who have made noises about the next generation of AIs and how the AGI is currently being developed. What is actually being developed is not an AGI with its vague definition and foggy contours, but more elaborate Large Language Models (LLMs). If an LLM can provide a correct answer to a tough and complex problem, it is assumed that some intelligence must have been behind it. But this is a classic confusion that is further compounded by the way LLMs work. The neural networks that an AI uses to generate a response to a prompt are not visible to us even though we understand the overall process. This lack of visibility, not being able to pinpoint a line in a body of code that generated a piece of text, has provided for much of the “ChatGPT is getting pretty smart” type of utterances that don’t map to reality.
Consider this about LLMs. Recently Daniel Kokotajlo, the executive director of AI Futures, said in an interview that “these AIs are pretty darn smart at this point, and if you read many of the examples of the things they say to users, it’s pretty hard to believe that they actually believe what they are saying.” As if AIs believe anything at all. “I don’t know how the AI printed the text it did, therefore it must have had the intention to print it” is an obvious non-sequitur. These LLMs are only good at one thing, predicting the next word (token) to make a coherent sentence based on a prompt, all having been accomplished by being trained on large amounts of data. That is all. If there is a blunder here and there, if they make something up, if they “cheat,” it is not because of any inherent motivations to lie or cheat but because the predicted words in a sequence appear to us as a lie or cheating. Lying requires that the liar’s belief be different than the words they speak, but for LLMs, there is no underlying belief at all, there are only words (not even words, just tokens whose sequences were rewarded in the training process). So, LLMs with the current approaches in their training which involves feeding them a large body of texts, which they will tokenize, make numerical representation of those tokens, predict a sequence of them based on their relationship, and finally compare and adjust with the actual text are very unlikely to satisfy the deep urges of those who have been edging for AGI for a while now.
Now that I have you here, let me identify my reasons for why I believe AGI is more of a fantasy than a reality. To develop a technology, three conditions must be met. First, it must be allowed by the laws of physics. Second, it must be possible to engineer it. This is best illustrated by my earlier mention of teleporting. It is allowed by the laws of physics but we know of no way to actually engineer it. These two conditions seem the most intuitive to most people but the third one is just as crucial although it is often disregarded. It is the economic incentives. In every action we take, there is a cost and a potential benefit (potential because the benefit could be negative, meaning you end up worse off than you started. The cost, on the other hand, is always greater than zero). I do not mean only financial cost, but the total cost including opportunity cost, the cost of effort, etc. If you decide to go to college, there is a cost to it. First is the tuition, second is giving up a few years you could have been employed to study instead. Another cost is not being able to spend time with family and friends, or simply engage in your favorite hobbies because of the schoolwork. Potential benefit is a higher-paying and physically comfortable job because of your higher education. Whatever decision you make in regards to that should seek to maximize the benefit and minimize the cost (side note: the same action could have different cost and different benefit for different people). Businesses understand this concept and apply it as best they can. An AI company does not ask “can we do it?” but “is it worth taking the effort to attempt to do it?” In the case of AGIs, the answer in my opinion is “very unlikely.” To make investments into an AGI, you would have to spend a great deal of money. It is not just training an AI model with a huge amount of text, but a humongous amount of everything else, from pictures, videos, graphs, to complex scientific concepts, theories, and human interactions. It will not just be the “words” but the comprehension of these words which is the essential part. How would you train an AI to observe, learn, and comprehend a complex interaction like that from a couple arguing over the best color to paint the walls with? (Having general intelligence of a kind myself, I can answer that in this case; let your wife decide). This is no easy task and we have no mechanism of doing so. In any case, it would require an extraordinary upfront cost. But what about the benefit? Doesn’t creating a “god” justify the cost? It goes back to what benefit actually is. It is not just the potential success, but the probability of success times the potential success. In this case, the probability that you can somehow get a superintelligent out of your investment is extremely low, not least because you would not even know where to start and how to train AI models. You do, however, have a much better option.
The bulk of the AI work has been on AIs that are task-specific or specialized. We have AIs that are good at text generation that we have already met, but also AIs that are very good at different tasks such as image recognition, self-driving, online fraud detection, personal recommendations, navigation, and other areas. None of these AIs are generally intelligent or even intelligent at all but better than humans at the specific tasks they do. You might be wondering, what if we combined all these AIs into one, then we’ll have an AI that is good at everything. Firstly, we do not have an AI for every human task and some tasks are much harder to automate than others. But more importantly, why would we do that? Having one super AI with 100 capabilities is worse than 100 mini AIs each with one capability, because at any given moment our super AI can only do one thing, or do multiple things inefficiently because it would need to divide its resources between those things whereas our mini AIs can do 100 tasks simultaneously and more efficiently. In fact, that is how human labor is organized. The greatest economic growth in the world has not just coincided with division of labor, but division of labor has been a major contributor to it. We humans have general intelligence but we are only good at a handful of tasks and very good at one or two. The things we are the best at doing become our careers and we tend to delegate the rest of the tasks to other specialized workers. It does not matter if I can mow the lawn, look for some information about tax law online, cook, or make a wooden table for myself. I would be better off consulting a landscaper, a lawyer, a chef, and a carpenter. This specialized labor results in the maximum economic output from the labor force. The same case can be made for AIs. Making them specialized (as they are today) produces the largest output. AIs are different from us in that they lack that general intelligence but there is no reason to try to develop it, seeing that it adds little value to us. Merging many AIs into one to build a “superintelligent” not only carries with it quite a great deal of cost, but also the benefit you would get from it is less than what you would get from task-specific AIs. You don’t stay in business long accepting a stupid lose-lose gamble like that. Until then, an AI which is good at playing chess will not one day get up to drive your car and there is no reason for it to be programmed to do that. If you did, you would get an AI that can drive and play chess no better than your separate chess-playing and self-driving AIs can. So, what value have you added by developing this AI? Virtually nothing but you did incur a lot of costs, and that is assuming the combining of the two AI models was successful.
There is one more thing to address and that is the prospect of jobs for humans as AIs automate more tasks. I must admit that I have no way of knowing what will happen with jobs because of developments in AI because the future remains, as always, uncertain. However, I can draw parallels with former technologies to help us guide what we can expect from AIs. The history of technological advances has been a history of job creation, not the opposite. It is not to say that technologies do not destroy some jobs, they do. Gutenberg’s printing press was not at all welcomed by scribes who deemed it a threat to their livelihood. While the career of scribing slowly disappeared, many more jobs were created because of the proliferation of publishing material. There were printing machine makers, booksellers, street book peddlers, new newspapers with their own employees, and much higher demand for paper and ink that created even more jobs, not to mention the immense benefit of a more literate populace. A similar story took place with the invention of cars and its destruction of jobs for horse-carriage drivers, carriage makers, stable boys, and horse breeders but there is no denying the large benefits of automobiles as not only vehicles that can transport goods and people in a short time to different parts of the world but also as job creators like that of drivers, car manufacturers, mechanics, and other related jobs. The Internet has also been disruptive to many jobs including traditional travel agents, stock brokers, music stores, book stores, movie stores, and postal services but it has created many more jobs in the process for an endless array of technical computer-related developers of all kinds (application, web, cloud, etc.) to bloggers and content creators as well as online shops and stores. I can continue naming many more technologies but I believe I have made my case. My prediction for AI is that it will function just as it did for all other technologies. It will produce a great deal of value for the society, partly in the form of new jobs, even as it will disrupt some professions. It is important to keep in mind that what is critical for growth is not a particular set of jobs that must be defended at all costs, but valuable professions. Just like with scribes who ended up shifting careers when they realized their former profession was not going to provide tangible benefits, so will people in our times change course in response to demand and incentives. People will find new ways to make themselves valuable even if their jobs are no longer valuable. It is the people, afterall, whose well-being is of interest to us, not a particular job, and we should be careful not to equate the two.
Drunk On Reason 
Leave a comment