History and Evolution of AI
Alan Turing
Although it is difficult to pinpoint, the roots of AI is generally traced back to the 1940s.
During the Second World War between 1939-1994, Alan Turing, a Cambridge mathematician, who came to be considered as the father of computing developed a code breaking machine called The Bombe. It was developed for the British government, with the primary objective of deciphering the Enigma code used by the German military during the War. The Bombe was used repeatedly to decode messages encoded using the Enigma. About 7 by 6 by 2 feet large with a weight of about a ton, it is widely regarded as the earliest working electro-mechanical computer.
The remarkable effectiveness of The Bombe in breaking the Enigma code, a task previously deemed impossible to even the best human mathematicians, made Turing wonder about the intelligence of such machines.
In 1950, Alan Turing published a ground breaking article titled "Computing Machinery and Intelligence," in which he introduced the concept of artificial intelligence (AI).
In this article, he outlined the potential for creating intelligent machines and, more notably, proposed a method to assess their intelligence. This method is now known as the Turing test. He presented this test as a sufficient condition for determining the existence of AI. Even today, the Turing test remains a fundamental benchmark for identifying the intelligence of artificial systems.
The Turing test involves human testers engaging in natural language conversations without restrictions via computer terminals. The conversation takes place between both human and AI natural language programs, the identities of which are hidden. If the evaluators are unable to distinguish between the responses of the human and the AI program, then the machine is considered to be intelligent.
The Turing machine has proven to be the most valuable formalization in theoretical computer science and is frequently utilized in this field. We will delve deeper into the Turing Test and explore its implications in an upcoming lecture.
Samuel’s Checker-Playing Program and Machine Learning
In 1952, Arthur Samuel developed the Samuel’s Checker-Playing Program, a significant milestone as it was the world's first self-learning program for playing games. The prevailing notion in computer science had long been that a computer can only perform actions that its programmer explicitly instructed it to do and cannot possess knowledge beyond what the programmer supplied.
However, this seemingly logical conclusion turned out to be incorrect because it failed to consider the possibility of a computer being programmed to learn autonomously. This breakthrough concept, known as Machine Learning, later evolved into a major subfield of artificial intelligence. Samuel's checker-playing program hence marked the inception of machine learning.
Initially, Samuel was able to defeat his own program, but after a few months of continuous learning, it is reported that he never won another game against it. This significant development marked the birth of machine learning as a transformative field within AI.
The Dartmouth Conference
The word Artificial Intelligence was officially coined a few years later in 1956, when Marvin Minsky and John McCarthy (a computer scientist at Stanford) hosted the approximately eight-week-long Dartmouth Summer Research Project on Artificial Intelligence (DSRPAI) at Dartmouth College in New Hampshire. The Dartmouth Workshop served to bring researchers in this newly emerging field together to interact and to exchange ideas.
the workshop reunited those who would later be considered as the founding fathers of AI. The objective of the project was to unite researchers from various fields in order to create a new research area aimed at building machines able to simulate human intelligence.
John McCarthy, on the Dartmouth faculty at the time of the Workshop, is credited with having coined the name Artificial Intelligence. In 1958, he developed the programming language L.I.S.P. or lisp, which was quickly adopted by the AI industry and gained enormous popularity among developers.
Frank Rosenblatt and the Perceptron
In 1957, Psychologist Frank Rosenblatt developed what he called the Perceptron. A digital neural network that was designed to mimic a few brain neurons. as an early Artificial Neural Network that could learn from data it became the foundation for modern neural networks. He tasked it with classifying images into two categories. He scanned images of men and women into the program and hypothesized that overtime the network would learn to differentiate between men and women or at least see the patterns that made them different. The attraction of the perceptron was due to a supervised learning algorithm, by means of which a perceptron could be taught to classify correctly. Thus, neural nets contributed to machine learning.
Unfortunately, the perceptron was not as successful as Rosenblatt had envisioned. As he only used a single layer of artificial neurons, making it extremely limited in what it could do. Computers of that day only had very limiting computing power enough to handle that simple setup. By 1969- Computer Science community abandoned the idea.
Expert systems
In the early 1950s, Expert systems emerged as a subset of AI, when the Rand-Carnegie team developed the general problem solver to deal with theorems proof, geometric problems and chess playing. Expert systems are computer programs aiming to model human expertise in one or more specific knowledge areas. The name came from the process of knowledge engineering, of having knowledge engineers laboriously extract information from human experts, and handcraft that knowledge into their expert systems. In an expert system, the knowledge of a particular subject area is represented in the form of rules and large knowledge bases.
Lead by prominent chemists Joshua Lederberg, and AI researchers Edward Feigenbaum and Bruce Buchanan, the first expert system, called Dendral was an expert in organic chemistry. DENDRAL helped to identify the molecular structure of organic molecules by analysing data and employing its knowledge of chemistry.
A second such expert system, called MYCIN, helped physicians diagnose and treat infectious blood diseases and meningitis. The system was successful in that it could diagnose difficult cases as well as the most expert physicians, but unsuccessful in that it was never fielded. Inputting information into Mycin required about twenty minutes. A physician would spend at most five minutes on such a diagnosis.
The Dartmouth Conference was followed by a period of nearly two decades that saw significant success in the field of AI.
An early example is the famous ELIZA computer program, created between 1964 and 1966 by Joseph Weizenbaum at Massachusets Institute of Technology. ELIZA was a natural language processing tool able to simulate a conversation with a human and one of the first programs capable of attempting to pass the Turing Test that we discussed before.
As a result of these inspiring success stories, substantial funding was given to AI research, leading to more and more projects. In 1970, Marvin Minsky gave an interview to Life Magazine in which he stated that a machine with the general intelligence of an average human being could be developed within three to eight years. this shows the vision that experts had in the advancement of AI technology.
AI winter
AI developed considerably between 1945 and 1975. However, there were relatively few major breakthroughs, much over-hyping of AI’s future prospects, and the cost of research into AI was increasing rapidly. The history of Turing Test and Expert systems all showed that people greatly overestimated AI's progress since early days. Towards the end of that era, experts had doubts as to how rapidly AI would develop.
As a consequence of the disappointingly slow progress made by AI, the American and British governments reduced funding for AI. this lead to the period of time described as “AI winter”, between 1975 and 1995, during which the commercial and scientific activities in AI declined dramatically.
The onset of the AI winter could be traced to the government’s decision to pull back on AI research. The decisions were often attributed to a couple of infamous reports, specifically the Automatic Language Processing Advisory Committee (ALPAC) report by U.S. Government in 1966, and the Lighthill report for the British government in 1973.
The ALPAC report
In the United States, AI research was motivated by the potential of machine translation (MT), especially during the Cold War when translating Russian quickly was crucial. In 1954, the first demonstration of MT was the Georgetown-IBM experiment. despite being limited to organic chemistry and having only a few rules and a small vocabulary,, it generated great public interest and made headlines, igniting hopes for electronic translation. This event was the most widespread publicity that Machine Translation had ever received. Many reports cited the prediction that interlingual translation could become a reality in a few years.
Stimulated by this attention and the Soviet government's response, US agencies began supporting Machine Translation research in 1956. However, progress was slow over the next decade, leading to the establishment of ALPAC to investigate the reasons behind this. In ALPAC's report conclusion, it emphasized that despite significant government investment, the results were disappointing. The report rightly noted that Machine Translation needed further fundamental research.
The Lighthill report
The Lighthill report, authored by Professor Sir James Lighthill in 1973, is a pivotal document in the history of artificial intelligence (AI). Commissioned by the head of the British Science Research Council, Lighthill's report evaluated the state of AI research. Despite his background in hydrodynamics and lack of prior AI expertise, his report had a significant impact, leading the British government to reduce funding for AI research in most universities, except for a few select institutions.
Lighthill's critique centred on AI's inability to live up to its early promises. He highlighted specific examples, such as airplane landing systems and chess programs, where conventional engineering methods often outperformed AI. Lighthill also criticized the need for substantial pre-existing knowledge in AI methods and the inability of AI systems to automatically acquire knowledge, rendering them less genuinely intelligent. His report pointed out the limitations of AI techniques, especially in dealing with large-scale, real-world problems. The consequences of this report were profound, ushering in the "AI winter," marked by reduced government funding and slowed progress in AI research.
After the AI winter, computer scientists shifted to smaller, practical, and commercially oriented projects. aided by increased computing power and the emergence of deep learning this lead to a resurgence in AI, often referred to as the "artificial intelligence renaissance."
The initial disappointments noted in reports like ALPAC and Lighthill's triggered the AI downfall. with unrealistic expectations and superficial study of AI applications contributing to the decline. as researchers often overestimated the capabilities of AI methods and made overly optimistic predictions. highlighting the importance of a deeper understanding of AI algorithms and cautious optimism in the field's development.
Japan's Fifth Generation Computer System project
During the AI Winter in the United States, Japan embarked on an ambitious project known as the Fifth Generation Computer System (FGCS) in to achieve the dream of artificial intelligence the 1980s.
Japan had previously lagged behind the United States in technology and sought to take the lead in computer technology by developing these new machines, which were not based on standard microprocessors but on multi-processor machines specialized in logic programming. Despite over a decade of research and a substantial investment of more than a billion dollars, the FGCS project did not succeed in creating truly intelligent computers. While the project didn't significantly advance the state of the art in AI, its grand vision and ultimate failure remain intriguing and instructive in retrospect.
One reason for the initial lack of progress in the field of AI and the fact that reality fell back sharply against the expectations,,, lies in the specific way in which early systems such as ELIZA and the General Problem Solver tried to replicate human intelligence. Specifically, they were all Expert Systems, that is, collections of rules which assume that human intelligence can be formalized and reconstructed in a top-down approach as a series of “if-then” statements.
Expert Systems can perform impressively well in areas that depend on such formalization. For example, IBM’s Deep Blue chess playing program, which in 1997 was able to beat the world champion Gary Kasparov is such an Expert System. Deep Blue was reportedly able to process 200 million possible moves per second and to determine the optimal next move looking 20 moves ahead through the use of a method called tree search.
Artificial Neural Networks
Statistical methods for achieving true AI have been discussed as early as the 1940s. during this period Canadian psychologist Donald Hebb developed a theory of learning known as Hebbian Learning that replicates the process of neurons in the human brain. This led to the advent of research on Artificial Neural Networks.
An artificial neural network is a computational model inspired by the human brain's neural structure. it consisted of interconnected nodes (neurons) that process and transmit information. It's used in machine learning to solve complex tasks, such as pattern recognition and decision-making, by learning from data and adjusting the connections (weights) between neurons during training.
Yet, this work stagnated in 1969 when Marvin Minsky and Seymour Papert showed that computers did not have sufficient processing power to handle the work required by such artificial neural networks.
In 1986, ANN pioneers David Rumelhart, Ronald J. Williams and Geoffrey Hinton co-authored a highly influential paper that popularised the backpropagation algorithm which is the process of computers learning from their mistakes and hence becoming better at a given task. This was used for training multi-layer neural networks, which became instrumental in the fundamental research that bought about the AI revolution.
Hinton and collaborators came up with a technique for a deeper, multilayered network. But it took 26 years before computing power and data capacity caught up and capitalized on the deep architecture. Today we call this multilayered approach a deep neural network.
In the early stages the problems preventing AI advancements were slow and inadequate computing power and the lack of data for training AI program. By 2006, both these were resolved as computers’ processing speed had grown significantly and massive amounts of data had been acquired through the advent of the internet.
In 2012, Hinton collaborated with two of his students for the ImageNet challenge. He implemented a multilayered neural network that was trained to recognize objects in massive image data sets, which was a major breakthrough in the field of computer vision.
With this turning point machines gradually became as good as humans at object recognition. By the late 2010s image recognition was commonplace,, even recognizing disease in medical imaging. Soon neural network AI was tackling video, speech, science and even games.
Deep Learning
Artificial neural networks made a comeback in the form of Deep Learning when in 2015 AlphaGo, a program developed by Google, was able to beat the world champion in the board game Go. Go is substantially more complex than chess (e.g., at opening there are 20 possible moves in chess but 361 in Go) and it was long believed that computers would never be able to beat humans in this game. AlphaGo achieved its high performance by using a specific type of artificial neural network called Deep Learning.
Deep learning is a subset of machine learning that utilizes artificial neural networks with multiple layers (deep neural networks) to automatically learn and represent complex patterns in data. It has achieved remarkable success in various applications like image recognition, natural language processing, and speech recognition due to its ability to model intricate relationships within large datasets.
Today Artificial Neural Networks and Deep Learning form the basis of most applications we know under the label of AI. They are the basis of image recognition algorithms used by Facebook, speech recognition algorithms that fuel smart speakers and self-driving cars. This harvest of the fruits of past statistical advances is the period of AI Fall, which we find ourselves in today.
A chronological timeline of important milestones in the history of AI
In 1950- Alan Turing published "Computing Machinery and Intelligence," which introduced the Turing test and opening the doors to what would be known as AI.
In 1952- Arthur Samuel developed, the world's first program to play games that was self-learning. Samuel’s Checker-Playing Program
1956 saw- the Dartmouth conference. The term artificial intelligence was coined by McCarthy
1958- Frank Rosenblatt developed the perceptron, an early ANN that could learn from data which became the foundation for modern neural networks.
In the same year John McCarthy developed the programming language LISP, which was quickly adopted by the AI industry and gained enormous popularity among developers
In 1959- Minsky and McCarthy established the MIT AI lab now known as Computer Science and Artificial Intelligence Laboratory (CSAIL)
In the same year Arthur Samuel coined the term machine learning in a seminal paper explaining that the computer could be programmed to outplay its programmer.
In 1965- the first expert system- Dendral, assisted organic chemists in identifying unknown organic molecules.
In 1966- Joseph Weizenbaum created Eliza, one of the most celebrated computer programs of all time, capable of engaging in conversations with humans and making them believe the software had humanlike emotions.
In 1974- Stanford Research Institute developed Shakey, the world's first mobile intelligent robot that combined AI, computer vision, navigation and NLP – later known as the grandfather of self-driving cars and drones.
In 1997- Deep blue chess machine developed by IBM beats the then world chess champion Gary Kasparov
In 2004- NASA's robotic exploration rovers Spirit and Opportunity autonomously navigated the surface of Mars.
2005- Honda's ASIMO robot, an artificially intelligent humanoid robot, was able to walk as fast as a human, delivering trays to customers in restaurant settings.
2009 marked the beginning of - the development of self-driving technology at Google X Lab
2011- IBM Watson beat jeopardy champion.
SIRI, Google Now and Cortana became mainstream as substantial voice-based interfaces
2015- Stephen Hawking, Elon Musk, and dozens of artificial intelligence experts signed an open letter on artificial intelligence calling for research on the societal impacts of AI.
2016- Google’s DeepMind defeated Korean AlphaGo champion Lee Sedol
2018- OpenAI released GPT (Generative Pre-trained Transformer), paving the way for subsequent LLMs.
Groove X unveiled a home mini-robot called Lovot that could sense and affect mood changes in humans.
2019- Google AI and Langone Medical Centre’s deep learning algorithm outperformed radiologists in detecting potential lung cancers.
2020- The University of Oxford developed an AI test called Curial to rapidly identify COVID-19 in emergency room patients.
In February 2020, Microsoft introduced its Turing Natural Language Generation (T-NLG), which is the "largest language model ever published at 17 billion parameters.
OpenAI introduced GPT-3, a state-of-the-art autoregressive language model that uses deep learning to produce a variety of computer codes, poetry and other language tasks exceptionally similar, and almost indistinguishable from those written by humans. Its capacity was ten times greater than that of the T-NLG.
2021- OpenAI introduced the Dall-E multimodal AI system that can generate images from text prompts.
2022- OpenAI released ChatGPT in November to provide a chat-based interface to its GPT-3.5 LLM.
2023- OpenAI announced the GPT-4 multimodal LLM that receives both text and image prompts.
In response to ChatGPT, Google releases in a limited capacity its chatbot Google Bard
a petition of over 1,000 signatures is signed by Elon Musk, Steve Wozniak and other tech leaders, calling for a 6-month halt to what the petition refers to as an "an out-of-control race" producing AI systems that its creators can't "understand, predict, or reliably control.