The field of artificial intelligence (AI) has its roots in the mid-20th century. The early development of AI can be traced back to the Dartmouth Conference in 1956, which is considered the birth of AI as a distinct field of research. The conference brought together a group of computer scientists who aimed to explore the potential of creating intelligent machines. Since its birth, AI’s journey has been long, eventful, and fraught with challenges. Despite early optimism about the potential of AI to match or surpass human intelligence in various domains, the field has undergone two AI winters and now, what appears to be, one hot AI summer.
In 1970, the optimism about AI and machine intelligence was so high that in an interview with Life Magazine, Marvin Minsky, one of the two founders of the MIT Computer Science and Artificial Intelligence Laboratory, predicted that within “three to eight years we will have a machine with the general intelligence of an average human being.” Although the timing of Minsky’s prophecy has been off by a long stretch, recent advances in computing systems and machine learning, including foundation models and large language models (LLMs), are creating renewed optimism in AI capabilities.
The first AI winter
AI winters refer to periods of time when public interest, funding, and progress in the field of artificial intelligence significantly decline. These periods are characterized by a loss of confidence in AI technologies’ potential and often follow periods of overhype and unrealistic expectations.
The first AI winter occurred in the 1970s, following the initial excitement that surrounded AI during the 50s and 60s. The progress did not meet the high expectations, and many AI projects failed to deliver on their promises. Funding for AI research decreased, and interest waned as the technology entered its first AI winter, facing challenges such as:
- Technical limitations: AI technologies had met significant technical obstacles. AI research had difficulty representing knowledge in ways that could be readily understood by machines and the R&D was exceedingly limited by the computing power available at the time. This in turn restricted the complexity and scale of learning algorithms. In addition, many AI projects were constrained by limited access to data and encountered difficulties in dealing with real-world complexity and unpredictability, making it challenging to develop effective AI systems.
- Overhype and unmet expectations: Early excitement and excessive optimism about AI’s potential to achieve “human-level” intelligence had led to unrealistic expectations. When AI projects faced major hurdles and failed in delivery, it led to disillusionment and a loss of confidence in the technology.
- Constraints in funding and other resources: As the initial enthusiasm for AI subsided and results were slow to materialize, funding for AI research plunged. Government agencies and private investors became more cautious about investing in AI projects, leading to resource constraints for institutions and researchers alike.
- Lack of practical applications: AI technologies had yet to find widespread practical applications. Without tangible benefits to businesses, consumers, or government entities, interest in the field faded.
- Criticism from the scientific community: Some members of the scientific community expressed skepticism about the approach and progress of AI research. Critics argued that the foundational principles and techniques of AI were too limited to achieve human-level intelligence, and they were doubtful about the possibility of creating truly intelligent machines.
For example, “after a 1966 report by the Automatic Language Processing Advisory Committee (ALPAC) of the National Academy of Sciences/National Research Council, which saw little merit in pursuing [machine translation], public-sector support for practical MT in the United States evaporated…” The report indicated that since fully automatic high-quality machine translation was impossible, the technology could never replace human translators. They said that funds would therefore be better spent on basic linguistic research and machine aids for translators.”
Another example is the Lighthill report, published in 1973, which was a critique of AI research in the UK. It criticized AI’s failure to achieve its ambitious goals and concluded that AI offered no unique solution that is not achievable in other scientific disciplines. The report highlighted the “combinatorial explosion” problem, suggesting that many AI algorithms were only suitable for solving simplified problems. As a result, AI research in the UK faced a significant setback, with reduced funding and dismantled projects.
Despite the multiple challenges facing AI at the time, the first AI winter lasted for less than a decade (est. 1974-1980). The end of the first AI winter was marked by a resurgence of interest and progress in the field. Several factors contributed to this revival in the early 1980s. These included new advancements in machine learning and neural networks, expert systems that utilized large knowledge bases and rules to solve specific problems, increased computing power, focused research areas, commercialization of and successes in practical AI applications, funding by the Defense Advanced Research Projects Agency (DARPA), and growth in data. The combined impact of these factors led to a reinvigorated interest in AI, signaling the end of the first AI winter. For a few years, AI research continued to progress and new opportunities for AI applications emerged across various industries. When this revival period ended in the late 1980s, the second AI winter set in.
The second AI winter
It is generally understood that the second AI winter started in 1987 and ended in the early 1990s. Similar to the first period of stagnation and decline in AI interest and activity in the mid 70s, the second AI winter was caused by hype cycles and outsized expectations that AI research could not meet. Once again, government agencies and the private sector became cautious about technical limitations, doubts in ROI (Return on Investment), and criticism from the scientific community. At the time, the expert systems proved to be brittle, rigid, and difficult to maintain. Scaling AI systems proved challenging as well since they needed to be able to learn from large data sets which was computationally expensive. This made it difficult to deploy AI systems in real-world applications.
Symbolic AI’s reliance on explicit, rule-based representations was criticized for being inflexible and unable to handle the complexity and ambiguity of everyday data and knowledge. Symbolic AI, also known as Good Old-Fashioned AI (GOFAI), is an approach to AI that focuses on the use of explicit symbols and rules to represent knowledge and perform reasoning. In symbolic AI, problems are broken down into discrete, logical components, and algorithms are designed to manipulate these symbols to solve problems. The fundamental idea behind symbolic AI is to create systems that can mimic human-like cognitive processes, such as logical reasoning, problem-solving, and decision-making. These systems use symbolic representations to capture knowledge about the world and apply rules for manipulating that knowledge to arrive at conclusions or make predictions.
While symbolic AI was a dominant approach in the early days of AI research, it faced certain limitations, such as difficulties in handling uncertainty, scalability issues, and challenges in learning from data. These limitations, along with the emergence of alternative AI paradigms, such as machine learning and neural networks, contributed to the rise of other AI approaches and the decline of symbolic AI in some areas. That said, symbolic AI continues to be used in specific applications and as part of hybrid AI systems that combine various techniques to address complex problems.
During the second AI winter, the AI fervor was not only waning in the United States, but globally as well. In 1992, the government of Japan “formally closed the books on its vaunted ‘Fifth Generation’ computer project.” The decade-long research initiative was aimed at developing “a new world of computers that could solve problems with human-style reasoning.” After spending $400+ million, most of the ambitious goals did not materialize and the effort ended with little impact on the computer market.
The second AI winter was a setback for the field of AI, but it also led to some important progress. AI researchers learned from their past experiments, and they developed new approaches to improve output and performance. These new approaches, such as deep learning, have led to a resurgence of interest in AI research ever since.
While the second AI winter was beginning to thaw in the early 1990s, other technical, political, and societal transformations were taking hold in parallel. The Cold War had come to an end with the collapse of the Soviet Union in December 1991 and, from the ashes, 15 newly independent nations rose. In the same year, Tim Berners-Lee introduced the World Wide Web—the Internet we know today. And in 1992, the Mosaic browser (later Netscape) was born. Users could “see words and pictures on the same page for the first time and to navigate using scrollbars and clickable links.” In the same decade, Linux, today’s most commonly known and used open source operating system, was launched. The first SMS text message was sent. Amazon.com was founded, disrupting the retail industry starting with books. Palm’s PDA (Personal Digital Assistant) devices gave us a glimpse of what’s to come in mobile computing. Google introduced its now dominant web search engine. And, yes, blogs became a thing, opening up opportunities for people to publish anything online.
As a result of all these changes, the world became more connected than ever, and data grew exponentially larger and richer. In the ensuing decade, the data explosion, key advancements to computing hardware, and breakthroughs in machine learning research would all prove to be perfect enablers of the next generation of AI. The AI spring which followed the second AI winter indicated renewed optimism for intelligent machines that can help radically improve our lives and solve major challenges.
Today’s hot AI summer
Fast forward to present where the application of AI is more than just a thought exercise. The latest upgrades to AI include a set of technologies which are preparing us for artificial general intelligence (AGI). Also known as strong AI, AGI is used to describe a machine’s intelligence functionality that matches human cognitive capabilities across multiple domains and modalities. AGI is often characterized and judged by self-improvement mechanisms and generalizability rather than specific training to perform in narrow domains. Also see Entefy’s multisensory Mimi AI Engine.
In the context of AI, an AI summer characterizes a period of heightened interest and abundant funding directed towards the development and implementation of AI technology. There’s an AI gold rush underway and the early prospectors and settlers are migrating to the field en masse—investors, the world’s largest corporations, entrepreneurs, innovators, researchers, and tech enthusiasts. Despite certain limitations, the prevailing sentiment within the industry suggests that we are currently living through an AI summer, and here are a few reasons why:
- The new class of generative AI: Generative AI is a subset of deep learning, using expansive artificial neural networks to “generate” new data based on what it has learned from its training data. Traditional AI systems are designed to extract insights from data, make predictions, or perform other analytical tasks. In contrast, generative AI systems are designed to create new content based on diverse inputs. By applying different learning techniques to massive data sets such as unsupervised, semi-supervised, or self-supervised learning, today’s generative AI systems can produce realistic and compelling new content based on the patterns and distributions the models learn during training. The user input for and the content generated by these models can be in the form of text, images, audio, code, 3D models, or other data.
Underpinning generative AI are the new foundation models that have the potential to transform not only our digital world but society at large. Foundation models are sophisticated deep learning models trained on massive amounts of data (typically unlabeled), capable of performing a number of diverse tasks. Instead of training a single model for a single task (which would be difficult to scale across countless tasks), a foundation model can be trained on a broad data set once and then used as the “foundation” or basis for training with minimal fine-tuning to create multiple task-specific models. In this way, foundation models can be adapted to a wide variety of use cases.
Mainstream examples of foundation models include large language models (LLMs). Some of the better-known LLMs include GPT-4, PaLM 2, Claude, and LLaMA. LLMs are trained on massive data sets typically consisting of text and programing code. The term “large” in LLM refers to both the size of the training data and the number of model hyperparameters. As with most foundation models, the training process for an LLM can be computationally intensive and expensive. It can take weeks or months to train a sophisticated LLM on large data sets. However, once an LLM is trained they can solve common language problems. For example, generating essays, poems, code, scripts, musical pieces, emails, and summaries. LLMs can also be used to translate languages or answer questions. - Better computing: In general, AI is computationally intensive, especially during model training. “It took the combination of Yann LeCun’s work in convolutional neural nets, Geoff Hinton’s back-propagation and Stochastic Gradient Descent approach to training, and Andrew Ng’s large-scale use of GPUs to accelerate deep neural networks (DNNs) to ignite the big bang of modern AI — deep learning.” And deep learning models fueling the current AI boom, with millions and billions of parameters, require robust computing power. Lots of it.
Fortunately, technical advances in chips and cloud computing are all improving the way we can all access and use computing power. The enhancements to the microchips over the years have fueled scientific and business progress in every industry. Computing power has increased “one trillion-fold” from 1956 to 2015. “The computer that navigated the Apollo missions to the moon was about twice as powerful as a Nintendo console. It had 32.768 bits of Random Access Memory (RAM) and 589.824 bits of Read Only Memory (ROM). A modern smartphone has around 100,000 times as much processing power, with about a million times more RAM and seven million times more ROM. Chips enable applications such as virtual reality and on-device artificial intelligence (AI) as well as gains in data transfer such as 5G connectivity, and they’re also behind algorithms such as those used in deep learning.” - Better access to more quality data: The amount of data available to train AI models has grown significantly in recent years. This is due to the growth of the Internet, the proliferation of smart devices and sensors, as well as the development of new data collection techniques. According to IDC, from 2022 to 2026, data created, replicated, and consumed annually is expected to more than double in size. “The Enterprise DataSphere will grow more than twice as fast as the Consumer DataSphere over the next five years, putting even more pressure on enterprise organizations to manage and protect the world’s data while creating opportunities to activate data for business and societal benefits.”
- Practical applications across virtually all industries: Over the past decade, AI has been powering a number of applications and services we use every day. From customer service to financial trading, advertising, commerce, drug discovery, patient care, supply chain, and legal assistance, AI and automation have helped us gain efficiency. And that was before the recent introduction of today’s new class of generative AI. The latest generative AI applications can help users take advantage of human-level writing, coding, and designing capabilities. With the newly available tools, marketers can create content like never before; software engineers can document code functionality (in half the time), write new code (in nearly half the time), or refactor code “in nearly two-thirds the time“; artists can enhance or modify their work by incorporating generative elements, opening up new avenues for artistic expression and creativity; those engaged in data science and machine learning can solve critical data issues with synthetic data creation; and general knowledge workers can take advantage of machine writing and analytics to create presentations or reports.
These few examples only scratch the surface of practical use cases to boost productivity with AI. - Broad public interest and adoption: In business and technology, AI has been making headlines across the board, and for good reason. Significant increases to model performance, availability, and applicability, have brought AI to the forefront of public dialogue. And this renewed interest is not purely academic. New generative AI models and services are setting user adoption records. For example, ChatGPT reached its first 1 million registered users within the first 5 days of release, growing to an estimated 100 million users after only 2 months, at the time making it the fastest-growing user base of any software application in history. For context, the previous adoption king, TikTok took approximately 9 months to reach the same 100 million user milestone, and it took 30 months for Instagram to do the same.
In the coming years, commercial adoption of AI technology is expected to grow with “significant impact across all industry sectors.” This will allow businesses additional opportunities to increase operational efficiency and boost productivity that is “likely to materialize when the technology is applied across knowledge workers’ activities.” Early reports suggest that the total economic impact of AI on the global economy could range between $17.1 trillion to $25.6 trillion.
Conclusion
The history of AI has been marked by cycles of enthusiasm and challenges, with periods of remarkable progress followed by setbacks, namely two AI winters. The current AI summer stands out as a transformative phase, fueled by breakthroughs in deep learning, better access to data, powerful computing, significant new investment in the space, and widespread public interest.
AI applications have expanded across industries, revolutionizing sectors such as healthcare, finance, transportation, retail, and manufacturing. The responsible development and deployment of new AI technologies, guided by transparent and ethical principles, are vital to ensure that the potential benefits of AI are captured in a manner that mitigates risks. The resurgence of AI in recent years is a testament to the resilience of the field. The new capabilities are getting us closer to realizing the potential of intelligent machines, ushering in a new wave of productivity.
For more about AI and the future of machine intelligence, be sure to read Entefy’s important AI terms for professionals and tech enthusiasts, the 18 valuable skills needed to ensure success in enterprise AI initiatives, and the Holy Grail of AI, artificial general intelligence.
ABOUT ENTEFY
Entefy is an advanced AI software and process automation company, serving SME and large enterprise customers across diverse industries including financial services, health care, retail, and manufacturing.