Generative AI sets itself apart from traditional AI by the fact that it is capable of generating new content such as visuals, audio and textual data. It may seem like technology has only recently emerged. However, this is not entirely true. The first generative algorithms date back to the origins of AI as a field of computer science. Yet, generative AI has only recently begun to evolve by leaps and bounds. Here's a look at the history of generative AI and why it's been developing so rapidly in recent years. Before heading to the discussion about generative AI, let’s take a moment and look at the development of AI as a whole. Many scientists from different fields were involved in the exploration of artificial intelligence at the beginning of the 20th century. Perhaps one of the most famous was the highly acclaimed mathematician Alan Turing. He has been involved with the challenge of artificial intelligence since at least 1941. Turing made one of his earliest mentions of "intelligent machinery" in 1947. In an eponymous paper, Turing explored the question of whether a machine could detect rational behavior. AI was born in 1956 when a workshop called The Dartmouth Summer Research Project on Artificial Intelligence was held. Researchers from different fields of science, including linguistics, philosophy and computer science, came together. They debated the potential of computing machines to simulate reasoning, intelligence, and creative processes. Among other events at the workshop, John McCarthy, an American computer scientist, proposed a name for a new academic discipline: artificial intelligence. Considerable progress was made over the next decades. Machines were becoming more affordable, cheaper, faster, and capable of storing more information. Machine learning (ML) algorithms were also improving. Arthur Samuel introduced the machine learning term in 1959 with the first self-learning program created to play checkers, which could independently play on its own. In the late 1950s, Frank Rosenblatt introduced the perceptron. It could be described as the first ever operational realization of a neural network. A perceptron represents a basic ML model that was designed to aid computers in learning from a diverse range of data. In the 1960s, John McCarthy developed the LISP programming language for artificial intelligence tasks. In the same decade, the first expert systems were developed to model a human's knowledge in a certain field. For example, Dendral is the first AI expert system for the purpose of identifying the molecular structure of an unknown organic compound. Among the first functioning examples of generative AI, the ELIZA chatbot was created in 1961 by British scientist Joseph Weizenbaum. It was the first talking computer program that simulated the work of a psychotherapist and could communicate with a human in a natural language. The rise of AI: deep learning and generative AI In the 1990s and 2000s, computer processing capacity has substantially grown. The DeepBlue chess computer system defeated the world chess champion in 1997, and Dragon Systems created NaturallySpeaking, the first publicly available voice recognition system. The rise of the Internet led to what has become an explosion in the amount of data being collected and processed. In the 2000s, the processing power of computing machines has reached the level essential for dealing with enormous data flows. New technologies and concepts have emerged that support the development of artificial intelligence. Machine learning, neural networks, and deep learning have become more widely accessible and have given new opportunities to develop smarter and responsive systems. Deep learning has been growing particularly fast in the 2010s. It is a type of machine learning that employs multi-layered neural networks that self-train on a large dataset. Modern generative AI is based mainly on deep learning technique, therefore generative AI also started rapidly developing in the 2010s. First generative AI As was already mentioned, one of the first primitive generative AI was ELIZA. It was a text chat bot created in the 1960s by Joseph Weizenbaum. ELIZA was one of the first examples of Natural Language Processing (NLP) and mimicked the work of a psychotherapist and could communicate with humans in natural language. ELIZA followed a simple pattern of recognizing keywords in text to later generate programmed generic responses. The chatbot's ability to communicate created an impression that the machine could understand human speech. However, the machine interpreted all words as character data, without giving them meaning as a human does. According to the developer ELIZA was just a parody of a psychotherapist and was completely non-intelligent. Development of generative AI Generative AI is a type of AI that can create realistic images and videos, generate text or music. To achieve this, generative AI models are applied. The purpose of such models is to generate new samples from what was already in the training data. Some of the first generative models were Hidden Markov Models (HMMs) and Gaussian Mixture Models (GMMs) that were devised in the 1950s. They produced successive pieces of data, such as speech. For instance, for HMMs, one of the first applications was speech recognition. The productivity of generative models, though, significantly boosted only after the rise of deep learning. In the field of natural language processing, recurrent neural networks (RNNs), which were introduced in the late 1980s, are used for language modeling tasks. RNNs can model relatively long dependencies and allow generating longer sentences. Long Short-Term Memory (LSTM), a kind of recurrent neural network, was later developed. One of the fundamental breakthroughs in generative AI is the creation of Generative Adversarial Networks (GANs) in 2014 by an American computer scientist Ian Goodfellow. It is an unsupervised machine learning algorithm that engages two neural networks that are in competition with each other. One network is a generative model that generates content and the other is discriminative that tries to figure out whether it is an authentic sample or not. Another type of model that has played a significant role in the development of generative AI is the transformer architecture model. It's a deep neural network algorithm presented only recently back in 2017. Transformers have powered many generative models in various domains. Similar to recurrent neural networks (RNNs), transformers are intended to process sequences such as natural language text. The transformer architecture is applied in NLP, which has led to the creation of large language models such as BERT and GPT. Vision transformer is a combination of visual features and transformer architecture to complete computer vision (CV) tasks. Recent breakthroughs A major breakthrough in generative AI and, in particular, in the development of NPL was the introduction of GPT (Generative Pre-trained Transformer) models. In 2018, the first version of GPT was created by OpenAI. It is a neural network that employs deep learning architecture to generate text, engage in conversations with a user and fulfill various language tasks. Its creation marked a turning point in the widespread use of machine learning. We can now apply this technology to automate and refine a diverse variety of tasks, from text translation and writing promotional materials to coding and researching complex topics. The value of such models resides in its processing speed and scale that it can operate at. GPT is a large language model built using a transformer algorithm that is trained in a self-supervised mode on a heap of textual data from the Internet. The model performs a language modeling task, i.e. predicts the next word (or part of a word) given the previous context. In 2023, GPT 4 was released, capable of generating up to 25,000 words of text, which is a significant improvement over previous versions. DALL-E is a machine learning model created in 2021 by OpenAI that generates photorealistic images from textual descriptions. GPT-3 has become the foundation for the creation of DALL-E in 2021. It is capable of generating high-quality and realistic images, as well as being able to perform some additional functions, such as adding, replacing, or removing certain objects, or generating alternative variants of a given image according to a textual description. The important aspect is that the model is continuously trained on new data.