Can Computers Be Creative?
A Deeper Technical Investigation into Machine-Generated Novelty.
Introduction
The notion of machine creativity has sparked intense debate and speculation in both the fields of artificial intelligence and philosophy. Central to this debate is the question of whether creativity is merely the ability to produce useful novelty, and whether such an ability necessitates consciousness or intention. This article will focus on the narrower, yet profound question: Can computers create novelty? This inquiry, steeped in historical and contemporary perspectives, challenges our understanding of both artificial and human creativity.
Historical Context: The Lovelace Objection
Ada Lovelace, often celebrated as the first computer programmer, posited that computers are incapable of originating anything new. They can only execute tasks they are programmed to perform1:
The Analytical Engine has no pretensions whatever to originate any thing. It can do whatever we know how to order it to perform. It can follow analysis; but it has no power of anticipating any analytical relations or truths. Its province is to assist us in making available what we are already acquainted with.
This skepticism towards machine creativity was later termed the "Lovelace objection" by Alan Turing (Turing, 1950). Turing, a pioneer of computer science, recognized this objection as a significant hurdle in the conceptualization of machine intelligence.
Turing's Prescient Vision: Self-Learning Machines
Turing, in 1948, proposed the possibility of machines that could learn autonomously2.
This was in his NPL report “Intelligent Machinery”, which was in effect the first detailed manifesto of artificial intelligence (Turing, 1948). “Intelligent Machinery” contains the first known suggestion that computing machines can be built out of simple, neuron-like elements connected together into networks. Turing proposed that networks could be “trained” (his term) to perform specific tasks, and he foresaw the need to develop training algorithms, now an important field;
And he pointed out how important learning is for the generation of novelty:
In responding to what he calls “Lady Lovelace’s objection” (Turing, 1950, p. 450), the idea that computers are deterministic and thus incapable of originality, Turing considers the role that learning might play in intelligence, and especially in the generation of novel or surprising behaviors.
This concept revolutionized the idea of what machines could do, blurring the line between a machine being explicitly instructed and its capability to learn independently. Modern machine learning, particularly in the form of Language Learning Models (LLMs), exemplifies this vision. These machines develop skills and produce outputs that are not directly anticipated or programmed by their human creators.
Analyzing LLMs: Beyond Stochastic Parroting
Language Learning Models, like GPT (Generative Pretrained Transformer), have often been criticized as mere "stochastic parrots." This criticism is rooted in the belief that these models only generate variations of pre-existing texts. However, this view is an oversimplification. Stephen Wolfram's3 analysis highlights that the quantity of potential outputs from an LLM vastly exceeds the number of existing texts. LLMs function by estimating a conditional probability distribution4 for the next token in a sequence and then sampling from this distribution. This process inherently generates novel token sequences, which can manifest as unique texts, code, music, or even images.
The Role of Randomness in Creativity
This method of generating novelty parallels the randomness found in evolutionary processes. The concept of the "Blind Watchmaker,” as articulated by Richard Dawkins, illustrates how random genetic variations can lead to the vast diversity of life on Earth. This process, governed by natural selection, is essentially a random yet guided mechanism that has created complex and novel biological structures and behaviors.
The Dichotomy of Randomness and Determinism
The discussion of randomness in the context of machine creativity leads to a broader philosophical question: Is our world fundamentally deterministic or ruled by randomness? Quantum theory introduces the concept of inherent uncertainty at the microscopic level, known as ontic randomness. Conversely, chaos theory demonstrates that even deterministic systems can exhibit unpredictable behavior due to their sensitivity to initial conditions5. This unpredictability in deterministic systems is a crucial factor in discussions about machine and human creativity.
The Mechanistic View of the Brain and Its Implications
If we adopt a mechanistic view of the brain, aligning it with deterministic physical processes, the notion of human creativity and free will comes into question. This perspective compels us to reconsider our definition of creativity. Are human creative endeavors truly novel, or are they the result of complex, deterministic, yet unpredictable brain processes?
Conclusion
In summation, when we enable computers to access sources of input beyond the initial programming, especially elements of randomness, they gain the potential to produce novelty that is not preconceived by their programmers. The evolutionary process serves as a testament to the richness of novelty that can emerge from mechanistic systems governed by random variations. If we redefine randomness as unpredictability within a deterministic framework, we must also reevaluate the concept of human creativity if we consider the brain to be a mechanistic entity. Thus, the potential for computers to exhibit creativity, in their unique mechanistic and unpredictable manner, emerges not just as a theoretical possibility but as a tangible reality. This perspective not only reshapes our understanding of machine intelligence but also challenges our perceptions of human creativity and ingenuity.
Countess of Lovelace, ‘Translator's notes to an article on Babbage's Analytical Engiro’, Scientific Memoirs (ed. by R. Taylor), vol. 3 (1842), 691–731.
Sprevak, Colombo (ed.), The Routledge Handbook of the Computational Mind (2019)
Stephen Wolfram, What Is ChatGPT Doing … and Why Does It Work? (2023)
This is essentially what all Generative AI models like GANs, Diffusion models, LLMs etc. do - they statistically estimate a conditional probability distribution. Generation means sampling from this probability distribution.
This apparent randomness due to unpredictibility is called epistemic randomness.