Computational Creativity

Introduction

This textbook explores means by which artificially intelligent systems exhibit behaviors that can be regarded as creative, in areas such as visual arts, music, storytelling, non-fiction narrative, architecture, product design, and computer programming. Such systems are tools for humans to advance human creativity, but increasingly these systems are better viewed as collaborators with humans in hybrid human-machine creation. Creativity will also be found in large collectives of agents, both human and machine, and creativity will be found in general AIs too.

The text is intended to support a standalone undergraduate course in computational creativity, or to augment a course in artificial intelligence, machine learning, cognitive science, and other disciplines. The book may also be used in conjunction with the Wikipedia entry on computational creativity and related pages, as well as a host of other sources, such as Veale, et al (2020). As a textbook, however, these pages include projects and exercises, sample syllabi, and other material and experiences relevant to the teaching and learning of computational creativity. The book can also be used for self-study or other informal education by motivated learners.

The philosophy behind the textbook includes an embrace of AI systems as tools for human creativity. Most notably, AIs such as chatGPT and Midjourney have changed the AI and societal landscape in the blink of an eye. The textbook seeks to provide materials that embrace these technologies as tools and collaborators for advancing student learning in still other creativity domains, as well as address the societal implications of computational creativity. It is our intent that exercises and projects include versions where AIs openly help design, implement, document, and evaluate a computationally creative system with student prompting, or an artifact in another creative domain such as visual art, music, dance, or narrative. We also include versions of exercises and projects that are intended to limit aid from an AI, just as instructors might limit aid from other people.

An example exercise, perhaps for a course in Computational Creativity, AI, or Software Engineering, is as follows.

Background: Read "A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT" (https://arxiv.org/abs/2302.11382) or listen to an instructor lecture on its content. Assignment: Collaborate with a large language model to specify, design, implement, and document a game of logic that uses knowledge of Boolean operations. The implementation can be in a programming language of choice. The game is intended to practice players in the full breadth of Boolean operations, as well as concepts such as minimal sets of connectives for expressing all Boolean functions. Submit (a) a paragraph description of the game and its rules, (b) an architectural design of the game program that involves major functions (e.g., moving, display, win/loss) and header comments for these functions, (c) a well commented code listing, (d) sample plays between a user and an AI, or two humans, or two AIs, (e) a summary listing of critical prompts used throughout the development process and their relationship to the prompt patterns found in the reading, and (f) comments on issues of interest, such as the appearance and impact of any apparent misconceptions on the part of the AI.

Before launching into material on various areas where computational creativity are relevant, we touch on a general introduction to the wide-ranging field.

Characteristics of creativity

Creativity, of various types and degrees, characterizes processes of creation. Creativity is a characterization in a multidimensional space relating to the extent to which a process (as embodied by some kind of agent) explores alternatives through thought and action, evaluates those alternatives, imagines what are possible results of actions, and empathizes and otherwise mentalizes the goals, capabilities, and feelings of other agents. We will not treat creativity as binary valued, but we will often implicitly reduce the multiple dimensions of creativity into a single dimension to talk about creation processes as more or less creative.

The results of creativity can be objects or thoughts, and the results are often characterized by varying kinds and degrees of novelty and value – how distinct is a result relative to a collection of other objects and imaginings of similar function, and what is the result’s pragmatic utility, beauty, or interestingness. You will sometimes hear inanimate objects referred to as creative, like “that is a creative vase”. You can assume that this is shorthand for saying that the vase has high novelty or value; or that the process that created that vase did significant exploration or used novel evaluation metrics, etc. If we call an agent creative it suggests that the agent, a human or AI artist or engineer, often carries out creative processes of creation, or produces results that have high degrees of novelty and value. This is the sense in which you will hear 'creatives' as people who create.

Is it possible for a creative process to produce mundane results? Sure, in large part because creativity lies in the eye of the beholder as well as the creator, and the space of objects against which the spectator evaluates a creation in terms of novelty and value is perhaps wider than the creator’s experience. The creator may have carried out processes that were quite unique to them, but not to a larger society. This distinction has been referred to as individual creativity and historic creativity, and of course it generalizes to a continuum between the individual and society as extremes.

Creativity occurs at different scales of time and activity, as well as of agency. It occurs in the operations at an assembly line station, however small we think of it (remember, creativity isn’t a binary), and at the level of the entire assembly line or some alternative paradigm for holistic construction. Non-human animals are creative, both as individuals and collectives, as are AIs, or so that is the assumption of this text. The topics mentioned here – exploration (search) vs exploitation, evaluation using measures such as novelty and value, projection or imagination, and scales of agency, time, and space are all amenable to computational treatment.

Computational Creativity and AI

Computational creativity is an area of artificial intelligence (AI), though some would say understudied. Most AI systems today, including computational creativity systems, are very focused and narrow applications, such as creation of visual art from narrative prompts, story writing, and music composition. In principle, such functionality in human affairs are very broad, drawing from the human creators' wealth of experience, but it could be said that the AI systems that mimic the human artists may take "shortcuts" in the creative steps, extracting features of the end results of creative processing and using these features in the creations of the AI. Nonetheless, the processing of computational creativity systems is impressive, typically in its processing of massive data sets that are typically the results of human creation. Some systems are multi-modal (e.g., taking narrative prompts as input and producing visual art, or vice versa from images to captions), which places them a step towards systems-level AIs. Systems level AIs include "intelligent" vehicles, Watson, and home assistants like Alexa. Further up the dimension of autonomy are person-comparable AIs, and these are still the stuff of science fiction.

One reason that is often given to interrogate your own thinking as you work on creativity and computational creativity projects is so that you will develop insights into the component processes that you use and coordinate in completing each project, and tasks like them in other parts of your life. Each of the component processes may themselves be decomposed further, and implemented by existing AI tools, at least in part. The hope of course is that you will imagine ways of implementing the sub-processes and complete process computationally so that you can be leaders, and the wisdom, in the wave of systems-level and person-level creative AIs that will probably come.

Our assumption is that creativity pervades intelligent processes of all kinds in human systems, and we will be discussing ways that it might pervade AI systems too. Learning, like creativity, is also a pervasive capability in human systems, and AI systems too, but even now (machine learning has been explored since the 1950s), most machine learning lies on the narrow end of the generality dimension. But learning in humans and AIs is a process, albeit spread over longer time scales, by which an agent explores and evaluates alternatives. Cognitive architectures and cognitive systems are areas of AI that study the processes of system-level AIs. We will often structure our discussions of creativity in systems-level AIs by considering the requisite capabilities outlined in our cognitive architectures reading. It is the goal of the book to describe the computational underpinnings of computational creativity, as well as describing the wealth of application areas for computational creativity.

And now for areas of computational creativity!

Curation

We interpret ‘curation’ broadly, as concerning collections of many types of things by novices and experts alike. Curation includes component activities of selecting, displaying, and educating about collections. Curation is an act of creation, as in creating an exhibit, and it begs some degree of creativity. Because curation involves collections of discrete things it is a good class of activity for introducing the utility of computational approaches. Our goal is to not simply talk about existing computational tools for carrying out curation, if any, but to imagine how computational tools might be applied to curation subtasks. This theme of discussing “what is” and “what could be” will run through other topics as well, though we may be less likely to know how to imagine the “what is possible” with respect to other media, though you may know more and are encouraged to "speak up" here on Wikibooks.

Suppose that a building for curating artwork has already been created, like the West Building of the National Gallery of Art, and a collection of paintings and sculpture is on site. A curator knows the dimensions of rooms, heights of walls, measurements of paintings and sculptures, and lots of metadata like artist, year, region, etc. What are the ways to organize the collection? Can the same methods and metrics be used to display gardens, zoos, natural histories, etc?

A basic AI functionality that might be useful for a computational curator is state-space search, where the states can be vertices in an explicit graph like a road network, or the states can be nodes or vertices (aka states) that are generated on-demand from an implicit graph. The "vertices" or states in AI problems are most typically quite complicated knowledge structures that are part of an implicit graph. State-space search is most typically used to explore alternatives in the 'problem-solving and planning' and 'reasoning and belief maintenance' functionality classes of a cognitive architecture.

Another basic AI strategy is representation and learning with artificial neural networks (ANNs), which are typically huge collections of simple interconnected processing units (akin to individual neurons). Powerful computation stems from the interconnection of these units and their coordinated activity. In our first treatment of ANNs the functionality that we discuss is most appropriate to three cognitive architectures functionality classes: 'recognition and categorization' , 'perception and situation assessment', and 'prediction and monitoring'.

Narrative

Interactive

Mark O. Riedl and Vadim Bulitko (2013). Interactive Narrative: An Intelligent Systems Approach, AI Magazine, Spring Issue, 67—77.

“The Story of Facade: The AI Interactive Drama“

Music

“Artificial Intelligence and Music Generation Systems: An Introduction to the State of the Art” (2020), Frontiers of Artificial Intelligence, Vol. 3. (This is long, so start early, and its one reason you don’t have an actual quiz on Wednesday of this week.)

The DALL-E of Music?

Shimon improvisational drummer videos

Dance

AI Choreographer: Learn to Dance with AIST++ Music Conditioned 3D Dance Generation

Performance Art

Affordance-based Generation of Pretend Object Interaction Variants For Human-Computer Improvisational Theater, pp. 140-147 of Proceedings of the 10th International Conference on Computational Creativity