Vol. 34 Issue 3 Reviews

Gerhard Nierhaus: Algorithmic Composition: Paradigms of Automated Music Generation

Hardcover, 2009, ISBN 978-3-211-77539-6, 287 pages, illustrated, 20 tables, index, €49.95; available from Springer-Verlag GmbH, Sachsenplatz 4-6, P. O. Box 89, 1201 Vienna, Austria; Web www.springer.com/springerwiennewyork/mathematics/book/978-3-211-75539-6.

Reviewed by Paul Doornbusch
Melbourne, Australia

Book CoverThere is little doubt that algorithmic composition really came into its own as computers became less expensive and more accessible. Formalizable and mathematical approaches to musical composition have had a longer history, but were less intensely applied before the development of commodity computers.

Gerard Nierhaus is a composer who teaches Algorithmic Composition and Computer Music at the Institute of Electronic Music and Acoustics (IEM), University of Music and Dramatic Arts, in Graz, Austria.

This book necessarily has a mixed content of music and mathematics, but the audience appears to be firmly students of algorithmic composition, and that includes beginning students. When I checked, the publisher had the book listed in its mathematics section, which is misplaced, I think, as the level of mathematics in the book is really only what a music student would desire. Despite its worrying problems with details (more later), students have a good basic text that will explain the fundamental ideas of style imitation in algorithmic composition. Although I find it short on real examples of interesting music, and perhaps long on superficial examples, the book does manage a broad overview of musical style imitation techniques and brings together a large range of disparate literature that has a bearing on the subject.

The introduction goes through the chapter layout of the book and discusses its intent. One of the limitations it takes on, and one that disappointed me, is that the works of individual composers, or individual works, are not dealt with. The book presents an overview of common or prominent methods of algorithmic composition in a systematic way, discussing their features, particularly for style imitation. After a rather extended historical overview, the chapters each discuss Markov models, generative grammars, transition networks, chaos and self-similarity (fractals), genetic algorithms, cellular automata, neural networks, and artificial intelligence.

The historical overview goes back to the Renaissance and further, discussing the development of philosophy, the abstraction and representation of numbers and counting in different cultures (back to about 3,000 B.C.), and so on. While the development of algorithmic thinking may depend on these being in place, I found the chapter overlong and the details and level of treatment boring and mostly irrelevant to the immediate goal. It may, however, be fascinating to students, and it certainly provides key developments and the names of those involved. The discussion on calculating machines is quite thorough, covering developments by John Napier, Blaise Pascal, and more, through to the handheld mechanical calculators of the early 20th century and including Charles Babbage’s analytical engine. The history of modern mathematics continues, discussing key ideas about the formalization of logic and the development of the modern computer. The historical discussion of computers seems rather idiosyncratic, initially, as it does not define the requirements of such a machine, and secondly as it claims that Konrad Zuse constructed the first computer. Modern computer history texts (for example, Electronic Brains: Stories from the Dawn of the Computer Age by Mike Hally, and others) say that Zuse created one of the first electromechanical digital calculators, and the distinction is significant because the definition of what exactly constitutes a computer is important. This book fails to notice the distinction, and many other aspects of computer history, and also manages to misspell the names of several key people or give them incorrect titles. The subsequent discussion on what can be computed is somewhat confused but the general model of a computer is useful. The chapter ends by introducing the computer in algorithmic composition and briefly mentions the work of Lejaren Hiller and Leonard Isaacson, Max Mathews, and Iannis Xenakis.

Each of the following chapters adheres to a similar structure: some history, the underlying theory including some basic mathematics, then a more detailed discussion citing examples, and a summary to close the chapter. This makes it easy for a student to find what they might want in the book, without reading too much, and it is a logical structure, particularly so for teaching. The author usefully makes the distinction early in the book between what he calls “genuine composition” and style imitation. Genuine composition is defined here as “a means of implementing compositional strategies for the creation of a new piece of art” (p. 3). Style imitation is defined as “the generation of musical material according to a given notion of musical style” (p. 3). I would propose an extension to the definition of genuine composition: a means of implementing compositional strategies with the intent of creating a new piece of art that offers a unique (and hopefully compelling) aesthetic musical experience. Students of computer music might confuse these two aspects of the practice, genuine composition and style imitation, so the distinction is important to make. However, it is not expanded upon later in the text, and this may leave readers confused. Each chapter ends with a summary, some discussion of the strengths and weaknesses of the technique, and, often enough, some evaluation of whether the technique is more appropriate for genuine composition or style imitation, although really only style imitation is discussed. A list of references is given at the end of each chapter, and these usually include some of the standard material, although by no means are all of the standard or classic references mentioned. There seems to be much fine research omitted, particularly research on genuine composition, but there are also other references included, many from other disciplines, which are very good to see.

The chapter on Markov models begins, as do most of the chapters, with a history and theory section, this time on the conditional probability system. Several typographical errors (incorrect arrows in a diagram, and obvious misspellings) mar the beginning of this chapter. After briefly mentioning the work of Hiller and Isaacson, and Xenakis, it goes on with variations of Markov models and spends several pages on their use in generating imitations of musical style. The book mentions in the initial introduction that for genuine composition with Markov models one needs to look at Hiller and Xenakis, but curiously it does not mention this point in the chapter itself, and the chapter barely spends two short paragraphs discussing their work.
Chapter 4 on generative grammars is the largest in the book at 37 pages. The initial history and theory section takes several pages and is dominated by the linguistic history of generative grammars and Noam Chomsky’s hierarchy of these. Some similarities between generative grammars and Markov models are made, as generative grammars are a hierarchical system of semantic units, and also discussed are the limitations of applying linguistic rules to musical composition. The chapter continues with the use of generative grammars in style replication and (standard) music analysis, particularly Schenkerian analysis, and goes on to examine in detail their use in style imitation for ethnomusicology and jazz.

Transition networks are covered next by the book. The similarities with generative grammars are noted, so again this chapter concentrates on musical style, with particular emphasis on the work of David Cope. Petri nets are also discussed, with their similarities to Markov models noted.
The beginning of the chaos and self-similarity chapter gives an appropriate background on the area, neither too long nor too short, and mentions the main players and concepts. A rather detailed discussion follows of fractals, Lindenmayer systems, and the concept of fractal noise is quickly covered. The rest of the chapter focuses on the application of Lindenmayer systems to musical content generation, usually pitches, with several examples in standard notation.
Genetic algorithms are an outcome of biological evolution theory, but their introduction within their chapter covers them in a highly computer-centric manner. Musical applications are discussed comparatively early, and a key element of genetic algorithms, fitness assessment, is discussed in detail. One of the points noted is that with genetic algorithms it is difficult to define fitness criteria for generated material that lies beyond the stylistic limitations of the starting material. Rhythmic generation receives its own section in this chapter, whilst it is more woven into the fabric of other chapters, perhaps because so much work with genetic algorithms is dominated by pitch. The chapter continues with the use of genetic algorithms in style replication, for example jazz, and in interactive real-time systems. Before closing, artificial life-forms are discussed where algorithmic composition is a by-product of artificial life-forms “living” and “interacting” in an artificial environment, and the possibility of using genetic algorithms to synthesize sound is unfortunately only raised and is considered as being beyond the scope of the book.

The introduction to the cellular automata chapter is straightforward and one is quickly led into the theory of one-dimensional cellular automata (CA) with minimal confusion. After a discussion of two- and three-dimensional CA, extended types are mentioned (continuous automata, n-dimensional CA, chance operations, and so on) before moving onto CA in algorithmic composition. A range of research in the area is examined, mostly from within the last 20 years. The claim that Peter Beyls was the first to use CA to generate musical structures is curious given that the earliest reference I can find to this is from 1989, and Iannis Xenakis used CA in many pieces (for example, Ata and Horos)from 1986 at the latest, and even this may not have been the first. The lack of any mention of Xenakis in this chapter is difficult to understand given how famous he is for using CA. Further discussed is the application of CA in real-time interactive systems, and the summary notes the difficulty of predicting the behavior of cellular automata as well as the difficulty of mapping the output to musical parameters. The decision not to negotiate the terrain of genuine composition in this chapter is regrettable given how much published research is available on it.

Using another analogy from biology, artificial neural networks model the structure of the brain. The introduction to this chapter leads smoothly from the basics and biology to computational models and includes such details of neural networks as connection weighting and network training. The application of artificial neural networks to algorithmic composition is initiated by a discussion of using them to harmonize melodies in the style of J. S. Bach. The discussion continues through a system to produce folk melodies, more work in the style of Bach, waltzes, and more. It notes how artificial neural nets are “superior” than Markov models for such endeavors, but there is no explanation of what “superior” means in this regard or of the testing or decision-making process used to make this claim. The end-of-chapter synopsis notes that artificial neural networks may be useful for automatic musical classification, and also notes the difficulty of training one to produce large-scale works. In addition, it draws attention to the fact that the output of an artificial neural network is practically never assessed for musical quality—the fact that there is an output is considered a success, regardless of how “good” it may sound. In closing, the author notes how artificial neural networks can achieve surprising results outside of the domain of their input, compared with generative grammars and Markov models.
Artificial intelligence (AI) is a logical progression from artificial neural networks and it is the last chapter before the final synopsis of the book. The chapter begins with a brief introduction and then a discussion of the main concepts as manifest in the ELIZA program before getting onto the applications of artificial intelligence to algorithmic composition. ELIZA is one of the earliest examples of an artificial intelligence program, created in 1966 by Joseph Weizenbaum; it will probably be a mystery to most computer users who are less than 40 years old. However, as ELIZA is a parody of a conversation with a psychotherapist and is based on keyword selection and rephrasing them into questions, it is considered a poor example of artificial intelligence (as exemplified by the Turing Test) and somewhere between a hoax and a gimmick in AI circles—even Weizenbaum was dismayed that people thought so much of it, so its dominance of this chapter is puzzling. A comprehensive discussion of Turing Tests in algorithmic composition can be found in the CMJ article titled “The Interrogator as Critic: The Turing Test and the Evaluation of Generative Music Systems” by Christopher Ariza (33:2, Summer 2009). The chapter continues with a discussion of what might constitute musical intelligence and a comparison between ELIZA and David Cope’s Experiments in Musical Intelligence system. Knowledge representation as a key concept in algorithmic composition and artificial intelligence systems is discussed, as are state spaces, search algorithms, and heuristic systems, citing the (groundbreaking at the time) mid-1980s work by Roger Dannenberg on automated accompaniment. Reasoning and rule-based system are covered in some detail and there is an overview of cooperating intelligent agents to round out the chapter. Artificial intelligence is an enormous field of study and this chapter does a decent job of covering the main ideas of the discipline and how they might relate to musical composition, although only in the context of style imitation.

The final synopsis starts by noting that different algorithmic methods might be used for different parts of a composition and that algorithmic composition is useful for style imitation or genuine composition. The notion of genuine composition is subsequently diluted by saying that it is dependent on the social context, or historical period of its creation, and that composers may have a proprietary style that they imitate. This may be valid for varieties of popular music, but it is invalid for what is often called “contemporary art music.” It then goes on to say that the procedures described in the book are used mainly for style imitation, and at least part of the reason for this is the field the author works in. I think the claim that the tools are used mainly for style imitation is unjustified, as it ignores a canon of algorithmically composed music which is original, successful, and compelling, from such composers as Clarence Barlow, Herbert Brün, Charles Dodge, G. M. Koenig, Iannis Xenakis, and so on, all of which have well documented algorithmic methods. For examples of references on genuine algorithmic composition, one can find many in Curtis Roads’ Computer Music Tutorial, and Christopher Ariza’s excellent treatise, An Open Design for Computer-Aided Algorithmic Music Composition: athenaCL, has 37 pages of references and a large selection of these are involved with genuine algorithmic composition. More philosophical algorithmic composition issues are examined in more detail as the chapter continues and other key ideas are discussed such as what constitutes musical style, to what extent algorithmic procedures are used in genuine composition, and how some forms of algorithmic composition require critical appraisal of the results as part of the creative process. The implication that some results are not critically assessed is extraordinary, and begs the question if composition of any kind, algorithmic or not, is possible without critical appraisal. Fortunately, this is expanded upon later.

The chapter also discusses the limitations of algorithmic composition and lays some of the blame for this on the limitations of the MIDI protocol, without mentioning that this is easily overcome by using Open Sound Control (OSC) and that many systems work happily with it (AC Toolbox, Max/MSP, Pure Data, Supercollider and so on), but MIDI is probably quite adequate for most style imitation purposes. Another way to overcome the limitations of MIDI is to use Csound output, and the previously mentioned systems and more offer this possibility, has existed for years even before MIDI as Music-N language output from historical algorithmic composition systems.

A later discussion on how some traditional instrumental compositional forms (such as fugues, motets, and so on) may be seen as algorithmic composition because they use rules, seems to miss the point that composers historically have always gone beyond the rules, or broken them, in their work—for example, it is sometimes the parallel fifths and octaves in Bach’s music which help to make a piece sound so good. The chapter tries to claim that algorithmic composers do not readily publish their methods and that the dominance of style imitation is a by-product of the publication of researchers, but the claim seems incorrect as there is a wealth of information on at least Xenakis and Koenig and every composer whom I have asked has readily been willing to discuss how their music is composed. Besides, can it really be that the volume of published material is more important than the quality of it or the importance of the ideas discussed? The justifications for not discussing genuine composition in the book do not stand up to scrutiny. While it is clear the book focuses on style imitation, arguments justifying this such as the proposed limitations of MIDI for algorithmic composition and the lack of literature on genuine composition are incorrect.

The section on strategies for data encoding and musical mapping are very welcome as mapping in particular is an often-neglected part of the process. The encoding and mapping discussion concentrates on pitch mapping and does not approach other musical elements or concepts, for example event density or shape operations, but rather discusses the problems of music encoding and data mapping for style imitation. Similarly welcome is the section on the evaluation of material. However, style imitation issues also dominate the arguments here. The arguments for algorithmic machine-assessment of generated material are interesting, but it makes me feel that the word “composition” in the term “algorithmic composition” is somehow being misused if human assessment is deemed too unreliable to be suitable for research purposes, as we are probably quite some way off asking a machine if our (algorithmic) compositions sound any good or not. The previously mentioned CMJ article on Turing Tests notes that, “use of the TT in the evaluation of generative music systems is superfluous and potentially misleading; its evocation is an appeal to a measure of some form of artificial thought, yet, in the context of music, it provides no more than a listener survey” (p. 49). Making algorithmic composition into a system or game of data manipulation to match a style criteria, the results of which are also assessed by the machine, seems to miss the point of musical composition: perhaps it needs another name. This section on assessment, and indeed most of the book, seems to contradict the concluding remarks of the chapter, which reads: “However—algorithmic composition is not a musical golem, usurping creativity from the human realm. The algorithm is a tool and means for the creative examination of the complex aspects of musical production” (p. 272). If musical creativity is so important, why is the emphasis in this book on style imitation rather than the greater creativity of genuine composition?

What would have been an interesting discussion in this final chapter is one focusing on how music theory and rules are often reductive in nature, taking common elements from a number of compositions or parts of a single composition, to make generalizations and propose the rules used. Musical composition, on the other hand, is at root an expansive and creative activity, often going beyond rules to achieve a desired aesthetic result. However, this is not addressed anywhere in the book. Despite much discussion of the semantics used to describe the various possibilities and varieties of algorithmic composition, the final chapter is an important one, and it really should appear at the beginning of the book, as it would make much of the book more understandable in its light, especially for students.

The basics of the key tools of algorithmic composition are adequately covered in this book, and as such it does a mostly competent job of presenting them fairly clearly, but only in the realm of style imitation and despite many detailed errors. One of the things I found while reading the book was that I wished for an included disc, or even a Web site, with the musical examples available for playback so that I could quickly listen to them. I hope that the publisher and author consider this, as it would improve the book as a teaching resource. I also found the numerous errors, misspellings, and typographical errors to be distracting, and potentially worrying.
I approached this book with a large degree of enthusiasm and optimism, unfortunately I left it feeling disappointed. The emphasis on style imitation in the book and the lack of discussion of “genuine composition” is, I feel, a great shortcoming. There was an important opportunity here to go beyond style imitation—to me, as a composer, it is the least interesting or useful aspect of algorithmic composition—and delve into the more difficult realm of creative composition of compelling musical works. Furthermore, the book does not delve into algorithmic sound synthesis (surely still micro-composition), or examine in detail how some composers have used algorithmic principles to create great works, let alone attack the core issue itself. While the systems outlined in the book may be applied in a range of ways from the superficial and trivial to the very sophisticated, nowhere is the problem addressed of what musical composition really is. This has been a problem with some other books on algorithmic style imitation, but to my mind, musical composition is something other than the application of a set of rules (particularly music theory rules), and it goes beyond this. Style imitation already has a large body of literature; I feel this book does a good job of pulling much of it together, but it has missed an opportunity to go further and add to the literature on algorithmic approaches to genuine composition.