Experience Driven Design of Creative Systems

The key contribution of this paper is to describe and demonstrate a novel application of grounded theory to the analysis of a human/machine music performance. Rather than attempting to measure the ‘creativity’ of our machine improviser, we instead proposed an investigation of the experiences of humans - in this case the designer, the performer and the listener. We report the design of an AI system chosen to perform in a specific creative context - a jazz-inflected musical performance in this case - and explore the specific experiences of these human actors through the performance itself. The performance is one which is a commonplace one where a single human musician interacts and performs with a single autonomous system. We describe this system which improvises by training pitch and event sequence models in real time from a live audio input and then uses a riffing behaviour to generate output in the form of note sequences with varying timbre. However, the main thrust of this paper is to propose a new methodology for understanding the role of the system through the interplay of experiences of audience, designer and performer throughout the performance, and describe how our time based media annotation system can be used to support that methodology. We present the results of this grounded ontology methodology applied to the text-based commentaries between system engineer, performer and listener. We argue that by developing an understanding of these inter-related experiences we can understand the desired and potential role of computational systems in creative contexts which can help in the design of new systems and help us curate new kinds of performance scenarios.