We will be issuing new CFPs early in 2012 for publication in 2013. Intense discussions are in process. These CFPs may concern media/climate change/environment issues, publishing itself, post-network politics and/or media business models (what these become if one “subtracts” financial Capital from the equation).

So please return around February/March 2012 or subscribe to our feeds. We are of course open to suggestions from the Fibreculture community (which includes you, if you’re reading this!). Our current guidelines (including those for Guest Edited issues) are here.

We are also working on our guidelines for future issues and processes. These will be tightened and streamlined somewhat in order to make us a little more sustainable in terms of workloads.

More importantly, we will be spending some of 2012 thinking deeply about what processes might best serve what has become the rather interesting complexity that is the publishing of humanities/contemporary media/transdisciplinary research right now. There has been such a rich proliferation of developments in publishing since the Fibreculture Journal began in 2002 (some of the most exciting now emerging from our publishers, Open Humanities Press). The entire Open Access movement seems delightfully to gone mainstream and there are a series of alternative movements already pushing things along even further.

Again, we are very open to suggestions from the Fibreculture community about where we might go next (although we have lots of ideas already!).

Incidentally, 2012 will be our 10th year in publishing! So I will indulge in a few short remarks about the past and future to finish.

As Editor, I think all those involved can be justifiably proud. I also remain deeply grateful to the many, many people who have been involved with the Fibreculture Journal over the last decade. By the end of 2012, we will have published 20 plus issues, with around 160 high quality articles. Given that we have a reasonably high rejection rate that’s something over 600 referees’ reports, along with 33 Guest Editors, and around 170 authors, all of these from all over the world. Our Board and Editorial Committee also have been incredibly supportive. We’ve become quite a crowd, and that’s before you get to our (we think tens of) thousands of readers. Of course, I must once again thanks our Journal Manager Mat Wall-Smith, and our previous Manager, Lisa Gye. Without them, none of this would have occurred, and none of the activity-to-come would even begin to be possible.

In short, because of all the work done by so many, everywhere I go now people seem to know the Fibreculture Journal. It’s been a wonderful and incredibly rewarding experience for me personally I must say.

Yet all has not been perfect. Let me try and turn my only real gripe, and it is big one, into something positive.

The gripe? It has to be said that many of the major research organizations/institutions around the world increasingly seem determined to treat those who actually produce journals with a kind of contempt (of course, they’re not all that fantastic in their treatment of authors/researchers either). Worse, they do this at the very same time they rely heavily on the work people do to produce journals. For example, they rely on journals to provide the central plank of research evaluation schemes and the like. It’s as if journals just appear out of the air, asking to be subject to some ridiculous form of bureaucratic subservience (yes, I’m angry about this, having been through multiple versions of it over nearly five years now). More generally, it’s as if what I and many others think is the quite wonderful flourishing of publishing is something deeply suspicious, always to be reduced to one or two major journals in pre-defined (and therefore often outmoded) “fields” (FCJ has even been one of the one or two journals on occasion, even if on another we have been classified as a “textiles” journal).

How to turn this into something positive? To put it simply, while of course we will retain our high academic standards, and will continue to double blind peer referee, etc, in other respects it is time to leave much of this institutional backwardness behind. If we’re thinking hard about this in 2012, it’s about how to do so. We’re thinking about how to embrace the energy, enthusiasm, deep social commitment and often cutting edge expertise of our communities in order to publish research that engages with key contemporary issues. We’re thinking about how to publish in a manner that gives that research the best chance it can have to engage with these issues. We’re thinking about how to further collaborate and cooperate with the flourishing of open access, online publishing. We’re thinking very deeply about how to avoid getting involved in fake competitions and false “excellence”. In short, where the institutional relation between research and the world falls short, or is counter-productive, we’re very happy to walk the other way with all our many friends (increasingly, we think, the majority).

Fibreculture Journal 19 deals with ubiquitous or pervasive computing, and with the recently emerged ambience of information. The issue details practices involving pervasive computing, both everyday practices and new art or dynamic architectural forms. Alongside these more practical concerns, the issue thinks through and with ubiquity. It takes ubiquity as something that has already arrived, something of immediate and increasing power. Simply put, ubiquity, by nature, is something with which we more often have to negotiate. Ubiquity’s ambiguous powers and possible futures also become key concerns. Finally, several articles consider the way that ubiquity might lead us to reconsider not only our past relationship to computing, but perhaps the very nature of computing and the human, vis a vis each other.

The issue was edited by Ulrik Ekman.

Articles include; Ulrik Ekman’s comprehensive introduction to the issues and ideas that move through ubiquity; Mette Ramsgard Thomsen and Karin Bech’s ‘Embedding response: self production as a model for an actuated architecture’; Anders Michelsen’s ‘ Pervasive Computing and Prosopopoietic Modelling: Notes on computed function and creative action’; Simon Penny’s ‘ Towards a Performative Aesthetics of Interactivity’; Christian Ulrik Andersen and Søren Pold’s ‘The Scripted Spaces of Urban Ubiquitous Computing: The experience, poetics, and politics of public scripted space’; Bo Kampmann Walther’s ‘Reflections on the Philosophy of Pervasive Gaming: With Special Emphasis on Rules, Gameplay, and Virtuality’; Matthew Fuller and Sónia Matos’ ‘Feral Computing: From Ubiquitous Calculation to Wild Interactions’; Malcolm McCullough’s ‘Toward Environmental Criticism’; Jonas Fritsch’s ‘Affective Experience in Interactive Environments’.

Lone Bertelsen, Ingrid Richardson and Andrew Murphie provided last minute copy-editing assistance. As always, we are deeply indebted to our anonymous referees.

More on issue 19 here.

Introduction

Digital technologies in new interactive environments are radically affecting the way we experience and make sense of the world. The advent of ubiquitous computing in particular has led to the development of advanced sensor technologies and microchips, moving the realm of computing from the desktop computer into broader contexts of interaction or interactive environments (Weiser, 1991). It seems, however, that there has been a shift from the original vision of ubiquitous computing in the light of the possibilities that a ubiquitous computational infrastructure has actually been shown to offer. Mark Weiser’s initial ideal of disappearing computers, so the technology resides in the background to let us focus on more important things, is no longer the only way of working with ubiquitous computing and technologies. Among others, Yvonne Rogers has proposed a new agenda for the design of UbiComp technologies focused on the creation of engaging user experiences, where the technology does not disappear but is actively used to create particular experiences (Rogers, 2006). In line with this, Petersen insists that objects and environments embedded with interactive technologies are becoming remarkable objects of lifestyle and identity (Petersen, 2004).

This tendency is currently being explored in interaction design, described by Winograd (1997) as a heterogeneous academic discipline concerned with designing digital artefacts, based on an understanding of the relationship between humans and digital technology (Winograd, 1997: 161). This understanding is subject to changes due to developments in digital possibilities, human needs, and situated social as well as physical contexts (Thackara, 2001). Consequently, interaction design is forced to continually develop new theoretical, analytical, and practical tools that may contribute to the design of new interactive environments. These allow us to experiment with how we might live with the technologies in the future.

Designing for living with ubiquitous technologies implies consideration of the experiential qualities that come into play to form our experience of designed interactive environments (McCarthy and Wright, 2004). The way the experiences offered by ubiquitous, interactive environments are conceptualised in interaction design largely determines the outcomes of the design processes. It therefore becomes necessary for interaction design to develop an understanding of, and a vocabulary to describe, how interactive environments and technologies might shape our situated affective experiences.

This article introduces the notion of affect as it is developed in the philosophy of Brian Massumi as a contribution to meeting this challenge. Massumi develops a philosophy of experience which is particularly useful for investigating how ubiquitous interactive environments might engage us affectively. I will suggest here that this is primarily through the creation of fields of experience. The concept of affect itself is inherently relational. It describes the correlation between a person’s ability to think and feel and the body’s ability to act in the world. Taking Massumi’s notion of affect as a starting point for the design of experience-oriented interactive environments forces us to focus on the affective tonalities and the relational events that might emerge through the interaction.

Before entering the conceptual development, however, I take a step back to relate ubiquitous computing to recent developments in experience-oriented interaction design. This sets the scene for introducing Massumi’s work on the concept of affect. Through an analysis of the interactive installation City Voices I develop an analytic understanding of how the installation changes our affective experience, bodily activation and capacitation in an interactive environment in a public setting. Following this analysis, I outline how taking the complexity of the concept of affect seriously in the design process can make it possible to sketch out new horizons for designing affectively engaging interactive environments.

Ubiquitous Computing and Experience-oriented Interaction Design

There can be no doubt that Mark Weiser’s The Computer for the 21 st Century has been seminal for setting the agenda for most of the work done when designing and conceptualising ubiquitous computing in industrial as well as research settings. His vision of calm technologies that stay out of the way and disappear into the background, appearing only when and where we need them, has spurred a great many both practical and theoretical developments that make technologically enhanced living convenient and comfortable (Weiser, 1991; see Rogers, 2006 for a comprehensive project overview). Yet, in an article from 2006, Rogers argues that there is an enormous gap between the ‘dream of comfortable, informed and effortless living and the accomplishments of UbiComp research’ (Rogers, 2006: 405). Instead, she proposes that the field of ubiquitous computing should broaden its scope and address other goals than the seamless integration of digital technologies in our everyday living. This leads to an agenda for designing UbiComp technologies for what she terms ‘engaging user experiences’ (Rogers, 2006: 406):

The 'excitement of interaction' that Weiser suggested forsaking in the pursuit of a vision of calm living should be embraced again, enabling users, designers and researchers to participate in the creation of a new generation of user experiences that go beyond what is currently possible with our existing bricolage of tools and media. We should be provoking people in their scientific, learning, analytic, creative, playing and personal activities and pursuit. (Rogers, 2006: 418)

The ideas presented by Rogers resonate with a general concern within the respective fields of digital art, digital aesthetics and interaction design. Bolter and Gromala (2004) use the concepts of transparency and reflectivity to describe two strategies in interface design. The aesthetics of the former is concerned with the disappearance of the media whilst the aesthetics of the latter invites and forces an active reflection on the part of the user in relation to the media. Recently, Munster (2006) has argued for an active understanding of the sensational and experiential aspects of interacting with new media. These should not be hidden away in the analysis and design, but further explored in facilitating new relations between people and digital technology:

Far from disposing of the senses, new media, whether commercially realized or experimentally proposed, point toward the synaesthetic disruptions and reconfigurations of bodily capacities and functions that might be made possible by digital technologies. (Munster, 2006: 19)

Interaction design today is investigating theoretical and practical ways to uncover new contexts of use, along with new conceptualisations of users to design for. People are not always rational or striving for efficiency or transparency in their interaction with technology; happiness, reflection, provocation, and desire also play a functional role in the design of new, remarkable interfaces, products and interactive environments (Dunne, 1999). The shift is also manifested in the rise of the experience-oriented domain of interaction design. Among others, McCarthy and Wright (2004) argue that experience-oriented interaction design can lead to the creation of innovative and creative forms of digital technology with human experience as the turning point.

A common concern in interaction design today can thus be said to be the articulation of a new vocabulary for addressing experiential concerns in design work along with an exploration of what that means for the future design of ubiquitous technologies. In this article I will argue that the concept of affect as developed by Brian Massumi can be beneficial for addressing and problematising this concern. The concept of affect is not foreign to recent developments in Human-Computer Interaction (HCI) and interaction design (see Fritsch, 2009 for a comprehensive overview). However, the Massumian notion of affect has yet to find its way into the discourse. In the following, I will introduce the vocabulary proposed by Massumi and argue that it might contribute to the challenges of designing affectively engaging, experience-oriented, interactive environments.

Affective Experience

Brian Massumi works at the intersection of philosophy, cultural theory, art, interactive architecture, politics and design. In his writings, he proposes a philosophy of experience which is inspired by radical empiricism primarily as this is exemplified in the work of William James. In radical empiricism, focus is on the relations between particulars. Relations that have the same ontological and epistemological status as the particulars themselves. Massumi talks about his own philosophical project as a form of expanded radical empiricism in which ‘experience is an additive "form of transition", a continued motion of intersecting process lines: a co-motion (commotion) of mutual non-exclusion’ (Massumi, 2002: 213).

According to Massumi, experience never stops. It can never be confined to a static, objective description. Experience is dynamic. However, it is possible to look into the formation of experience – how experience works, what makes us experience – bringing into play the relational complexity advocated by a radical empiricist philosophical approach. To arrive at a description of these general conditions of emergence of experience, Massumi argues that it is necessary to investigate that which comes before an actual experience, the preindividual. The notion of the preindividual is taken from French philosopher Gilbert Simondon to denote that which has not yet taken any determinate form, but modulates the formation of experience and the potential for action in a given situation. The preindividual is felt and experienced without being registered consciously. This does not mean, however, that it does not have an important impact on what we experience consciously.

Thinking with Massumi in relation to interaction design offers a way to describe the dynamics that govern the transition from the non-conscious or preindividual dimensions of the experiential to the experienced. Thinking this way involves thinking about the workings of what Massumi in several places refers to as the field of experience (Massumi, 2002: 4). The field of experience addresses the potential for experience as much as the outcome, which is a qualified experience. Massumi specifically addresses this field through the notion of affect and the role it plays in our experiences in-the-making [1]. However, working with affect demands conceptual caution. It does not offer definite answers but rather what Massumi himself refers to as a ‘field of questioning’:

The notion of affect does take many forms, and you are right to begin by emphasizing that. To get anywhere with the concept, you have to retain the manyness of its forms. It's not something that can be reduced to one thing. Mainly because it's not a thing. It's an event, or a dimension of every event. What interests me in the concept is that if you approach it respecting its variety, you are presented with a field of questioning, a problematic field where the customary divisions that questions about subjectivity, becoming or the political are usually couched in do not apply. (Massumi and McKim 2009: 1)

When developing his notion of affect, Massumi takes as a starting point Gilles Deleuze’s reception of Spinoza’s idea of affect. This is affect as something which can be defined in terms of the capacity to affect or be affected. This definition makes it possible to think of affect in active terms, avoiding a strict focus on passive reception. Massumi stresses that the importance of the capacity to affect and the capacity to be affected are two facets of the same event. Capacitation here refers to a preindividual and virtual potential for action experienced affectively which may or may not translate into an actual(ised) action line. The power to affect and be affected governs a transition where a body passes from one state of capacitation to an augmented or diminished stat of capacitation, a transition which is felt. The felt quality of a given experience is that which characterises the feeling of the transition as the body moves from one power of existence to another, which is separate from the actual capacitation understood as potential for action. Massumi argues that positive affects are those that make us feel alive and act in the world. Negative affects have the opposite effect, reducing our possible activity in the world and making this reduction felt.

In the Massumian affective account, a body is defined by what capacities it carries from step to step. The charge of affect is not something fixed and it plays out differently in any given situation. Instead of working with pre-determined and static notions of the experiential, and in keeping with the radical empiricist approach, focusing on affective experience stresses the relations that occur in the middle of a field of experience. It is necessary to understand relational events that play out differently every time, take up the past differently, creating new potentials for the future. Massumi underlines that an account of affect has to ‘…directly address forms of experience, forms of life, on a qualitative register’ (Massumi and McKim, 2009: 1). The affective position accounts for the way an experiential field – a potentialising set of conditions of emergence – effects our preindividual but felt relation to the world. In the following section this will be explored through the concepts of affective tonality and proprioception. This will allow a more elaborate description of the workings of the preindividual.

Affective Tonality and Proprioception

Massumi develops the concept of affective tonality from Alfred N. Whitehead who argues that we always apprehend the affective tone of a given situation. The affective tonality is not something residing in either the subject or the object. It emerges in the actual encounter while simultaneously modulating the occurrence [2]:

What Whitehead calls affective tonality is something we find ourselves in, rather than finding in ourselves. An embracing atmosphere that is also at the very heart of what happens because it qualifies the overall feel. (Massumi, 2007: 82)

According to Massumi, the capacity to be affected and to affect is formed by the affective tonality of a given situation. However, rather than determining our lines of action the affective tonality is tied to tendencies that will always play out differently and event-fully in every situation. Some tendencies will actualise and some will remain in the virtual, but they still remain part of the really felt affective experience. Massumi emphasises that the body’s tendencies are also like this. They are activated as the body moves into and through situations. This leads to a relational complexity that provides conditions of emergence for more qualified experiences:

…the body is that region of in-mixing from which subjectivity emerges. It is the coming together of the world, for experience, in a here-and-now prior to any possibility of assigning categories like subject or object. (Massumi, 2009: 3)

Starting from affect in the description of the body is ‘an invitation for an indefinitely constructive thinking of embodied, relational becoming’ (Massumi and McKim , 2009: 2). The tendencies in the body take many forms ‘as instincts, inclinations, teeming feelings, masses of memories, conscious and nonconscious’ (Massumi and McKim, 2009: 2). The question becomes one of exploring how this crowding is moved into a constitution, as a qualified experience. This question is ultimately a question of the emergence and formation of a subject: ‘The subject of an experience emerges from a field of conditions which are not that subject yet, where it is just coming into self’ (Massumi and McKim, 2009: 3).

The notion of proprioception is also crucial to the Massumian experiential vocabulary. It is defined as the sensibility proper to the muscles and ligaments, as opposed to tactile sensibility (which is ‘exteroceptive’) and visceral sensibility (which is ‘interoceptive’). Proprioception is preindividual yet actualised, really felt bodily but not consciously reflected upon:

Proprioception effects a double translation of the subject and the object into the body, at a medium depth where the body is only body, having nothing of the putative profundity of the self nor of the superficiality of external encounter. (Massumi 2002: 59)

Massumi argues that proprioception is always a primary phase in every sensation. It is the body’s self-feeling of a posturing for movement, nonconscious by nature, barely active before unfolding into action. It can only be felt, lived out, transduced into an ensuing action-line. Proprioception is an enactive awareness of the body’s own movement, affecting the body’s actual unfolding. The body’s capacity for action or activation is therefore this bodily field of potential, where affect is what connects the virtual series with their actual event-ful unfolding as real, preindividual tendencies forming an experiential field which might transform into personally qualified experiences (Massumi, 2002). In sum, proprioception actively works in and forms an experiential background.

A Massumian approach to working with experience-oriented interaction design would account for the fields of experience offered by interactive, ubiquitous environments. It would uncover how they work affectively, offering particular conditions of emergence for the interaction. It would take into consideration the affective tonality of these environments, in terms of how they might activate and capacitate bodies as they move into and through interactive situations. It would describe the relational complexity of the events forming and being formed by the affective experience of moving through and with an interactive environment. In all this, an affective point of view necessitates looking into potential situations of emergence and the re-conditionings of the emerged in relation to becoming:

Conditions of emergence are one with becoming. Re-conditionings of the emerged define normative or regulatory operations that set the parameters of history (the possible interactions of determinate individuals and groups). (Massumi 2002: 10)

In the following, through the analysis of the interactive installation City Voices , I will try to show how the preceding vocabulary can be used to describe the affective experience offered by a particular interactive environment.

City Voices as a Field of Experience

City Voices was a citizen involvement project in the form of an exhibition. It focused on gathering values and ideas for the design of the future Multimedia House, a 28,000 m2 building with surrounding areas, to be built at the harbour in Aarhus, Denmark from 2012-2015. The exhibition was designed by the architect and interaction design firm KOLLISION (www.kollision.dk).

Figure 1: City Voices at the main Library (left) and at the Aarhus Centre for Contemporary Art (right).

The project consisted primarily of two interactive installations; one exhibited in the main library of Aarhus and one in the Aarhus Centre for Contemporary Art (Fig. 1). Each installation involved an interactive table on wheels. When one moved the tables, a map displayed on a table top screen was panned, making the table a peephole into a larger digital map. A compass was wired to the tables, so the digital map would also relate to the physical placement of the table in terms of N-S-E-W directionality. The user would navigate the digital map by moving the table around in the physical room. It was possible to choose between three different digital maps to be displayed on the table; a local map of Aarhus, a map of Denmark, and a world map. On the table, parts of the maps would be displayed that could be navigated by moving the table. On the wall the user could see the full map and the selection of the map presented on the table (Fig. 1., Fig. 2).

On each of these maps a number of scenarios were visualised on the interface. Each one contained a voice from the future discussing a personal experience in relation to the finished Multimedia House. The idea was to inspire the users to voice their own opinions and discuss their own experiences and expectations. A microphone on the table allowed the users to record their own scenario and leave it for the next user to encounter. During the process the users’ scenarios would be accumulated for all to hear along with the original scenarios. These user scenarios could be accessed through the two interactive tables and the project website.

Figure 2: People interacting with the table (left) and a close-up of the map displayed on the table,
which you could only navigate by moving around in the physical room (right).

City Voices uses ubiquitous technologies in the design of an interactive environment experimenting with the affective tonality of the public setting. The installation is a post-desktop interface encouraging bodily movement in order to explore the digital content and physical constraints of the interaction with the system. The interactive tables are not facilitating the search for information or disappearing into the background. Instead, they actively foster curiosity and engaged exploration of the content of the exhibition by providing new means of interaction with the underlying digital system, contributing to a different affective experience of the technological setup. When you interact with City Voices , you are immediately engaged in the exploration of an experiential field activated by the installation. The need to move the table around in the exhibition space activates you proprioceptively. The table itself becomes an interface between your body and the system; your bodily movement is necessary to navigate the system, and this alters the affective tonality of the interaction. You explore actual movements in space, choreographed or performed by the installation. Simultaneously, the movement is registered proprioceptively and by the system itself processing the data. There is a doubling of a registering of movement. This is both visualised in the system and felt in the interaction, or more accurately occurring in-between as part of the affective tonality.

The interactive tables and the coupling of the digital content and physical interaction with their co-emerging spatial realities profoundly alter the affective tonality. You interact with the table in a room based on a partial map of the city visualised on the screen on the table. Simultaneously you also orient yourself towards the larger map on the wall. Trying to navigate all three scales of interaction smoothly is almost impossible. You need to continuously stop and situate yourself in relation to the scale with which you are trying to connect. Attention shifts between interacting with the table, orienting yourself towards the visualisations on the walls and moving about in the room creating a complex relational space of interaction. Interestingly, though, the installation is rarely used by only one user. More people can easily gather in the room or around the screen, creating the scene for social interaction which in turn affects coordination of the movements and explorations.

City Voices seeks to engage people in the future planning of the city landscape by providing cues as forms of scenarios. These are meant to trigger a participation in the project that involves leaving a voice message. It remains possible that the interaction with the setup might in the future effect a change in the cityscape when the Multimedia House is built. This is capacitation on a different experiential level, yet it still feeds into the affective experience within a defined interactive setup. At the same time, the engagement with City Voices goes beyond the immediate and already doubled spatio-temporality of the interaction with the installation here-and-now. It facilitates a possible long-term relationship or more correctly a possible re-occurrence or retro-activation of the interactions preserved in the data collected and presented both on the two interactive tables and on the shared website.

What exists virtually as part of the future of the cityscape is activated and partly actualised through City Voices as relational events. The installation tries to make existing and new relations felt differently. City Voices, like the name suggests, gives people a ’voice’, it creates the potential for strong affective ties by capacitating people, giving them an opportunity to voice their opinions and feelings. In this way, the installation ’bridges’ the affective with emotivity, emotions, narratives and communication. All these thresholds of the experiential continuum are somehow activated and distributed in the physical, social and affective spaces the installation creates.

In this analysis, the concept of affect and the resulting vocabulary work to describe the experiential field that is offered by the particular interactive setup City Voices . People are invited to activate and affect the setup; in turn, there is an affectation through movements, interactions and reflections. How this affectation or activation is lived or felt depends on the specific situation; the relational events play out differently every time. However, the engaged exploration and the experiential field facilitated by the interaction design does offer the potential (and an invitation) for activation and capacitation. In this respect, City Voices is an experimental exploration of how a designed urban interactive environment might take a wholly relational and interactional form by altering the affective tonality of interaction, in a public setting, and with digital and ubiquitous technologies. The technological exploration investigates new forms of proprioceptive engagement in the interactional events. Finally, the installation provides conditions of emergence and cues for re-conditions of the emerged that might or might not translate into more stable or qualified forms of becoming.

Concluding Remarks

The analysis of City Voices can be seen as an example of how the design of an interactive environment – and the analysis of the experiential field it offers – might be conceptualised in affective terms. Rather than disappearing, the digital technology in City Voices is used actively to engage users in the exploration of the installation, thus experimenting with the affective tonality of the cityscape, the interactive situation and the possible relational events that might emerge. Ubiquity in this setting results in a complex interaction design of a possible space stirring with activation and affective capacitation.

The Massumian understanding of affect as developed in this article has the potential to offer important insights concerning the theoretical foundations for practical design experiments with experience-oriented interaction design. However, in keeping with the Massumian philosophy of experience, it is important to stress that affective experience cannot and should not be used as a blueprint for design. Arguably, the notion of affect with its attached vocabulary raises more questions than it answers. This is exactly the point; the concept of affect offers to interaction design, as it does to philosophy and cultural theory, a ‘field of questioning,’ a problematic field ‘starting in the middle with the full complexity and dynamics of fields of experience.’ This, to me, seems to be the right place to start any experience-oriented design process.

Biographical Note

Jonas Fritsch holds a PhD in interaction design and is currently Assistant Professor at Aarhus University. He works on a multitudinous thinking-together of interaction design and affect theory in conjunction with practical design experiments carried out at the Centre for Digital Urban Living (www.digitalurbanliving.dk) and the Centre for Advanced Visualization and Interaction (www.cavi.dk) in the Department of Aesthetics and Communication. He holds an MA in Information Studies with a supplementary degree in Aesthetics and Communication from la Nouvelle Sorbonne, Paris and is a member of the SenseLab, Concordia University.

Notes

• [1] Importantly, in a Massumian vocabulary affect is distinct from emotions or feelings that can be seen as a recognized outcomes of affect (Massumi, 2002: 61).
• [2] In line with this Erin Manning argues that the affective tone can also be described as the relational concernedness of the emerging world, again highlighting the radical empiricist approach and the relational as integral to a description of the affective experience (Manning, 2009).

References

• Bolter, Jay David & Gromala, Diane. Windows and Mirrors: Interaction Design, Digital Art and the Myth of Transparency (Cambridge, Mass.: MIT Press, 2003).
• Dunne, Anthony. Hertzian Tales – Electronic Products Aesthetic Experience and Critical Design (RCA Research Publications, 1999).
• Fritsch, Jonas. ‘Understanding Affective Engagement as a Resource in
• Interaction Design’, proceedings of the Nordic Design Research Conference , NORDES’09: Engaging Artifacts , 2009.
• Manning, Erin. Relationscapes: Movement, Art, Philosophy (Cambridge, Mass.: MIT Press, 2009).
• Massumi, Brian. Parables for the Virtual: Movement, Affect, Sensation (Durham/London: Duke University Press, 2002).
• Massumi, Brian and McKim, Joel. 'Of Micropolitics and Microperception [interview],' in Inflexions 3 (Nov. 2009).
• Massumi, Brian (2007): The Thinking-Feeling of What Happens. In Massumi, B., Mertins D., Spuybroek, L., Marres, M. and Hubler, C. (eds). Interact or Die: there is Drama in the Networks (NAI Publishers, 2007), 70-91.
• McCarthy, John C. and Wright, Peter. Technology as Experience (Cambridge, Mass.: MIT Press, 2004).
• Munster, Anna. Materializing New Media – Embodiment in Information Aesthetics (Dartmouth College Press, Hanover, New Hampshire: University Press of New England, 2006).
• Rogers, Yvonne. ‘Moving on from Weiser’s Vision of Calm Computing: Engaging UbiComp Experiences.’ In Dourish, P. and Friday, A. (eds.): Proceedings of UbiComp 2006 (Springer-Verlag Berling Heidelberg, 2006) 404-421.
• Petersen, Marianne Graves. ‘Remarkable Computing – the Challenge of Designing for the Home.’ (Late Breaking Results Paper, Proceedings of CHI 2004 , 2004) 1445-1448.
• Winograd, Terry. ‘From Computing Machinery to Interaction Design’, in Denning, P. and Metcalfe, R. (eds). Beyond Calculation: The Next Fifty Years of Computing (Springer-Verlag , 1997) 149-162.
• Thackara, John. ‘The Design Challenge of Pervasive Computing’, Interactions , (May-June 2001) 46-52.
• Weiser, Mark (1991): The Computer for the 21st Century. Scientific American.
• Zournazi, Mary ‘Navigating Movements: an Interview with Brian Massumi.’, 21C Magazine 2002. Retrieved from http://www.21cmagazine.com/issue2/massumi.html.

Background

The rise of the ambient brings new directions in environmental criticism. Here ‘the ambient’ means a continuum of contexts where information has been embedded locally, to facilitate being brought in and out of focal attention. How may design for that state contribute to necessary shifts of worldview? Do new relations of embodiment, touch, tagging, and display change environmental awareness, no so much of global problems as of simply being in and part of the world?

This word ‘environmental’ has been badly abused; it seems to mean everything to some people and nothing in particular to others. So long as whatever this word meant was considered large and far away, absent of technology and beyond human agency, it was difficult to imagine much less act upon. So long as information technologies have been considered placeless and nonmaterial, they have hardly been part of such imagination.

Information has always had environmental effects, nevertheless. Abstract systems of access and distribution have physical, even geographic manifestations, and sometimes direct mappings. One most dramatic example of these might be America’s Jeffersonian grid, a system of survey that encouraged commodification of land. More recent information technologies not only map environments but also interpret them. Data visualization techniques have advanced public environmental education. Then too, information infrastructures themselves have ecological footprints. Any of these relations of the digital and physical might suggest background for any of these essays in this collection. But here, let us narrow the scope to a question of coupling embodied information to shifts in environmental criticism.

For as humanity now changes not only its means, but also its ends, economic valuation shifts dramatically, and in the process creates a greater role for design. As the upkeep of non-fiscal capital (e.g., natural, human, or cultural) becomes a more viable enterprise, design emphasis shifts from extent of material production toward more intensive, systemically integrated patterns of living. As in a traditional culture where waste was less of a possibility, the new environmental economy produces a sense of simple elegance that is everywhere. Here the wish for distraction declines, and the sense of participation increases. It is a simple truth of urbanism that people take care of things and places that they value aesthetically.

Given these overarching concerns, here are four threads for consideration in this essay:

• 1) Consider a turn in environmental history, which admits not only of urbanism but also of ambient media.
• 2) Ask whether an information environmentalism is desirable, even possible, or whether it is mainly censorship.
• 3) In the context-sensitive practices of the expanding discipline of interaction design, what might already be environmental, in some larger sense, at least implicitly so?
• 4) Under what themes does the discipline develop a critique of these values, attitudes, and practices? How might this follow from developments in literature and the arts?

1. A Turn in Environmental History

First, a turn in the field of environmental history generally concerns itself more with artifice, and specifically investigates the relation of built environments and cultural values. Although such interest was once perhaps countercultural it now enters mainstream debate.

Environmental history asks how human affairs transform the living world. This has of course become the central question of our time. So that others might ask better how else humanity might live, environmental history first describes the effects of particular ways in which particular peoples have already lived. While lands may indeed shape cultural dispositions, environmental history reverses these relations to look at how cultures and destinies have shaped the land. Like much else in early environmental work, it has simply tried to document impact.

Such studies of setting and circumstance may seem normal to any discipline with ethnographic roots. Interaction design has been one of these from the start. Yet even quite recently all this emphasis on place would have been considered unusual, nostalgic, and perhaps misguided, in just about any field. Circumstance was something to transcend. Modernity was the general belief that humanity could impose abstractions on the world until those became the world, at least in human experience, and especially through the use of technology.

To admit of circumstantial impacts was not what most historians did anyway. As Donald Worster declared: ‘In the old days, the discipline of history had an altogether easier task. Everyone knew that the only important subject was politics and the only important terrain was the nation-state’ (Worster, 1988: 289).

Environmental history thus belongs to a more fundamental turn toward the study of everyday life. This breakthrough has generally been credited to the Annales school of economic historians (e.g., Bloch, Lefebvre, and Braudel in France). Much of this took a voice that was countercultural, in which material circumstance was mainly evidence of political marginalization. Environmentalism inherited that voice.

Those of us born into an age where awareness of environmental crisis is an ongoing way of life may have difficulty imagining that it was once uncommon to admit environmental sensibilities of any kind (Buell, 2003). It took a while for enough people to shift frame of reference from what were almost entire instrumental, pragmatic forms of value to admit intrinsic forms of value, or to understand humanity as an intrinsic part of a living world. But meanwhile it was being forgotten, by some quite saturated in technology, how it was just that immersion that so shocked early environmental critics. Hence the resonance of California essayist Henry Miller’s epithet (1970): ‘the air-conditioned nightmare.’

Considering the first decades of ‘doing environmental history,’ Worster emphasized how in the origins of this field, artifacts had been excluded.

The social environment, the scene of humans only interactive with each other in the absence of nature, is therefore excluded. Likewise is the built or artifactual environment, the cluster of things that people have made and which can be so pervasive as to constitute a kind of ‘second nature’ around them. (Worster: 293)

This is where the field necessarily began.

Thus another early step toward an environmental history of ambient information was to admit of any positive role for design. Without this, the countercultural origins of environmentalism too often reduce into naïve wholesale oppositions of nature (good) and artifice (bad). There are people who believe that any human intervention detracts from a place – even, say, the Golden Gate Bridge.

More general histories of the city did not lack awareness of environment. Many a mythology began from the founders hallowing a chosen spot. Geography was destiny. City air, for all its stench, was said to make men free. Yet it was the stench that demanded a new breed of historians. Environmental history of the city was history of plague and pollution. Industrial urbanism seemed not only separate from the world, but completely in opposition to it.

Now, much of this has been changing. For one indicator of this change, note ‘Green Manhattan’ (Owen, 2004). Hardly an ideal place, and more like the early environmentalists’ very idea of what to avoid, Manhattan turns out to be more green than most of America when its huge carbon consumption is normalized per capita. Relative to their exurban counterparts, New Yorkers have less stuff, perhaps more fun, and lower ecological footprints, all without risk of cholera. And not to get ahead of the story, but many of the benefits of mobile and embedded media incidentally first appear in places of highest social density.

It is instead the exurban domains where environmental knowing seems most absent. Somewhere in suburbia somebody has never even gone for a walk. Also it is in the rural domains where the worst perversions of the biosphere are perpetrated by agribusiness. To early environmentalists, the extent to which exurban artifice would disable common sense was not yet conceivable.

Thus the next step in environmental history was to treat the relationship of planetary crisis to settlement patterns, as if not all urban artifice were bad, and some ways of living healthier than others. Do poorly built environments undermine the very ability to think environmentally? It is ever more useful to ask how artifice can dramatize human agency and belonging in the world, not just provide conquest, distraction, or escape.

Urban environmental history, once considered an oxymoron, asks how settlement patterns present and reflect choices about worldview. Although often beginning from the ecological impact of economic production, it can proceed to questions of consciousness concerning just what about a production or its side effects is considered necessary. It asks what has been valued as commons. Whatever its foundations in pristine nature, it also studies questions of liveability.

This has enhanced a tendency within media, which have often been self-referential. Given the inherent quality of digital media to include both information about itself and links to other media productions, media studies increasingly investigate information as a world in itself. For some young people steeped in electronics, history goes back only so far as electronic recordings.

Histories of information, which became abundant during the web boom, have thus emphasized long term change in literacy, and not the environment. Yet literacy itself can be situated. Once it was solely in monasteries. Commercial television was first rolled out in taverns, to take another example (McCarthy, 2001: 5-7). Where you read a book affects how you read that book. The locations in media which actually get used deserve more study. This provides one way toward environmental criticism.

2. Information Environmentalism?

Second, a question of information pollution has long existed but now accelerates. Media facades especially stir public debates, not only legal but also philosophical. A sense of unprecedented distraction keeps growing.

To send your first-grader on a school field trip, you might have to fill out twenty lines on a form, over breakfast that morning. At work your inbox is stuffed, not only with spam (which can be filtered), but also with reply-to-all-recipients email from your colleagues (which cannot). Over lunch, there’s no fashion crime in wearing a hands-free phone on one ear. To get the attention of a colleague who is playing his iPod loudly enough that you hear it too, you might have to raise your voice. After work you might have trouble finding a bar without television. You could carry a ‘TV-B-Gone’ universal remote, which scans through the IR spectrum in which most TV on-off signals lie, but you might be nervous about where you can get away with using it, and of course that would be yet one more gizmo to manage.

Twenty years have passed since Richard Saul Wurman coined the term ‘information anxiety.’ What does it take, he asked, to put the verb ‘inform’ back into the noun ‘information?’ (Wurman, 1989: 38). Context, for one thing. Few people claim that the ubiquity of wisdom has been rising so rapidly as ubiquity of data. Since those twenty years are the very ones in which information technology exploded into everyday lives, this expression may be worth reexamining. How has concern for ‘information anxiety’ canceled or reinforced other more basic environmental sensibilities? The more that its designs are embedded, the less the field of interaction design may ignore these issues.

Information anxiety feels like an inability to keep up, an urge to research your every move, or a duty to provide useless data. It comes at you over call lists, customer notices, and ubiquitous advertising. But not all information anxiety is junk. It is also your own inability to restrain your grazing on news, digging into the practices of your trade, or researching exhaustive minutiae of some hobby like knitting, that you supposedly do to relax.

In reconsidering Wurman’s original thesis, the continuum of attention assumes more importance. One man’s signal is another man’s noise. ‘What we choose to read and what we choose to ignore are, therefore, some of the most critical decisions that we make, yet they are invariably made with little thought, almost unconsciously’ (Wurman, 1989: 204). The technical capacity to shift focus of attention has accelerated since then. Conversely, the social impetus to stay informed has been rendered absurd, both by the explosion of data and by the unforeseen capacity to search much more, and retain much less. Thus the relationship of environment and intent comes to the surface. To what does the context invite you to intend. How does the inscription of datascapes onto physical architectures invite intentions about being there instead of tuning out. How does the infoglut discount foreground messages in favor of a more ambient ‘affect.’

‘Ambient information anxiety’ demands a new concern in the human sciences. Educators, ethnographers, sociologists, and psychologists now debate the intellectual and emotional impacts of a fulltime, ubiquitous, multitasked feed. Architects and urbanists need to find their take on these issues as well. How does design of the physical environment mitigate (or worsen) information anxiety? When is it pollution?

In Technics and Civilization, Mumford observed that ‘the first mark of paleotechnic industry was the pollution of the air’ (Mumford, 1936: 167). The neotechnic phase, which anticipated what would later come to be called cybernetics, shifted ‘from destruction to conservation’ (255). For example, in nineteenth century Pittsburgh, smoke and fire were discussed aplenty, but to construct them as pollution was to think of them as a correctable nuisance; and this only happened much later. In a familiar cycle within technological determinism, pollution is a social construct.

Today a mother at the supermarket takes great care to scrutinize the ingredients on the labels of the foods she is buying for her family. Yet on returning home she may flip on the television without any similar vigilance. Obsessed with material pollution, she is oblivious to information pollution.

Nonmaterial pollution does exist. Most towns have regulations against nighttime noise, for example. At a loud party in the wee hours, it would be no surprise for the police to show up. Yet if there are no similar policies against light pollution, you might be able to light your driveway as brightly as technologically possible without concern that your neighbors would call the police.

Desktop data glut has been described as information pollution, that makes foragers likely to ‘hunt elsewhere’ (Nielsen, 2003). Shenk was the first to call it smog: ‘Data smog gets in the way; it crowds out quiet moments, and obstructs much-needed contemplation. It spoils conversation, literature, and even entertainment. It thwarts skepticism, rendering us less sophisticated as consumers and citizens. It stresses us out’ (Shenk, 1997: 31).

Yet to discuss information pollution is likely to contribute more of it. Also, it is subjective: information pollution fouls one person’s thought, not everyone’s waters. Above all, because one person’s signal is another person’s noise, it is much more difficult to agree on what is a common, noxious nuisance. The ethical questions raised differ from both those of free speech and those of clean air. Many of these concern whether information glut declares itself, and allows people to opt out (Greenfield 2004). The ethical questions can seem more culturally biased, as in what makes a society with many implicit forms of common sense much more pleasant than a nanny state with many explicit rules preventions. So how can it be shown that glut fouls conviviality, which per Ivan Illich (1973) is all that which occurs socially without need for overt information? Such questions tie into ethics, politics, rhetoric, and other such fundamental categories in philosophy. It is a lot easier just to keep building, buying, and consuming more information feeds.

3. Contextual Practices

Third, then, this essay asks where current practices and common wisdom in the young field of interaction design show environmental awareness of any kind. Where might designed embodiment in ambient information actually increase awareness of the many issues raised? Such general questions seem warranted in a field where the pursuit of calming is a given (Weiser and Brown, 1997). It is the goal of this essay to provide intellectual background and future questions to that sensibility, and not to survey current practices in detail. How do context-sensitive design practices, which already emphasize symbolic environments, now make the step toward ambient environments?

Interaction design has to be intrinsically environmental at some level because it expands the symbolic context of actions. Since its inception, the field has looked at data input and output in relationship to interface form and structure. It has looked at tasks in relationship to workflows and organizations. It has looked at architectures of access. More recently it has looked at socially networked frames of reference, especially amid habitual and even institutional contexts. And now, as in the focus of this volume, it is looking at physical embodiment in urban space.

Something about urban computing shifts the disciplinary emphasis from equipment to perceptions. It replaces a bias toward adding ever more tools and technological features with a quest toward a sensibility inflected toward maintaining some cultural commons. It differs from larger and necessary agendas in ‘eco-tech,’ that is, embedded information systems for improved resource economy, especially in building technology. It emphasizes proximity of particular people and annotation of particular places. It admits that knowledge of how to inhabit a city may be lost in the rush to personal entertainment. Indeed, distraction may be the state on which it must operate. The ethical issues at hand are those of distraction.

As the literature of urbanism has long emphasized, individuality in places that are of interest to particular inhabitants produces a cumulative urban experience, such as the experience of districts. The personalization of mobile media feeds does not produce this cumulative experience. This has often been understood though psychogeography. Physical experience introduces aspects of orientation, memory, and territory, largely because the body imposes a schema on space. While these do increase the sense of being in the world, the question arises whether they reinforce or interfere with existing senses of embodiment and environment. On the other hand, new techniques of embedding, touch, and diversified display formats introduce a new domain of using the world itself as an access structure. In contrast to more innate relationships of body and memory, this new domain invites a more exploratory and less territorial approach to spatial mental mapping. That is because the world is now so much more overlaid and annotated with cultural productions of different spatial ordering. This much-championed hybrid introduces challenges of ‘findability’ (Morville, 2004) to which disciplinary approaches are quite different from the conventional literature on cognitive mapping. Thus new relations of embodiment, tangible interface, and ambient information increase the environmental implications of interaction design in two complementary ways.

On the one hand, the discipline could increase its concern for shifts in focal attention. Early interaction design carefully studied focus amid tasks. Pervasive computing began from the introduction of peripheral attention (Weiser and Brown, 1997). The ambient is a continuum of possibilities for focal shift. Information theorists contend that complexity and emergence become central to any study of awareness amid ambient, noisy, glutted communications. ‘When information is understood as a process rather than a product, the line separating it from noise is difficult to determine,’ Taylor explained. ‘The interplay of noise, which is informative, and information, which is noisy, creates the conditions for emerging complexity, which is the pulse of life’ (Taylor, 2001: 123).

On the other hand, the need to tune and personalize one’s range of focal possibilities becomes a central strategy of information anxiety management. This challenge was evident to Wurman twenty years ago, but it has accelerated, and in doing so it has implicated more contexts. The discipline might now read ‘almost unconsciously’ as ‘embodied, haptically-oriented, socially cued.’ One key to the ambient is the increasing belief that much mental activity does not rise to the level of conscious deliberation.

Thus the overall social and practical contexts of action have become much more central to interaction design, and often surpass matters of mechanical or workflow-oriented usability as the key success factor. This makes the field more design-oriented. What one chooses to take on as subject matter matters as much as how efficiently that is engineered. For subject matter beyond the graphical user interface, the discipline works on product-service systems, neighborhood technologies, material and energy flow representation, and more. Because waste is almost intrinsically ugly, exposing and correcting it is a cultural, and not merely moral or economic act. Environmental description, advocacy, and education allow more people to understand and shape such policy. Criticism suggests that these are cultural acts. Hopefully this issue provides a review of many such projects. To see the change, one need only consult a CHI proceedings of ten years ago, when it was widely thought that only the virtual would count.

4. Toward New Genres

Fourth, the invitation here is to develop terms and criteria by which to give more discernment and design authority to these implicitly environmental computing practices. And the philosophical question is whether those design experiences cultivate (for a highly networked generation, many of whom seldom played outdoors as children) a sense of connection to the physical world.

The first step is to demonstrate that locality is more than location. Proximate networks and not just GPS coordinates become the basis for locative media work. Locality and participation seem key to many an emerging green economics. How does interaction design become valuable to this?

Orientation to the physical everyday came first to history and letters. In the humanities, environmental criticism generally has origins in literary studies. Most university courses on environmental criticism concern literature first, and perhaps the high visual arts. Even without ‘media,’ a traditional culture created its environment mythologically; modern environmentalism might trace its origins to Virgil. Literature can blur distinctions among mythology, politics, environment, and perception. Was Mark Twain, educated as he was by the river, an early environmentalist?

However, in recent decades, literary studies have often divided over environmental criticism. To the relativistic language games of postmodernity, any insistence on worldly referents seems naïve, and any insistence on a master narrative (e.g., planetary change) demands opposition. Even spatial disciplines such as architecture long followed this literary lead. A better lead, however, appeared in cultural landscape studies. These ask how a society’s values may be known as much from its settlement patterns as its literature or objects of art. Much of this has been directed to the macro scale, and sprawl, especially in America. But where it crosses with social history of technology, a useful genre of environmental criticism emerges. For example, this is evident in the parallels commonly drawn between pervasive computing and early electrification. It is the ethnographic perspective of some of this work that makes it relevant to interaction design. For example McCarthy’s 2001 study of ambient television looks at the influence of information pollution (our term, not hers) on gender roles and the spaces of conviviality and travel. As more kinds of ambient information technology become similarly normative, and no longer principally a technical challenge, surely they invite such similar critique.

This is mainly a matter of building a culture. That perhaps begins with awareness of context. Only with a conversation that is accumulated and perpetuated does the larger body of work acquire enough coherence to warrant criticism. The word ‘criticism’ implies familiarity with the issues, and how to reconsider what you have been thinking anyway. For instance the original Macintosh was described as the first human-computer interface that was good enough to criticize. With design for embodied, tangible, and diversely displayed information, is pervasive computing at last good enough to criticize?

One of the must fundamental sets of categories for critique has been introduced by Borgman (1999) as information ‘about,’ ‘for,’ or ‘as’ the world. Information ‘about’ the world characterizes much pre-modernity. Even biologists concentrated on descriptions. Information ‘for’ the world implies also some generative understanding, and is the main work of design. Modernists are notoriously spotty at seeing how things are because they are so busy imagining how else things might become. That begins an environmental disconnection. Then there is information ‘as’ the world, ‘a deluge that threatens to erode, suspend, and dissolve its predecessors’ (Borgman, 1999: 2). Herein lies the main environmental disconnection. Media-saturated people somehow become reality-challenged. Realism involves making sense of information types, especially in relationship to unmediated objects, structures, and surroundings, especially those that cannot be turned off, put away, or ignored.

Here let us limit the scope to information in environments, neither just describing them elsewhere like stories and maps, nor becoming them virtually like games and cinema, but, ubiquitous and embedded from within, engaged at annotating, operating, identifying, linking, recording, embellishing, polluting, populating, and filling the peripheries of what the best philosophers agree upon as reality.

Afterword

Under philosophical shifts required by planetary change, environmental critics no longer oppose environment and culture, nor conflate environment with nature. Instead the critical challenge has becomes to recognize the origins of crisis, and prospects for change, in cultural attitudes toward any and all concepts of environment in which humanity plays a part. This includes built environment, managed biosphere, information sphere, and ambient cultural experience. How each of these disposes people to understand their place in the world now demands exposition. Bio-centric forms of environmentalism may always lead the way; yet to make such abstraction accessible to more imaginations, culturally-centric forms must also play a role.

Information and environmental histories have not yet interwoven enough. Now, as information media become ubiquitous and interfaces become physically tangible and socially transparent, this separation may change. ‘Environmental history of information’ may no longer yield a null search on Google, as it did at this writing.

Biographical Note

Malcolm McCullough is associate professor of architecture at the University of Michigan. He has also served on the faculty of Carnegie Mellon and Harvard. He has written two widely-read books on architecture and interaction design – Digital Ground (2004) and Abstracting Craft (1996) – and has engaged in speaking and editorial review across a wide range of disciplines, from urbanism through applied arts to interaction design.

References

• Borgman, Albert. Holding on to Reality (Chicago: University of Chicago Press, 1999).
• Buell, Lawrence. From Apocalypse to Way of Life : Environmental Crisis in the American Century (New York: Routledge, 2003).
• Greenfield, Adam. ‘All watched over by machines of loving grace: Some ethical guidelines for user experience in ubiquitous-computing settings.’ Boxes and Arrows. www.boxesandarrows.com. 2004.
• Illich, Ivan. Tools for Conviviality. (New York: Harper and Rowe, 1973).
• McCarthy, Anna. Ambient Television: Visual Culture and Public Space (Durham: Duke University Press, 2001).
• Miller, Henry. The Air Conditioned Nightmare (San Francisco: New Directions, 1970).
• Morville, Peter. Ambient Findability (Sebastopol, CA: O’Reilly, 2005).
• Mumford, Lewis. Technics and Civilization (New York: Harcourt, Brace, 1936).
• Nielsen, Jacob. ‘Information Pollution.’ Alertbox. www.useit.com. 11 August 2003, <http://www.useit.com/alertbox/20030811.html>.
• Owen, David. ‘Green Manhattan– Why New York is the greenest city in the U.S.’ The New Yorker. 18 October 2004.
• Shenk, David. Data Smog – Surviving the Information Glut (San Francisco: Harper Collins, 1997).
• Taylor, Mark. C. The Moment of Complexity – Emerging Network Culture (Chicago: University of Chicago Press, 2001).
• Weiser, Mark, and John Seely Brown. ‘The Coming Age of Calm Technology.’ 1996. 15.08.2011. <http://nano.xerox.com/hypertext/weiser/acmfuture2endnote.htm>.
• Worster, D. ‘Appendix: Doing Environmental History’ The Ends of the Earth: Perspectives on Modern Environmental History (New York: Cambridge University Press, 1988).
• Wurman, Richard Saul. Information Anxiety (New York: Doubleday, 1989).

Introduction

In ‘The Coming Age of Calm Technology’, Mark Weiser and John Seely Brown are clear in their assertions. What really ‘matters’ about technology is not technology in itself but rather its capacity to continuously recreate our relationship with the world at large (Brown and Weiser, 1996). Even though they promote such an idea under the banner of ‘calm technology’, what is central to their thesis is the mutational capacities brought into the world by the spillage of computation out from its customary boxes. What their work tends to occlude is that in setting the sinking of technology almost imperceptibly, but deeply into the ‘everyday’ as a target for ubiquitous computing, other possibilities are masked, for instance those of the greater hackability or interrogability of such technologies. Our contention is that making ubicomp seamless (MacColl et al, 2002) tends to obfuscate the potential of computation in reworking computational subjects, and thus societies, modes of life, and inter-relations with the dynamics of thought and the composition of experience and understanding.

Even though we might recognize this inherent demand in the bifurcation of ‘calm’ technology into the new feral logics, ubicomp’s reliance on the development of artificial intelligence – through techniques of identification, naming, tracking, sorting, monitoring and responding – has been precipitated, though if not always inspired by, a research agenda inherited from military programs inaugurated during World War II onwards (Heims, 1980; Agre 1997). This same ‘military’ heritage continues to define much of the research agendas of significant academic and industrial institutions, directly influencing their epistemological orientations. Doing ‘calm’ requires then an overemphasis on the machine, keeping the deployed human component stable and unexcited and focusing essentially on the development of artificial intelligence frameworks that in turn calmly interpret and act on the calmed user. Such an approach can be seen to operate in opposition to a deeper understanding of human-computer symbiosis, one that may even step out of this dyad to engage with a wider understanding of systems and ecologies. What is crucial to recognize here is that this agenda, infatuated with an ontological positivism simple enough to teach a machine, establishes distinct conceptions of ubiquity that ramify into computational technology, human action and cognition and the environments in which these occur. As it stands, many of the problems experienced by ubiquitous computing as a progressive research program can be found in the reliance on such an ontology and its apparently stable concatenations of misidentification and mismatched response, delimitating the potential of new logics into simplified models of calculation.

Despite its partial ‘military’ heritage, ubiquity is effectively bifurcating from this calculative approach, challenging much of the practice of computer programming and other layers of the ordering of computers. This is now diversified by: networks, embedded systems, new graphical user interfaces, the World Wide Web and wireless devices, networks of sensor-actuators, and the multi-form variations introduced to computing by its deep embedding in the social technical, biological, aesthetic and political dimensions of life. This renewed context has, particularly within the field of computer science, shifted from a view of ‘computation as calculation’ to one of ‘computation as interaction’ (Stein, 1999; Wegner, 1996; Murtaugh, 2008). It has deviated from the core of its first-order programming philosophy, from computation as ‘number crunching’ to an object based and distributed approach (Wegner, 1996; Gouldin and Wegner, 2003). The growth of computation into a new feral state has also affected the ways in which user relations with the new artifactual ecologies can be understood, extending previous research frames. Here, the emphasis on the ideal of ‘computation as calculation’ has not only affected the field of computer science, it has also influenced the field of cognitive science (Dupuy 2000) and those concerned with computation as metaphor, instrument, field or infrastructure.

Once again, ubiquitous computing and its propensity for distribution as part of environments, opening new space for variable kinds of users and contexts, has shifted the idea of cognition as analogous to the workings of computational devices to an idea of cognition as situated, embedded and distributed. This idea has much in parallel with the field that Heinz von Foerster and others named ‘second-order cybernetics’ (Poerkson and von Foerster, 2002). Indeed, amongst other contexts, this field has a lasting influence, or even undergoes a revival in some of today’s laboratories of computational design, one example being in the development of robotics (Bishop and Nasuto, 2005). One of the main goals of second-order cybernetics was the attempt to study complex systems, including humans and machines, under a new light. It attended in particular to the way in which they formed patterns of reflexivity, gained a recursive sense of self and of the wider processes in which that self co-composed. Crucially, they also emphasized the study of systems that are analytically indecomposable, such as memory, or which grow as part of a ‘conversation’ (Pask, 1976), or other form of structural coupling (Maturana and Varela, 1998), making the neat distinction between subject and object, observer and the observed, simply a neat, but disabling perceptual gimmick.

One of the founders of the cybernetic movement, Norbert Wiener, posed the need to interlink the new worlds of ‘automata’ with distinct social interests and concerns, where the ‘decentralization of authority should accompany the decentralization of computation’. This project would soon be obfuscated by scientists such as John von Neumann, whose vision of their use tended to relegate computational power exclusively to the interests of the ‘military and industrial establishments’. This limitation inaugurated at the same time cybernetics’ disengagement from domains – unfortunately – characterized as ‘humanist’, finally closing the computer into itself (Eglash 2000). As a consequence, the inherited ‘military’ agenda has contributed to the divide between cultural approaches to technology and the means for a technical imaginary and those that characterize themselves as ‘scientific'. Today, an expansion of the scope of interdisciplinary approaches within the distinct domains that study human-computer interaction and computing cultures more widely is moving towards a reversal of this ‘trend’. Here, ubiquitous computing has played a decisive role. Its propensity for distribution throughout the environment has opened space for a new range of contexts, users and artifacts, raising new complex questions that extend beyond the premise of perfecting artificial intelligence. What once was the concern of an engineering driven discipline is now delivered to a wider field of intelligences and skills beyond any discipline.

Interaction, the Extension of Computation beyond Calculation

Both the fact and ideation of ubiquitous computing and its characteristic embeddedness has driven the need for an exploration of new human-computer relations, shifting, in one description, the research motto from ‘proactive computing’ to ‘proactive people’ (Rogers et al., 2006). The shift arises not only from a new conception of human/machine symbiosis – of the sort originally theorized and supported by Licklider (1960), towards mutual proactivity – but also from a challenge from within the field of computer science itself. Essentially, this shift has been accompanied by a transformation of the understanding of what computation is and how it might be done, dislodging a model in which computation is seen as a series of fixed functions that are outlined to achieve a certain specified goal. Computation as something ‘centralized, sequential and result-oriented’ and primarily focused on the execution of calculation, moves tentatively or eagerly, but perhaps inexorably, towards an understanding of ‘computation as interaction’ (Stein, 1999: 1). Here, dynamic relations become key. A computation does not simply equal the achievement of a result, measured by an ever increasing metric of acceleration towards that result, but tends towards a collaboration with the user or other elements in a wider ecology, understood through a connective multiplication of the capacities of each entity in the computational composition.

Where the ‘calculative’ approach has persisted – in the disavowal of computational artifacts beyond their formal description as variations upon calculus – this has tended to make difficult or even prevent the recognition of any external influence, or of the multi-layeredness of computational situations. Concerning the embeddedness of computational gadgets throughout the environment, it has particularly influenced the ways in which certain dogmatic computational frames have pushed forward a ‘result-oriented’ rather than ‘process-oriented’ programming styles (Stein, 1999). Such imperatives have often dissuaded engaged professionals from elaborating any form of ‘computer criticism’ that could provide a richer frame for understanding computational artifacts and their contexts (Papert, 1990). At the core of such reluctance sits the bulk of our conceptions concerning the practice of programming, ideas that are embedded in strong cultural formations of thought and rationality. An example that throws light on both epistemological styles and their consequences for computation, the Logo programming environment for children – a pioneering project of its kind – was built upon the recognition that many computational environments, for the purpose of education still preserve canonical dogmas, such as the fixation on outcome or result achieved by pre-determined means. On the contrary, the Logo environment has shown that many children favor a style that contradicts the formal canonical one identified with planning and reasoning, opting for a more ‘intuitive’ mode; both are conceptually valid (Papert, 1990). The example of Logo is problematic in the sense of its limited domain of actual use, but the way in which it opened the possibility for different programming idioms to come to fruition and to be recognized as being idiomatic, with different perspectival weightings and affordances, is significant. Today, different programming methodologies, scripting environments, frameworks, and the increasing sense of coding and computation as part of popular cultures and interesting subcultures open computation to a multiplication of conceptual and idiomatic registers (Mackenzie, 2006).

In fact, such conceptions of programming based on ratiocination have a long history in the development of the computational field itself. These find their philosophical legacy in a historic battle between rationalists and empiricists. Descartes’ figuration of reasoning as the base of existence was followed by Kant’s denial of the contingency of true knowledge, and in turn by those contemporaries who, in their most useful formulations, find the rational folded into and entangled with the aleatory, intuitive and experiential in ways that defy the capacities of expectation of either model. Still, much of modern Western science has been influenced by rationalist mathematical modes of thought, these even more intensified by the successes of technological development, the faith on the dependency of such rationality, and a corresponding ability to recognize a world which is increasingly reformatted into rationality's own image.

The history of Western thought, however, is also a history of successive collapses of total formalizations (Kline, 1980). Kurt Gödel’s ‘incompleteness theorem’ that exposed the fissiparous faultiness of rationalism when turned upon itself is only an example, but a crucial one in the history of computing. The thesis opened the way for a new epistemological agenda where the idea of incompleteness would prevail, a decisive element in bringing the paradigm of interaction to centre stage for the development of computing. Alan Turing famously took up Gödel’s ‘incompleteness theorem’ and in 1936 published a paper ‘On Computable Numbers with an application to the Entscheidungsproblem’, showing that ‘mathematics could not be completely modeled by computers’ (Goldin and Wegner, 2003: 1). However, whilst acknowledging its foundation in the tracing of limits to computability, the Turing machine was adopted by computer scientists at large, providing a seductive model with which to address problems. Computer science took the transformation of ‘inputs into outputs’ as a defining characteristic of computation, a transformative process already known to the mathematical field and best described by formal algorithmic operations (Goldin and Wegner, 2003: 1).

Approaching the 1990’s, computer science faced new challenges. With a new logic of ubiquity, from the World Wide Web to wireless devices, phones and other more cranky assemblages, new questions appeared. Logical-axiomatic transformations (as understood through the ‘inputs into outputs’ of the Turing machine) as the sole answer to computational problems (those problems to which computing was addressed and distinct therefore from 'computable' ones) would no longer suffice. By that time, many already felt the need to extend computation into new paradigms, as the novelty of machinic formations evolving out into diverse ecological conjunctions allowed the exploration of, for instance, physical effects rather than solely logical ones. Today, one result of this is the development of ‘interactive-identity machines’, simple transducers characterized by non-algorithmic behavior that use the computing power available in the surrounding environment, substituting mathematical reasoning with empirical development (Wegner 1996). Here, the new ‘interactive’ paradigm appeared as most attractive, not only since it best explicated possible relations with ‘real’ environments that could not be completely predetermined by a list of configured inputs, but also because it provided the best extension for the already achieved capabilities of the Turing machine. The twist being that the ‘incompleteness’ now supplied by the interacting environment is a feature that is not under the machine's control, or not integral to its axiomatic nature, but generated out of its points of (intensive and extensive) conjunction with the world.

The new interactive ecologies could not rely on formal computable algorithms. These engaged in a constant switching back and forth, between the figure and ground of computation in the machine, and in the potential for computability within the environment. There was an increasing demand for developing new dynamical models (Stein, 1997) to new interactive algorithms (Mackenzie, 2009), extending to ‘computation beyond calculation’. Most important, ‘interactive computing’ allowed the necessary resistance to the canonical style, rejecting computer science’s legacy, finally leaving space for a new epistemological orientation. Of course, it is not that suddenly a computer becomes something more than a technically describable object, rather, computers were turned inside out. Their processes and affiliations routed themselves through non-computable processes producing new moiré patterns of resonance, interaction, and interference.

The Interactive Paradigm, a Cybernetic Revival

We can also trace the origins of interactive approaches to the ‘Macy conferences’ held between 1946 and 1953, in New York, when a group of researchers met with the goal of discussing ‘Circular Causal and Feedback Mechanisms in Biological and Social Systems’. Here, the term cybernetics was taken on, following the title of mathematician Norbert Wiener's (1948) book on control and communication (Eglash, 2000). Two of the interdisciplinary researchers present, mathematician Norbert Wiener and neurophysiologist Warren McCulloch inaugurated what later became characterized as the ‘first-order' cybernetic movement. The research motto: the idea that the dynamic entities, which maintain certain kinds of consistency over time, such as the human nervous system, can be characterized by internal feedback processes that maintain constant stability.

Part of the research agenda of cybernetics was an emphasis on inter-disciplinary applicability through the creation of meta-categorical concepts, such as feedback, that crossed both living and non-living entities. Cybernetic approaches, based on powerful degrees of abstraction, were later applied to the development of new kinds of artificial intelligence, now framed by a connectionist approach that emphasized structures of simpler but interconnected neural units (Bishop and Nasuto, 2005) instead of intelligence deriving from one main central processor. This ‘bottom up approach' was applied to both machinic computation and human cognition. Aspects of this approach are paralleled by what is often recalled as ‘second-order cybernetics’, epitomized in, but not limited to the work of physicist and philosopher Heinz von Foerster and participants in the Biological Computing Lab at the University of Illinois. Second-order cybernetics is characterized by finding ways in which feedback loops travel outside of the boundaries of an entity, affecting its behavior in reflexive and non-determinable ways. Here, von Foerster introduces the idea of ‘subjective dynamic construction’ (Poerksen and von Foerster, 2002: 23) into the complex web of feedback loops within a given system. This abstract framework relocates the position of the observer or agent of a given system into its dynamics. If for first-order cybernetics the agent was decoupled, with second-order cybernetics this would no longer be possible. Such a position re-describes, with a different scale of reference, the relevance of situating computational artifacts as ‘external scaffoldings’ (Griffiths and Stotz, 2000), where agent and environment are no longer fully separated but mutually articulating.

In more recent times, there has been a growing interest in using von Foerster’s ideas in the field of robotics. Key factors have been the importance of environmental situatedness and non-representational embodiment as the driving forces in the development of intelligent behavior. Additionally, ‘second-order cybernetics’ opened the door for what later would be called a ‘dynamic systems theory of cognition’ and the ‘enactive theory of perception’ (Bishop and Nasuto, 2005). For both, cognition is a highly interactive process where agent and environment have important and active roles. Building on, but also questioning, earlier theories of environmental perception, such as those of James J. Gibson (1977), which are well-known in design, a dynamic systems theory of cognition recognizes the importance not only of the relation between an agent and its environment, but also the ‘state-space’ that relates such entities and the permutations of actions that compose them. ‘Enactive perception’ recognizes the role of embodiment in the process of cognition. The agent only perceives to the extent its perceptual system enables it to do so. For both approaches, cognitive processes are determined by agent-environment coupling dynamics (Bishop and Nasuto, 2005). Reality is no longer an established edifice but rather subject – but not reducible – to an individual’s interpretation. In the case of research performed within ubiquitous computing and its dependence on artifacts that track, monitor and respond to a set of human activities, the idea of agent-based reality puts in question, once again, the idea of a predetermined computational entity.

The transformation of computation from ‘calculation’ to ‘interaction’ also has a direct relation to the development of some of the main ideas developed by von Foerster, Gordon Pask, Gregory Bateson and others associated with second-order cybernetics. Turning intelligence inside-out, this model opposed the idea of simple internal representation, proposing instead one of external and distributed pragmatics working with affordances. Such conceptions have in turn influenced computation itself, recomposing the idea of programming as a bottom up process, a move that returns to influence ideas about intelligence. Instead of something being sited in the ‘head’ (Noë, 2004), it has an emergent quality depending on high levels of interaction. Again, there are strong levels of filiation. The ‘bottom up model’ was first developed by cyberneticians such as Wiener, based on ‘corresponding’ connectionist approaches to cognition. Second-order cybernetics challenged this view, by locating the agent in relation to the same system (Bishop and Nasuto, 2005). This further promotes a ‘constructivist epistemology’ where the observer, perspectively and enactively, constructs her own experience and cognition becomes a continuous dynamic process.

At this point, it needs to be acknowledged that a new conceptual approach to the materiality of the world is also devised. First, this provides a means of recognizing the position of both agent and environment in a complex dynamic ecology, where the extensions or limits of either are not clear-cut, or are rendered effectively meaningless. Second, it provides a means of abstraction in which behaviors and patterns that cross categorically distinct entities can be recognized. Finally, it was a form of thinking, based strongly in practice, which opened up hybrid insights into forms of life, providing a new methodological approach when researching the ‘social’ and the ‘natural’ (Eglash, 2000), or let’s rather say, with apprehension of their inadequacy as terms, the ‘natural’ and the ‘artifactual’. Concerning the design of computational artifacts, such an agenda presupposes the relative importance of both user and artifact, now appearing in the light of more tricky, multivalent relations. It also presupposes that the user is as much responsible, or at least implicated – in multiple, non-predictable ways – in the construction of their own artifactual ecology. Such a constructive process calls for an approach that places computation in a new light, reemphasizing the importance of a research agenda that marks the expansion of computation through distinct layers of interactivity. This entails a transformation, not only for computer science itself, but also in the conception of new computational artifacts and environments.

The implication of the user in the expanded computational environment is however not a story of simple unfolding opportunity. The history of cybernetics is both a history of systems of control and of a mode of understanding and synthesizing systems of recursive self-organization. A society of extended interaction is certainly being put into place, the computer is spread out into the world, computation is enfolding itself into all layers of life: but in many cases as systems of control. Writing from an island whose society puts more faith in surveillance cameras than in the citizens they watch we have no reason to welcome the spread, distributed, or open arms of control. Instead, we might usefully look towards coupling an understanding of interaction with a critical and inventive politics. As an applied science, cybernetics became in part, like early strains of HCI, a means of integrating humans with machine systems (Hables Grey 1995) or those of the poetry of management (Beer 1975). In the nineteen-eighties, one of the resulting figures, that of the cyborg, became subject of wry critical celebration, most famously in the work of Donna Haraway (1991). The current wave of re-engagement with cybernetics tends to emphasize not so much its application in systems of control (Holmes 2007) but its empirical work amongst epistemic questions with the production of experimental devices (Haque 2007).

From Cybernetics to Interactive Designs

Such epistemic questions have instigated the need to understand interactive technologies in their actual use, blurring the modern divide between the ‘social, symbolic and subjective from the material, real, objective and factual’ (Latour, 2008: 6) avoiding any absolute bifurcation of interaction between users and artifacts and the relations between them. This has been the true contribution of the field of participatory design, one that opened the field of computation, finally setting up the interactive paradigm to reach out into the ‘situated, interpretive and messy’ aspects of human nature (Harrison et al., 2007). Forms of participatory design have long questioned modernity’s formalism and rationality as raised on the research edifices of WWII and beyond. It has questioned computer science’s formal approach and the rational structuring of cognition by incorporating user and environment, cultural, economical and political contexts. In this sense, participatory design makes possible, if it does not always deliver, a ‘third and hybrid space’ (Muller, 2003: 2) between users and artifacts, inaugurating a dialogical plane for action. As in ‘distributed cognition’ (Hutchins 1999, 2000), both user and artifact come to play determining roles in the mix of effects and interactions, with the explicit recognition of the processes of thought in relation to design. For designers at large this is of substantial relevance, since new forms of computation are bound up with the encouragement of new forms of sustainability, development and experimentation. Such considerations are especially potent when we consider the contexts in which users and technologies are embedded, along with the variable kinds of access different users have today – to the panoply of computational artifacts, processes and services (Crang et al., 2006).

In this same context, it seems pertinent to attempt to consider innovation. Here, the ideal still ostensibly driving most technological research agendas is not so much innovation anymore. Instead, it is uses (Shapin, 2007) or even misuses. For this same reason, participatory design philosophy and methods have been taken on by some companies in the development of user-centered approaches to innovation (von Hippel, 2005). Its principles have been applied, in different interpretations, in a number of ways and sites ranging from the development of industrial artifacts to that of software. However, these are not without their problems. Improvements that only travel inwards to a company will soon make users rightly cynical. The world of marketing is not innocent of simulating 'demand' in order to generate it and interaction will inevitably face many kinds of shaping by interested parties and social and libidinal forces (Lyotard, 1993). Nevertheless, distributed innovation as it articulates the reflexive extension of interaction is a potent feral force that proliferates on the outskirts of policies and regulations, amongst distinct communities of users/designers. This same shift has provided a new distributed conception of innovation and knowledge, with a significant move to test and extend the ways in which Free Software derived development methodologies and principles might produce effects in other domains of production and use, such as non-executable data. For reasons such as this, we use the term 'interactive designer' as much as that of 'interaction designer'.

As a counterattack to this potent feral force, many new approaches to the development of regulations and copyright policies have tended to ‘imprison’ the same users by relegating full control to corporations or owners of ‘intellectual property’. These regulations, and sometimes the closure of designed objects, tend to retard innovation by limiting the scales and kinds of access to technological and cultural artifacts and processes. The rise of new computational technologies tends however to set this back, essentially by providing information and innovation opportunities to communities that may extend beyond 'traditional' geographical or economic constraints. The affordability – often at the cost of abusive forms of production, of computational gadgets and the ease with which people start to manipulate them, and thus perhaps break out of the role of simple user – is opening some space for a new era of technological innovators. Indeed several researchers have suggested that a democratic approach to technological design might have the tendency towards increasing opportunities, or instigating new forms of economic development (von Hippel, 2005). Approaches to ubiquitous computing that develop significant modes of interaction will necessarily also meet with the question of the way in which data of all kinds moves about, whether it is shared, sold, wrapped in protective seals, or is treated as simply part of the wider ecology of materials.

Feral Computation and Design

It now seems pertinent to ask: what might design facilitate or enter into combination with in this process? Can design catalyze a process of material reshaping, further influencing those described above? First, we need to extend this venture and consider relations between designer, the artifact and systems it engages with (that now might appear in a planned or unplanned manner). This is perhaps even more relevant when seeking feral qualities in the hardware domain. Here we note the recent and widespread increase in interest in tinkering, making idiosyncratic technical objects, and the extension of this into a renewed perhaps quasi-popular engagement with electronics through platforms such as Arduino (see http://www.arduino.cc/) and others that provide a means of recognizing and working with a quite palpable hunger for a more difficult, unpredictable, customizable and intelligent inter-relationship with electronic design, often that more commonly found at the level of prototyping. Such moves provide not only interesting options for design curricula but also for those engaged in design beyond formal education, as useful information and experiences move in ways that build self-organizing practices. This in turn establishes possibilities for a new generation of designers, and more broadly of designing practices. By such means, as machinic platforms, knowledge and skills, coupled with modes of thinking that provoke ubiquitous curiosity rather than calmness, certain principles of hackability are extended to a wider range of users who in turn modify them. We can only envisage potential future uses when bits and pieces become more robust, or usefully weak enough to make interesting connections, and are globally widespread, opening paths for new modes of commodity distribution, hackability and self-production. Here, design is involved in a more complex web of experimental devising, moving beyond the provision of simplified solutions, engaging in the construction of a system that continuously reshapes the questions that were initially waiting to be answered.

The qualities of ferality can also be found in applications such as ‘Squirrel’. Designed by Shannon Spanhake, (of the California Institute for Telecommunications and Information Technology) this small device incorporates a battery, sensor chip, Bluetooth and accompanying software, ‘Acorn’. The device allows users to monitor and read levels of carbon monoxide in the air through their cell-phones (Ramsey 2007). This extends the potential of pollution monitoring, once only in the hands of small public and private groups of specialized research enterprises. It is now opened up to users with the potential for hacking the politically enclosed debate. In a similar vein, Mexican artist-engineer Gilberto Esparza shows us the possibility of designing parasitical robotic artifacts, simple life forms that feed on power generated by human societies. The parasites ‘clgd’ (colgado) and ‘dblt’ (diablito) live suspended on telephone cables and re-circulate their energy while interacting with the surrounding environment through the emission of unusual sounds. The ‘ppndr-s’, a redaction of the name pepenadores, are small robot parasites that live amongst the accumulated remainders of human disposals, carrying out simple tasks such as removing, scattering and sweeping (for more information go to: www.parasitosurbanos.com). These artifacts live out of surpluses and engender a sense of the uncanny in urban situations, feral robots that couple with rickety media ecologies and the leakages and disarray of cities. Thinking of new forms of relation between natural processes and those that are deeply synthetic in a way that can open them up to wider understanding, or at least more tractable difficulty, Haque Design & Research presents ‘Natural Fuse’, an eco-physical network that employs the capacities within the environment, such as plant carbon-sinking, to create a recursive system where energy can only be derived from electricity networks if sufficient plant matter is present in the circuit to absorb the carbon produced by the energy used, (for more information see: www.haque.co.uk). In all these examples, hackability is taken into a new speculative dimension. Hackability itself becomes feral and no longer solely pertains to the material and infrastructural domain, in which it is measured by the relative technical openness of a system, but also to a performative one, in turn proposing new user modes for appropriating and experimenting with a wider sphere of interests. Finally, acquiring alternative sets of dimensions, scales and sizes of intervention, these new user modes call upon us to reinvent what the city is, what the ‘proper’ way of allocating energy might be and to circulate the capacity to know in new ways, demanding in turn new kinds of agency and knowledge circulation.

Rather tending to contradict mainstream perspectives on the sites of technological innovation, this call has been heard from economically impoverished countries, where salvaged machines imported from the West, or sourced direct from fabrication in the East, have, due to material scarcity, been incorporated into a tradition of maintenance and redesign. By transforming technologies in their own terms users come to generate a body of knowledge unforeseeable by the ‘original’ designer. The story of Morris Mbetsa, a self- taught inventor from Mombasa, seems pertinent. Mbetsa invented the ‘Block & Track’, an anti-theft device and vehicle tracking system based on mobile phone technology. The device uses a combination of voice, dual-tone multi-frequency (DTMF), signaling and SMS text messaging technology to control a vehicle's electrical system providing the user the ability to activate or disable the ignition in real time (AfriGadget 2008). Such ingenuity, in ‘developing’ countries, is not simply a means for survival, but a crucial resource for educational, economical and cultural development, which shapes its own conditions of emergence.

Interaction, Society and New Distributed Systems

To shift to another scale, such reflections become even more relevant if we consider the pace at which computational artifacts have become simple everyday objects for many children making ubiquitous or distributed forms of computing 'everyday' experiences in ways that test their stability. New forms of computational literacy suggest that the relation of users not only towards technology, but also to knowledge production itself, will change in unforeseeable ways. Not disinterestedly perhaps, Microsoft is amongst those keen to argue that developing forms of knowledge will increasingly require more and new forms of ‘computational literacy’ (Microsoft Research, 2005), to which new forms of critical and inventive intelligence will need to arise. There is a fundamental and pressing need for the design of research tools that might enable or constrain new forms of experimentation and knowledge production. Even though, for Microsoft’s vision of the future, computationally literacy will imply profound mathematical thinking (Microsoft Research, 2005), it seems rather probable that computational literacy will also imply a form of ‘concurrent’ thinking (Stein, 1999) that is able to couple recognition and understanding of formal processes with their other experiential scales and dimensions. Computation, in the light of such an interactive conception, will of course have implications for education, challenging it with the evidence of new modes of experimentation. Here, materials will need to be modular and flexible, in order to elicit a constructive approach to both content and knowledge development. Finally, interactivity will imply novel forms of enquiry: by providing programmable and flexible tools that encourage ingenuity and reflexivity.

To achieve these goals much has to be done, for example when reflecting on the question concerning the implementation of information technology in school curricula. Whereas much effort has been directed towards the distribution of computers amongst pupils and ensuring fast connections to the World Wide Web, little has been discovered concerning the actual forms in which information technology might develop and enhance useful learning models (Gärdenfors and Johansson, 2005). We once more return to ‘feral’ understandings of computation, cognition and design, and it seems that the same kinds of thinking might usefully be applied to education. The sphere of knowledge production, and therefore also partially, perhaps rather too partially, of education, is relevant to the question of interactive design in the expanded sense. Education's involvement in the production of knowledge practices and of intellectual, organizational and spatial subjectivation has accompanied, if implicitly, much of the debates concerning the understanding of computation (specifically artificial intelligence and its figuration of the position and kinds of intelligence) and the studies of cognition sketched earlier in this paper. Add to this the commonplace that: ‘there is nothing more political than education’ and the models we choose in the development of knowledge will have implications for the development of design and of societies.

Now, if the capacities for thought are distributed between people, artifacts and the ecologies they are in, make and wreck, the question of learning becomes a crucial one in the imagination of what is computable as much as what is thinkable. The consideration of education and knowledge in society suggests that ubiquitous intelligence already forms an uneven and fissiparous challenge to the simple proliferation of ubiquitous computing on all levels of life, and will itself demand or instigate new social and organizational approaches. The interactive approach, in becoming feral, articulates not only computation, but modes of social organization, learning and alternative modes of collective innovation. Here, we recall a cybernetic approach to systems development where:

… one of the key ideas the general theory embodies is the principle of recursion. This says that all viable systems contain viable systems and are contained within viable systems. Then if we have a model of any viable system, it must be recursive. That is to say, at whatever level of aggregation we start, then the whole model is rewritten in each element of the original model, and so on indefinitely.’ (Beer, 1973: 4)

This would mean that a feral approach to technological development would entail less formalised approaches to society at large. It would at one scale perhaps imply the formation of cybernetically reflexive collectives, a ‘recursive public’ composed of modular, but ungridded, elements, where distinct modes of subjectivation, technicality and organization co-cooperate in the devising of their own infrastructural logics (Kelty, 2008). Finally, design and cybernetics might find a common path, design with its emphasis on ‘participatory’ approaches and cybernetics in its consideration for the development of complex systems where the system cannot be separated from acts of observation and involvement.

For designers, considering such a principle of recursion means accepting a major responsibility in the development of systems, and also the responsibility of constantly reflecting back to their own sphere of activity, accepting and testing the task of actually designing design, and in so doing accepting their rather more minor role. Doing so requires an acknowledgement that creations devised by distinct professions might actually slip out, escaping description by initial intentions (Fuller and Haque, 2008). This implies a deviation from formal and discipline-oriented professional approaches to a more multivalent view, encouraging design to become more than a purely sectional interest in the life of objects to actually consider and take part in the political, social and cultural implications of its activity (Wood, 2007). Such a position imagines an expanded role for design as an activity, but also, conversely, it suggests a recognition of the wild expansion of the scope of people, thoughts and things involved in the activity of design and the selection, development, deprecation and implementation of technological systems.

Some Closing Remarks

Throughout the present work we have moved rather too rapidly – but hopefully suggestively – between distinct fields of enquiry, from computation, cybernetics and finally design all under the rubric of an ‘interactive paradigm’. We have attempted to argue that this paradigm, emerging from computation’s enfolding into the world, has prompted conceptual developments leading to new questions and formulations. However, central to our theme is the fact that none of this would have been possible without accounting for the development of new artifactual, technical and ubiquitous realities, where distinct artifacts have gone feral, interweaving with the subjective, social and ecological fabrics of our cities, lives and societies. The idea of ubiquitous computing, its characteristic embeddedness, but also partial containment, within diverse multi-scalar settings of many different kinds, has suggested the exploration of human-computer relations in ways that can no longer be seen as simply calm, but also partially wild, if they do not drive us so.

By transforming our understanding of what computation is and how it might be done; shifting from a view of computation as ‘centralized, sequential and result-oriented’ or as ‘computation as calculation’, towards an understanding of ‘computation as interaction’ (Stein, 1999), we are able to alter preconceptions concerning the practice of programming and computing itself. This implies a shift from the views of a science that once took the transformation of ‘inputs into outputs’ as essential, to one that now emphasizes a process orientated view of computation that extends beyond prescriptive algorithmic understanding (Wegner, 2003). Due to its strong link to cognitive science, inaugurated through the behaviourist school during WWII, and vice-versa, such transformations have found wider implications for computation and beyond. For cognitive science particularly, this has meant the proliferation of new approaches to the study of the human mind. Starting from the premise that the idea of mind itself extends beyond the confinements of ‘head and skull’ (van Gelder, 1995; Chalmers and Clark 1998), the importance of the organism's environment can be recognized as crucial in establishing the complexities of any considered cognitive unit. The relations I establish with my surrounding world are as much relevant in the study of my cognitive capacities as the workings of my brain. By considering its extension beyond the single body of an organism, such an approach challenges the webs of rationality and formalism that can only be sustained when we consider the intellect as something separate from the world I am constantly immersed in. It does not mean however that such boundaries cannot themselves be effective when enforced or entrained by other, wider, means. All such boundaries, in themselves, have meaning.

Such conceptual workings, that transverse both computing and cognition are more readily understood by revisiting their filiations to a third approach: the works of second-order cybernetics and its study of complex systems, including both humans and machines. This is a field of study and practical experiment epitomized by the introduction of the idea of ‘subjective dynamic construction’ into the complex web of feedback loops within a given system. Such ‘subjective construction’ cannot be made an account of without considering the characteristic embeddedness of any organism, entity or practice. Ultimately, this has lead to the development of the recognition of human-computer relations as highly situated in a way that their cultural, subjective and political dimensions cannot be ignored.

A proper conceptualization of the message handed down from second-order cybernetics might finally open new approaches to the study of human-computer interaction at large. Essentially it allows us to recognize that this same relation extends beyond the tight formal coupling once devised by behaviorist and formalist approaches to both cognition and computation. By considering the messy and informal aspects (‘subjective construction’) of human computer interactions, previous Fordist models of design seem to become obsolete and the trajectory lead in part by the ‘participatory’ approach sheds new light into the design of alternative ecologies of computation and interaction where user, artifact and designer, play roles that are shiftingly determinant rather than fixed.

If we initiated our discussion by speaking of ubiquity as key instigator of a new interactive paradigm, one that brought novel approaches to computing, cognition and design itself, we have attempted to finalize this dicussion by considering how ubiquity might embody this same feral quality. In this light the feral artifact is one that has been able to escape a domesticated and captive state. It is not a sedative, exuding calmness for an already understood, tracked and pre-empted subject whose needs and requests have been mapped out and whose life as such corresponds to that of a finite state machine. To design for a feral context is to work with processes of subjectivation that are not simply described by input and output, no matter how far away they are from traditional sites of computation. Instead, such a design works with artifacts, processes, ecologies, people, computation and politics, in a way that is able to explore the multiple potentials of the fields of interaction, calculation, control and cognition. When computing escapes from being unable to recognize itself outside of the formalisable, becomes promiscuous, starts finding itself recomposed in combination with packs of other epistemic currents, it becomes feral. When design recognizes that its expertise is distributed amongst people, institutions, media, infrastructures and in the dissensus, joy and confusion of contemporary lives and their technological avidity and starts to find means of generating a rigor in the chaotic, it too starts to lose its domestication. In doing so, recursively, it promises to produce something that renders their complex artifactuality sensible and open – with all kinds of difficulty – to design: to think with things.

Biographical Note

Matthew Fuller is author of various books including Media Ecologies, materialist energies in art and technoculture, Behind the Blip, essays on the culture of software and the forthcoming Elephant & Castle. With Usman Haque, he is co-author of Urban Versioning System v1.0 and with Andrew Goffey, co-author of the forthcoming Evil Media. Editor of Software Studies, a lexicon, and co-editor of the new Software Studies series from MIT Press, he is involved in a number of projects in art, media and software and works at the Centre for Cultural Studies, Goldsmiths, University of London. http://www.spc.org/fuller/

Sónia Matos is a designer and lecturer at Edinburgh College of Art who works at the intersection of ethnography, conceptual tool design, participatory interventions and writing. For the past five years, and as part of her doctoral studies at the Centre for Cultural Studies, Goldsmiths College, University of London, Sónia has dedicated her research to the study of a whistled form of language known as the Silbo Gomero, one that is still alive in the small island of La Gomera in the Canarian Archipelago – this body of research has culminated in the design of learning support materials for the Gomeran community. Her work has been presented at the 23rd British HCI Conference, the Chisenhale Gallery, London, as part of the '21st Century' research based program and more recently as part of the 'Prototypes for Transmission' organized by Constant – Association for Art and Media. She is currently preparing a new research project regarding food, design and ecology in the Azorean Archipelago.
.

References

• AfriGadget. ‘18 year old self-taught electronics “genius” invents mobile phone-based vehicle anti-theft system’, (2008), http://www.afrigadget.com/2008/07/16/18-year-old-self-taught-electonics-genius-invents-mobile-phone-based-vehicle-anti-theft-system/
• Agre, Philip E. Computation and Human Experience (Cambridge, U.K: University Press, 1997).
• Beer, Stafford. ‘Fanfare for Effective Freedom’, (1973), http://grace.evergreen.edu/~arunc/texts/cybernetics/Platform/platform.pdf.
• Beer, Stafford. Platform for Change (London: John Wiley and Sons, 1975).
• Boden, Margaret A. Minds and Mechanisms. Philosophical Psychology and Computational Models (Brighton: The Harvester Press, 1981).
• Bolter, Jay David. Turing's Man. Western Culture in the Computer Age (Chapel Hill: The University of North Carolina Press, 1986).
• Brown, John Seely and Weiser, Mark. ‘The Coming Age of Calm Technology (1)’, (1996), www.ubiq.com/hypertext/weiser/acmfuture2endnote.htm
• Chalmers, David and Clark, Andy. ‘The Extended Mind’, (1998), http://consc.net/papers/extended.html
• Diamond, David. ‘MIT Everyware’, (2003), http://www.wired.com/wired/archive/11.09/mit_pr.html
• Dourish, Paul. Where the Action Is: The Foundations of Embodied Interaction (Cambridge, Mass.: The MIT Press, 2004).
• Dupuy, Jean-Pierre. The Mechanization of the Mind: On the Origins of Cognitive Science, trans. M. B. DeBevoise (Princeton, NJ: Princeton University Press, 2000).
• Eglash, Ron. ‘Cultural Cybernetics: the Mutual Construction of People and Machines’, (2000), http://www.rpi.edu/~eglash/eglash.htm
• Fuller Matthew and Haque, Usman. ‘Urban Versioning System v1.0’ (New York: Architectural League of New York, 2008).
• Gärdenfors, Peter and Johansson, Petter. (2005). ‘Introduction to Cognition, and Communication Technology’, in Peter Gärdenfors and Petter Johansson (eds.) Cognition, Education, and Communication Technology (Mahwah, London: Lawrence Erlbaum Associates, Publishers, 2005), 1-20.
• Gibson, James J. ‘The Theory of Affordances’, in Robert Shaw and John Bransford (eds.) Perceiving, Acting, and Knowing: Toward an Ecological Psychology (Hillsdale, NJ: Lawrence Erlbaum, 1977), 67-82.
• Peter Wegner and Dina Goldin ‘Computation Beyond Turing Machines’, (2003), http://www.cse.uconn.edu/~dqg/papers/cacm02.rtf.
• González, Gilberto Esparza. ‘Urban Parasites’, www.parasitosurbanos.com/parasitos/proyecto.html
• Crang, Michael; Crosbie, Tracey and Graham, Stephen. ‘Variable Geometries of Connection: Urban Digital Divides and the Uses of Information Technology’, Urban Studies, 43.13 (2006): 2551-2570.
• Gornick, Naomi and Grout, Ian. ‘A Dialogue on the Future of Design Education’, (2008), http://attainable-utopias.org/tiki/academicpapers.
• Griffiths, Paul and Stotz, Karola. ‘How the Mind Grows: a Development Perspective on the Biology of Cognition’, Synthese, 122 (2000): 29-51.
• Hables, Chris Gray. The Cyborg Handbook (London: Routledge, 1995).
• Haque, Usman. ‘The Architectural Relevance of Gordon Pask’, in Lucy Bullivant (ed.) 4dSocial: Interactive Design Environments (London: Architectural Design, 2007).
• Haraway, Donna. Simians, Cyborgs and Women: The Reinvention of Nature (London: Free Association Books, 1991).
• Heims, Steve J. John von Neumann and Norbert Wiener: From Mathematics to the Technologies of Life and Death (Cambridge, Mass.: The MIT Press, 1980).
• Holmes, Brian. ‘Future Map, or How the Cyborgs Learned to Stop Worrying and Love Surveillance’, (2007), http://brianholmes.wordpress.com/2007/09/09/future-map/
• Hutchins, Edwin. ‘Distributed Cognition’, (2000), http://eclectic.ss.uci.edu/~drwhite/Anthro179a/DistributedCognition.pdf.
• Kelty, Christopher. Two Bits. The Cultural Significance of Free Software (Durham, NC: Duke University Press, 2008).
• Hollan, James; Hutchins, Edwin and Kirsh, David. ‘Distributed Cognition: A New Foundation For Human-Computer Interaction Research’, (1999), http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.33.6906
• Kline, Morris. Mathematics, the Loss of Certainty (Oxford: Oxford University Press, 1980).
• Latour, Bruno. ‘A Cautious Prometheus? A Few Steps Toward a Philosophy of Design (with Special Attention to Peter Sloterdijk)’, (2008), www.bruno-latour.fr/articles/article/112-DESIGN-CORNWALL.pdf
• Licklider, Joseph Carl Robnett. ‘Man-Computer Symbiosis’, in editor? IRE Transactions on Human Factors in Electronics, HFE-1 (1960): 4-11.
• Lyotard, Jean-François. Libidinal Economy, trans. Iain Hamilton Grant (London: Athlone Press, 1993).
• MacColl, Ian; Chalmers, Matthew; Rogers, Yvonne and Smith, Hilary. ‘Seamful Ubiquity: Beyond Seamless Integration, Equator Research’, (2002), http://www.equator.ac.uk/
• Mackenzie, Adrian. Cutting Code, Software and Sociality (New York, NY: Peter Lang, 2006).
• Mackenzie, A. ‘Intensive Movement in Wireless Digital Signal Processing: from Calculation to Envelopment’, Environment and Planning A, 41 (2009): 1294-1308.
• Maturana, Humberto and Varela, Francisco. The Tree of Knowledge, the Biological Roots of Human Understanding (Boston: Shambhala, 1998).
• Microsoft Research. ‘Microsoft Research Report. Towards 2020 Science’, (2005), http://research.microsoft.com/enus/um/cambridge/projects/towards2020science/downloads/t2020s_report.pdf.
• Muller, Michael. ‘Participatory design: The third space in HCI’, in Andrew Sears and Julie A. Jacko (ed.) Handbook of HCI (Mahwah, NJ: Erlbaum, 2003), 1061-1083.
• Murtaugh, Michael. ‘Interaction’, in Matthew Fuller (ed.) Software Studies- a Lexicon(Cambridge, Mass.: The MIT Press, 2008), 143-149.
• Bishop, Mark and Nasuto, Slawomir. ‘Second Order Cybernetics and Enactive Perception’, Kybernetes 34: 9/10 (2005): 1309-1320.
• Noë, Alva. ‘Experience without the head’, (2004), http://socrates.berkeley.edu/~noe/EWTH.pd
• Papert, Seymour. ‘Computer Criticism vs. Technocentric Thinking’, (1990),>http://www.papert.org/articles/ComputerCriticismVsTechnocentric.html
• Papert, Seymour and Turkle, Sherry. (1990). ‘Epistemological Pluralism: Styles and Voices Within the Computer Culture’, Signs 16 (1), From Hard Drive to Software: Gender,Computers, and Difference (1990): 128-157.
• Pask, Gordon. Conversation Theory: Applications in Education and Epistemology (Elsevier, 1976).
• Poerksen, Bernhard and von Foerster, Heinz. Understanding Systems: Conversations on Epistemology and Ethics (Heidelberg: Carl-Auer-Systeme-Verlag und Verlangsbuchhandlung, 2002).
• Preece, Jenny; Rogers, Yvonne and Sharp, Helen. Interaction Design – Beyond Human-Computer Interaction (Hoboken, NJ: Wiley, 2006).
• Ramsey, Doug. ‘Tracking Pollution and Social Movement: Love Fest for Calit2 Technologies at 'Make Fest 2007',(2007),http://www.calit2.net/newsroom/article.php?id=1080
• Rogers, Yvonne. ‘Moving on From Weiser's Vision of Calm Computing: Engaging UbiComp Experiences’, in Paul Dourish and Adrian Friday (eds.) UbiComp Proceedings 2006(Berlin/Heidelberg: Springer-Verlag, 2006), 404-421.
• Sengers, Phoebe; Steve Harrison and Tatar, Deborah. ‘The Three Paradigms of HCI’,(2007),
• http://people.cs.vt.edu/~srh/Downloads/HCI%20Journal%20TheThreeParadigmsofHCI.pdf.
• Shapin, Steven. ‘What Else is New? How Uses, Not Innovations, Drive Human Technology’,(2007), www.newyorker.com/arts/critics/books/2007/05/14/070514crbo_books_shapin
• Smith, Linda and Thelen, Esther. A Dynamic Systems Approach to the Development of Cognition and Action (Cambridge, Mass.: MIT Press, 1994)
• Stein, Lynn Andrea. ‘Non-Algorithmic Computation: Research Overview’, (1997),http://faculty.olin.edu/~las/2001/07/www.ai.mit.edu/people/las/papers/
• Stein, L. A. ‘Challenging the Computational Metaphor: Implications for How We
• Suchman, Lucy. Human-Machine Reconfigurations: Plans and Situated Actions (2 ed.) (Cambridge:Cambridge University Press, 2007).
• Wegner, Peter. ‘The Paradigm Shift from Algorithms to Interaction’, (1996) http://www.cs.brown.edu/people/pw/papers/ficacm.ps.
• Wegner, P. ‘Computation Beyond Turing Machines’, (2003),http://www.cs.brown.edu/~pw/
• Wiener, Norbert. Cybernetics: Or the Control and Communication in the Animal and the Machine(Cambridge, Mass.: The MIT Press, 1948).
• Wood, John. ‘Metadesign’ in Rachel Cooper (ed.) Design for Micro-Utopias. Making the Unthinkable Possible (Hampshire: Gower Publishing Limited, 2007), 167-184.
• van Gelder, Tim. ‘What Might Cognition Be, If Not Computation’, The Journal of Philosophy 92 (7) (1995): 345-381.
• von Hippel, Eric. Democratizing Innovation (Cambridge, Mass.: The MIT Press, 2005)

Introduction

Presently there are a number of gameplay definitions, ranging from Sid Meier’s famous assertion of ‘interesting choices’ to Richard Rouse’s concept in which the vibrant iterations of user input and machine output is the decisive factor (Rouse, 2005). Further, ‘gameplay’ seems to drift indefinably between psychological categories, as in the emphasis on the concept of ‘flow’, and more formal conceptualisations often derived from computer science. However, the discussion of how the realm of pervasive games necessitates both playing and gaming, play-mode and game-mode, seems to oblige a slightly different gameplay terminology. One level of gameplay is the actualisation of a specific stratification of rules, strategies, and interactions as well as the realisation of a certain amalgamation of commands, plans, and paths. A second level, which is equally or even more important since it explains the inherent fascination of playing games, is the interconnection of playing in an open environment and focusing on presence and gaming within a fixed, discrete framework (the ‘board’) and paying strict attention to progression.

The purpose of this article is to inspect the theoretical consequences of moving from a traditional, ludological concept of computer games to an extended ludology of pervasive games. In the process of unfolding these consequences our general understanding of play and games—these straightforward and yet highly convoluted terms—will hopefully also be sharpened. Such an assessment should rivet reflections that take the heightened emphasis on physical space as well as the contingency of socio-cultural activities in pervasive games seriously. It should further critically investigate the notion of the virtual. This is done in the third section where I employ Gilles Deleuze’s Bergson-inspired claim, that the virtual is indeed more ‘real’ than any present actualisation, as a backdrop against the hypothesis that, rather than invoking a common sense dichotomy between a tangible reality and the informational bits and bytes of the computer, we should instead focus on the discontinuous relation between virtual play and actualised gaming as the pivotal modus operandi of pervasive games. The article can be read as a continuation of the work on game theory laid out in Walther (2003, 2005, 2006, 2007a, 2007b, 2007c). Consequently, the noticeable lack of empirical references in this article is not the result of a normative disapproval of the many and very interesting experiments with pervasive technology that flourish these days, but on the contrary must be regarded as a rather rigid attempt to enlighten the deep axioms and often ambivalent key conceptions underneath the world of digital games.

Rules and Gameplay

In line with economic game theory we can define games as complex, rule-based interaction systems consisting of these three key mechanisms: absolute rules, contingent strategies, and possible interaction patterns. Game rules are absolute in the sense that while the players may question the rationality of the rules at hand, they are nevertheless obliged to obey, to ‘play by the rules’. Rules are therefore absolute commands (Neumann & Morgenstern, 1953) and unquestionable imperatives. They transcend semantic issues, cultural signification, moral agendas, etc. This does not, incidentally, preclude the fact that game rules are discussed in a cultural or ethical milieu.

In contrast to rules, strategies are contingent, nonabsolute entities since they count as the more or less detailed plans for the execution of turns, choices, and actions in the game. Other strategies than the ones actually carried out could have been outlined and performed. Both in the shape of short-term tactics and as long-term schemes, strategies are contingent. In economic game theory, a strategy is an overall plan for how to act in the assembly of different states that the game may be in (Juul, 2006). Game theory studies the affiliations of the rules and the strategic behaviour in competitive situations (Smith, 2006). Finally, interaction patterns are the moves and choices, which become part of the game being played, thus interfering with the restrictions and options of the game. As the implementation of game strategies tend to cluster in selected regions of the possibility space of the game (in approximation of what is known as the ‘dominant strategy’ in game theory) forming a path through the game space,[1] we may even insinuate that the interaction patterns, taken as a whole, are the game itself—especially if we view it from the perspective of the player (Holland, 1998). Interaction patterns are the possible as opposed to necessary combinations or the emergent outcome of rules and strategies. This differentiation can be listed even more briefly:

• - Rules are commands
• - Strategies are plans for game executions
• - Interactions patterns define the actual path through the game and specify the topography of human-computer (or player vs. rule) dynamics

Clearly, the interaction patterns work as ‘middle ground’ as they occupy a domain located between the machine that upholds the rules (the computer) and the human player who has to find and optimise the best way to accomplish the goal of the strategy.

The notion of game play, which we shall pursue also as regards the relation between free play and confined gaming, involves all three levels of a game, which also explains the difficulty in defining the concept properly. We can refer to the following definition as the ontological or formal definition. The definition is ontological because it assumes at least the minimal and necessary (axiomatic) existence of some quasi-material, algorithmic entity:

[Gameplay: Definition 1]  Game play is the actualisation of a specific stratification of rules, strategies, and interactions as well as the realisation of a certain amalgamation of commands, plans, and paths.

For a player, a successful game play means a delicate balance between knowing the rules and mapping one’s strategy in accordance with both rules and the possible actions of opponents. Games should be equally challenging and rewarding, hovering between boredom and anxiety thereby assuring a space of flow through the network of choices. For a computerised game system, a successful game play implies a balance between fixed rules and the control of player input in variable settings.

What defines a rule? A rule, being algorithmic in its core design, consists of a simple, unequivocal sentence, e.g. ‘you are not allowed to use hands while the ball is on the pitch’. Here, a rule constitutes the possibility space of a game by clearly stating limitations (not use hands) as well as opportunities (the ball is on the pitch). It is always possible to define a game both in negative and positive terms: rules limit actions; they determine the range of choices in the possibility space; they encircle the arenas to be played in; yet they also frame what can be done.

At this point, I am speaking of all games, both traditional games, including sports, and computer games. Heroes of Might and Magic rests on rules stored in and processed by a computer. Chess or Monopoly, by contrast, relies on rules not accumulated in the database and algorithms of a computer but written down on paper and stored in the player’s mind during the play. In a game of soccer, for example, a referee administers such rules ultimately by reference to the FIFA Handbook. Implicit rules that are normally considered exterior to the ‘real’ rules (e.g. a time restriction in chess matches) must be engaged explicitly in digital games. These rules have to be programmed as well. Weather conditions or the general physics of a soccer game are usually taken as ‘out-of-game’ features in the real world. When we simulate a soccer game in a computer, however, the rules of soccer and the general physics (including random variables such as surface granularity, crowds, time of day, etc.) must be built into the rule algorithms and the input-output control of the computer.

Rules specify the constitution of the playing ‘deck’ or, more broadly, the playing ‘field’. In games, behavioural patterns inside this field are limited, constrained, and highly codified (Huizinga, 1994; Caillois, 2001; Walther, 2003). Rules are guidelines that direct, restrict, and channel behaviour in a formalised, closed environment so that artificial and clear conditions inside the ‘magic circle’ of play are created (Salen & Zimmermann, 2004). The outside of this circle, reality or nonplay, is essentially irrelevant to game play. Confronted with unambiguous rules, strategies (or tactics) might entail best practice solutions variable to the given rule constraints. Hereafter, interaction patterns map the various player interventions and can hence be viewed as a texture of moves and choices overlain on top of the possibility space of the game. Furthermore, interaction patterns can refer to the social and competitive intermingling of players during the fulfilment of the game. In that respect, the patterns correspond to the outcome of absolute rules and social dynamics.

The formal organisation of games can be regarded as a parameter space. In this space, the current state of the game counts as a point and ultimately a dimension in the parameter space. A played game has therefore n possible state dimensions. In Tic-Tac-Toe, for instance, the nine squares constitute the parameter space of the game and thus the possibility domain for the arrangement of the board pieces. The rules of the game define the possible edges in the space connecting states, and the total number of discrete points in the parameter space represents the total number of games states. Rules define the possible game as in the initial framing of the game, whereas a particular game is a path through the state space. This latter particularity rests, consequently, on a variability space upon which one can also measure the optimum rate or success probability of the system. The crucial factor is that there can be no variability or multiple paths through the possibility space of a game without the compulsory parameters of the game. Hence, the parameter space constitutes the transcendental level of the game, whereas the particular game path expresses the contingent realisation of the space.

This dialectic between parameter space and actual game path (or variability space) also sheds some light on why games are complex; basically it is because there is an uneven relation between the unchanging set of rules and the actual and changing realisation of a particular game. This asymmetrical tie between rules and realisation (or rules and strategies) can be termed game emergence. Most often it is impossible to predetermine the actual moves and outcome of a game only by knowing the set of rules. Also, most games are games of imperfect information (Nash 1997). At the outset, the rules of chess are simple, and yet the wealth of distinct chess playing tactics is quite enormous. A child can memorize chess rules, but to master all grand openings in the actual game is probably a lifetime achievement.

Play-mode and Game-mode

It is however a characteristic feature of many types of framed differentiations, or ‘fiction’ (in the widest possible sense, and notwithstanding if this feature counts as an impossible Utopia), that they wish to expand the gaming space, physically and mentally, often by reconfiguring the social landscape of worlds into a dense grid of game objects, game goals, and game worlds, thus obscuring the demarcations between the real and the virtual. Very often, stories, movies, games, and other narratives play with these demarcations.

What, then, is most important? Is it the game itself, or is it rather the social and geographical infrastructure that supports it? Who (or what) has the upper hand? Is it the relational complexities of the characters or other personified ‘avatars of story’ (Ryan, 2006) or perhaps the fluent vectors of the game world? In the following we will explore this tension between the telic game orientation and the presence of a world surrounding the former by drawing upon the recent paradigm of pervasive computing (Walther, 2007b; Walther, 2007c). [2]

There is a fine line between being there, somewhere, and being there with a purpose. The mundane space that a human subject inhabits is not by nature geometrical; rather, it is structured in accordance with matter-of-fact actions. In such a spatial environment, the various orientations are related to directions (practical vectors), places, ranges of space, and things, in contrast to dimensions, points, lines, and absolute objects. The space for action is a praxis-architecture or a phenomenological space that is not defined by length, height, and width, but rather by territory, proximity, and distance (Nielsen, 1996). A personal space zeroes in on the required equipment and relations to institute meaning, whereas a geometrical space is continuous and unbounded.

Territorial exploration, whether in the real, physical world or in the flowing realm of one’s fantasy, involves the incessant modification of intentions. It is an advanced procedure of trial-and-error set in a socio-semantic circumstance. You go right. Not interesting. You move to the left. Wait, here’s something. You rush straight forward, and the result is immediate and loads of action. Such movement requires both cognitive mapping and a basic perception of metric coordinates. However, the elusive co-existence of being present and intentionally moving around for a reason is also known as rules. It subsumes at least three important characteristics of structured, goal-oriented activity: momentum + direction (vectorization) + a valorized and quantifiable endpoint or outcome. Mapping a place through adventurous discovery in order to figure out the story underneath the space, and possibly inventing new ones in the same process, is all about playing. Learning to move and advance in a space filled with discrete norms of orientation, i.e. a parametrical space, meaning that you can do this but not that, and that you can go here but not there, is the art of gaming.

Thus, there are two firmly interwoven modes of game epistemology: there is play-space and there is game-space. Accordingly, there is play-mode and game-mode. Together they form the much hyped and commonly misunderstood term ‘game-play’. We call those games that mix up the tangibility of everyday spaces with the closed information spaces found in digital computers pervasive games. Such games may be the next generation in computer games. Make people move around. Don’t tie them in front of the screen. Moreover, these games are particularly captivating because they deliberately place the relation between rules and world voyaging, gaming and playing, the parameters of games and the variables of play, at the nucleus of the very rule system itself. In other words: you learn how to master the rules of the game by playing them out in the real world. Pervasive gaming is game-play out in the open.

In the play mode, one does not want to fall back into reality (although there is always the risk of doing so). In the game mode, it is usually a matter of climbing upwards to the next level and not losing sight of structure. Play is about presence, while the game is about progression (Walther, 2003). Play-space could be a city, and game-space could be the rules and informational network dictating what can and cannot be done during game-play. Or, to rephrase this in abstract terms, play-space could be a fictitious world, with its binding rules, and game-space the rules and missions within this world: the teleology of the protagonist, the endpoint of his trajectory.

Play is also about uncertainty, and herein lies the irreducible element in play which, according to Roger Caillois, makes it inaccessible to mathematics (Caillois, 2001: 173). Complete transparency derived from calculation and perfect strategy means the disappearance of player interest together with the pleasurable uncertainty of the outcome.

Look at people playing. One notices that there is always the inherent but beguiling hazard of being ‘caught’ in reality. Nothing is more distressing for play than the aggressive intermission of reality which at all times jeopardizes play as play or simply threatens to terminate the privileges of play. Then it’s back to normal life—which may be, incidentally, a giant gamespace in its own right, as McKenzie Wark suggests (Wark, 2007; cf. also Galloway, 2006). This is, of course, a structural feature of all play and of all game-play. This is true of chess and soccer. It is also apparent in Doom and Myst. Interruption and termination must be avoided at all costs—in the continuous pursuit of having fun—but, since they are inescapable, they must be built into the very ‘being’ or ontology of playing games.

Now, consider pervasive gaming, game-play out in the open. As a player I rush down a street in order to amass my next item to be uploaded via my PDA so that my game-buddy at home can keep track of my doings and goal-seeking so far. It’s 4 pm, there is heavy traffic, and I am momentarily barred from reaching the corner with the alacrity I wished for.

We witness a growth in the design of game systems that use ubiquitous computing techniques to propel forward player experiences that connect objects within the real world with objects of the computational world. SuperFly, by the Swedish game company It's Alive Mobile is a good example. The player's aim is to become a virtual celebrity. The projects Can You See Me Now? and Uncle Roy All Around You, both created by the UK performance group Blast Theory, use hand-held digital devices, GPS location tracking, and online agent technology in an attempt to use location and mobility as game features from within the real world. While one player stays at home and moves a virtual character around a representation of a real city, other players speed around the real streets, trying to hunt down the virtual quarry.

In chess, there are no strident interruptions between two or more discrete fields. I move my queen independently of physics, be it weather, traffic jams, or the occasionally bad habits of my fellow citizens. In a game of soccer, you block your opponent, and he tries to tackle you. However, a nice set of training principles that look for ways to avoid the physicality of blocking is always an option. That is what the refinements of dribble are all about. In pervasive game-play, mixing play-space and game-space, ‘real’ problems, as the ones described above in my own thought experiment, remain real problems. If not, the aesthetics of producing eloquent game mechanics turns into a matter of ethic. I do not, in the quest of fulfilling the game’s teleology, knock down the old—real—lady on the sidewalk only because she is refraining me from targeting the ‘pac man’ further down the road a little bit faster.

Therefore, we must be careful in judging the fun factor of game-play. It is not only the city, the social and geo-graphically expanded context, in itself, that is the locus of enjoyment in pervasive game-play. Yes, I can go explore, and yes, I meet people, and yes, the site of navigation has become much wider than a trivial board. Nevertheless, the bouncy guarantee of space might indeed become the constraints of the game. Serious gamers do not want to waste their time looking for ‘interesting’ places to explore. They much rather want to understand the structure so as to move forward revealing new game areas or climb upwards in the hierarchy of levels.

As we shall see later on, this veiled and all-important prerequisite of playing games, including the pervasive ones, is contained within the notion of ‘the virtual’. Being a conditional causa sui of the actualised game, and the gameplay that unfolds in the being-present, the virtual explains in its pure form the dual and much overlooked nature of gameplay: the virtual is the ‘past’ of the ‘now’ of gaming since we must always silently remember the enjoyable and playful offspring of a game, and, at the same time, the virtual drives the articulated though momentarily non-fulfilled target point of gameplay that is the ‘future’.

Why? Because play is centred in a discovery of open spaces that invite observation through the duration of temporality. Gradually, one learns how to pilot inside play, and since the completion of more and more successful tasks takes time, it corresponds to the distinctive forms that keep differentiating the play system into finer grades of subsystems. One inhabits spaces like these via certain as-if structures; one assumes a role and lives out characters whether in the form of other players or agents that one can adapt as a player. The gamut of play equalises a measurement of its geometry—how big is the playing field, and where are its borders? And these lengths and widths become in turn the source of gaming's internalisation of both geometrical space and discrete progression.

In contrast, play seems to focus on investigations of semantics, since the task is, not only to measure its space, but furthermore to elaborate upon its modes of interpretation and means for re-interpretation. Not only do we explore a world while playing, its potential meaning and the stories we can invent in that respect also drive us. Play spaces tend to expand, either in structural complexity or in physical extent. This expansion is further reflected in the praxis of play, for instance when players argue over the exact thresholds of a play domain. Another feature that distinguishes playing from gaming is the notion of presence, as I pointed out earlier. Obviously, the sensation of presence is tightly interwoven with phenomenological concepts like ‘immersion’ and ‘flow. Play commands presence. We have to be there—not only be there, but also be there. This is the double meaning of Heidegger’s dasein: dasein (being there) and dasein (being there). We go with the flow; or, rather, while swallowed by the presence of playing we are in the flow, as Mihály Csíkszentmihályi claims (Csíkszentmihályi, 1990). A game’s success is intimately tied to the organisation of space and time. Gamers need to trust this organisation. Since a game hinges on a certain finite structure in order to promote infinite realisations of it—the correlation of rules and tactics—the very articulating of presence so important for play must already be presupposed in a game. One already knows in a game that the mission is to keep on gaming, which really means, in my vocabulary, to keep on playing, that is, to prolong the sensation of presence. The energy can then instead be directed towards elucidation of the game’s structure: ‘How do I get to the next level?’ and not ‘Why do I play?’ This keeping on, knowingly or unknowingly, is the virtuality of games.

Although one should indisputably respect the ethical boundaries of pervasive games that transport game-play out in the open—just as one should bear in mind that the metaphysics of fictional worlds often goes beyond the natural laws and moral confines of everyday life; one does not want to hang on too long for the old lady to cross the street. While waiting, the question above might turn up thus threatening to disintegrate the exquisitely balanced halves of gaming (to progress) and playing (to be present).

Thus, we can put forward the second definition of gameplay. In continuance of the first one, we can refer to the following definition as the epistemological or player-oriented definition:

[Gameplay: Definition 2]  Gameplay is that kind of player activity that intentionally involves the asymmetrical relation between world exploration and level progression.

What do the novel features of pervasive games, including play-mode and game-mode, mean with respect to rules, as we have defined them above?

• - Rules are commands. However, since pervasive games often take place in a physical context that is characterised by socio-cultural contingency, rather than the discrete controlling of parametrical progression, rules should therefore be seen as deep commands within a variability space that, simultaneously, allow for the more or less intentional modification of rules.
• - Strategies are plans for game executions. However, while strategies in strictly confined ‘boards’ with clearly defined edges may be easier to control (and hence to realise), strategic actions in large pervasive games building on the complexities of both real and computational space tend to evolve into even more complex sub-strategies and perhaps unforeseen interaction patterns.
• - Interaction patterns define the actual path through the game and specify the topography of human-computer (or player vs. rule) dynamics. True, but while the topography of traditional computer games rests on a topological level structure which enables, for instance, a virtual teletransportation from one ‘dimension’ to another, the topography or path of large pervasive games—typical of urban gaming—is rather based on the geometrical constraints that are bound to the actual physicality and linked with the socio-cultural contingency of the gameworld.

In the following we shall qualify the notions of play-mode and game-mode further by relating them to Deleuze’s concept of the virtual.

Virtuality

A prevailing notion of ‘virtuality’, which was especially dominant in the cyber theories and virtual reality oriented writings of the nineties, sees it as a kind of spatial and epistemological liberation from Cartesian geometry (Ostwald, 1997; Lunenfeld, 2000). The kind of space that Descartes had in mind, in his Meditations on the First Philosophy, is organised in accordance with Euclidian mathematics and categorises spatial elements in a general and absolute sense. In Euclidian space there is no resistance; it is an ethereal morphology containing absolute, non-contingent relations. A typical cyber theory goes on to claim that the spatial form of the computer (and its spatial representation) more willingly obeys the laws of differential topology, which describes spatial singularities or catastrophes. Cartesian space is analogue with its emphasis on measurable planes, geometrical continuity, and friction-less dimensions, whereas the topological space of Henri Poincaré is digital and discontinuous. The effect of the latter is the displacement of spatial orientations (such as up/down and inside/outside) and the novel possibility of constructing latitudes with infinite dimensions that threaten to adjourn the space as object.

Moreover, the virtual has generally been considered as the apex of a media and artistic evolution. Thus, an important distinction between static images in which the perspective is tied to the observer, and virtual reality implemented, interactive environments, is the user’s liberated command over spatially distributed point of views. Ever since the Italian Renaissance and the theories and works of art by Alberti, Leonardo, and Botticelli, the painted arts have tried to master a linear vista so as to create depth, visual consistency, matte relations, and smooth continuity. But the idea of artistic creativity in the Renaissance nevertheless rests on the artist and the way he manipulates the experiences and senses of an audience. The celebrated trompe l’oeil is a technical culmination of this conceptual and visual striving for perfection that can be regarded as a pre-form of augmented reality: a frame within the frame or an illusion within the illusion itself. Later, in photography, the point of view is still linked to physical setting of a camera in concrete space-time. Motion picture and television finally release the perspective into moveable and oscillating events in time. Not only do images move around in front of the camera lens—the camera is itself mobile in relation to the actual viewpoint. Yet, this dynamic mode of representing and presenting space-time has a certain limitation in perspective because the photographer or director completely determines the POV. It is only with the advent of virtual reality ‘that the user can have substantial visual control of the scene’ (Bolter, 1996: 113).

Gilles Deleuze’s definition of virtuality and the virtual that he inherits more from Aristotle and Spinoza than from Plato and Descartes, and which he elaborates particularly in Bergsonism, Difference and Repetition and The Logic of Sense, is remarkably different from the above. Interestingly, he seems to arrive at a classification of the virtual—including an assumption of metaphysical grounding, to which I will return—not because of the concept in itself, but rather as a response to a problem with ontological implications posed by structuralism. How can the underlying structures or systemic constructions exist if they do not belong to either the mind of a subject or the material world of objects? Deleuze’s answer is that these structures, which act as conditional guidelines for present actualisations, represent something that is, ontologically speaking, ‘in-between’ subject and object, and this is exactly virtuality. How is it possible, Deleuze asks, for something to be a condition of being, to be a catalyst of an actualised articulation or presence, while not being discernible or measurable, or being located in one particular person’s mind, or otherwise embedded in a material world? The difficulty is to establish the exact ontology or existence of such relational, determining structures, i.e. to insist that this ontology is eo ipso the most comprehensive form of reality and, at the same time, a detached and left behind reality. Part of the answer lies in Deleuze’s deliberate Kantian design. Similar to Kant’s notion of space and time as unifying forms imposed by the subject that do not, in themselves, exist in space and time, Deleuze holds that pure difference is non-spatio-temporal—reality without actualisation, ideality without abstraction.

The reason why ‘the virtual’ commonly appears obsessively contrary to reality is because we fail to acknowledge virtuality as the ‘real’ condition of actuality, and also because we one-sidedly prioritise the actual as the being-present over the virtual as pure Being. The virtual is neither non-existence nor ontic occurrence; rather, it is a real system of differential relations that creates actual spaces, times, and sensations. Deleuze writes in Bergsonism:

We have […] confused Being with being-present. Nevertheless, the present is not; rather, it is pure becoming, always outside itself. It is not; but it acts. Its proper element is not being but the active or useful. The past, on the other hand, has ceased to act or be useful. But it has not ceased to be. Useless and inactive, impassive, it IS, in the full sense of the word: it is identical with being in itself (Deleuze, 1988: 55).

Could it be that ‘play’ belongs to the virtual and ‘game’ to the actualisation of being? And what does that mean in relation to the philosophy of pervasive gaming that seems to be more attentive towards explorative play than the traditional emphasis on level progressive gaming? Let us pursue this thought in the following.

At least in traditional computer games, play-mode works as the virtual or relational condition for the present activity and the ontological materialisation we call ‘the game’. This condition is more than the underpinning algorithms and digital codes but must be considered the abstract, or even ideal, base of games as such. There is a condition, a raison d’être upon which the materialisation of each game depends and that condition is play. One could argue that play is an ontological praxis; but it is certainly also an ontological condition. Continuously, new games arise and old ones cease to be—not just in an inventive but also in a pragmatic sense. Contrary to this constant activity in game-time, the inexorable pastime of playing games, ‘play’ has the strange quality of being discontinuous in time: somehow it is there all the time, as a completion of the idea incarnated in a specific game, and, at the same time, it is not there since it cannot be found in the integral whole of any particular object. The game is the discrete being or entity that seems to discriminate the conditional past (play) as abstract and fictional. However, as Deleuze states in chapter four of Difference and Repetition, ‘Ideas and the Synthesis of Difference’:

The virtual is opposed not to the real but to the actual. The virtual is fully real in so far as it is virtual […] ‘Real without being actual, ideal without being abstract’; and symbolic without being fictional. Indeed, the virtual must be defined as strictly a part of the object—as though the object had one part of itself in the virtual into which it plunged as though into an objective dimension (Deleuze, 1994: 208f.).

Virtuality is not the unreal in a stark contrast to the actual existence of either ideal subjectivity or material objectivity but, rather, the determined structure or ‘embryonic’ element that completes the actualisation more than makes it whole. Following from this we can say that a game is a discrete, unconnected and actualised entity that belongs to one and the same relational structure or system—the latter being ‘play’ or the Bergsonian pure past. While the game is experienced in time as a discontinuous, ontological emanation, play accounts for the process in which we, unknowingly, move from the virtual domain of ideal relations and singularities that characterise a system to uniquely actualised entities. Whereas games constantly move about and reorganise themselves into discrete actualities with epistemic qualities, the ‘purity’ of play withstands this perpetually passing-by through a strong sense of simultaneity. Play, then, completely determines a game but is only a part of the object.

There is of course the danger that ‘past Play’ becomes a secret transcendental teleology of ‘present Game’, or that the virtual-as-Substance turns into a Spinozian metaphysic. In temporal terms, the actual (which we call game-mode) corresponds to chronos, the pure present, whereas the aion, or the virtual (which we call play-mode), is the pure past and the condition of chronos. A certain impassivity therefore clings to virtuality as an abstract ontological memory that insists in all actuality and yet can never be said to contribute anything to the concrete instances of our chronological time. We would then fall prey to the Spinozian idea that everything that exists is a modification of the one substance.

Does this imply that games cease to exist the moment they occur and that, by contrast, play is composed of the real substance that somehow endures despite the constant fragility of relational organisation? No, insofar as play is the ‘past’ of gaming’s ‘presence’, this does not mean that play contributes nothing to actuality and disappears as a tragic but true Being. Rather, the two levels, substance and actualisation, playing and gaming, the ontology of the virtual and the epistemology of the present, exist in a continuous state of flux, a nomadic unrest that perpetually reproduce the relational encounters from moment to moment and from one game state to another. This is especially true in the later writings of Deleuze—the one who turns to chaos and complexity theory—where the distinction between the virtual and the actual progressively disappears, and we instead get interrelating processes where events reproduce certain patterned relations or organisations.

Taking off from Deleuze we can pursue the proposed link between the virtual past and the being-present of gameplay. I suggest reading the Bergsonian framework, not as a comprehensive metaphysics of time, but as a conceptual scheme of causal conditioning. The following figure illustrates the double realm of virtuality.

Fig 1.Virtuality and Actualisation in Game and Gameplay

First, the virtual accounts for the underlying past of the actualised game—resulting in present gameplay—as it separates itself from the other of play, i.e. reality or nonplay. This is how we described the virtual in the above. However, there is a second and just as important characteristic of virtuality in that it also occupies the teleological domain of games. Here we find the level oriented progression and goal of gaming and, in a way, this very modality of the virtual signifies the irrefutable ‘arrow’ of play and games. Traditional videogames as well as novel pervasive games always possess this telic attribute—heading, with or without detours, for the quantifiable outcome and the Utopia of winning. Furthermore, the two poles of virtuality, both pointing directly towards the now of gameplay, are intimately coupled with the transitional localities in games. Dangerously near the edge where a game exactly stops being fun to play, play-mode is a feeble confine because of the latent likelihood of stepping out of the magic circle and back into reality. Game-mode bears a similar stamp of transitional delicacy due to the all-pervading thread of blocking the uni-directional gameplay thus freezing the player between two levels and denying him the bliss of advance, access, and the potent omen of triumphing.

Concluding remarks As I see it, the Deleuzian outline represents a step forward from the premature and overtly hyped understanding of virtuality as the ontologically ‘un-real’. Rather than to focus on whether the virtual, in the traditional cybertheoretical sense, dictates an essentially different metaphysic or a whole new set of phenomenological qualities, we should enquire into the differentiating and imposing forms of play and games as they mutually determine the logic of virtually conditioned actualisations in time and space. This way, we do not need to come up with any new metaphysic or phenomenology. Furthermore, this approach emphasises the virtual as the transcendental, non-spatio-temporal, and driving force behind the praxis of play with nevertheless actual and real implications. The virtual is not play, but it in-forms its being. The virtual is not (the) game, but it shapes and sharpens its imminent horizon. Virtuality becomes a conceptual tool with which to describe both the playful past and the goal-oriented future of gameplay—thus being a kind of synonym for the what if and as if of the homo ludens. In fact, this is an anti-Deleuzian claim; rather than the call for a ‘Body without Organs’ that thrives in playspace like a madman amidst the open-ended, smoothed immanence of nomadic anarchy (Deleuze, 1987: 149ff.), the underlying thesis here is one of an inevitable return to a striated, grid-fixed game universe. Ultimately, ‘flight lines’ becomes telic lines; not necessarily unmovable guidelines, but guidelines nonetheless.

Gameplay is the actualisation of rules, strategies, and interaction patters as well as a non- equlibrious poise between explorative play and level-oriented gaming. Pervasive gaming, as we have seen, revitalizes exploration as the blood and bones of naturalised gameplay; and yet it never completely abandons the telos of the discrete, parametrical, and competitive. The virtual domain, including the past of play and the future of games, is hardly ever questioned as such in gameplay; the conditioning forces always ensure the unsaid and proper framework within which play and games can take place. This way, the virtuality of games is perhaps not just the logico-formalistic precondition of gameplay as it hovers between the mimicry of play and the teleological, rational desire of progression, but a violently imposing ‘discrete ideology’, as Slavoj Zizek would say (Zizek, 2008). [3] The constant reterritorialisation of the virtual that happens in the course of playing games is thus a play within the virtual much more than it is a play with the virtual. And, as it happens, Zizek hates Kung Fu Panda: ‘If you ask me for really dangerous ideological films, for ideology at its purest, I’d say Kung Fu Panda. I saw it five times because my son likes it. The movie is extremely cynical in that you know they make fun of all this ideology, of Buddhism and these things, but the message is even though we know it is not true and we make fun, you have to believe in it. It’s this split of you know it’s not true but just make like you believe in it’ (New York Entertainment, 2008).

Biographical Note

Bo Kampmann Walther is Associate Professor at the Center for Media Studies, University of Southern Denmark, and has written, taught, and lectured extensively on new media, computer games, and (mediated) sports. Read more at http://www1.sdu.dk/hum/bkw

Notes

[1] It is a commonplace of screen-based, contemporary computer games to equate this dominant strategy with graphical trajectories so as to ensure a dominant path in the otherwise seemingly ‘free’ world of the game. See Espen Aarseth’s analysis of Half-Life 2 for a comprehensive documentation of this aspect (Aarseth, 2005).

[2] I should stress that when I am referring here, and in the remainder of the article, to ‘pervasive gaming’ I exclude persistent games (Everquest, World of Warcraft) as well as Alternate Reality Games and MMORPG’s, on the one side, and mobile games (existing computer games reshaped for cell phones and other handheld devices) on the other. Thus, pervasive gaming represents a technological paradigm that relies on adaptronics, wearable, mobile, or embedded software and hardware in order to facilitate a ‘natural’ environment for gameplay that ensures the explicitness of computational procedures in a post-screen setting. As a result, pervasive games frequently use GPS, various types of wifi (and Bluetooth), and signal triangulation techniques. In true pervasive gaming the physical environment must be prepared, technologically, for mobile, location-oriented gameplay. Consequently, there is a fine line between the natural environment that facilitates the played game and the natural environment that is responsive to gameplay.

[3] I am not claiming that Zizek should be against games on the above grounds—in fact, he isn’t (cf. http://scrawledinwax.com/2008/09/15/wax-scrawls-zizek-on-ideology-and-video-games-crashing-parties-and-sms-art/). I am merely suggesting that a certain Adorno-Foucault inspired critique of the underlying instrumentalism of competition and masculinity clearly predominant in the culture of play and games will find it relatively easy to pinpoint the ‘unknown knowns’ of our object. Sadly, any thus far proposed counter strategy that would work against the ideological powerhouse of gaming—attempting to subvert the violent logic of games; desiring to compose non-teleological gameplay; invoking ‘tragedy’ rather than the usual potent endings of games—is precisely against the point, that is, outside the magic circle.

References

• Aarseth, E. ‘From Hunt the Wumpus to Everquest: Introduction to Quest Theory’, in F. Kishino et al. (ed) ICEC 2005, LNCS 3711, 2005.
• Caillois, R. Man, Play, and Games (Urbana: University of Illinois Press, 1958, 2001).
• Csíkszentmihályi, M. Flow: The Psychology of Optimal Experience (New York: Harper and Row, 1990).
• Deleuze, G. A Thousand Plateaus (Minneapolis: University of Minnesota Press, 1987).
• —. Bergsonism (New York: Zone Books, 1988).
• —. The Logic of Sense (New York: Columbia University Press, 1990).
• —. Difference and Repetition (New York: Columbia University Press, 1994).
• Holland, J. John. Emergence. From Chaos to Order (Oxford: Oxford University Press, 1998).
• Huizinga, J. Homo ludens. Vom Ursprung der Kultur im Spiel (Rowolt: Reinbek, 1938, 1994).
• Juul, J. Half-Real. Video Games Between Real Rules and Fictional Worlds (Cambridge, Mass.: MIT Press, 2006).
• Lunenfeld, P. Snap to Grid. A User’s Guide to Digital Arts, Media, and Cultures (Cambridge, Mass.: MIT Press, 2000).
• Nash, J.F. Essays on Game Theory (London: Edward Elgar, 1997).
• Neumann, J. von and O. Morgenstern. Theory of Games and Economic Behavior (Princeton: Princeton University Press, 1953).
• New York Entertainment: What Did Slavoj Zizek Think of ‘Kung Fu Panda’ Anyway? December 9, 2008.
• Ostwald, M.J. ‘Structuring virtual urban space’, P. Droege (ed.) Intelligent Environments (Amsterdam: Elsevier, 1997).
• Rouse, R. Game Design: Theory & Practice (Plano: Wordware, 2005).
• Ryan, M.-L. Avatars of Story (Minneapolis: University of Minnesota Press, 2006).
• Salen, K. and E. Zimmermann. Rules of Play: Game Design Fundamentals (Cambridge, Mass.: MIT Press, 2004).
• Smith, J.H. ‘The Games Economists Play – Implications of Economic Game Theory for the Study of Computer Games’, Game Studies 6, 1 (2006), http://gamestudies.org/0601/articles/heide_smith
• Walther, B.K. ‘Playing and Gaming. Reflections and Classifications'’ Game Studies 3, 3 (2003), http://www.gamestudies.org/0301/walther/.
• —. ‘Atomic Actions – Molecular Experience: Theory of Pervasive Gaming’, ACM. Computers in Entertainment, 3, 3 (2005).
• —. ‘Pervasive Gaming: Formats, Rules and Space’, Fibreculture 8 (2006), http://journal.fibreculture.org/issue8/issue8_walther.html
• —. ‘Self-Reference in Computer Games: A Formalistic Approach’, in W. Nöth and N. Bishara (eds.) Self-Refence in the Media (Berlin: Mouton de Gruyter, 2007a).
• —. ‘Pervasive Game-Play: Theoretical Reflections and Classifications’, in C. Magerkurth and C. Röcker (eds.) Concepts and Technologies for Pervasive Games: A Reader for Pervasive Gaming Research vol. 1 (Aachen: Shaker Verlag, 2007b).
• —. ‘Pervasive Gamespaces: Gameplay Out in the Open’, in F.v. Borries, S. P. Walz and M. Böttger (eds.) Space Time Play: Computer Games, Architecture and Urbanism. The Next Level (Basel, Boston, Berlin: Birkhaüser, 2007c).
• Zizek, S. Violence (New York: Profile Books, 2008).

Introduction

As I write this, at the end of 2010, it is sobering to reflect on the fact that over a couple of decades of explosive development in new media art (or ‘digital multimedia’ as it used to be called), in screen based as well as ‘embodied’ and gesture based interaction, the aesthetics of interaction doesn’t seem to have advanced much. At the same time, interaction schemes and dynamics which were once only known in obscure corners of the world of media art research/creation have found their way into commodities from 3D TV and game platforms (Wii, Kinect) to sophisticated phones (iPhone, Android). While increasingly sophisticated theoretical analyses (from Manovich, 2002 to Chun, 2008 to Hansen, 2006, more recently Stern, 2011 and others) have brought diverse perspectives to bear, I am troubled by the fact that we appear to have advanced little in our ability to qualitatively discuss the characteristics of aesthetically rich interaction and interactivity and the complexities of designing interaction as artistic practice; in ways which can function as a guide to production as well as theoretical discourse. This essay is an attempt at such a conversation.

Over the first (roughly) two decades of practice, interactive aesthetic strategies were developed and adapted to the constraints of digital technologies—themselves under rapid development through this period—and substantial technical R+D as well as aesthetic research, was undertaken by artists. In fact, I propose that the reality that technical and aesthetic developments were inseparable was a defining characteristic of the work of that period. Less obviously, digital and interactive practices were (only) slowly assimilated into the corpus of fine-arts practices and cross-fertilised with more traditional aesthetic approaches.

Within the context of arts practices, a space for development of interaction less overwhelmed by the instrumental individualistic modalities of interactivity characteristic of the computer industry opened up, informed by generative systems and artificial life and discourses of emergent complexity and self-organisation which link back to cybernetic conceptions of responsive systems. These negotiations were characterised by a not entirely clearly elucidated swing between preoccupation with user subject experience and a (more sculptural?) emphasis on the behaviour of the artefact, which offers less instrumental and more bio-mimetic approaches. [1]

The domain of Interactive Art functioned over this period as a space of free invention, an anarchic research realm less application-driven and relatively free of market directives. In this context, in many cases, ideas appear long before they were recognised as research agendas addressed in academic and corporate contexts. In that period, all kinds of experimental interaction modalities were realised by artists—many deploying custom technologies in code and/or hardware. The necessity to develop tools was double edged—on the one hand it permitted an organic development process in which the specifications of the technologies arose from theoretico-aesthetic requirements (and sometimes vice versa). On the other hand, the implicit task of running an engineering R+D lab with limited technical skills and usually very limited budgets was fatiguing. Without recognising these special conditions, it is not possible to grasp the significance of work arising in that period. In some cases, such work remained unknown outside a relatively closed tech-art community, and was independently recreated in other contexts. In other cases, the transfer was more explicit and sometimes it resembled a plundering. We might usefully examine the period to discover the roots of ubiquitous technologies and the motivations for such. Over the last decade, much of the substance of interactive art research has found its way into commercial digital commodities and thereby suffered a confusing erasure of its historico-aesthetic significance. It is an odd feeling to see, in a few short years, systems which were perceive as having rich aesthetico-theoretical presence trivialised as mass produced commodities deployed in a paradigm of vacuous ‘entertainment’.

Echoing the fundamental computer science hardware/software binary of computer science, mainstream digital discourses were undergirded by a commitment, stated or unstated, to a dualistic polarisation of materiality and the ‘digital’, especially in the early years. This led to a deep and polarising discursive tension with the embodied and holistic perspectives of traditional fields of practice, rendering one camp ‘luddite’ and the other ‘techno-fetishist’. Recognition of the centrality of the negotiation of materiality and embodiment within digital practices is, in my opinion, fundamental to understanding the history of interactive art, and provides a purchase with which to understand transitions to ubiquity. The advent of cheap, ‘over-the-counter’ technological bundles (microcontrollers, sensors, programming environments…) has made the practice far more amenable to technological novices, and has also created a new set of aesthetico-technological challenges in the sense that these commodity widgets are designed to fit a narrow consumer need and thus have all kinds of decisions built into them (image ‘improving’ algorithms, digital video formats…) that are often difficult to isolate, let alone work-around. One of the happy upshots of the transitions outlined in the points above is the increasing presence of work by a second generation of artists in the field which more fluidly combine, for instance, material aesthetics of (even formalist) sculpture and installation with digital and interactive systems. Some examples of such work are given later in this paper.

Interactivity and the Emergence of Digital Cultural Practices

I identify a two-decade period—roughly speaking 1985-2005—as the pioneering experimental period of (computer based) interactive art. Crucial to the understanding of work in this period is the blindingly rapid development of the technological context. At the beginning of the period the graphical user interface was a novelty, the internet barely existed, the web was a decade away, interactivity was an intriguing concept. The production of acceptably high resolution illusionistic digital pictures (still frames) was an active research area and a megabyte of RAM was something luxurious.

The period neatly brackets the emergence of most of the major technological milestones which now undergird digital culture and ubiquitous computing: WYSIWYG, digital multimedia, hypermedia, virtual reality, the internet, the world wide web, digital video, real-time graphics, digital 3D, mobile telephony, GPS, Bluetooth and other mobile and wireless communication systems. It was a period of rapid technological change, euphoria and hype.

In what follows I discuss several works of the period which in my mind, stand as markers for significant moments in the development of interactive digital cultural practice. I have chosen these works in part because I have experienced them directly and in most cases, I am privy to both the goals of the artists and the internal workings of the systems. I include three of my own works here, because in developing them I was engaged in the development of ideas and approaches in question, and also because I happen to know intimately the motivations and development of designs and prototypes in these works. [2]

There are numerous other works one might usefully discuss. It is a regrettable fact that, due in part to rapid changes in the technologies and the absence of appropriately skilled and resourced staff in appropriately set-up collections, I estimate that 75% of this work built in the last 25 years is lost entirely, and only a very small percentage of it persists in working form. It is sobering to reflect that a photograph 100 years old, if kept dry, is perfectly accessible to view, but the vast majority of commercial digital systems only a decade old are unusable, to say nothing of the many custom systems. [3]Occasionally, a heroic effort is conducted to rescue such work before it decays completely. The exhibition Eigenwelt der Apparatwelt by Woody Vasluka for Ars Electronica in 1992 was one such case in which works were rescued from basements and attics and restored. [4]

Conspicuously, I here avoid discussion of desktop-based works. My reason for this is principled. It has always been my opinion that such work—whatever its creative and experimental value—on the level of interaction, interface, and physical instantiation, took too much for granted. That is, by willingly adopting the constraints of the commodity desktop interface, assumptions about the nature of interaction which were reified in physical and system architectures were implicitly endorsed and solutions appropriate for sedentary mathematico-symbolic deskwork were uncritically adopted. This, in my opinion, has had the effect of perturbing and constraining creative possibilities in regrettable ways.

Interactive Art before the PC

While the notion of a performative and processual aesthetics of interaction has been bandied about for twenty years or so, looking at interactive art over that period reveals little in the way of development in the formal qualities of interaction per se. The interactional logic captured in Edward Ihnatowicz’ Senster of the early 70s remains paradigmatic. [5] Senster also neatly framed the agendas of reactive robotics, biomimetic robotics, social robotics, twenty five to thirty years ahead of the institutional curve. Around the time of the Senster, Myron Krueger pioneered machine vision-based embodied screenal interaction in several works, the most well-known being Videoplace (1975). There is little in the Wii or the Kinect which was not prototyped in the several iterations of Videoplace forty years before. [6]

Figure 1. Senster on display at the Philips Evoluon, Eindhoven, 1970-1974

In terms of autonomous behavioural repertoire, Grey Walter’s Turtles of the late 1940s set a standard for machine behaviour seldom exceeded since. These devices, built with minimal (and by today’s standards, rudimentary) technology, displayed behaviours seen today in artworks and robotic toys. The turtles were made not as artworks but as cybernetic experiments into electronic ‘brains’. It is important to recognise that, consistent with the cybernetic context he worked in, the conception of ‘intelligence’ which Walter sought to simulate was thoroughly situated and pre-cognitivist.[7] The complexity of the creative agency of Gordon Pask’s Musicolor of the late 1940s is likewise seldom attained (Pask, 1971). The Turtles and Musicolor were entirely analog, and Senster had the digital processing power, roughly, of an Arduino.

These works then, provide a pre-digital or at least pre-consumer-digital reference point for interactive art. It is important to recognise that they arose within the discursive context of cybernetics—as opposed to the cognitivist regime of late C20th computing—and prior to the overwhelming presence of the digital commodity industry. And if this perspective explains to some extent their inventiveness, then we might have to view the last quarter century as a period of increasingly successful attempts at rehabilitation from the disembodying discourses of the cognitivist and the ‘digital’.

It has been observed that the advances in Artificial Intelligence over the period of roughly 1965-1985 can be attributed almost entirely to advances in hardware engineering—faster processors and more RAM have permitted larger scale brute force search, etc., as over the period formal procedures and basic techniques did not advance significantly. This assertion seems to have some justification, given that the computer upon which Deep Blue’s chess program ran in 1997 was ten million times faster than the Ferranti Mk1 upon which Dietrich Prinz’s chess program ran in 1951. [8]

Could it be that the design and aesthetics of interaction are subject to the same conditions of increase of technical capability without significant development of ideas? The rapid advance in bells and whistles permits interfacial cosmetic niceties undreamable 20 years ago. [9] The techno-fetishism of higher resolutions and faster bit-rates serves the needs of an industry which depends on obsolescence (as perceived or as inbuilt material breakdown) to remain profitable, often deploying novelty to obscure a void of significant advancement. The ‘all the time, everywhere’ catchphrase of pervasive computing (like the prospect of the same trivial and pathetic ‘content’ consuming more bandwidth as HD and 3D television) can be seen as a marketing device for bit-peddlers. Increasingly, physical devices are loss-leader market capture devices, as printers are for ink cartridges.

Grounding Interaction

Having elucidated some of the historical realities of the practice, in this section of the paper I will outline some of the theoretical challenges. The first is also historical and has to do with sites of practice. Much of the work arose in the context of the plastic and visual arts, and as such, a theoretical void was encountered (or, more regrettably, not encountered). That is to say: whatever the theoretical tools available to address matters of form, colour, expression, and embodied sensorial engagement, those traditions had little to say about ongoing dynamic temporal engagement because traditional art objects do not behave. Within the traditions of aesthetics of the plastic arts, these are fundamentally novel issues. Recognition of this reality suggests an as-yet incomplete project of identifying prior fields of practice which have some theoretical relevance to this realm. [10] Questions like ‘how does the act of interaction mean?’ and ‘how are such valences to be manipulated for enriched affective practice?’ are fundamental questions in the aesthetics of interactive art.

Long ago R. Buckminster Fuller (1970) asserted ‘I seem to be a verb’ and this cybernetically inspired locution was itself informed by the doubly continuous nature of the analog electronic signal: as temporally continuous and amenable to infinite resolution. The sentiment expressed by Fuller also informed much of the emerging art practices of that period which emphasised process—performance, site specific art of various kinds, etc. More recently the terms dynamical, processual, procedural, performative, situated, enactive and relational, each in their own milieu, capture the evanescent process of contextualised doing. There appears then to be a groundswell in the theorisation of cultural practices sympathetic with phenomenological approaches, and by extension with emerging post-cognitivist cognitive science. I see this trend here as a critical paradigm shift which has major implications for the theory and practice of interactive art.

Being a radically interdisciplinary realm, scientific and humanistic narratives collide in interactive art and the science wars have been a constant backdrop to work and theorisation in the field. Yet to the extent that interaction is an embodied process, neat separations of biology and culture are collapsed. My position here is evolutionary and materialist: interaction makes sense to the extent that it is consistent with or analogous to the learned effects of action in the ‘real world’. Our ability to predict, and find predictable, behaviours of digital systems, is rooted in evolutionary adaptation to embodied experience in the world. We are first and foremost, embodied beings whose sensori-motor acuities have formed around interactions with humans, other living and non-living entities, materiality and gravity. We understand digital environments on the basis of extrapolations upon such bodily experience-based prediction. This is easy to understand in mimetic environments such as Second Life, but is equally present in more basic mouse-screen levels of interaction.

Our most common interactive modalities subscribe to or enlist assignments or connections which are deeply sensorimotoric in nature, and perhaps draw upon DNA hardcodings. Aside from the trivially Pavlovian modality (press the button and get the reward/food pellet) what are the key interactive modalities in artworks? In installation and robotic work, many examples exploit a zoomorphic ‘puppy paradigm’ of approach and withdrawal, trust and fear. This is always beguiling—for a while. Is the charm of this modality somehow ‘natural’ to us as humans, perhaps hardcoded into our DNA as parenting animals? This question opens a field of inquiry at the intersection of neuroethology and interactive aesthetics. Whatever the case, the next question is how to move aesthetic development of the field beyond this biological or cultural ‘ground zero’.

It is a seldom noted corollary to the panegyric around ‘the virtual’, that many interactive art projects focus on the dynamics of embodied interaction (Penny, 2011). This central aspect of human being-in-the-world was conspicuously poorly addressed in conventional sit-at-a-desk computer systems. Because of the sensitivity of artists to persuasive sensorial immediacy and embodied engagement, interactive art practice pioneered research into dimensions of interaction which remained opaque to institutional and commercial labs for many years. In this spirit, one must acknowledge that people making interactive art were doing ‘affective computing’ a decade ahead of academic and industrial recognition of such issues.

In the contemporary context, this situation has changed in two ways. Interface technologies are far more diverse, complex and subtle. Not long ago, microphones and cameras were exotic add-ons to computers. The embedded miniaturised accelerometer has become ubiquitous and has contributed to the development of all sorts of gestural and body-dynamic driven applications. [11] And yet, first generation interactive modalities involving pointers and keyboards hang on skeuemorphically. Of all the things I do in my life, only some of them map well onto sitting at a desk in front of a glowing surface, poking at buttons, nor is this situation improved one iota when the context is miniaturised so the buttons are smaller than my fingers and I have to put my reading glasses on to look at the screen. For all the expansion of wireless networks (etc.) we have not progressed very far in interactive modalities. As in AI, advancement in the field can, as often as not, be attributed to advances in hardware.

More subtly, the first generation to have lived with digital devices during infancy is now approaching adulthood. This generation acculturated to, for instance, multi-modal on-screen interfaces. These people’s neurology must have, to some extent, formed and developed around such systems. That is, the metaphors and behaviours of digital systems, like any aspect of language and culture inculcated in infancy, have generated isomorphic neurological structures—digital metaphors instantiated at the level of cellular biology. [12]

Who or What Is Interacting? : Analysis of Interactive Systems

Conversations regarding the aesthetics of interaction sometimes take on a weirdly schizophrenic quality due to the fact that some speak from the point of view of user experience and some from the point of view of system design. The question ‘is it interactive?’ can have wildly different answers depending on this point of view. One can maintain, as some do, that viewing a photograph is ‘interactive’. Such positions are clearly nonsensical if one is looking from the perspective of the artefact/system, and are destructive of the goal of building a richer critical discourse about interactive systems. The photograph does not change in any way due to changes in its environment. A human viewer might have varying experiences due to personal associations, varying proximity or lighting conditions, but there is no interaction in the sense of an ongoing sequence of mutually determining actions between two systems possessing agency, or as interacting components (user and machine) in the larger user/machine system. The bifurcation in such conversations is whether the critique addresses the experience of the ‘user’ or the behaviour of the system.

While there is undoubted value in probing the nature of the interactive aesthetic experience on the part of the subject, study of the design of the system as an armature upon which the experience occurs must provide a necessary complement. Such a design-centric approach engages issues such as designer authoriality and the position of the system as a literature—as discussed by Mateas and as is becoming central in some aspects of software studies (Fuller, 2008). Other (Artificial Life) approaches address the system as quasi-organism, in autopoietic or enactive sensori-motor loops with user(s) (etc.) (Penny, 2010).

To focus on the machine, the fundamental requirement of an interactive system is that it correlates in a meaningful way, data gathered about its environment (usually a user’s behaviour) with output. That is, the system must present effects which are perceived by the user as being related to their actions. Without this there is no perception of interactivity. From the perspective of system design, successful interaction comprises two mirrored parts: first, the sensing and interpretation aspects of the system must gather relevant information about the world and interpret it ‘correctly’. Secondly, action of the system must be contrived such that they are perceived to be related to the events in the world which were sensed and interpreted. Meaningful interaction thus requires that several functions be correct and coordinated correctly. Sensors must be chosen correctly and calibrated correctly to capture relevant environmental electro-physical variables and such data must be interpreted correctly. Well-designed associative systems result in the generation or production of output whose content, location and dynamics makes sense to a user as a meaningful correlate of their own behaviour.

But this does not mean that only literal or instrumental modalities can be meaningful. Temporal immediacy permits aesthetic deployment of sleight of hand. In the world, if I knock a glass and it falls to the floor splintering, I assume a physical and temporal causality. Assumption of causality based on temporal order can be designed-in and exploited in interaction design. As in film montage, diverse elements and events can be connected by an associative or inferential temporal sequencing. The aesthetic manipulation of temporal process is inherent in interaction. It is worth noting that the verb in ‘interaction design’ (as opposed to chair design or car design) implies process.

Lay and Virtuosic Systems

The distinction between systems designed for untrained public (lay) interaction, and systems designed for use by trained interactors is a technical binary which tacitly characterises interaction design in the fine arts. [13] Because art practice is predicated on public exhibition and an imperative of some degree of public accessibility, and because interfaces are often unfamiliar (not a condition experienced in the closed environments of university and corporate research labs), the task of providing ‘intuitive’ access to unfamiliar modalities was (a usually unremarked) part of the design task of artists. When interactive digital artwork began, there was some focus on celebrating the novelty of digital interaction itself (i.e. mouse-screen coordination and virtual on-screen ‘buttons’) while at the same time constructing an easy way into such interaction paradigms, usually via deployment of objects, images and structures familiar from pre-digital forms. In the early years of interactive art, substantial design effort was required to create a context in which an untrained member of the public might be drawn into a work which was framed and constrained in such a way that they could be expected to do something which wasn’t completely ‘out of the ballpark’, while simultaneously not being tediously instructive and didactic.[14] Within a decade, users had thoroughly acculturated to the screen/keyboard/pointer, desktop paradigm. Since then, interaction design, in the arts and elsewhere, has bifurcated between deploying well known interaction paradigms (which are at this point ‘intuitive’) and the development of novel modalities. In the latter case, artists must engage a meta-design task of introducing the user to the special modalities of the work, without making such introduction itself laborious or instrumental. No one wants to do a tutorial for an artwork. [15]

‘Virtuosic systems’ comprise an entirely different category of interactive systems as they are designed for virtuosic performance by a highly trained ‘player’. Such systems are most familiar in the context of music and digital musical instrument design. These systems are often highly idiosyncratic, predicated on a fused trajectory of player training and system development—often the designer/developer is a performer. Such systems re-affirm a conventional binary of (active) performer and (passive) audience, whereas systems for ‘lay’ interaction conflate the roles of performer and observer—the performance is the performer observing herself perform.

Temporality and Poetry in Interaction

The very existence and success of commercial and commodity interactive digital multimedia (and their rhetoric) have, one might suggest, impeded aesthetic progress in the field, because they created confusion between interactivity for instrumental purposes and interactivity for cultural purposes. The interactivity of conventional software tools (say a word processor) should ideally be ‘transparent’ and instrumental. What is meant by ‘transparent’ and ‘intuitive’ in such discourse is that the behaviour of the system is consistent with previously learned bodily realities. A lack of clarity on such issues is typical of the tendency of technical research areas to undertheorise.[16] In Heideggerian terms, instrumental software should be ‘ready-to-hand’. To the extent that it is noticeable, it is bad. This, one might argue, is exactly the opposite of what aesthetic interaction ought to be—it should not be predictably instrumental, but should generate behaviour which exists in the liminal territory between perceived predictability and perceived randomness, a zone of surprise, of poetry.

To the extent that every digital interactive event is analogical, interaction is always poetical, and the construction of instrumental systems involves reduction of the poetry quotient. And in many cases, the focus of the artist has been precisely to probe the qualities of this analogising. This is most often obvious in augmented and mixed reality projects where the behaviours in the digitally constructed environment maintain certain consistencies but invert, erase or otherwise distort such correspondences. For instance, user representation may be abstracted (it is already reduced to two dimensions), but temporal correspondences make it abundantly clear what aspects of the image correspond to what body part or gesture.

Such correspondences leverage deeply embodied understandings of a sensorimotor nature—specifically the affiliation of proprioceptive and visual feedback. In a way analogous to the example of the Blind Man’s Stick as explored by Merleau-Ponty, Gregory Bateson and more recently Lambros Malafouris, the real-time computer vision representation of the user almost instantly obtains a prosthetic functionality, testifying to the remarkable speed of neurological mapping across modalities (Merleau-Ponty, 1965: 245; Bateson, 1972: 434; Malafouris, 2008). And indeed, without this, we would not be able to drive a car or use a screwdriver. While realtimeness is not easily subtracted from the stick, it was a technical challenge for vision-based work, and as such the symptom acquired a name—latency. The very existence of latency—unavailable in the case of the stick—has led to exploration of the sensorimotor requirements of the perception of realtimeness. The persuasiveness, the plausibility of interaction is intimately temporal, and indeed, sensory modalities have more or less extended conceptions of ‘now’. For instance, ‘immediacy’ (the reciprocal of latency) is much more immediate in hearing than in vision.

The question of interaction has cognitive and phenomenological dimensions which have ramifications for the development of adequate aesthetic theory for the practice. In interactive work which arises from a tradition of plastic or visual arts, conventional aesthetic language imposes an axiomatic subject-object distinction upon the artwork/interactor system. This has the effect of obscuring the very (relational) nature of the experience. A distinction must be drawn between two paradigmatic modalities of interaction deployed in cultural practices, which we might identify as ‘instrumental’ and ‘enactive’. The instrumental mode, typified by HTML links and its hypertextual predecessors (all the way back to Hypercard) regards the enaction of a link as simply a way to get from A to B, a connection which ideally is instantaneous and is not marked as an event in itself. A and B are the objects of concern—they are objects, and nothing else is of concern. Effects such as fades or wipes, borrowed from video and cinematic language, tend to be distractions or at best signify a change of temporal or spatial context, register etc. As noted above, Buckminster Fuller’s assertion ‘I seem to be a verb’ was itself informed by the doubly ‘continuous’ nature of the analog electronic signal. Both of these conditions are artificially curtailed by the discrete nature of digital data—a fact which, via rhetoric of object-oriented programming and the like, may have informed the object-centric nature of instrumental interaction. This erasure of temporal process is somehow typical of the ways we tend to explain experience—note the cinema ‘frame’ and its metaphoric extension into analog and digital video realms creates the sense that time is composed of a sequence of stoppages.

The lesson of performativity is that the doing of the action by the subject in the context of the work is what constitutes the experience of the work. It is less the destination, or chain of destinations, and more the temporal process which constitutes the experience. To call it ‘content’ would be again to slip into objectivising language. In what follows I will deploy the concept of ‘enactive cognition’ of Varela, Thompson and Rosch (1992) as it captures the ongoing ‘structurally coupled’ nature of experience (as they say) in ‘laying down a path in walking’ and it is precisely this (performative) aspect of the aesthetics of interaction which demands theoretical elaboration. [17]

Interaction: Embodiment, Gesture and Affect

Over the ‘heroic’ period in question, machine vision for interactive artworks was pursued by several artists, with David Rokeby’s Very Nervous System being an early case (though Krueger’s Videoplace is the true pioneering work). [18] Very Nervous System, first shown at Venice Biennale in 1986, responded to the dynamic of user movement with stereo audio. Rokeby notes:

The installation is a complex but quick feedback loop. The feedback is not simply ‘negative’ or ‘positive’, inhibitory or reinforcing; the loop is subject to constant transformation as the elements, human and computer, change in response to each other. The two interpenetrate, until the notion of control is lost and the relationship becomes encounter and involvement… The installation could be described as a sort of instrument that you play with your body but that implies a level of control which I am not particularly interested in. I am interested in creating a complex and resonant relationship between the interactor and the system. [19]

The fact that video cameras and real-time video are now a normal part of contemporary computers and gaming systems obscures the fact that into the late 90s, machine vision was regarded as a non-trivial technical research problem. Getting video into a computer in real time required special and expensive peripheral hardware. This makes Rokeby’s achievement all the more remarkable. VNS ran on an Apple IIe, a machine which would have struggled to render a single 640×480 ray traced image in 24 hours. He managed with its tiny processor to do both real-time machine vision and real-time stereo audio output. He achieved this extraordinary result because, in the first instance, his project was concept driven—he knew what he wanted out of the technology and he had an adequately deep understanding of the analog and digital electronics and coding that he was able to pare away unneeded functions. [20]Rokeby’s approach was also ‘dynamical’ and attended to temporal rather than pictorial pattern. While a conventional approach would analyse sequential frames pixel by pixel, laboriously identifying and labelling presumed relevant ‘objects’; with an artist’s education, David recognised both that frame-wise thinking was an impediment and that light values on pixels do not indicate objects in the world in any unproblematic sense.

Much of the thinking behind academic and industrial machine vision research still labours under the naïve conception that frames are a fundamental aspect of reality (rather than a skeuemorphic convention) and likewise that ‘lines’ in images can be unproblematically associated with objects or edges in a physical space. Computer science libraries and journals are replete with papers on topics like ‘edge detection’; in some cases the authors seem quite unaware that a video image depends on optics developed in film cameras, and cameras themselves were designed to implement the graphical perspective, a conventionalised geometrical system for representing spatial depth on a plane. [21]

David was interested in temporal bodily dynamics and recognised that variation through time was the data that was critical and pictorial resolution was far less important. Colour was therefore dispensable. In fact, David’s 1986 ‘camera’ consisted of 64 cadmium sulphide light dependent resistors in the back of a wooden box, with a plastic Fresnel lens on the front. The resistors, being slow moving devices, effectively damped the system. David in fact summed the voltage values for the resistors, to produce one value per time step, and tracked the pattern of change in the line generated through time. The parsimonious elegance of this solution is characteristic of the technological solutions which technically adept artists of the day were compelled to realise due to the double constraint of the condition of the technology of the day and the limitations of art budgets. VNS is a prime example of what I have referred to previously as ‘machine parsimony’: an ethic of technology design which is elegant and economical—at the cost of being application specific. [22]Such an approach is antithetical to the conventional commitment to ‘generality’ and general-purpose tools, which tends to be profligate in its use of resources, and encourages a plug’n’play approach to coding, which while emulating the look of purist modularity, is antithetical to its spirit. In the context of digital cultural practices, the slippage in computer science discourse around this notion of generality is most unfortunate. While Turing’s mathematical formulation of the ‘general purpose machine’ has undeniable value in its context, the notion has oozed weirdly into other frames of reference. This has as much to do with marketing rhetoric sleight of hand and with economies of scale in the computer industry as it does any principled argument.

Around the same time as VNS, another Canadian artist (and a teacher of Rokeby) Norman White debuted his Helpless Robot (1987).[23] Savagely funny, the Helpless Robot has no motive power and its sensor suite is rudimentary. It depends on its verbal persuasiveness to entice humans to do its work for it. As a person is drawn into helping the helpless robot, the device becomes increasingly impolite and abusive, creating a situation from which the humiliated human helper must sheepishly escape. Under the hood, Helpless Robot speaks over 500 phrases organised by emotional categories such as boredom, frustration, arrogance, and overstimulation. As such, it is an early example of interactive art practices pre-empting affective computing research which began in institutional contexts a decade later.

In 1990, Luc Courschesne developed Portrait One, an interactive video portrait.[24] The subject, a young woman, spoke to the user, who responded by selections from multiple choice options on screen via mouse clicks. The system ran under Hypercard on a Mac II which serially drove a 12’ laserdisc player. Computer and video images were overlaid visually in a neatly contrived reflective box—there was no electronic integration of computer and video images. By contemporary standards the technology was vestigial and even by the standards of its day, it was not particularly technically ambitious. The ambition and the success of Portrait One was in the realm of simulation of affect. While users of far more technically sophisticated (immersive VR etc.) works came away nonplussed, people came away from Portrait One in love. [25] Courschesne achieved with limited means and high aesthetic intelligence what others with far more resources were unable to do. His insights into interactive dramaturgy, narrative construction and simply, the subtleties of acting, drove Portrait One and later projects. The work thus offered a poignant implicit critique of more technophilic enterprises.[26]

Arising from sculptural and installation sensibilities, my own Petit Mal—an Autonomous Robotic Artwork (begun in 1992 and first exhibited in 1995) sought to move ‘serious’ interaction off the desktop, out of the shutter-glasses into the physically embodied and social world. [27] I undertook the task of building a robust mobile autonomous machine for cultural purposes. I saw the device, technically, as a vindication of a ‘reactive’ robotics strategy and a critique of conventional AI based robotics, as well as an experiment in artificial sociality. The device behaves robustly with the public (albeit in a constrained environment) continuously for 10-12 hour days. Socially, it elicits play or dancing behaviour in users. Interaction is driven by curiosity and seemingly, a desire to pretend that the thing is cleverer than it is. People willingly and quickly adjust their behaviour and pacing to extract as much action from the device as possible, motivated entirely by pleasure and curiosity. [28]Petit Mal implements a non-instrumental kind of ‘play’ which is quite incommensurable with conventional computer game logic. [29]

Beyond Pages by Masaki Fujihata (1995) was a work which subtly and exquisitely merged the book paradigm with interactive graphics, held together by carefully crafted interaction design supported by not particularly adventurous technologies of the time—data projection from underneath combined with a tracking stylus for interaction.[30] As one drags the stylus across the image of a large book displayed on a projection surface inset into a desk, pages flip. The high-resolution sound samples synchronised with the image contrive the feeling that these are heavy, course pages. On each page an object is depicted, and stylus action moves or changes the object. Propelled by the cursor, a rock shuffles across the page. Large crunchy bites are taken out of an apple by an invisible eater. Japanese characters on the page are voiced. In an elegant set of visual puns, an image of a light switch on the page turns on the (physical) lamp on the desk. A door handle on the page opens the door in the video image projected on the adjacent wall.

Figure 2. Installation view of Beyond Pages

As noted above, during this period, artists were (necessarily) at pains to demonstrate the new modalities of digital interaction by grafting them on to (representations of) understood artefacts and devices. After twenty years, as often as not, the tables are turned. It is the digital interface to which users are naturalised and the works serve as a window onto a preceding technology. It may not be long before users play with Fujihata’s Beyond Pages to get a sense of what books were like.

The Implicit, Enactive, Performative Body

Consistent with its interdisciplinary history, the analysis of interactive art has a strongly dialectical quality. On one hand, a bone-headedly Luddite approach has all but ignored the fact that machine mediated interaction is a novel context, and that without some familiarity with the technology, discussion of the work is superficial. On the other hand, technocentric approaches tend towards instrumentalisation of the user and the trivialisation of precisely the phenomenon which is in need of explication. Ultimately some critical purchase must be made upon the behaviour of the complete (user/machine) system.

This difference might be conceptualised in terms of ascribed modalities of meaning construction—is meaning gathered as a result of the extraction of representational tokens, or is it enactively constructed in doing? In his insightful work in this area, Nathaniel Stern (2011) marshals insights from performance studies to formulate a conception of the ‘implicit body’, a Deleuzian ‘body in motion’, as an object through which to discuss interaction. Central to his analysis is an awareness of the dynamical condition of interaction—a perspective I would identify in cognitive terms as ‘enactive’. As an interdisciplinary intervention, projects like Stern’s have the salutary effect of balancing the weight of a technocentric and instrumental approach which often has the effect of rendering the user as a ‘robot’ or a Pavlovian subject, capable of a limited range of behaviours elicited by specific stimuli. Crucial to Stern’s analysis is an understanding of temporally and spatially ongoing embodiment as the locus within which meaning is created. Such a position is in Agre’s (1987) terms ‘deictic’, and in Pickering’s (1995), ‘performative’. This confluence (of Varela and Deleuze, Agre and Pickering—and one might add, Evan Thompson, Ezequial Di Paulo, Bruno Latour, Nathaniel Stern, Karen Barad and others) points to a new ontological perspective from which interaction, and the interactor, can be usefully reformulated.A perspective from which, I would argue, advances in interaction design practice might be made, and indeed, in practice have been made, though theoretical analysis has been slow to recognise it.

Fugitive (Penny, 1997) deployed a custom machine vision system for detecting bodily movement and large gestures of a user in a 10m circular space. [31] This behaviour drove selection of video from a structured database of video and sent it to a motion-controlled video projector which displayed the images in varying locations on the wall of the cylindrical room. My goals in the interaction scheme of Fugitive were precisely to resist the tendency toward scopophilic focus on the image, and rather to draw the attention of the user to the temporal continuity of their own embodiment. This was in part motivated by a critique of Virtual Reality. In conventional VR, the disembodied gaze had the ability to ‘move’ on preordained paths within a pre-structured architectonic environment.

In Fugitive, the subject is the subject. This presented novel design challenges as one had to construct the images and the interaction to counter the normal assumption that when looking at an image, it is the image, rather than the looking, which is important. I made explicit at the time, in Fugitive, the continuity which structured the experience was the subjective temporal continuity of the users’ embodiment, or more correctly their kinaesthetic awareness. Virtual ‘worlds’ arose and collapsed on the basis of that continuity.

One way of thinking about interactive artworks is that they might provide a context in which engagement with the work constructs a condition which requires further action in order to be resolved; in which artefacts and effects are arrayed spatially and temporally in such a way as to encourage the formulation of novel ideas. The temporality of the process is unavoidable, and its design constitutes a kind of synthetic enactionism. The arrangement of such artefacts and effects in a way that optimally stimulates such processes (bearing in mind questions of demographics and cultures) is then, the cognitive dimension of the task of interaction design for aesthetic purposes.

Terminal Time and AI-based Art

In the later 90s an eminently interdisciplinary grouping—Steffi Domike (a documentary filmmaker) Paul Vanouse (a media artist) and Michael Mateas (an AI specialist)—produced a project which, while only marginally ’interactive’ and not exactly adapting to a user in real time, deployed relatively sophisticated AI techniques to build documentary films ‘on demand’ and according to the political profile of the particular audience. [32] In Terminal Time the audience was polled in multiple choice form on its positions on various issues regarding technology, religion, science, politics, gender issues (etc.) their response gathered by a simple applause meter (microphone). Based on this data, the system assembled a documentary on the history human civilisation, combined from hundreds of music and film clips and short voice-overs, which was then presented to the audience. Terminal Time stands as a witty commentary on contemporary political media—the audience gets the history it deserves. Terminal Time generates as output, a formally conventional linear narrative, it trades in a currency of symbolic tokens, it addresses its multiple audience as a demographic mass and has no concern with the ongoing embodied behaviour of it audience.

As experiences, conventional interactive systems tend to fall into two camps—ones that are simple and therefore easy to understand, and which therefore either become familiar and automatic or boring, and others which are so complex as to be baffling and cause all but the most dedicated to fatigue. By ‘conventional’ I mean that the interactions themselves are hardcoded. This is a kind of meta-stability. To move beyond this stalemate, in 1997, I proposed the idea of a dynamical ‘auto-pedagogic interface’—an interface which observed and learned from its user(s) how well they understood the system, and ramped up complexity gradually as a result of constant monitoring. [33] In this context, Terminal Time represents a phase shift in the sophistication of strategies of database combinatorics in artworks. To that date, interactive works had usually taken recourse to only the most simplistic of selection procedures (lookup tables, pseudo-random choices). Terminal Time deployed a wide palette of sophisticated AI techniques to dynamically construct the work for every iteration. This strategy was based in Mateas’ reflexive view of AI. Consistent with the critique of the time, he rejected the larger project of AI regarding simulation of human intelligence. He insightfully observed that in the context of that discipline, some very intelligent people had developed sophisticated tools for automated reasoning and such tools were ripe for application in interactive art. Mateas (2001) refers to this practice as ‘Expressive AI’.

Phenomenologically-based critiques of AI, the elucidation of the frame problem (and the related symbol-grounding problem) and the more recent critiques by Agre have themselves ‘framed’ the domain in which (good old-fashioned) AI can be effective—a domain of logico-mathematical manipulation of symbolic tokens. Philip Agre is so eloquent on this subject that a full quotation is justified (but only on the left):

…the privileged status of mathematical entities in the study of cognition was already central to Descartes' theory, and for much the same reason: a theory of cognition based on formal reason works best with objects of cognition whose attributes and relationships can be completely characterized in formal terms. Just as Descartes felt that he possessed clear and distinct knowledge of geometric shapes, Newell and Simon's programs suffered no epistemological gaps or crises in reasoning about the mathematical entities in their domains (1997: 143). [34]

AI foundered on extraction of such tokens from the world and the re-connection of the result of such processing with the world. Indeed, even to speak of connection with the world as constituted by input and output is to subscribe the worldview which has been found to be defective. Thus the useful applicability of AI techniques to interactive art and its challenges, as it has been framed here, is limited. Interactive art captures and epitomises many of the contexts in which AI has been troubled—embodied engagement with an explicitly open-ended world of multivalenced cultural properties. Interactive art is the New York of AI—if it can make it there it can make it anywhere.

There is a domain of cultural practice where the constraints on AI techniques are not an impediment. On the net, the applied fields of data-mining and machine-learning have demonstrated significant successes. The popular name for autonomous agents in game-worlds is ‘AIs’. It is worth noting why this is. It is because on the net, the processing of the material world into bits has already been done (by people). Everything on the net is pre-digested for CPUs into a numerical value, one just has to find and get it. This relieves such systems of much of the tricky (far more tricky than AI rhetoricians admitted) task of interpretation of sensor data derived from the electro-physical world.

Giver of Names (David Rokeby, 1998) presents as novel a reading of artificial intelligence (and especially the area of ‘natural language’ analysis and generation) as VNS did of machine vision. [35] Giver of Names speaks cryptic sentences produced by a metaphorically-linked associative connectionist database of objects, ideas and sensations. The system is seeded by shape and surface recognition information derived from a machine vision system which views (arbitrary) objects placed on a pedestal by visitors. These utterances are fed back in to the system, generating a stream of solipsistic musings which by turns sound poetic, philosophical or like the ravings of an obsessing schizophrenic. Here the preoccupation is not with interactivity, but with the behaviour of a solipsistic self-reflexive autonomous system.

Artificial Life, Autonomous Agents and Virtual Ecologies

Another work which inventively addressed concerns of the period is El bal de Fanalet/ Lightpools, by Narcis Pares, Roc Pares and Perry Hoberman (1998). El bal de Fanalet/ Lightpools combined multiple spatial tracking and interactive artificial life-based graphics with artefacts derived from Catalan popular culture. [37] El bal de Fanalet Lightpools captures three important themes of 90’s interactive art. The first was the articulation of the notion of ‘the virtual’ which was a central term in the discourses of the period, and the exploration of embodied interaction with the ‘virtual’. The second was an exploration of ‘artificial life’, the possibility of implementation of life-like forms and behaviours in computational systems. A third concern for artists working with interactive systems was the limitation of conventional computer platforms and sensor systems to single user application. There was much interest in devising viable multi-user systems which attend to the small and large group social dynamics so important in human interaction. A set of more practical concerns involved how to instrument members of the public so such presentations were viable in public spaces. Extensive ‘suiting-up’ was labour intensive and time consuming and required complex logistics and support. El bal de Fanalet/ Lightpools addressed all these concerns with some elegance.

Figure 3. El Bal De Fanalet/Lightpools showing ‘lightpools’ and ‘falanets’

Pares, Pares and Hoberman implemented a world where users could grow virtual organisms which would then interact with users and other entities in the ‘virtual’ realm. Aspects of these modalities were commonly addressed by other artists. Char Davies and Sommerer and Mignonneau come to mind. But El bal de Fanalet/ Lightpools is particularly successful in addressing and resolving a combination of these issues.

Users wielded a fanalet, an interface device modelled on the eponymous paper lantern used in a traditional Catalan dance festivity. The signification of the lantern was deployed metaphorically to conceptually join the physical world inhabited by the users with the virtual world ‘under the floor’. Here three (at least) separate conceptual realms are joined—the world of the fanalet and the dance, the world of fish etc. in ponds, and the realm of artificial life creatures. The ‘glue’ that held these metaphors together was real time integration of user gesture with projected content. Tight gestural correspondence and control of latency gave actions an immediate sensorimotor veracity. The ‘lamp’ interface device contained a spatial tracker and the tracker localised the area where light would shine from an overhead projector. The virtual realm was ‘exposed’ in the ‘lightpools’ associated with each fanalet, in the way that at night a lantern might illuminate an area of a lake or pond. Movement of the fanalet by the user encouraged movement of the virtual creatures, and new behaviours emerge when users merge their lightpools. Here is a case example of complex interactive metaphors being intelligible by virtue of their relation to real world scenarios and dynamics. The question of how these metaphors can be stretched while remaining coherent is a rich area for cognitive-linguistic research in the field. [38]

Works such as El bal de Fanalet and Petit Mal reflect a general interest among the community at the time in Artificial Life. Alife marked a radical swing in computation and AI circles from symbolic approaches to systems modelled on biology, self-organising and dynamical systems and emergent complex order. Colonial animals—slime molds, sponges, termites, ants and bees—became the poster-children for emergent complex behaviour from simple elements. Cellular automata were seen to model such behaviour computationally. Evolutionary, genetic and ecological metaphors and methods (Holland, 1975; Ray, 1992), neo-neural-net connectionism and Darwinian computational environments (genetic algorithms) demonstrated significant success (Sims), as did bottom-up and reactive robotics (Brooks, 1991; Steels, 1996). In the arts, there was a remarkable flowering of new practices based in these ideas (Penny, 2010).

My own work Sympathetic Sentience (Penny/Schulte 1996) was inspired by such developments. [39]My goal was to build a minimal physically instantiated system of multiple communicating units which demonstrated emergent complex behaviour. Sympathetic Sentience was constituted by twelve independent, more or less identical, wall mounted sound producing devices. These communicated in a serial loop via custom infra-red hardware. Alone, each unit could emit only an occasional chirp, but in a loop, each device voiced a constantly changing but acceptably ordered rhythm and melody. The system on its own generated complex and constantly changing output and was self-stabilising, it possessed a native capacity for quasi-stability—after initial build-up, the chirp/silence ratio stabilised but fluctuated around 50%, it never went silent or fully saturated. (In cybernetic terms it was a homeostat of sorts). The system achieved this result with little but a simple instable pulse generator, an XOR gate and a short time delay in each unit. The system did have minimal, suppressive ‘interactivity’—a user could unknowingly interrupt an IR beam and thus impose a gap or silence in the melodic passage, which would then slowly fill up again. The focal concern was not with human agency, but with the agency, autonomy and creativity of a minimally complex system.

A more recent work which combines emotion modelling with embodied interaction is Sniff by Karolina Sobecka and Jim George (2009).[40] Sniff is member of the Alife genus of on-screen interactive semi-autonomous agents which must count Weizenbaum’s Eliza as their ancestor. Sniff is a lively pup, or rather, a behavioural portrait of a lively pup rendered on a large back-projection screen. Sniff interacts with visitors via machine-vision based sensing. Sniff is rather finely physiologically modelled and 3D animated, but in a deft artistic touch, Sniff eschews the baroquity of texture maps and synthetic fur and is presented as a fine-resolution polygonal model in white lines, in a featureless black space. The contrast between the verisimilitude of Sniff’s behaviour and the abstraction of its visual representation heightens the persuasiveness of its physiological and behavioural modelling. Sniff is a rather subtle canine behaviour-portrait, it will identify with one of several visitors a primary interactor, and has a sophisticated behavioural memory.

Figure 4. A user interacting with Sniff in a storefront installation of the work

Performative Ecologies by Ruairi Glynn (2008) is a more abstractly mechanical work which attempts to realise embodied agents which learn from their environment and improve their behaviour. It consists of three identical, exhibitionistic and somewhat insecure motorised sculptures which dance for their supper, as it were. Each has a tail part—a motorised illuminated rod which it can turn and light up variously, and a head part, an infra-red video camera on a motorised neck. When functioning as designed, each unit identifies a visitor using face recognition software, and performs a dance for her with its tail, all the while monitoring the attentiveness of the visitor, using gaze tracking. The longer the visitor attends, the better the dance scores. Later, using genetic algorithms, these devices develop new dances from the best scoring previous dances, to be tried out on the next day’s visitors. In a limited way, these devices move beyond the ‘look-up table’ paradigm of interaction where all behavioural possibilities are pre-scripted, to a mode which can truly produce novelty or surprise. [41]

The Role of Epistemic or Performative Action in Interaction

A work of very different scale and historical reach is Rafael Lozano Hemmer’s Displaced Emperors (1997), a part of his ‘relational architecture’ series of works.[42]The projection installation took place on the multi-storey exterior of a Hapsburg villa in Linz, Austria. A user, equipped with a custom triangulating sonar tracker on the back of her hand, makes a waving gesture in the direction of the castle, and the image of a giant hand moved across the façade. Using synchronised image projections, this waving appeared to wipe the exterior facade of the building away, revealing period interior rooms. These room views were in fact taken in the Chapultepec Palace in Mexico City, the residence of the short-lived Mexican Hapsburg dynasty. The waving/wiping gesture collapsed one physical location onto another, and directly—via the immediacy of sensori-motricity—induced reflection of the historical peculiarities of the colonial period.

We should make a distinction between interface and interaction modalities which are deployed as a mechanism for exploring ‘content’, and modalities which themselves contribute to the accumulated meaning or experience of the work. As discussed above, in some interactive work, interactive modalities are taken as transparent and given: the dynamics of interaction were conceived as a means to an end which was primarily found in the ‘content’ of the work (as if interaction dynamics were not always part of the ‘performative’ content). In other cases, such as Displaced Emperors, the dynamics of interaction play a key role in the overall construction of meaning. We might suggest that content-centric interactives are retrogressive, and simply articulate the representational idiom of painting, film and video, while works in which the dynamics of interaction are themselves the ‘content’ of the work occupy a more progressive position.

The recognition that the process of interaction is both embodied and quintessentially performative provides a position from which to build out an aesthetic theory. Andrew Pickering’s (1995) formulation that the representational and performative idioms are distinct and perhaps incommensurable is relevant. Originally applied to questions of scientific knowledge and practice, this binary is ripe for application to arts practices, for conventional plastic arts artefacts are representational artefacts par-excellence. I want to assert that interactive cultural practices, while deploying representational components, prescribe a performative ontology—some more than others. Again—to the extent that the mechanisms of interaction are naturalised, automatic, ‘intuitive’, ready-to-hand—they do not play a significant part in the epistemological circuit of the work. But to the extent that I have to bend this way, climb that ladder, or stand with my feet in cold water, the doing of the work, the embodied and performative dimensions are and must be designed as (often major) components in the overall meaning of the work. Extensive work remains to be done in assessing how interactions mean. Here the distributed cognition research of Edwin Hutchins (2006) and David Kirsh and Paul Maglio (1994) offer ways in to this inquiry, as both researchers are concerned with how bodily action plays a role in cognitive processes involving manipulation of artefacts and exploitation of images.

Building such meaning has several aspects. Making a user complicit in the construction of an unfolding experience is a powerful technique for establishing engagement and commitment. If seeing a video in which a person is pushed off a balcony is disturbing, pushing them yourself—or being put in the position of choosing to—is an order of magnitude more so. (Unless of course such action is routinely trivialised, as in first person shooter games). Aside from ethical issues, basic sensorimotor realities anchor the actions of interaction in a way that dissociated contemplative vision can never achieve—if indeed it were possible. As embodied beings, we conduct our path through the world in the form of sensorimotor circuits which have no beginning or end. Contrary to received wisdom, humans rarely perceive then act, but understand the world in a synaesthetic and proprioceptive fusion of sensing and action, often acting to perceive the action to calibrate the action. An aesthetic theory of interaction, then, must include a choreographic understanding of user action.

It is not just doing, not just the awareness of doing, but the cognitive dimensions of doing which are important dimensions of interaction design. Kirsh and Maglio (1994) demonstrated a phenomenon they call ‘epistemic action’. Epistemic action describes the gathering of knowledge through action in the world. That is, certain kind of things can only be thought, or can be thought faster, more efficiently or more richly, by the manipulation of objects in the material world. Doing math on paper is an example which trades in symbolic notional systems, moving the scrabble letters about to suggest words is somewhat more materialised. An array of utensils and ingredients on a kitchen bench might suggest action, the shape of the knife facilitates certain kinds of actions (and not others), but does the arrangement and design of implements support mental computation? Kirsh and Maglio’s Tetris research addresses time constrained problem-solving. The challenge of applying such models to interactive art practice is that only the most tedious kinds of interactive art, and only the most tedious approaches to interactive art, treat it as a puzzle-solving task. In more exemplary cases, the conditions are set for an inquiry whose outcome, while framed, is open-ended and designed to be generative, allowing the possibility of hypothesis formulation, rather than simply resolution. Cognitive science, with its quantitative scientific roots, has understandably shied away from such questions, seeing them as out-of-ballpark. This is one of the ways in which scientifically based practices find unresolvable discontinuities in the arts.

Similarly, I have several times been asked by HCI professionals if I have undertaken user-studies of my works, and the question has always seemed absurd. The idea that the ‘outcome’ of the experience is not proscribed is fundamental to the rationale of the practice. It would be an amusing dinner party game to design such a questionnaire. ‘On a scale of 1-10, rate your surprise’ … ‘How satisfactorily did the work challenge your unquestioned assumptions?’… ‘What did it make you think about that you had not expected to think about?’ Would it be possible to analyse epistemic action in the domain of interactive art, where the task might be formulated not so much as getting the right answer, but asking the right question?

Becoming Becoming

Through this essay, I have argued in various ways for the relevance of the processual in thinking about interaction. I noted that such sensibilities underlay much 60s art, and played a role in the design discourse of that techno-guru of the period Buckminster Fuller. Here again we might observe the prescient but inchoate way these ideas come up first in the arts. Yet how can a felt embodied idea be anything but inchoate, without the development of a discursive skin? In their day, works like Senster and Videoplace could not be appreciated for want of a discursive context, so likewise it has been with ‘process’ in general.

In the years since, such a discursive skin has indeed begun to surface, in patches as it were. In Artificial Intelligence, Agre and Chapman (1987) proposed Deictic programming which contested the objectivist ‘gods eye view’ assumptions of conventional AI. Situated and reactive robotics (Brooks, 1991) were similarly inclined as were discourses of ‘emergence’ and iterative processes in Artificial Life, Self Organisation and Dynamical Systems Theory (Kelso, 1995). In cognitive science, Enactivism, situated and distributed cognition challenges cognitivist attitudes. [43] In the humanities, the rise of performance theory (in some forms) contests the excesses of the linguistic turn. In science studies, Actor Network Theory and Pickering’s Mangle both destabilise conventional objectivist science discourses. In art theory, we saw the rise of relational aesthetics, and in philosophy, a resurgence of interest in Spinoza and Bergson, brokered by Deleuze (2001) and later Massumi (2002), and a resurgence of pragmatism (James, 1907; Dewey, 2005). This is quite a patchwork, and some of the pieces might feel a little strained. But across diverse fields in the late twentieth century, such approaches have lurked on the margins of positivist disciplinary discourses. To go beyond a call for the recognition of the fundamentally embodied and distributed nature of cognition, the upwelling of arguments (in such diverse fields as feminisms, performance studies, science studies, and cognitive and neurosciences) which contest presumed axiomatic binaries of subject/object, world/representation suggests a fundamental and large scale change in ontology which one might be excused for identifying as a paradigm shift.

Contemporary cognitive science, informed in part by Heidegger, Husserl and Merleau-Ponty, has brought into question not only cognitivist representationalism, but the fundamentally Cartesian binaries of mind/body, self/world and subject/object. The deep ontological ramifications of all this is captured with some precision by Karen Barad (2007), in which she argues that the very construction of subject and object are historically contingent, and proceeds to propose a radically materialist and performative ontology which sees ‘phenomena’ as primary and subjects and objects contingently forming or falling out of a process of ‘intra-action’. Such an approach would be consistent with the performative ontology of Pickering, the ‘laying down a path in walking’ of Enactivism and the ‘ongoingness’ of O’Reagan and Noë (2001). This ontological reformulation has direct relevance to the theorisation of, and the creation of, interactive artefacts.

In Rafael Lozano Hemmer’s Displaced Emperors the gesture of the swipe of an outstretched hand wipes the façade off the palace, revealing (images of) the rooms inside. The façade, in both a real and metaphorical sense, is destroyed, performatively implicating the user in a violent act. The user, standing in front of the imposing edifice, makes a grand gesture. They could make a wriggle or a flick, but they tend to make an expansive full arm gesture. How such a style of gesture is implied or specified is a mysterious aspect of the design of the scenario—but it happens. And the value of that gesture—grand and empowered, is an integral part of the experience. Perhaps the gesture is implied by the immediately prosthetic sensori-motor linkage of the gesture with an image of a (giant) hand, which strokes the surface of the façade as a shadow would. The embodied sensorimotor understanding of the movement of my shadow as action at a distance is exploited here. The strength and scale of the arm is amplified, as with a sword or a staff, and this powerful force-amplified prosthetic arm wipes the masonry of the façade away—at least it appears that way, sans the noise and rubble and dust. Here then, embodied and gestural engagement constitutes something like epistemic action—but in the process, nothing is resolved. Through enactive or performative action the user is immersed in a context loaded with the paradoxes of colonialism.

Autistic and Solipsistic Machines

In recent years a new class of devices has arisen which offer a perversely witty response to the preoccupation with the user. These works possess complex behaviour and are so autonomous as to have no connection with human viewers. In such works, the visitor engages in a conventional mode of passive observation of an autonomous machine negotiating its environment. The biological or ecological implications suggest a kind of synthetic ethology. Two recent works exhibit a resurgence of a minimalist/formalist sculptural aesthetic. They also have in common a commitment to materiality and to integrating electro-physical realities as part of the larger computational a system. That is, the elegance of their formalism has extended into the machinic/performative dimension of their existence.

Der Zermesser by Leonhard Peschta (2007) is a large but minimal geometrical robot which engages in a sensitive relation with architectural space, expressing its response as perturbations of its tetrahedral form.[44]One might call it ‘architaxis’. Each node is instrumented and motorised such that it can both move and adjust the lengths of is vertices. Der Zermesser works on a slow, non-human timescale, like a mollusc on a rock. This work is exemplary of a long-delayed reunion of aesthetic systems of conventional plastic arts with interactive art. The geometric minimalism of Der Zermesser recalls formalist sculpture, that epitome of modernism, and indeed, it could be said to implement formalist aesthetics as a machine, in the same way that computer hardware implements Boolean algebra. And certainly the formalisms of coding and computer engineering do have immediate sympathies with the modes and methods of formalist sculpture. The formal coherence and elegance of Der Zermesser is carried through to its electronic aspects—the geometrical nodes are also motor, sensory and computational nodes. The tetrahedron, the first platonic solid, the strongest polyhedron, is an icon of engineering efficiency, and is also manifested in a myriad of biological forms, evidence of the effectiveness of evolutionary design. Der Zermesser captures this duplicity as an adaptive geometry.

The Conversation, by Ralf Baeker (2010) is a self-referential and homeostatic colonial machine—a model of a colonial organism rendered in electro-physical machines. [45] The Conversation is a closed-loop computational system combining electro-mechanical, analog and digital electronic components, presented in an elegant and minimal sculptural form—a ring made up of a 99 solenoids. The solenoids pull on wires connected to three concentric rubber rings in the centre of the device. With a machine parsimony based in an understanding of electro-physical phenomena, Baeker monitors fluctuations in the magnetic pull of the solenoids via their current consumption (where others might have added an array of sensors). A floating metal ring—a platonic apparition, paradoxically pulsates with the vibrancy of the living. If the metal ring is the cell wall, the three fluctuating red rubber rings define the nucleus, with radiating strings relaying tensions back and forth between the two. Crucially, the physical system is not ‘driven’ by code, but in dynamical and cybernetic sensibility, the moment by moment condition of the material array drives resonating cycles of oscillations and dampings within the system.

Conclusion

The general historical picture I hope to have drawn here places the development of an aesthetics of interaction within a developing technological context. During the ‘heroic period’ of interactive art, big questions such as: ‘how can we deploy computational capability in artworks?’ and ‘how can we integrate computation with material, sensorially immediate practices?’ motivated work. The technological challenges of interaction, like the other technical challenges of the 90s—problems of wireless communication, computer graphics and of machine vision—are now effectively resolved. Culturally, the novelty of the scenario of the machine which responds to a user in real-time has clearly worn off. In digital cultural practices, exploration of the modalities of interaction has been fairly thorough, though there is always room for inventive exploration of the subtle complexities of the poetics or aesthetics of interaction. In my opinion, future development of the aesthetics of interaction might usefully be framed as three areas of concern: the material artefact, the code/machine system, and the dynamics of interaction.

For two decades, the computational has been more or less ‘pasted on’ to artefacts and social structures. And with the technology, the rhetoric of cognitivist computation has also usually been more or less uncritically pasted on. Works like Giver of Names, Der Zermesser, and The Conversation, demonstrate an integration of sculptural and embodied dimensions with a sensorial and performative orientation to coding, into a coherent aesthetic and theoretical approach (we heave a sigh of relief and whisper ‘at last!’). In their integration of materiality and computation, these works reject a dualist computationalist separation of software and hardware, information and matter, control and action. For me this signals a new maturity in the cultural and theoretical grounding of work of this kind, where ‘interactivity’ is subsumed into a wider field of autonomous machine behaviour. As I have proposed elsewhere, in our current era of ubiquitous computation, the universe of live data which was once called ‘the virtual’ is increasingly anchored into physical and social context via a diversity of digital commodities. The technologies, techno-social structures and modalities of interaction which permit this (re)union were workshopped and prototyped in ‘media arts’ research and elsewhere over the past quarter century.

A second generation of practitioners naturalised to the digital are pursuing a more organic interrelation between machine behaviour and sensoriality/materiality than their forbears could, in part due to technological developments but also due to a maturation of the fine arts context. Meantime, an increasingly code and hardware-literate community of artists are able to deploy more sophisticated aesthetic code-machines. A deepening and theoretically substantiated conception of interaction and enactive cognitive process inhering in a performative ontology promises more rich and subtle systems. And lastly, the conception of ‘interaction’ has been expanded beyond user-machine, to larger ideas of behaviour between machines and machine systems, and between machine systems and the world. This leads to a kind of machinic ecology, and potential useful application of actor network theory.

I have proposed that across a diverse range of disciplines, we are on the cusp of a veritable Kuhnian paradigm shift toward a performative ontology. Such moments are, in the terms of Hakim Bey and Peter Lambourne Wilson, truly ‘temporary autonomous zones’ (Bey, 1991) and open up temporarily at moment of flux. Like all good technical ideas, it has a heady whiff of metaphysical liberation about its tattered edges. Over the last fifty years or so, such shifts have appeared (on the horizon so to speak) only—mirage-like—to evanesce. For example, it is salutary to note how aspects of the cybernetics which could be recuperated to the positivist establishment lived on, control theory being a case in point. In any case, in such a Kuhnian shift, the intractable is rendered trivial by an orthogonalising shift of perspective. In my opinion, the practice and performance of interactive art itself is an integral part of that ontological shift, and that shift offers leverage on theoretical questions which have seemed vexing under previous theoretical approaches.

Biographical Note

Simon Penny is Professor of Arts and Engineering at University of California Irvine. His practice has included artistic practice, technical research, theoretical writing, pedagogy, and institution building in Digital Cultural Practices, Embodied Interaction, and Interactive Art. He makes interactive and robotic installations utilising novel sensor arrays and custom machine vision systems which address critical issues arising around enactive and embodied interaction, informed by traditions of practice in the arts including sculpture, video-art, installation, and performance; and by ethology, cognitive science, phenomenology, human-computer interaction, ubiquitous computing, robotics, critical theory, cultural studies, media studies, and Science and Technology Studies.

Notes

[1] I have explored related themes in two recent papers: ‘Trying to be Calm,’ and ‘Desire for Virtual Space’ (2012, 2011). The current paper addresses related issues across the same time period. As such the three might be usefully read together.

[2] Discussion of these works can be found at www.simonpenny.net (works).

[3] This technological obsolescence is a profound impediment to pedagogy and historical research. It is a regrettable fact that much contemporary media art history and theory discusses works which the authors themselves have not actually experienced, and often relies on poor documentation and/or other textual accounts. This situation would be regarded as intolerably unscholarly in conventional art history. Imagine the contempt an art historian would be held in if they presumed to write on, say, the ceiling of the Sistine chapel without having actually seen it. So much worse then, is relying on photographs (!) of a work which was originally dynamic and multi-media.

[4] Eigenwelt der Apparatwelt, Pioneers of Electronic Art, Ars Electronica 1992 exhibition catalogue, 1992.

[5] See http://www.senster.com/

[6] See http://www.youtube.com/watch?v=dqZyZrN3Pl0

[7] See http://www.youtube.com/watch?v=lLULRlmXkKo (Quaint BBC coverage)

[8]Dietrich Prinz wrote the first chess playing program for a general purpose computer. The program first ran in November 1951 on the Manchester Ferranti. Prinz was the head of programming at Ferranti.

[9] Steve Wozniak recently commented when asked if, in the late 80s, he foresaw internet porn, that the early Apples did not have enough memory to render a (colour) image, to say nothing of real-time video streams. (Radio interview on ‘Wait, wait, don’t tell me’, December 2010, personal notes)

[10] See my forthcoming essay: ‘Emergence, Improvisation, Interaction’, Lewis and Piekut (eds) Oxford Handbook of Improvisation Studies.

[11] Likewise, sophisticated machine vision procedures, such as ‘body-segmentation’—the nerdy moniker for inferring skeletal anatomy from single or multiple images—which were non trivial and cutting-edge technical problems circa 2005, are now part of commodity devices, i.e. the Kinect.

[12]As far as I am aware, this assertion remains speculative. Given the extensive evidence in developmental neurology for similar neurological adaptation, I am persuaded by this idea. As we move into the generations of those ‘born digital’ there is a call for neurophysiological research into structural and behavioural differences between those naturalised to digitality and those who came to digital systems later in life.

[13]As indicated, I am here focusing my attention upon interaction design in the fine arts, and referring to practices which extend beyond the desktop or hand held device. I am well aware of the rich complexities of the domain specific interaction modalities in networked interactions from virtual communities (Second Life) to massively multi user gaming (World of Warcraft) to dimensions of social media from Facebook to Twitter, but these are beyond the scope of the current text.

[14]Stories of such unfamiliarity from the early 90s are common, such as that of the young woman driven to tears by the computer mouse. She had reasoned, perfectly reasonably, that the mouse should ‘face’ the screen, i.e. with the ‘tail’ toward her, which inverted all her actions with respect to cursor movement.

[15]I use the term ‘user’ here (grudgingly) and in what follows to refer to anyone interacting with an interactive artwork. Many including myself have deplored (the instrumentality of) the term, and have sought alternatives and coined neologisms, but alternatives have been clunky and have not achieved general acceptance.

[16] See Agre (1997). A more subtle point is that to the extent that such modalities are intuitive in this sense, neurological impact will be minimal. But as digital behaviours themselves become ‘naturalised’ at the level of culture and of embodied cognition, we will see a drift in what is taken to be ‘intuitive’.

[17] This phrase is the title of chapter 11 of The Embodied Mind: Cognitive Science and Human Experience by Francisco J. Varela, Evan Thompson, Eleanor Rosch (MIT press, 1992) The authors note ‘Our guiding metaphor is that a path exists only in walking…’ (241). The term has been taken up in aspects of contemporary cognitive science and Philosophy of mind, notably the work of Kevin O’Regan and Alva Noë (2001).

[18] See http://www.youtube.com/watch?v=GALMmVZ49Pc

[19] See http://homepage.mac.com/davidrokeby/vns.html

[20] In computer science and engineering circles, the idea of generality or general applicability is a cardinal virtue. This ethos can lead to systems which do many things badly and nothing well. The approach of many artists (including Rokeby) is to achieve a specific goal with quality and elegance. An approach which is fundamentally pataphysical. ‘Pataphysics will be, above all, the science of the particular, despite the common opinion that the only science is that of the general’. Alfred Jarry

[21] The geometrical basis of perspective was laid by the Arab mathematician Alhazen (al-Hasan Ibn al-Haytham) in 1021. There is nothing ‘natural’ about perspective, it is not the ‘true’ way to represent the world, and it is not an integral biological aspect of human vision.

[22] I developed the concept of ‘machine parsimony’ in the mid 90s to describe this kind of ‘application specific’ development which did not subscribe to the requirement for generality.

[23]See http://www.year01.com/helpless/statement.html

[24] See http://www.medienkunstnetz.de/works/portrait-one/

[25] I witnessed this at SIGGRAPH91 in Las Vegas.

[26] It is no accident that these three pioneering works all arose in Canada. Canada was remarkably proactive in funding and otherwise supporting ‘media-art’ research in the period and in the small international community of interactive art—Canadian artists (etc) were ubiquitous and recognised as doing generally progressive advanced work.

[27] See http://www.simonpenny.net (works—petit mal).

[28]Interestingly, the only demographic who were unwilling to interact were adolescents.

[29] The conception of ‘play’ as encapsulated in typical computer games is a decidedly narrow slice of modes of play and has an instrumental cast to it. When, for instance, the paradigmatic ‘first person shooter’ is stripped of its colourful monsters and futuristic weapons, computer game-play resembles the worst qualities of industrialised labour: constrained and highly repetitive tasks executed in social isolation, a tight harnessing of user and machine, rewards linked to high rates of production, to say nothing of the covert inculcation of military skills.

[30]See http://www.iamas.ac.jp/interaction/i97/artist_Fujihata.html, http://www.youtube.com/watch?v=6Ek2DW7aV68

[31] See http://www.simonpenny.net/works/fugitive.html, http://www.simonpenny.net/works/fugitive2.html

[32]Terminal Time http://www.terminaltime.com/

[33]See my ‘Agents as Artworks and Agent Design as Artistic Practice’ (1999). This kind of adaptive technology harks back to cybernetic precepts but has rarely been pursued in artwork. A work which has pursued this to some extent is Performative Ecologies by Ruari Glynn (2008) (discussed below). 

[34] Agre’s remarkable paper is the most insightful inquiry into the culture of AI I know of.

[35] See http://homepage.mac.com/davidrokeby/gon.html

[36]See http://www.galeriavirtual.org/varis/lightpools.ogg

[37]El bal de Fanalet/ Lightpools used the GAMS triangulating sonar sensor developed by Will Bauer and Rafael Lozano-Hemmer, also deployed in Displaced Emperors (Lozano-Hemmer, 1997).

[38] See for instance the work on concept blending by and Fauconnier and Turner (2008).

[39]See http://www.simonpenny.net/works/sympathetic.html, http://www.simonpenny.net/works/sympatheticII.html

[40] See http://vimeo.com/6400266

[41]Sniff and Performative Ecologies were included in Emergence, a show of Artificial Life Art curated by Simon Penny and David Familian at the Beall Center for Art and Technology, University of California Irvine, December 2009 – April 2010. Regrettably Performative Ecologies did not function as designed during the exhibition.

[42]See http://www.lozano-hemmer.com/displaced_emperors.php 

[43] Enactivism, situated and distributed cognition are three major variants of post cognitivist cognitive science.

[44] See http://www.youtube.com/watch?v=2kRuxutzluE

[45] See http://www.no-surprises.de/the_conversation 

References

• Agre, Philip. ‘Towards a Critical Technical Practice: Lessons Learned in Trying to Reform AI’, Geoffrey C. Bowker et al. (ed.), Social Science, Technical Systems, and Cooperative Work : Beyond the Great Divide (Mahwah, N.J.: Lawrence Erlbaum Associates, 1997).
• Agre, P.E. and D. Chapman. ‘Pengi: An implementation of a theory of activity’, Proc., AAAI-87 (Seattle, WA, 1987): 268-272.
• Barad, Karen. Meeting the Universe Halfway: Quantum Physics and the Entanglement of Matter and Meaning (Duke University Press Books, 2007).
• Bateson, Gregory. Steps to an Ecology of Mind (University of Chicago Press, 1972).
• Bey, Hakim. T.A.Z. the Temporary Autonomous Zone, Ontological Anarchy, Poetic Terrorism (San Francisco: Autonomedia, 1991).
• Brooks, Rodney. ‘A Robust Layered Control System for a Mobile Robot’, A.I. Memo No. 864, Massachusetts Institute of Technology, Artificial Intelligence Laboratory, September (1985).
• —. ‘Intelligence without Reason’, A.I. Memo No. 1293, Massachusetts Institute of Technology. Artificial Intelligence Laboratory, April (1991).
• —. ‘Elephants Don't Play Chess’ Robotics and Autonomous Systems 6 (1990): 3–15.
• Deleuze, Gilles. Spinoza: Practical Philosophy (San Francisco: City Lights, 2001)
• Dreyfus, Hubert L. What Computers Still Can't Do: A Critique of Artificial Reason (Cambridge, Mass.: MIT Press, 1992).
• Chun, Wendy Hui-Kyong. Control and Freedom: Power and Paranoia in the Age of Fiber Optics (Cambridge, MA: MIT Press, 2008).
• Dewey, John . Art as Experience (New York: Perigee Trade, 2005).
• Di Paolo, Ezequiel. A. 'Extended life', Topoi 28 (2009): 9-21.
• Fauconnier ,Gilles and Mark Turner. ‘Rethinking Metaphor’, Cambridge Handbook of Metaphor and Thought (New York: Cambridge University Press, 2008): 53-66.
• Fuller. R. Buckminster. I Seem to Be a Verb: Environment and Man's Future (New York: Bantam Books, 1970).
• Fuller, Matthew (ed.). Software Studies – A Lexicon (Cambridge, Mass.: MIT Press, 2008).
• Hansen, Mark B. N. New Philosophy for New Media (Cambridge, MA: MIT Press, 2006).
• Holland, John. Adaptation in Natural and Artificial Systems (Michigan: University of Michigan Press, 1975).
• Hutchins, Edwin. ‘Imagining the Cognitive Life of Things’, Lambros Malafouris and Colin Renfrew (eds), The Cognitive Life of Things: Recasting Boundaries of the Mind (2006): 7-9.
• <http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.104.6035&rep=rep1&type=pdf>
• James William. Pragmatism: A new name for some old ways of thinking (New York: Longmans, Green and Company, 1907).
• Kelso, J.A. Scott. Dynamic Patterns: The Self-Organization of Brain and Behavior (Cambridge, Mass.: MIT Press, 1995).
• Kirsh, David and Paul Maglio. ‘On Distinguishing Epistemic from Pragmatic Actions’ Cognitive Science 18 (1994): 513-549.
• Latour, Bruno. Reassembling the Social: An Introduction to Actor-Network-Theory (Oxford: Oxford University Press, 2007).
• Malafouris, Lambros. ‘Beads for a Plastic Mind: the ‘Blind Man's Stick. (BMS) Hypothesis and the Active Nature of Material Culture’, Camb. Archaeol. J. 18.3 (2008): 401-414.
• Manovich, Lev. The Language of New Media (Cambridge, MA: MIT Press/Leonardo Books, 2002).
• Massumi, Brian. Parables for the Virtual: Movement, Affect, Sensation (Durham: Duke University, 2002)
• Mateas, Michael. ‘Expressive AI: A hybrid art and science practice’, Leonardo: Journal of the International Society for Arts, Sciences and Technology 34.2 (2001): 147-153.
• Merleau-Ponty, Maurice. Phenomenology of Perception (New York: Routledge and Kegan Paul, 1965).
• O’Reagan, J Kevin, and Alva Noë. ‘A Sensorimotor Account of Vision and Consciousness’ Behavioral and Brain Sciences 24 (2001): 939–1031.
• Pask, Gordon. ‘A Comment, A Case History and a Plan’, J.Reichart (ed.) Cybernetics, Art and Ideas (London: Studio Vista, 1971): 76-99.
• Penny, Simon. ‘Trying to be Calm: Ubiquity, Cognitivism, and Embodiment’, Ulrik Ekman (ed.) Throughout (Cambridge, Mass.: MIT Press, forthcoming 2012).
• —. ‘Agents as Artworks and Agent Design as Artistic Practice’, Kerstin Dautenhahn (ed.) Human Cognition and Social Agent Technology (Amsterdam: John Benjamins Publishing Company, 1999): 395-414.
• —. ‘Twenty Years of Artificial Life’, Digital Creativity 21.3 (2010): 197-204.
• —. ‘Desire for Virtual Space : The Technological Imaginary in 90s Media Art’, Space and Desire. Scenographic Strategies in Theatre, Art and Media (Zürich : Zurich University of the Arts, 2011).
• Pickering, Andrew. The Mangle of Practice: Time, Agency, and Science (Chicago: University of Chicago Press, 1995).
• Ray, Thomas S. 'Evolution, ecology and optimization of digital organisms', Santa Fe Institute working paper 92-08-042 (1992).
• Stern, Nathaniel. ‘The Implicit Body as Performance: Analyzing Interactive Art’, Leonardo 44.3 (2011): 233-238.
• Steels, Luc. 'Perceptually Grounded Meaning Creation'. in Tokoro, M., (ed.), ICMAS96, AAAI Press (1996).
• Steels, Luc and Brooks, Rodney. The artificial life route to artificial intelligence: Building Situated Embodied Agents (New Haven: Lawrence Erlbaum Ass, 1994).
• Thompson, Evan. Mind in Life: Biology, Phenomenology, and the Sciences of Mind
• (Harvard: Belknap Press of Harvard University Press, 2010).
• Varela, Francisco, Evan Thompson, Eleanor Rosch . Embodied Mind The Embodied Mind : Cognitive Science and Human Experience (Cambridge, Mass.: MIT Press, 1992).
• Vasulka, Woody. Eigenwelt der Apparatewelt, Pioneers of ElectronicArt (Ars Electronica 1992 exhibition catalogue). <http://www.vasulka.org/Kitchen/PDF_Eigenwelt/Eigenwelt.htm>

Artworks

• Baeker, Ralf The Conversation 2010
• Courschesne, Luc Portrait One 1990
• Fujihata, Masaki Beyond Pages 1995
• Glynn , Ruairi Performative Ecologies 2008
• Ihnatowicz, Edward Senster 1971
• Krueger, Myron Videoplace 1975
• Lozano-Hemmer, Rafael Displaced Emperors 1997
• Mateas, Michael, Steffi Domike, Paul Vanouse Terminal Time 1999
• Pares, Narcis, Roc Pares and Perry Hoberman El bal de Fanalet/ Lightpools 1998
• Pask, Gordon Musicolor 195
• Penny, Simon Petit Mal – an Autonomous Robotic Artwork 1995
• Penny, Simon, and Jamie Schulte Sympathetic Sentience 1996
• Penny, Simon Fugitive 1997
• Peschta, Leonhard Der Zermesser 2007
• Rokeby, David Giver of Names 1998
• Rokeby, David Very Nervous System1986
• Sobecka, Karolina and Jim George Sniff 2009
• White, Norman Helpless Robot 1987

Introduction

This article treats the philosophical underpinnings of the notions of ubiquity and pervasive computing from a historical perspective. The current focus on these notions reflects the ever increasing impact of new media and the underlying complexity of computed function in the broad sense of ICT that have spread vertiginiously since Mark Weiser coined the term ‘pervasive’, e.g., digitalised sensoring, monitoring, effectuation, intelligence, and display. Whereas Weiser’s original perspective may seem fulfilled since computing is everywhere, in his and Seely Brown’s (1997) terms, ‘invisible’, on the horizon, ’calm’, it also points to a much more important and slightly different perspective: that of creative action upon novel forms of artifice. Most importantly for this article, ubiquity and pervasive computing is seen to point to the continuous existence throughout the computational heritage since the mid-20th century of a paradoxical distinction/complicity between the technical organisation of computed function and the human Being, in the sense of creative action upon such function. This paradoxical distinction/complicity promotes a chiastic (Merleau-Ponty) relationship of extension of one into the other. It also indicates a generative creation that itself points to important issues of ontology with methodological implications for the design of computing. In this article these implications will be conceptualised as prosopopoietic modeling on the basis of Bernward Joerges introduction of the classical rhetoric term of ’prosopopoeia’ into the debate on large technological systems. First, the paper introduces the paradoxical distinction/complicity by debating Gilbert Simondon’s notion of a ‘margin of indeterminacy’ vis-a-vis computing. Second, it debates the idea of prosopopoietic modeling, pointing to a principal role of the paradoxical distinction/complicity within the computational heritage in three cases:

• a. Prosopopoietic aspects of John von Neumann’s First Draft of a Report on the EDVAC from 1945.
• b. Herbert Simon’s notion of simulation in The Science of the Artificial from the 1970s.
• c. Jean-Pierre Dupuy’s idea of ‘verum et factum convertuntur’ from the 1990s. 

Third it concludes with a brief re-situating of Weiser’s notion of pervasive computing on the basis of this background.

First part – Pervasive Computing – Beyond Use as Interaction

One of the best kept secrets of new media success in the past twenty years or more is the creative role of the human user. This claim may outrage any serious observer of new media – after all, isn’t new media about human user interface, user interaction, media (user) culture, appealing graphics and applications (to the user) and so on? What about Web 2.0 and social software? Isn’t the history of computer use since the 1940s one long process of accommodation of the human user in still more sophisticated manners?

The short answer is no.

The creative action of human use as a principal instance of human ontology, that is an entity that may be different in principle from computed function, yet important for such function, has never been queried in the history of computing. The conception of computed function has never been approached from the perspective of a form of Being ontologically different from computing, yet of crucial importance. Often it has simply been ignored. Think about the ongoing thrust of etymology: the term for socially inclusive software, ‘social software’ – networked communities of humans – is still ‘software’, and so on. The predominant assumption (and a tenet of a basically cybernetic origin) has been that human use is ontologically predicated on implications of the technical organisation of computed function. In fact, in the history of computing the human has grosso modo been rendered an entity to be ontologically surpassed or even eradicated by the diffusion of computed function into all aspects of the human’s Being. From computer chips in model trains, intelligent buildings, and smart materials, to robotics, to models of the brain's functionality, all sorts of computer based modeling, and further, to recent ideas of a connectionist network sociology (Watts, 2003; Barabási, 2003); the attempt to fully implicate the ontology of the human Being within computing has been significant.

A preliminary sketch may indicate three common frameworks for understanding human use in this regard (use/interaction etc.) in the post war era:

• (a) Human use derived as directly as possible from computed function in well defined measures of design (e.g., Jakob Nielsen’s ‘usability’ [Nielsen, 1993]),
• (b) Use derived from perceptual aspects of approaches to computed function (e.g., Donald Norman’s ‘affordance states’ [Norman, 2001 [1998]]),
• (c) Use derived from computed function as social context (e.g., Susan Leigh Star’s assumption of new media as ‘community building media’ [Starr, 1995]).

Put differently, the technical organisation of computed function has been posited as ontological determination of use in the sense of human use by a human Being, that is, of this Being’s body, psyche-soma, intelligence, behavior, emotions, social predication and so on. In sum, this is use in whatever form we know.

Pervasive Computing – Creating an ‘Unknown Artificial World’

The broad origin of this determinism is what we, for want of a better term, may call a computational heritage originating in the mid-20th century (from Alan Turing’s response to David Hilbert’s ‘Entscheidungsproblem’ in the 1930s onwards). This heritage today encompasses a stunning mass of issues, disciplines and knowledge closely related to formalisations (e.g. algorithms), languages/codes (e.g. programming languages), techniques (e.g. CPUs), artifacts (e.g. computers) and smart materials (e.g. piezo-materials). In addition to this are further perspectives arising within new disciplines and theories such as self-organisation, connectionism and complexity – all contributing to a paradigm of ‘complexification’ (Casti, 1994, 2002; Eve et al., 1997; Mainzer 1994; Watts 2003; Barabási, 2003). Added to this, there are the immense and diverse movements into the ‘posthuman’ (Hayles, 1999) along with constructivist social and cultural theory and philosophy, exemplified by the thought of Niklas Luhmann and Gilles Deleuze, to mention two well-known thinkers. Further, there is the appearance of ‘classic’ (Toronto School) and new media and communication theory (Michelsen, 2005, 2007).

However, considering the scope of computed function in today’s society there is an impending need for a look at the human Being in different terms. Put differently: if the claim of pervasive and ubiquitous forms of embedded digital computing is correct what does that entail for the human Being which ‘embeds’ this? To begin to answer this, I turn to a different understanding of the artificial.

Ezio Manzini has argued for an entirely novel state of the artificial per se. He argues that there exists a discrepancy between the human creation of the artificial and the lack of insight into the distinctive character of the resulting effects. ‘To man the artificial is a completely natural activity,’ (Manzini, 1994: 44), but the resulting artificiality appears as ‘an unknown artificial world that we must examine to discover its qualities and laws’ (Manzini: 52). Manzini argues that it is necessary to establish positive connotations for the artificial.

In this article the current interest in pervasiveness and ubiquity will be used as occasion for discussing such connotations, beyond an ontological determination by computed function. The article's point of departure is Cornelius Castoriadis’s idea of a new form of self organisation pertaining to the Human Being per se: ‘a human strata of the real’. This strata is distinguished by a strong ontological imagination – ‘the imaginary institution of society’ (Castoriadis, 1987). All phenomena within such a strata reside ontologically within human creativity, what he terms ‘humanity’s self-creation’:

Humanity self-creates itself as society and history – there is, in humanity’s self-creation, creation of the form of society, society being irreducible to any ‘elements’ whatsoever … This creation takes place ‘once and for all’ – the human animal socializes itself – and also in an ongoing way: there is an indefinite plurality of human societies, each with its institutions and its significations, therefore each also with its proper world. (1997: 339)

In Domaines de l’homme (1986), Castoriadis writes that a new type of distinction/complicity between ratio and creativity follows from the idea of humanity’s self-creation. He argues that it is necessary to undertake the project of distinguishing between and thinking together the ‘ensidic [rational] dimension and the proper dimension of the imaginary’ (1986: 16–17). Folllowing from this, it is possible to envisage an ontological ‘distinction/complicity’ between the technical organisation of computed function enacted in a human strata of the real and a strong ontological imagination of a human Being (Michelsen, 2007). There is a need to undertake a project of distinguishing between and thinking ‘together’ the ratio of computed function and the proper dimension of the imaginary in an ontological sense of creative action upon such function.

For all these reasons, the following will account for what I have called a paradoxical distinction/complicity between the technical organisation of computed function and the human Being in the sense of creative action upon such function. This paradoxical distinction/complicity will be described as the opening of a certain form of aisthesis based on the intertwining of function and creative action as two instances which are ontologically distinct, yet deeply complicit. This is what I will term a chiastic aisthesis, following Merleau-Ponty’s famous idea of the ‘chiasm’ of the ‘visible and the invisible’ (Merleau-Ponty, 1983). Furthermore, the approach here will be conceptualised in general terms as a methodology of prosopopoietic modeling developed from the classical rhetorical device of prosopopoeia (Greek: προσωποποιία).

Pervasive Computing – Paradoxical distinction/complicity

A first observation of the paradoxical distinction/complicity may be found in the work of the French pioneer of cybernetics, Gilbert Simondon, from 1958. Simondon writes about the relation between the aesthetic and the cybernetic organisation of ‘technicity,’ what he terms the ‘ensemble’ (Simondon, 1989 [1958]) (all translations of Simondon’s French text are mine. A.Michelsen). For Simondon, the problem of ‘the aesthetic’ has implications for certain acts of primary importance. The aesthetic form of thought, the aesthetic object – fundamentally the aesthetic , as he argues in a dense overview – points to a peculiar origin, a ‘remarkable moment’ [point remarquable] or ‘position’, ‘event’, of cultural issues (Simondon, 1989 [1958]: 179ff). The aesthetic suggests a ‘bifurcation’ of the natural world by way of a remarkable moment which makes explicit a cultural world of humans, and further a human form of Being. Importantly, this world stands out as a particular bounding of the aesthetic in a human mode which is non-dichotomous, i.e. not within the received idea of a subject-object dichotomy. It is a form of organisation relying on a dimension of Being – the cultural, the human etc. – which is in-between ‘pure objectivity and pure subjectivity’ (187).

The idea of the ensemble thus indicates a different organisation bounded not by computed function but by an appraisal of the human which is open towards computed function. This allows one to conceive of the human in an ontological sense which is on one hand different from the technical organisation of computed function, and on the other hand related to this organisation. It is possible to envision two ontologies without creating a conflict with the cybernetic tenet, because they refer to principally different forms of organisation which are nevertheless open to each other in the sense of an intertwining in-between ‘pure objectivity and pure subjectivity’.

For Simondon it is important to avoid a definition of the ensemble (11) as an extended determination by computed function. On the contrary, a computing machine exhibiting a range of options is not an automat determined by a vocabulary of closed functions. It is a machine which qva its technological definition introduces a principal margin of indeterminacy relying on human intervention, defined by Simondon as ‘perpetual innovation:’ (12)

(…) the being of the human intervenes as regulation of the margin of indeterminacy, which in the final consequence is adapted to a more comprehensive exchange of information.

The technical organisation of computed function thus resides within an enforcing openness because of innovative human steps emerging in the margin of indeterminacy. We may argue this way: at one extreme of a spectrum the technical organisation of computed function is ‘hard’ and almost impenetrable for the human, e.g. in the aggregate automated workings of code. At another extreme the human is almost wholly aloof in a self-enclosed imaginary flux which has nothing to do with forms of computing. However, in the intertwining middle there emerges an enforcing openness between computed function and creative action. Here computed function becomes defined by the margin of indeterminacy and creative action rendered computational.

Computed function is thus co-defined by a ‘regulation’ in the shape of ‘perpetual innovation’, that is, integrally related to the different instance of a human Being. This is what is unacknowledged or figured as ontologically surpassed in the computational heritage.

Simondon thus indicates a paradox. There exists a paradoxical distinction/complicity between the technical organisation of computed function and the human Being in the sense of creative action upon such function. In one perspective they are wholly distinctive; in another they are complicit. The result is that computing as we know it reside within a paradox.

Simondon thus argues not only for a paradoxical distinction/complicity beyond ontological determination of the human Being by computed function. He further indicates that: a. The distinction/complicity entails a margin of indeterminacy in between the purely objective and the purely subjective, i.e., beyond modern Western dualism. b. The ensuing ‘regulation’ can be seen as an intertwining of two distinct forms of organisations at the emergent level of a bifurcation. c. Here, cybernetically speaking, ‘a more comprehensive exchange of information’ is possible between innovative – or creative, forms of action, and the technological organisation of computed function. d. All that has to do with use is appearing in this intertwining middle. I want to suggest that it does so as a specific form of aisthesis.

From Pervasive Computing to Chiasm

This can be taken further by involving the notion of the ‘chiasm,’ outlined in Maurice Merleau-Ponty’s famous draft of what was to be the posthumuous work of The Invisible and the Invisible (Maurice Merleau-Ponty, 1983 [1968]) from the same decade as Simondon’s work on the technical object

Here Merleau-Ponty elaborates the famous idea of the visual as a ‘flesh’ of the world based on a ‘chiasm’ of something ‘visible’ and ‘invisible’. While his well known early work on the ‘phenomenology of perception’ from the 1940s argues that ‘our own body is in the world as the heart in the organism: it keeps the visible spectacle constantly alive, it breathes life into it and sustains it inwardly, and with it forms a system’ (Merleau-Ponty, 1986: 203), he takes radical steps in the second posthumous phase which allow for thinking the aisthesis of a paradoxical distinction/complicity:

With the first vision, the first contact, the first pleasure, there is initiation, that is, not the positing of a content, but the opening of a dimension that can never again be closed, the establishment of a level in terms of which every other experience will henceforth be situated. The idea is this level, this dimension. It is therefore not a de facto invisible, like an object hidden behind another, and not an absolute invisible, which would have nothing to do with the visible. Rather it is the invisible of this world, that which inhabits this world, sustains it, and renders it visible, its own and interior possibility, the Being of this being. (Merleau-Ponty, 1983: 151)

As research into the ‘paradox’ in the 20th century has shown – notably Willard Van Orman Quine’s investigations (Quine, 1966), a paradox does not rule out meaning per se. Some paradoxes, Quine argues, are only insolvable in their time. That is, they are only to a certain extent anomalies, e.g., Zenon's paradox. At a later stage they may be verified, and thus become veridical, even if some still maintain – so far – their anomaly. The chiasm in our context may be reviewed as a paradox under its way to verification.

This is exactly what Merleau-Ponty argues. The chiasm is a verified paradox based on the transversal structure of the flesh which organises the visible in forms of meaning beyond the seeing subject vis-à-vis the visible object. Moreover, this is a form which in its intertwining becomes inner and outer in the same moment, that is, a fact of the world and an idea. It is an organisation, or an ensemble, which organises potential and actualises forms by an inherent dynamic involved with the intertwining.

We can look at the history of computing from this perspective. Take the history of the internet. From the first drafts of internet architectures in the late 1950s and 1960s by Baran and Davies, up to the breakthrough of the World Wide Web in the mid-1990s, important initiations of creative action upon computed function have emerged. In Janet Abbate’s words, the network architectures developed from the 1950s onwards did not only accommodate a variety of computed functions, they were also paradoxically open in a very productive sense. Their later success was closely connected to this feature, ‘the ability to adapt to an unpredictable environment’ (Abbate, 1999):

No one could predict the specific changes that would revolutionize the computing and communication industries at the end of the twentieth century. A network architecture designed to accommodate a variety of computing technologies, combined with an informal and inclusive management style, gave the Internet system the ability to adapt to an unpredictable environment. (Abbate: 6)

From the Home Brew Computer Club (1975-1977) and its focus on amateur designs for a centralised technology that moved away from corporate, military and research uses, to the spread of cell phone technology in the past decade, such unpredictable environments have played a massive role for any new formation of use. Every turn of use has presupposed and implied perpetual innovation in the sense indicated by Simondon, i.e., human use – what Abbate, perhaps somewhat vaguely, terms ‘an unpredictable environment.’ And vice versa, every instance of human use has triggered a huge development of computed function. In direct contrast to many assumptions of the computational heritage, the human has not been an ontological entity to become eradicated. It has bounced back with every cusp of new use, to the point where current new media growth is almost wholly reliant on a deep human presence, as we see in phenomena such as Facebook and Youtube.

It is possible, indeed necessary, to argue that every time computing has leaped forward with new successes, this has resided paradoxically within an instance of the Human Being in the sense of creative action upon computed function in a still more comprehensive manner.

The current focus on the notion of pervasive computing is no exception. Notions such as ‘pervasive’, ‘ubiquituous’ and ‘embedded’ are yet another example of Simondon’s paradox, emergent as chiasm. This is clearly apparent in Mark Weiser and John Seely Brown’s manifesto The Coming Age of Calm Technology from 1997 (Weiser and Seely Brown, 1997). Here they envision a ‘third wave’ of computing – ‘beyond calculation’ in the Turing machine sense – which will be more at the users disposal. It will take on a membrane-like, active set of relationships between artifact and human, which, at the time of writing (1997), is primarily ‘post the personal computer of the desk top (PC)’:

The Third wave of computing is that of ubiquitous computing, whose crossover point with personal computing will be around 2005-2020. The ‘UC’ era will have lots of computers sharing each of us. Some of these computers will be the hundreds we may access in the course of a few minutes of Internet browsing. Others will be embedded in walls, chairs, clothing, light switches, cars – in everything. (Weiser and Seely Brown: 77)

In the ‘’"UC" era’ there will emerge a chiastic relationship between almost omnipresent computed function –in everything, experienced as seamless by the user. ‘Lots of computers’ will be ‘sharing each of us’. There will emerge a membraneous interaction between new ubiquituous artifice – an ‘artificial environment’ (Manzini) – on the basis of which two forms of organisation will chiastically engage with each other. This will be what Weiser and Seely Brown discuss as the absent, yet omnipresent ‘calm’ which ‘engages both the center and the periphery of our attention and in fact moves back and forth between the two’ (Weiser and Seely Brown: 79):

… it should be clear that what we mean by the periphery is anything but on the fringe or unimportant. What is in the periphery at one moment may in the next moment come to be at the center of our attention and so be crucial.

This is ‘encalming’ (80) because: a. It allows us ‘to attune to many more things’ since computed function is at a distance yet present, that is, absent yet present, ‘the periphery is informing without overburdening’, and b. It allows us to operate in-between center and periphery, thus placing the peripheral ressources of computed function in a ‘calm through increased awareness and power:’

… without centering, the periphery might be a source of frantic following of fashion; with centering, the periphery is a fundamental enabler of calm through increased awareness and power. (italics my emphasis)

Despite Weiser and Seely Brown’s explicit reference to the debates within HCI, e.g., Gibson, Norman etc, around issues such as affordance, what is suggested here is in fact in almost complete accordance with Simondon’s idea of a particular bounding of the aesthetic in between ‘pure objectivity and pure subjectivity’ (Simondon, 1989: 187). One may even argue, without taking the parallel too far, that the actual event of encalming is exactly the emergence termed by Simondon as ‘perpetual innovation’. Moreover, this entire conjecture is deeply chiastic, almost to such an extent that it may be translated directly into Merleau-Ponty’s draft description of a transversal structure of the flesh which organises the visible in forms of meaning, beyond the seeing subject vis-à-vis the visible object. This flesh is simply the dimension which allows for the intertwining middle described by Weiser and Seely-Brown. The options enabled by UC make for a chiastic organisation residing with this dimension.

Second Part – A Methodology of Prosopopoietic Modeling

In the second part of the paper I will debate this chiastic organisation in the sense of its methodological implications for the design of computing, inspired by Bernvard Joerges’s interesting note on the need for a prosopopoiesis of large technological systems (e.g., like the internet) (Joerges, 1996). With prosopopoiesis Joerges refers to the rendering meaningful of large technological systems in a human mode. Prosopopoiesis will be seen as a proper methodological definition of what I have termed a paradoxical distinction/complicity between the technical organisation of computed function and the human Being in the sense of creative action upon such function. It follows from the verified paradox of the chiasm, based on a transversal structure of flesh. It opens up certain approaches to novel forms of artifice. Importantly, this establishes positive connotations for the artificial in the terminology of Manzini. That is, it encourages the type of designs that allow for the chiasm that Weiser and Seely Brown discuss. In blunt terms, it allows for the designs which will render computed function a billion user large success in the second half of the 20th century.

Put differently, there exists a real chiasm – intertwining, of the paradoxical distinction/complicity – which makes up a history of use, i.e., creative action upon technological organisations of computed function (in the last resort, of the universal Turing machine idiom with all that it entails). This is guided methodologically by prosopopoietic modeling.

Prosopopoietic Modeling – Methodology

Joerges argues that large technological systems institute a ‘Macht der Sache’ [power of things] in a social context which needs to be rendered meaningful in their technical organisation by a ‘showing’ (15ff) in a human mode as ‘Körper der Gesellschaft’ [embodiment of society]. For this he uses the term I have already begun to discuss, ‘prosopopoiesis,’ derived from the notion of prosopopoeia in classical rhetorics. A prosopopoeia is a rhetorical device in which a speaker or writer communicates to the audience by speaking as another person or object. Joerges gives an example of this in the technological organisation by which the rain embodies the roof in the appearance of a delicate sound pattern intelligible to the aisthesis of a human underneath, who becomes enactively aware of the technical organisation of the roof (265ff). The rain appears via a particular gestalt – a prosopopoeia – that makes sense to a human: a sound to ear and so on.

Large technological systems must enact a response in the user in order for the systems to function (had the roof-user not been able to identify the roof, he might not creatively have found shelter, so to speak). This is paradoxical in the sense that it transforms the technical organisation into a much different but no less relevant range of emergent events involving the system 'roof' and sound. The fall of rain on the roof is thus potentially full of meaning which the prosopopoiea expands poietically by creative action. This literal implication is needed in order to situate the system fully. The roof has to become situated creatively as meaning for humans in order to function. Or more radically, the function of the roof is immediately situated within a plenitude of creative meaning and there is not really any function, in fact no roof, without it.

Prosopopoiesis may thus be described as a method for modeling events within Simondon’s paradoxical distinction/complicity as a margin of indeterminacy opening for innovation of a range of possible uses. To be precise, it situates use as any form of human use. Or, differently, it situates the negotiations between Weiser and Seely Brown’s center and periphery in something which may be rendered a method for enabling specific designs. Prosopopoiesis becomes a methodological locale for creative action upon computed function where we may discover the details of the chiasm intertwining ‘pure objectivity and pure subjectivity’ (Simondon). In the following I will pursue some further indications in three steps:

• Prosopopoietic modeling – John von Neumann’s Draft: the basic traits of prosopopoiesis, discussed via Paul Ricoeur’s idea of heuristic inscription and the complexity of calculus in the Turing machine.
• Prosopopoietic modeling – Herbert Simon’s simulation: a response to prosopopoiesis in the computational heritage. Here I will focus on Simon’s idea of simulatory interface.
• Prosopopoietic modeling – verum et factum convertuntur: negotiations from within the computational heritage: Here I will discuss Jean-Pierre Dupuy’s observation of modelling as product and transcendence of human finitude.

Prosopopoietic Modeling – von Neumann’s Draft

According to Wikipedia, ‘A prosopopoeia (Greek: προσωποποιία) is a rhetorical device in which a speaker or writer communicates to the audience by speaking as another person or object’ (Wikipedia. The Free Encyclopedia: Prosopopoeia):

This term also refers to a figure of speech in which an animal or inanimate object is ascribed human characteristics or is spoken of in anthropomorphic language. Quintilian writes of the power of this figure of speech to ‘bring down the Gods from heaven, evoke the dead, and give voices to cities and states …

In order to enact meaning vis-à-vis an ‘inanimate object,’ the object in question must bring about a gestalt which may relate to the creative action of a human. In the computational heritage, this basic trait may be said to have first emerged as a creative action upon the complexity of calculus in the Turing machine. The Turing machine was one of the most sophisticated artifacts ever invented, not least because of its potential for modelling everything compatible with the universal Turing machine form. Yet it also appeared difficult for humans to approach directly. A number of complicated steps had to be taken in order to understand and utilise the machine, from the construction of storage techniques to the development of programming as such (Ceruzzi, 1998). The user, early on and later, had to establish a number of procedures going beyond anything hitherto seen.

Concrete computed function (e.g., the chaotic cablings of the ENIAC, the first general-purpose electronic, digital and reprogrammable computer from the mid-1940s) had to be creatively related to use by a heuristic force of fiction, residing within a figure of speach which might introduce, ‘a general function of what is possible in practice’ (Ricoeur, 1994: 127). Use, in other words, was not a piece of cake. Creative action upon the machine facing the user for a long time required something close to a complete reinvention of it. This was the first encounter of prosopopoietic modeling. It was not a haven or a promise, but a downright requirement. And if computing today may appear more ‘plug n’ play’ it is only because of decades of toiling and juggling, not only with specific forms of prosopopoeia, but with an excruciating construction of whole fields of interrelated prosopopoeia – what to the user appears seamless as ‘the web.’

The first appearance of prosopopoiesis was consonant with the first programming of the ENIAC and other early Moloks of pioneering computer design by an engineered intuition of how to e.g. combine cables in order to calculate. Despite, or exactly because of the eccentric expertise required for these operations, they immediately posited the issue of prosopopoietic modeling: of methods to enact chiastic aisthesis.

A famous early instance is John von Neumann’s draft of a principal logical design of a computer. He wrote this as the First Draft of a Report on the EDVAC, from 1945, after a few days of inspection of the of the ENIAC. In order to grasp the portent of this machine von Neumann applied a ‘figure of speech,’ by describing the ENIAC's technical organisation of computed function via an instrumentarium of logic. Von Neumann saw through the technical mass of relays and combinatorics, to a momentous and automated logic figuration – i.e., a future logic that was the precondition for making explicit the universality of the Turing machine.

This was a radical rethink of the ENIAC-inventors' (Mauchley and Eckert) accomplishment, in terms of what is still known as the ‘architecture’ of the serial computer. The mechanical workings inside the ‘black box’ of Mauchley and Eckert’s ENIAC – the ‘inanimate object,’ i.e., the concrete mechanically instantiated calculus of the Turing machine, were by von Neumann’s logic tables opened towards a methodological coupling of Mauchley and Eckert’s mechanical engineering with logic in the sense of creative action upon these mechanics. To paraphrase Paul Ricoeur, mechanical workings were opened to a mode ‘in which we radically rethink what family, consumption, government, religion and so on are’ (Ricoeur: 132). Logic was thus applied as creative action, not only upon the specificity of the ENIAC in 1945 but upon the universal Turing machine, via a chiastic aesthesis which offered the specialised wartime artifact of the ENIAC the broader promise indicated by Turing. Prosopopoiesis may thus be taken as another term for a creative heuristic of the universal Turing machine, attempting to tackle methodologically the paradox of distinction/complicity that any user would encounter.

From the very beginning of the computational heritage – for example as summarised in the emphasis on universality of the universal Turing machine, it was clear that the mechanical calculus was implying a new ratiocination. In this, human proof in the traditional sense of compelling human embedded logic was superceded by embedded technology. When it came to the Turing machinics – mathematics and algorithmics embedded in digtal relay-structures, the machine could in plain words calculate much more than any human and, importantly, in modes with a comprehension, scope and intrinsic purport which a human could only, at best, address intuitively.

That was the birth of mechanical approaches to ‘complexity’ which until then had been a vague metaphor (Merriam-Webster dates complexity to 1661, at the time of the origin of the modern ratio in a sense somehow of mechanical computing, c.f. ‘reckoning’ in Hobbes’s usage from the same time). It came into being, we may argue, because the instrumentarium of logic was methodologically applied as prosopopoietic modeling. To put it bluntly, the computer posited to the world from the beginning a paradoxical dinstinction/complicity between a mechanically embedded machinics much beyond any human grasp which was nevertheless deeply reliant on human creativity.

In fact the paradox was so compelling that the rendering operative of the Turing machine was only possible when creating it heuristically by way of prosopopoiesis. The Turing machine – as artifice, would only make sense if prosopopoietically modeled, whether in Turing’s own explanatory metaphor of the famous ‘tape’, or by Mauchley and Eckert’s concrete cablings, or by von Neumann’s logic tables in the First Draft of a Report on the EDVAC, or – consequently, in any other way created by humans.

This machine could only be entered into a general function of what is possible in practice for humans once such a prosopopoietical method was established. There was no way around it. Had this not happened, we may polemically ad, the whole issue of mechanical computation could have remained an eccentric side effect of the American wartime economy. Indeed all the more or less eccentric historical precedents had turned out to be exactly this – Pascal, Babbage, Lovelace etc., up to the advent of Turing, Shannon, and von Neumann.

The success of computing would raise the issue of from where and how to create: of how to procede, grasp, deal, to imagine creatively, the artifact of computed function ‘first’ invented by Mauchley and Eckert’s mechanics. And further, this prosopopoiesis would remain challenging for the coming decades, in abject contrast to any naive idea of technique as ‘ready to hand’ (Heidegger would revise his assumptions in the postwar essay on technology).

In the first years of computing this task was left in the concrete to the cybernetic pioneers – often in quite practical ways, when fumbling with chaotic cablings such as the ENIAC’s in the mid-40s, and later in the 1950s and 1960s extending this to a new expert culture of cybernetics and systems theory.

By then the prosopopoiesis relating to the creative actions of experts had for a long time been involved in wild speculations which would imagine in diverse and increasingly public ways the Turing machine calculus as a ‘general function of what is possible in practice.’ Animated machines, electronic brains, an entire world system trespassing the realm of God in the making; all of this as a ‘mode in which we radically rethink what family, consumption, government, religion and so on are.’ Not for nothing is the title of Norbert Wiener’s last book God & Golem, Inc. A Comment on Certain Points Where Cybernetics Impinges on Religion (Wiener, 1964).

Thus prosopopoietic modeling appears as a very first methodological precondition for making sense of an inanimate object, or better, for an apparant ‘anima’ of its Turing machine calculus (!). However, such deliberate fancy would enact the ‘planned rationalism’ of a technical organisation of computed function into a ‘situation’ of much broader human use in the coming decades, in the words of Lucy Suchman writing when user-centric design was clear in sight (Suchman, 1987).

A first description of the prosopopoiesis needed to methodologically situate the Turing machine calculus is thus also a first heuristic force of inscription by creative action upon computed function, no more, no less – from the outset a chiastic aisthesis, as indicated by Simondon. The very first inventions of specific ways of computing (from Konrad Zuse’s electromechanical machine Z3 and the other ‘firsts’ to the ENIAC) had from the very outset to be re-invented as ‘a general function of what is possible in practice’ (Ricoeur, Op.cit.: 127) by paradoxically rethinking what is taking place inside the complexity of Turing machine calculus by way of prosopopoietic modeling. The cybernetic pioneers struggled bravely to get everything right. Yet they also testified to the dynamic of prosopopoiesis in their creative production of prosopopoeia, of methods relying on the creative enaction of figures ‘of speech in which an animal or inanimate object is ascribed human characteristics or is spoken of in anthropomorphic language’ (Wikipedia). And over the coming decades all sorts of paradoxical issues which had little to do with computing senso strictu came into being, as seen in examples from Kubrick’s 2001: A Space Odyssey (1968) to William Gibon’s Neuromancer (1984) (Bukatman, 1993). See the graphic below for a summary of the argument:

Prosopopoietic Modeling – Simon’s Simulation

At this point we may argue that prosopopoietical modeling has several features:

a. It enables a chiastic aesthesis, emerging in-between two distinct yet intertwined ontological forms of organisation, the Simondonian pure subject and object (see the part to the right in the graphics above).

b. It moves the emphasis of use away from the technical organisation of computed function in whatever complex form, and toward a heuristic force of inscription which makes relations possible by methods derived through prosopopoietic modeling.

c. It makes possible a new approach to understanding use in this regard, in the computational heritage, by moving towards creative action upon computed function and a re-evaluation of the ontological form of the human Being.

A further step of my argument may be presented by taking up Herbert Simon’s discussion of ‘simulation at the front end of use’ in the programmatic treatise The Sciences of the Artificial (Simon, 1996 [1969]). Here, it may be argued, heuristic inscription is tackled via the announcement of methods of simulation. Simon argues that the computer makes up a particular form of complexity which cannot and may not be put wholly at the disposal of the user. What is needed is a ‘meeting point – an "interface" in today’s terms – between an "inner" environment, the substance and organization of the artifact itself, and an "outer" environment, the surroundings in which it operates’ (Simon 1996 [1969]: 6).

To Simon the intrinsic complexity of the inner environment of the artifact (i.e., the resources of the universal Turing machine) need not be wholly disclosed. Appropriate measures of simulation at the front end directed towards the user's attention may be enough. This argument is in strict accordance with the deterministic positions of the computational heritage: the technical organisation of computed function is simply too comprehensive for any form of prosopopoiesis beyond simple simulatory presentations. That is, beyond what would be known from the 1960s as GUI, or ‘graphic user interface,’ in combination with other interactive devices such as mouse, keyboard etc, based on Ivan Sutherland, Douglas Englebart’s and others experiments throughout the 1960s. In the late 80s and 90s this would evolve into the comprehensive design issues of usability, affordance and context, further user-centricity etc.

Simon’s solution thus appeared to fix prosopopoietic modeling through strict methods for displays of the technical orgnisation of computed function. At the time of his writing, in 1969, such methods were well under way to become commonplace, exemplified by new norms of e.g. usability. What would be less clear to Simon and the computational heritage was that this simulation was a very restricted form of chiastic aisthesis prosopopoietically subsumed under displays of the technical organisation of computed function. It would result in a bland and ironic substitute for the complexity of the Turing machine calculus, parading over decades as the desktop. It was of course still a creative action upon the technical organisation of computed function, but it left the intrinsic complexity at the mercy of a prime metaphor from 19th century bureacracy, the desktop – or, in different words, linear print as a frame for enacting complexity.

While affirming the basic traits of prosopopoietic modeling, simulation nevertheless became a veil to cover the paradoxical distinction/complicity, merely bearing witness to the fact that the technical organisation of computed function would far supersede the human capacity for ratiocination, leaving it at the front end as the infamous ‘dumb’ end user. But it was prosopopoietic modeling nevertheless. Simon’s book can be read as a methodological manifesto which affirms the prescient assumption of chiastic aisthesis by Simondon, if by disregard.

Prosopopoietic Modeling – Verum et Factum Convertuntur

The reduction of prosopopoietical modeling to a graphic simulation of a desktop would be quite convincing to prime users of computed function throughout the 80s and 90s. Nevertheless, even in this it demonstrates the paradoxical distinction/complicity between the technical organisation of computed function and the human Being in the sense of creative action upon such function in fully fledged operation. Today, however, this modeling appears as an increasingly daft entry to fast internet access etc. Simulation will hardly fit the context of radical heuristic inscription and the surging variety of prosopopoeia which have emerged throughout the 1990s and 2000s.

By 2011 the cell phone, wireless and devices such as iPods, smart phones etc. have long made widespread prosopopoietical modeling possible, as we see in such phenomena such as web 2.0 and social software. It becomes apparent that the technical organisation of computed function today is working only when allowing for extended creative action.

In his book on the origins of cognitive science Jean-Pierre Dupuy considers indirectly the importance of the new perspectives of modeling which appeared with the universal Turing machine calculus. Dupuy places it in a more reflected relation to the issue of prosopopoietic modeling. (Dupuy, 2000)

Dupuy emphasises the importance of complex ‘modeling’ discovered by the computational heritage with a profound reference to Giambattista Vico’s dictum of ‘Verum et factum convertuntur.’ This means that humans can ‘have rational knowledge only about that of which we are the cause, about what we have ourselves produced’ (27-28):

A model is an abstract form … that is embodied or instantiated by phenomena. Very different domains of phenomenal reality … can be represented by identical models, which establish an equivalence relation among them. A model is the corresponding equivalence class. It therefore enjoys a transcendent position, not unlike that of a platonic Idea of which reality is only a pale imitation. But the scientific model is man-made. It is at this juncture that the hierarchical relation between the imitator and the imitated comes to be inverted. Although the scientific model is a human imitation of nature, the scientist is inclined to regard it as a ‘model,’ in the ordinary sense, of nature. Thus nature is taken to imitate the very model by which man tries to imitate it. (29-30)

The creation of a complex model is paradoxically both a product and a transcendence of human finitude. The model abstracts from phenomenal reality ‘the system of functional relations,’ putting aside everything else, and in its mechanical sphere of calculus, by computed function, such a model comes to obtain a life of its own, ‘ (…) an autonomous dynamic independent of phenomenal reality’ (31).

Now, according to Dupuy, this principle of Verum et factum gains a particular emphasis from the 1930s onwards. With Alan Turing and Alonso Church’s alignment of computation and mechanics a new significance of the artificial is conjectured, the issue of ‘effective computability:’

It seems plain to us now that the notion of effective computability that was being sought [in the 30s], involving only blind, ‘automatic’ execution procedures, was most clearly illustrated by the functioning of a machine. It is due to Turing that this mechanical metaphor was taken seriously. In his remarkable study of the development of the theory of automata, Jean Mosconi makes an interesting conjecture about the nature of the resistance that this idea met with in the 1930s: ‘Considering the calculating machines that existed at the time – the perspectives opened up by Babbage having been forgotten in the meantime -, any reference to the computational possibilities of a machine was apt to be regarded as arbitrarily narrowing the idea of computability … If for us the natural meaning of "mechanical computability" is "computability by a machine," it seems likely that until Turing came along "mechanical" was used in a rather metaphorical sense and meant nothing more than "servile" (indeed the term "mechanical" is still used in this sense today to describe the execution of an algorithm).' (35)

Thus the ideas of Turing and Church not only expand on the notion of computed function, they also further the notion of what sort of technical organisation computed function can be seen to be. This is the road which takes the universal Turing machine into forms of modelled complexity, and further to a paradigm of ‘complexification’ (Casti) – highlighting the paradoxical distinction/complicity. Verum et factum embodies an acceptance of an integral need for prosopopoeia. It also seems to indicate that complexity modeling is only an underlying aspect of prosopopoietical modeling, of a generalised, methodological, and in fact little reflected creation of figures, ‘of speech in which an animal or inanimate object is ascribed human characteristics or is spoken of in anthropomorphic language.’

The machine may create something, not in its capacity for incorporated mathematics or in its capacity of calculating mechanics (Turing machine calculus in various forms), but in its capacity of effective computation, thus foregrounding ‘effect’ in terms of models which also reside chiastically with the human. Not only the machine but also the human will henceforth have all the options of computation as ‘an autonomous dynamic independent of phenomenal reality’ – i.e., as an element of a chiastic aisthesis. For all its capacity for effective computation, any technical organisation of computed function is only meaningful in the methods which will effectuate its Turing universality as ‘Verum et factum’: ontologically transcending and ontological relying on creative action upon computed function.

In Conclusion – Weiser’s Notion of Pervasive Computing Today

Reading through my notes above, I think Mark Weiser’s pursuit of pervasive computing takes on a specific quality. Texts such as ‘The portable Common Runtime Approach to Interoperability’ (1989), ‘The Computer for the 21st Century’ (1991), ‘Libraries are more than information: Situational Aspects of Electronic Libraries’ (1993a), ‘Some Computer Science Issues in Ubiquitous Computing’ (1993b), ‘The Last link: The Technologist’s responsibilities and social change’ (1995), and ‘Designing Calm Technology’ (1995) read like an engineer struggling to expand the technical organisation of computed function into something else. Indeed Weiser himself points to the challenge ‘to create a new kind of relationship of people to computers, one in which the computer would have to take the lead in becoming vastly better at getting out of the way so people could just go about their lives’ (Weiser, 1993b: 1).

For one thing, this captures the fact that they expose the paradoxical distinction/complicity between the technical organisation of computed function and the human Being in the sense of creative action upon such function. The machines must be designed so that creative action is rendered possible. Or differently: there is no design, no machine without creative action. This is a huge step away from Simon’s simulatory strategies, once in accordance with Dupuy’s argument for ‘verum et factum’, as discussed above. In order to transcend the finitude of the technical organisation of computed function, the technical organisation must pervasively transcend the realm of human finitude as ever-present, yet absent – made operative by prosopopoietical modeling.

Second, however, this move away from the determinism of the computational heritage is only a partial step. It also installs a benevolent super-determinism, which in the period after the mid-90s appears increasingly outdated. The constant development of prosopopoietical modeling restates the paradox in its own way. While the engineering aspects are focused on quite restricted locales – the artificial definition of lap tops and cell phones etc., we see billions of prosopopoiea which simply appear as a blunt cultural instituting of new media.

What we see in all this, I believe, is a ‘cultural turn’ of new media, one which by now is generally acknowledged, and which has turned the engineering of Weiser, not to mention the simulatory techniques of GUI into quite useable but minor issues of computed function today. In this respect Weiser appears as a highly sympathetic engineer addressing us from a finished history: or better a history which from the beginning was misconceived, since all technological development resides with the paradoxical distinction/complicity. The world now indicated is a social world of new media, the perspetive of which we have just begun to see. But it is a world that increasingly exploits the paradoxical distinction/complicity that for so long was a problem for the computational heritage, and which continues to highlight prosopopoietical modelling.

I will close by advocating an increased interest in such, e.g., in regard to new generations of new media for the developing world, where a cultural paradigm may make wonders of use (Michelsen, forthcoming). That is, there is a much keener focus on the options appearing when starting out from creative action upon computed function, not, vice versa, in whatever form and capacity. If the Turing machine in the mid-20th century was still a real enigma to be translated by heavy expertise, real enigmas today may reside much more with the latter part of the distinction/complicity: that is, with our enigmatic and human creativity.

Biographical Note

Anders Michelsen is Associate Professor, Ph.D., at the Department of Arts and Cultural Studies, University of Copenhagen (2500 students/ 40+ scientific employees). From 2002-2009 he coordinated the first master program in Visual Culture Studies in Denmark. From 2007– 2011 he was Head of Studies at the Department of Arts and Cultural Studies. Michelsen’s research topics and interests lie within the interdisciplinary cultural and social research field emerging from, a. contemporary culture and art in processes of globalization, including visual culture b. imagination, creativity, and forms of innovation, c. design and technology, especially as related to social roles of culture and intercultural communication, d. new media, information and communication technology '4' development (UN headline: ICT4D). He is co-chairing the Danish-Somaliland NGO PeaceWare-Somaliland which is projecting the first tele psychiatric system in Africa, in the Republic of Somaliland: http://peaceware-somaliland.com. His Ph.D. thesis (2005) explores the ‘Imaginary of the artificial: Confronting the imaginary of the artificial: from cyberspace to the Internet, and from the Internet to us. He is presently writing book on ‘visual culture and the imaginary’ – on creative imagination and innovative imaginaries in collective and subjective forms. Michelsen is author/co-author of 2 books and 2 publications, he has co-edited 8 anthologies and journal issues (most recently Thesis Eleven 88.1 – http://the.sagepub.com/content/vol88 /issue1/). He is currently co-editing a three volume collection with contributors from Europe, US, Latin America, The Middle East, Asia and Australia, on the notion of visuality and visual culture, Transvisuality. Dimensioning the visual in a visual culture. Liverpool: Liverpool University Press 2012-13. He has authored 67 (71) papers published in Denmark and internationally, in Australia, Brazil, Denmark, Germany, Italy, Norway, Spain, Sweden, and the U.S..

References

• Abbate, Janet. Inventing the Internet (Cambridge, Mass.: MIT Press, 1999).
• Barabási, Alberto-László. Linked. How Everything Is Connected to Everything Else and What It Means for Business, Science and Everyday Life (London: Plume, 2003).
• Bukatman, Scott. Terminal Identity. The Virtual Subject in Postmodern Science Fiction (Duke University Press, 1993).
• Casti, John L..Complexification: Explaining a Paradoxical World through the Science of Surprise (New York: HarperPerennial, 1994).
• Casti, John L. ‘Complexity’ (Encyclopedia Britannica [database online] http://search.eb.com/eb/article?eu=108252, 2002).
• Castoriadis, Cornelius. Domaines de l’homme. Les carrefours du labyrinthe II (Paris: Seuil, 1986).
• Castoriadis, Cornelius. The Imaginary Institution of Society, trans. Kathleen Blamey (Cambridge, UK: Polity Press, 1987).
• Castoriadis, Cornelius. World in Fragments: Writings on Politics, Society, Psychoanalysis, and the Imagination (Stanford: Stanford University Press, 1997).
• Ceruzzi, Paul. A History of Modern Computing (Cambridge Mass.: MIT Press 1998).
• Dupuy Jean-Pierre. The Mechanization of the Mind. On the Origins of Cognitive Science (Princeton: Princeton University Press, 2000).
• Eve, Raymond A., Horsfall, Sara and Lee, Mary E. (Eds). Chaos, Complexity and Sociology. Myths, Models, and Theories (Thousand Oaks: SAGE Publications, 1997).
• Hayles, N. Katherine. How We Became Posthuman. Virtual Bodies in Cybernetics, Literature, and Informatics (Chicago: The Chicago University Press, 1999).
• Joerges, Bernward. Technik. Körper der Gesellschaft. Arbeiten zu Techniksoziologie (Frankfurt am Main: Suhrkamp Verlag, 1996).
• Mainzer, Klaus. Thinking in Complexity. The Complex Dynamics of Matter, Mind, and Mankind (Berlin: Springer Verlag, 1994).
• Manzini, Ezio. Artefacts. Vers une nouvelle écologie de l'environnement artificiel (Paris: Les Essais, Centre Georges Pompidou, 1991).
• Merleau-Ponty, Maurice. The Phenomenology of Perception. (London 1986 [1962]).
• —. The Invisible and the Invisible. Followed by Working Notes, Ed. Claude Lefort. (Evanston: Princeton University Press, 1983 [1968]).
• Michelsen, Anders. ‘Medicoscapes: on mobile ubiquity effects and ICT4D – The
• Somaliland Telemedical System for Psychiatry’ Digital Creativity (Forthcoming)
• —. ‘Autotranscendence and Creative Organization: On self-creation and Self-organization’ ‘Autopoiesis: Autology, Autotranscendence and Autonomy’ Thesis Eleven. Critical Theory and Historical Sociology 88 (2007): 55-75.
• —. ‘The Imaginary of the Artificial: Automata, models, machinics. Remarks on promiscuous modeling as precondition for poststructuralist ontology’ New Media, Ole Media. Interrogating the Digital Revolution (New York: Routledge, 2005), 233-247.
• Nielsen, Jakob. Usability Engineering (Boston: Academic Press, 1993).
• Norman, Donald A. The Design of Everyday Things (London: Mit Press 2001 [1998]).
• Quine, Willard Van Orman. The Ways of Paradox and Other Essays (Cambridge, Mass.: Harvard University Press, 1976).
• Ricoeur, Paul. ‘Imagination in Discourse and in Action’ Rethinking Imagination. Culture and Creativity (London and New York: Routledge, 1994), 118-135.
• Varela, Francisco J., Thompson, Evan, and Rosch, Eleanor. The Embodied Mind, Cognitive Science and Human Experience (Massachusetts: MIT Press, 1993).
• Simon, Herbert. The Sciences of the Artificial (Cambridge Mass.: MIT Press, 1996 [1969]).
• Simondon, Gilbert. Du mode d'existence des objets technique (Editions Aubier, 1989 [1958]).
• Star, Susan Leigh (ed.) The Cultures of Computing (Oxford: Blackwell Publishers/The Sociological Review, 1995).
• Suchman, Lucy. Plans and Situated Actions: The problem of human-machine communication (Cambridge: Cambridge University Press, 1987).
• UPA – Usability Professionals Association http://www.usabilityprofessionals.org/usability_resources/about_usability/what_is_ucd.html
• Wiener, Norbert. Cybernetics: or Control and Communication in the Animal and the Machine (Cambridge, Mass.: MIT Press, 1991 [1948]).
• Wiener, Norbert. The Human Use of Human Beings. Cybernetics and Society (New York: Doubleday & Company Inc., 1954 [1950]).
• Watts, Duncan. Six Degrees. The Science of the Connected Age (New York: W.W. Norton & Company, 2003).
• Weiser, Mark. ‘The Computer for the 21st Century’ http://www.ubiq.com/hypertext/weiser/SciAmDraft3.html.
• Weiser, Mark and Reich ,Vicky. ‘Libraries Are More than Information: Situational Aspects of Electronic Libraries’ (1993a) http://www.ubiq.com/hypertext/weiser/SituationalAspectsofElectronicLibraries.html.
• Weiser, Mark. ‘Some Computer Science Issues in Ubiquitous Computing’ (1993b) http://www.ubiq.com/hypertext/weiser/UbiCACM.html.
• Weiser, Mark. ‘The Last link: The technologist’s responsibilities and social change’ Computer-Mediated Communication Magazine 2.4 (1995): 17 http://www.ibiblio.org/cmc/mag/1995/apr/last.html.
• Weiser, Mark and Seely Brown, John. ‘The Coming Age of Calm Technology’ Beyond Calculation. The Next Fifty Years of Computing (New York: Springer Verlag, 1997), 75-85.
• Weiser, Mark and Seely, Brown John. ‘Designing Calm Technology’ (1995) http://www.ubiq.com/hypertext/weiser/calmtech/calmtech.htm.
• Wikipedia: Prosopopoeia, http://en.wikipedia.org/wiki/Prosopopoeia
• Wikipedia: ENIAC, http://en.wikipedia.org/wiki/ENIAC.

The computer is moving out into physical and urban reality. Since Mark Weiser's call for a ‘computer for the 21st century’ in 1991 a migration from the screen and the desktop towards integrating computers and networks into our surroundings has been a part of contemporary computer science research; for example, in augmented reality, ubiquitous computing (ubicomp), and pervasive computing. A number of technological developments (such as big screens, new smart materials, GPS, RFID tags, and ever faster and cheaper wireless networks) have helped carry the research agendas out into ordinary reality.

This article will discuss how we experience the urban space of ubicomp. It will do so by introducing the concept of ‘scripted space’ in order to discuss how ubicomp is related to new developments in public urban space. Focusing on the experience of the urban we will argue that scripted space is a concept that highlights the written, coded quality of ubicomp. As opposed to that suggested by such titles as ‘The Disappearing Computer’ (Streitz, Kameas and Mavrommati, 2007), we believe that embedding the computer into the environment will not render it transparent or invisible. Rather, computing will continue to be an important part of our conscious experience. Furthermore, we will concentrate on the present rather than speculating on future potentials, technologies, and uses. As argued by Genevieve Bell and Paul Dourish, ubicomp has arrived; the future is here, though it looks different than originally envisioned by the IT designers and technologists (Bell and Dourish, 2007). It is especially remarkable that ubicomp is predominantly a ‘messy’ cultural interface and not, as envisioned, a ‘seamless’ tool for work. The many parallel technologies, networks, gadgets, and software developments have become part of ‘cultural scripts of many sorts’ – for example, via the cultural developments surrounding mobile phones, social software, and web 2.0 (Bell and Dourish, 2007: 141).

Scott McQuire discusses whether this development has the potential to change public space and reverse its demise. In continuation of the work of Paul Virilio, Richard Sennett, and Jürgen Habermas he argues that public space has lost to the public image consumed in private at home, and that the urban has lost to the suburban. McQuire quotes Philip Kasinitz, writing that suburban spaces ‘are notably rich in private spaces and poor in public ones’ (McQuire, 2006: p. n.p.). But this might be changing: ‘With the emergence of mobile devices, media consumption is increasingly occurring in public. It could be argued that the 'media event' is in the process of returning to the public domain’ (McQuire, 2006: p. n.p.).

Drawing on a workshop study of digital layers in the midsized Swedish city of Lund, the article will consider how this might happen. It will conclude by discussing the potential of scripted space as a new public domain by mapping out a matrix and suggesting writerly scripted spaces as a way of furthering this development.

Scripted Space

The concept of scripted space has been suggested by Norman M. Klein (Klein, 2004) but he does not elaborate on the concept in relation to ubicomp. Etymologically, the term ‘script’ refers to something written. It is mostly understood in a literary sense as either a ‘manuscript’, meaning the handwritten original, or a set of instructions and dialogue for a performance. In IT and software, drawing upon these connotations, scripting refers to the user’s own writing of instructions for the computer. In an operating system, experienced users may add scripts that overwrite certain files, for instance, or mount net volumes. These scripts can later be activated automatically by clicking on an icon. In authoring software like Adobe Flash, users may add ‘action scripts’ which determine dynamic behaviours of objects in an animated drawing (e.g., in a game where the score is increased whenever a ball hits a hole). In the written set of instructions, scripts have a semiotic attribute. The signs, however, are rarely read but are experienced as a performative attribute. The meaning of scripts is established in the performance of the instruction and not in the coded, incomprehensible instruction itself.

Our understanding of scripted space is based on the fact that ubicomp literally means adding scripts to the space surrounding us by programming smart things, architecture, infrastructure, and PDAs, and by connecting the scripts of these devices through wireless networks. Ubicomp adds simultaneously a semiotic and a performative attribute to the urban space. Recognising ubicomp as a scripted space highlights how digital scripts are added to the coupling of space and signs already carried out through urban planning, architecture, advertising, street signs, and so forth. As such, the scripted space of ubicomp does not result in an entirely new configuration of urban space, but is seen as continual development of the urban experience, which gets intensified by the adding of scripted environments and objects. The scripted space is in other words related to earlier developments of the spectacular modern city (McQuire, 2006; Pold, 2004) and denotes an urban experience scripted through architecture, visual media, signs, and semiotics (figure 1). 

Figure 1: Scripted Space

Besides being a continuation of the ways urban space has been planned, scripted space also has a more contemporary dimension which is linked to the computer. Scripted space has a non-visual, coded, encrypted side to it and suggests that there are computed transactions and control structures behind the facade, surface, or interface of the city.

This aspect has also been described by Bruno Latour and Emilie Hermant. They describe and photograph the development of Paris from ‘la ville lumière’ to ‘la ville invisible’ – from the illuminated, spectacular city that we still visit and cherish to an invisible city with an informational layer folded into its spectacular surface (Latour, Hermant and Shanon, 1998). The invisible city is a city where computers and information workers are constantly working to describe and script the city; Latour and Hermant document processes where the elements of the city (cafés, monuments, streets, etc.) are constantly under surveillance and correction, described and prescribed by networks of computers and information workers. What becomes clear from their detailed description of Paris (published before the current surge of ubicomp) is that the city is constantly being produced (re-engineered and mediated) by computers in complex networks and that it takes considerable amounts of work and material infrastructure as well as large numbers of highly skilled workers to maintain this. 

Subsequently, the scripted space as the info-layer of the city is a complex and networked software structure where algorithmic scripts organise and control the city and our experience of it. As expressed by Nigel Thrift, the computer is ‘extending its fugitive presence through object frames as diverse as cables, formulae, wireless signals, screens, software, artificial fibres and so on’ (Thrift, 2004: 584). 

Following Scott McQuire, it is this mediation and scripting that holds the potential return of an urban public space. When the ‘media event’ returns to the urban public domain via ubicomp one should not merely consider it as disappearing into the city, invisibly and transparently embedded in the experience of the urban. Rather, the coded, scripted, or essentially written quality of ubicomp should be highlighted. Of course the scripts of ubicomp are hidden behind screens, devices, and interactions. In this way, they are not as accessible or readable as the texts in books or on signs. They are (as scripts or code in general) first and foremost experienced in their execution. The scripts of ubicomp add new layers to the already multilayered urban scripted space – which is already more or less impossible to read in its entirety, but is nevertheless experienced through urban daily life. But how is the urban scripted space experienced? And how does it relate to a public domain?

The Experience of Scripted Spaces

Obviously, a scripted space is experienced in many different ways, but an important aspect is the feeling that something is going on behind the facade and that there are powers controlling and structuring what is seen and experienced. The immediate experience of the urban is disturbed by the feeling that there is something unreadable, but still scripted, programming the space. 

In the novel The Crying of Lot 49, Thomas Pynchon describes the narrator's panoramic view of a southern Californian sprawling suburb in ways that illustrate this experience quite well. The narrator watches a dull suburb, where ‘nothing was happening,’ and she thinks of ‘the time she'd opened a transistor radio to replace a battery and seen her first printed circuit. The ordered swirl of houses and streets, from this high angle, sprang at her now with the same unexpected, astonishing clarity as the circuit card had.’ Since the novel was first published in 1966, Pynchon uses a radio circuit board, rather than a computer, as a metaphor for the hidden information structure. Elsewhere in the novel, though, computers are remarkably present, and the novel revolves around hidden information structures, networks and almost paranoid investigations of these. The urban space is hardly identifiable, but simultaneously it seems to communicate a hidden plot – a concealed information infrastructure. The space is not remarkable and cannot be read in spite of its ‘intent to communicate’ and ‘hieroglyphic sense of concealed meaning’: ‘There'd seemed no limit to what the printed circuit could have told her (if she had tried to find out); so in her first minute of San Narciso, a revelation also trembled just past the threshold of her understanding’ (Pynchon, 1979: 14-15).

This ‘hieroglyphic sense of concealed meaning’ is an essential part of the experience of the scripted space, and the novel describes a very early example of an informational urbanity which is experienced not as readable signs, but as ‘signs of signs,’ as hieroglyphs that the urban dweller knows are readable for those with the right code but that cannot be accessed by outsiders. One feels that there is something going on beyond the surface, but it is difficult to see or read the scripts. Rather than becoming ‘natural’, the scripted space maintains a highly mediated character, even though the urban dweller does not necessarily see through it. A similar view is found in Norman Klein's description of the experience of scripted space in places such as Las Vegas, Disneyland, and the entertainment architecture of The Jon Jerde Partnership (e.g., malls like San Diego's Horton Plaza and urban areas like the Universal City Walk in Los Angeles). Spaces where ‘each square foot must pay off’ and that bring on an effect of ‘happy imprisonment’ and ‘ergonomic control’ (Klein, 2004: 332). Again, the urban dweller feels the control and the imprisonment – she knows that she is being manipulated but still enters freely and enjoys the user-friendly ergonomics.

The urban dweller’s experience of concealed meaning, happy imprisonment or ergonomic control is comparable to other experiences of scripted spaces. Computer games seem to offer an important contemporary exploration of this experience of a scripted space. The player can interact with scripts in different ways. On a functional level, a script can be activated logging in to a computer game, setting control menus, etc. However, the player’s most dominant experience is of course the aesthetic experience. On an aesthetic level the player activates scripts to play with them. As suggested by game theorist Ted Friedman the ‘way computer games teach structures of thought – the way they reorganise perception – is by getting you to internalise the logic of the program’ (Friedman, 1999: p. n.p.). In this internalisation the computer’s scripts simultaneously work in two ways. At once, they are they are an ally helping the player in controlling the environment and an obstacle to be overcome, an opponent. The player must collaborate with the computer in order to find a way to overcome it. A game experience on an aesthetic level is essentially about playing with and against the scripts as long as it remains a compelling activity.

A third option is however also possible in the computer game. Friedman’s description of an internalisation of the computer’s scripts has a distinct political level. Winning a game can only happen by accepting the terms of the game. In Friedman’s case, the game Civilization II (1996), the player can only win by assuming the role of a colonising nation state; art and religion serve a purely functional role: to keep the people pacified and so forth. The game thus addresses controversial historical and ideological assumptions. To Friedman, the player must submit to this political level but experience from other games suggests different strategies. Working as a designer for Maxis on their game Sim Copter (released the same year as Civilization II), the artist and founding member of The Yes Men, Jacques Servin (a.k.a. Andy Bichlbaum) illegally incorporated an easter egg into the game. Triggered by a cheat code or on certain dates, Servin’s secret script will flush the game world with men in trunks kissing both each other and the pilots in the game. Servin’s easter egg indicates a political response to the ideology of the scripted space. Here, the scripted space is subject to debate and will be reconfigured, used in unintentional ways or even hacked.

How do these functional, aesthetic and political levels and experiences of the urban scripted space manifest themselves? We took part in a workshop exploring such issues in the mid-sized Swedish city of Lund. [1]

Scripted Spaces in Lund

The forefront of scripted spaces might be more or less 'virtual' spaces, such as computer game spaces, which are heavily commercialised and 'themed', or it might be post-urban spaces, such as Las Vegas and Los Angeles, or high-tech spaces, such as Singapore or Tokyo. But as demonstrated by Latour and Hermant, this mediation of space also spreads beyond the technological 'sci-fi' spaces to affect the experience and quality of space in more ordinary cities (Latour, Hernant and Shannon, 1998). Ubicomp is increasingly becoming part of an everyday urban life – also outside the techno-sci-fi capitals, at least in the richer parts of the world, where the population is readily equipped with ubicomp technology and networks. Lund is an example of this.

In general, Lund consists of many layered traces of history, culture, commerce, personal and social narratives. We see ways of using and mis-using the urban space, and we see signs of transactions and interactions whose full meaning eludes us. We see signs of class and ethnic struggles, signs and sign systems that make people feel welcome or alienated, safe or threatened. Of course, our interpretation of and reaction to these signs depend on who we are and what we want – for example, whether we are residents or non-residents, our purpose, and our financial and cultural capital. Hence, a native to Lund who knows her way around will read the city scripts differently from a tourist. She will be able to read and interpret more of the scripts and normally also be less conscious of her readings and perhaps unconsciously overlook the stuff she is not able to interpret. However, this changes neither the basic hieroglyphic nor the everyday-like quality of the scripted space.

As in the computer game, we found behaviour that indicated the importance of staying connected to the scripted space, scriptory control, and the potential for renegotiating the scripts themselves. Focusing on the digital layers of Lund, we saw three different interrelated ways of using public space. First, IT is used to create what we will label a log-in space, often designed to keep unauthorised access out. The digital layers are primarily visible as gateways under surveillance so as to  restrict access. Only those able to locate the access points, who also know the passwords and have the keys to log in, can access the gated areas and hidden scripts. We find many traces of surveillance, encrypted and secured information structures, and keypads for logging in. Sometimes we find traces of information structures that appear enigmatic to us as unauthorised foreigners but that are still clearly communicating ‘a hieroglyphic sense of concealed meaning’, such as signs indicating hidden infrastructures, security alarms, and surveillance cameras. We find many physical structures guiding and welcoming us, combined with electronic layers sorting and disciplining the clientele – for example, the signs in alluring shop entrances indicating the credit cards accepted, in this way welcoming those with the right cards and excluding those without money (figure 2). Another example of a log-in space could be the list of available wireless networks on a laptop or PDA, which often require log-in passwords and authentication (figure 3).

Figure 2: Credit cards in shop entrance.

Figure 3: Wireless network log-in.

Second, IT is used to create what we will call a mobile personal computing space. We observed many laptops, media players, mobile phones, and other devices allowing people to carry their private data, soundscape, media- and communication-devices with them in public space (figure 4). Accessing, configuring, and inhabiting public space through personal interfaces thus compete with, for instance, newspaper reading, which previously defined public life in a café as a private behaviour in public space. Also, text messages setting up social encounters can be considered to play an important role in the way both non-residents and residents make their way around urban space. When we add to this websites advertised on stickers or posters, virtual maps, and other web 2.0 services, it becomes clear that the mobile personal computing space is not only about the isolated individual (figure 5). It is the individual’s interface to the public space. However, as implied by, for instance, Apple's successful range of ‘iProducts’ (iBook, iPod, iPhone), this interface is still privileging the individual and delivers personalised views and configurations of urban, social space. 

Figure 4: Mobile personal computing space: a lap top in a café.

Figure 5: Sticker on a pole advertising a website.

Third, IT is used for a hypertextual connectivity and linking of local nodes (people, places) to other nodes and the network. It is important to be well-connected, to be on-line, and to be accessible. In this fashion, most of the urban dwellers we passed in Lund probably had mobile phones in their pockets, as well as credit cards and other keys allowing them access to the scripted space. Cafés and shops have websites and email addresses, and some even provide wireless internet access, thereby offering the possibility to connect and get on-line (figure 6). Many urban dwellers probably also have their own networks of social links, favourite sites, and perhaps even exchange tags and preferences. In this way, an urban space and its dwellers are connected to each other, and to other spaces. Compared with earlier ways of congregating and inhabiting urban space, this social formation is thus often strangely invisible to outsiders in physical urban space.

Figure 6: Kiosk advertising internet access with an @.

The Scripted Space as the New Public Domain

The advent of digitally scripted objects in an urban setting can be seen as a movement of the public domain from the private home back into the urban setting. Log-in space, mobile personal computing space, and hypertextual connectivity each suggests a different approach to the scripted space as a new public space. Mapping out the observations from Lund, we find a matrix of different potential approaches to the scripted space (table 1).

In a home (or an office) setting, individual access to a scripted space takes place via the desktop and the internet – whose historical predecessor could be either the reading or the inscription of the personal notebook, manuscripts, letters, and the like. Today, the movement of this activity into an urban setting takes place via PDAs, laptops, phones, media players, and so on, all of which are made for individual use in an urban setting – creating a personal reading and inscription of space. Using an MP3-player, a laptop in a café, or hypertextual connectivity and linking via an iPhone; getting a personal schedule, information about a future meeting point in town, the opening hours of a store; receiving messages, and participating in an urban game – all of this contributes to creating an individualised, personal space within the urban setting. Though this kind of public behaviour could be seen as purely functional, it should also be seen as an individualisation of the urban setting where public space becomes a playing field for private activities – an ‘individual game space’. A historical predecessor to this kind of public participation is the café, where city dwellers read newspapers, are situated behind glass windows to see and be seen, or plan their city activities through immediate access to city information (through city magazines such as TimeOut, gallery and movie lists, etc.).

A public variant of access to the scripted space is nevertheless also evident. Quite often, the data accessed is not merely controlled and static but dynamic and open. In other words, hypertextual connectivity and linking is not just about accessing information but also about actively contributing information – as labelled in the web 2.0 paradigm (O'Reilly, 2005). Interfacing to the scripted space is not merely individual but also collective and public. A historical predecessor to these cultural forms could be a participatory newspaper culture, the writing of chronicles, letters to the editor, and so forth. This reflects an idea of public space as not just a setting for individual performative activities but also a space for debate and knowledge dissemination – a political level of scripted space equivalent to the idea of the agora.

What is striking about the observations from Lund is the emphasis on using digital technologies for personalised access and interfaces to the urban space and the public domain. Most often, digital technology in urban space is predominantly exclusive. This is of course the case with the creation of log-in space, but also to some extent with the mobile personal computing space. However, the hypertextual connectivity and linking seem to suggest a desire to connect to the social. Surprisingly, today, we usually see participation in weblogs, WIKI-editing, and so on as a participatory culture that does not involve the urban setting. Though WIKIs and weblogs may be accessed and read from an urban setting, there has been little effort to actively promote the urban setting as a context for active participation. Surely, examples can be found – such as user-generated content of urban screens and media facades, and video and textual tagging of the cityscape (e.g., as in WIKI maps) – but in the context of an average urban setting, they are in no way dominant. This of course signifies an area open to new forms of digital civic participation and will probably develop into new ways of creating social relations and potentially new ways of developing cultural interfaces in scripted spaces.

Conclusion: Writerly Scripted Space

The digital urban and scripted space is at once a functional, aesthetic and political space. It manifests itself not as a grand spectacle but most often as a space where one can log-in (or be left out), a space for mobile personal media consumption and local/global connectivity. Beyond being able to operate the urban scripts for functional use (logging in, configuring network settings, activating links, etc.) the ability to play and emancipate the digital scripts is crucial for the experience of the scripted space. The digital, scripted layer of the urban can be accessed to serve as the point of departure for individual experiences of the city as a personal, public playground (a borderless café) or, less frequently, the basis of social encounters, communication, and debate (a borderless agora).  This activity that defines participation in the urban public domain depends on the ability to access and internalise the scripts. It is, however, not the only way to participate in the public, scripted space of the city. 

The city dweller may implement, experience, and use urban scripts, but what happens when she is not satisfied with the foundation of participation in urban public life, the scripts themselves? This experience is a call for action where the city dwellers take control of the urban scripts and become ‘writerly’ by creating, hacking, and rewriting the city. 

’Writerly’ (or ‘le scriptible’) is the term Roland Barthes used to characterise the particular way of reading associated with the open texts of modern literature. Writerly texts address the reader as an active producer of meaning rather than merely a passive consumer reconstructing the text's 'proper' meaning through a way of reading predetermined by the author – as in the classical characterisation of ‘readerly’ texts (Barthes, 1970: 10). In certain interpretations, hypertext itself is equivalent to this liberation of the reader (e.g. Landow, 1992). It seems, however, more accurate to evoke a digital writerly, open text as a textual corpus where the reader is enabled to not just experience the effect of the script but also to affect the script (by e.g. programming). Following Barthes, the German media theorist Florian Cramer thus advocates a ‘writerly computing’ where the user is transformed from a consumer whose use is staged by the software interface to a producer who actively takes control of the scripts and has read and write access (Cramer, 2003: 100).

Open, writerly texts/computing is the basis of all experimental digital artistic practices. This is where the computer is not merely seen as a medium for messages but as open to interpretation, imagination, and further production by the user. Though often not a regular sight in average cities, such practices that critically explore the new public domain of scripted spaces are numerous and include the inversion of city surveillance (‘sousveillance’) or other ways of rewriting or 'hacking' the city (Greenfield and Shepard, 2007; Fuller and Haque, 2008). Turning users from consumers to producers of scripts may challenge existing hierarchies of meanings, roles, and experiences that differentiate acceptable from unacceptable practices in urban scripted space. This will also mean a return of an urban public domain and a new paradigm of computing remarkably different from the dominant ideas of the disappearing computer.

Acknowledgements

This research has been funded by the Danish Council for Strategic Research, 09-063245, (Digital Urban Living).

Thanks to Stacey Cozart Madsen for English proof reading.

Biographical Note

Christian Ulrik Andersen is Ph.D., Associate Professor and chair of Digital Aesthetics Research Centre at Aarhus University, Denmark. He researches within digital aesthetics, software cities and computer games. Together with Søren Pold he recently edited the book "Interface Criticism – aesthetics beyond the buttons" (2011). He is also a researcher at the Centre for Digital Urban Living, Aarhus University.

Søren Pold is Ph.D. and Associate Professor of digital aesthetics at Aarhus University, Denmark. He has published in Danish and English on digital and media aesthetics – from the 19. c. panorama to the interface – e.g. on electronic literature, net-art, software art, creative software, digital culture and digital urban living. Together with Christian Ulrik Andersen he edited the anthology "Interface Criticism – Aesthetics Beyond the Buttons" (2011).

Notes

[1] The workshop was part of the ACM conference NordiCHI 2008, a conference on Human-Computer Interaction, held on 19 October 2008. (http://www.nordichi2008.org/index.php?option=com_content&task=view&id=66&Itemid=90). The workshop, attended by IT researchers, was organised by the Digital Urban Living Research Center, Martin Brynskov, Peter Dalsgaard, Christian Dindler, Jonas Fritsch, Kim Halsskov, and Ole Sejer Iversen. All participants went to the centre of Lund and were asked to look for and take photos of three different layers in the urban space (social, digital and physical) and organise these with reference to three different dimensions (information, services, aesthetics) on a printed map of Lund. The participants were asked to focus on a transitional, mainly pedestrian space with shops, hotels, cafés and bars.

References

Barthes, Roland. S/Z (Paris: Éditions du Seuil. 1970).

Bell, Genevieve., and Dourish, Paul. 'Yesterday's tomorrows: notes on ubiquitous computing's dominant vision', Personal and Ubiquitous Computing, 11.2 (2007): 133-143.

Cramer, Florian. 'Exe.cut[up]able statements: The Insistence of Code', in Ars Electronica: Code – The Language of Our Time, eds. G. Stocker and C. Schöpf (Ostfildern-Ruite: Hatje Cantz, 2003), 98-103.

Friedman, Ted. 'Civilization and its discontents: Simulation, subjectivity, and space', in Discovering discs: Transforming space and genre on CD-ROM, ed. G. Smith,  (New York: New York University Press, 1999), 132-150

Fuller, Matthew., and Haque, Usman. Urban Versioning System 1.0 (Vol. 2). (New York: The Architectural League of New York, 2008).

Greenfield, Adam, and Shepard, Mark. Urban Computing and its Discontents (Vol. 1). (New York: The Architectural League of New York, 2007).

Klein, Norman M. The Vatican to Vegas (New York: New Press, 2004).

Landow, George P. Hypertext the convergence of contemporary critical theory and technology (Baltimore, Md: Johns Hopkins University Press, 1992).

Latour, Bruno, Hermant, Emilie and Shannon, Susanna. Paris ville invisible (Paris: La Découverte, 1998).

McQuire, Scott. 'The Politics of Public Space in the Media City', First Monday Special Issue #4 (2006), http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/1544/1459 

O'Reilly, Tim. 'What Is Web 2.0 – Design Patterns and Business Models for the Next Generation of Software', http://www.oreillynet.com/pub/a/oreilly/tim/news/2005/09/30/what-is-web-20.html

Pold, Søren. Ex libris medierealistisk litteratur, Paris, Los Angeles & cyberspace (Odense: Syddansk Universitetsforlag, 2004).

Pynchon, Thomas. The crying of Lot 49 (London: Pan Books, 1979).

Streitz, Norbert A., Kameas, Achilles and Mavrommati, Irene. The disappearing computer : interaction design, systems infrastructures and applications for smart environments (Berlin: Springer, 2007)

Thrift, Nigel 'Movement-space: The changing domain of thinking resulting from the development of new kinds of spatial awareness', Economy and Society, 33.4 (2004): 582-604.

Introduction

Ubiquitous computing positions a world where computation is embedded into our surrounding environment. Rather than retrieving information and communication from distinct devices (PCs) removed from contexts and activities, ubiquitous computing proposes that the mediated can become an integral part of our environment, seamlessly implanted, much in the way that text-based communications are incorporated now (Weiser, 1991). As such, ubiquitous computing places us in a world where material reality merges with computational reality, and where the built environment holds a strata for computation, a readiness towards the digital and the mediated.

This paper asks what the architectural consequences of ubiquitous computing can be. Architecture is concerned with the designing and making of spatial contexts. The central objective of architectural design practice is to define spatial ambitions and ally these with the given design criteria. In this way it holds a different point of departure to ubiquitous computing. The aim for ubiquitous computing is to support the incorporation of interactive experiences in ways that can, for example, extend and support the existing framework of a work situation or the home. Where ubiquitous computing seeks models for embedding the digital in a meaningful and seamless manner, it also privileges the idea of the everyday, of the environment and surroundings of its users as fundamentally antecedent to the digital tool. Reconciling the cultural and technological ambitions of ubiquitous computing with the practice of architecture therefore brings forth new challenges for the development of concepts and techniques by which digital events can become “embedded”, that is, can become part of the conception of architecture rather than an afterthought.

This article reflects on the two interactive installations Breathing Room and Slow Furl, following a practice-based research method. The paper explores the issues involved from two perspectives. Firstly, it queries how embedding ubiquitous computing and the idea of responsiveness into architectural practice can lead to new models of interactivity. Referencing the concepts of self production and recursiveness in robotics and early cybernetics, it seeks not to fix the user in a situation of apparent control but rather to position the environment as a whole within a continual condition of self invigoration. We propose the term pulse as a measure of temporal extension, in order to think temporality and movement as part of an architectural language of design.

The second perspective concerns the inherent challenge to the material practices of architecture that result from working with responsiveness. Architecture has traditionally been cast within the image of the permanent and the static. Embedding the ability to sense and act is therefore one of several recent events in architecture that questions its tectonic conceptualisation. By proposing instability as a concept for material thinking this article references the emerging field of transitive materials. Materials are thought performatively both in respect to their structural and their computational facility. 

The idea of ‘embeddedness’ poses questions not only in respect to the use of and behaviour within a ubiquitous computing space or to the infrastructural necessities that this might require, but also, more fundamentally, to what it means for computation to become an integral part of the built environment. The article asks the following questions. What happens to the relationship between the spatial conceptualisation and occupation of an environment when computation becomes an integral part of our built environment? How does a consideration of architecture as interactive challenge traditions of spatial conceptualisation and lead to a rethinking of core architectural concerns such as stability and structure? What happens to our material practices as the ability to sense and actuate become core properties of an ‘architecture of change’? 

Breathing Room and Slow Furl: Textile architectures of actuation and response 

The ambition for the two architectural installations Slow Furl and Breathing Room is to imagine a textile architecture that acts upon and reacts to changes in their environments. Conceived as robotic installations at an architectural scale the installations are speculative probes querying the territory of robotics, architecture and ubiquitous computing, while suggesting new spatial concepts for a soft space.

Figure 1: Slow Furl, textile view.

The installations use textiles as a material for construction. A primary interest in textiles within these projects is their inherent pliability. As compressive materials, textiles allow for easy state shifts such as folding, pleating or stretching. But textiles are also interesting from a manufacturing point of view. Textile technologies are also methods of material production: knitting, weaving, lacing, or crocheting are structural techniques by which single fibres are brought together to make fabrics, generating their particular properties of pliability and flex. Controlling these techniques as well as the specific properties of the fibres allows the specification of the material performance. Understanding textiles as a technology as well as a material, the installations propose new ways of embedding the computational into the built environment. 

In Breathing Room and Slow Furl the textile is understood as a matrix for embedding computation. By integrating conductive fibres into the textile membranes the architectural skin itself becomes part of the electronic circuit of the robotic environment. The installations use this externalised circuitry to create a strata of soft switches (Berzowska, 2004) by which the state of the fabric, its position and folds, can be registered. As the installations move, the conductive paths within the material change, thereby creating shifts within the electronic state. These state shifts act as a sensing matrix of material interaction.

The membranes clad dynamic substructures, or armatures, that hold their particular motility, informed by the materiality and structure of their construction. The armatures are actuated using simple motors steered in turn by an array of micro controllers. The micro controllers use the soft switches as input steering the actuation of their movement cycles. In Breathing Room and Slow Furl computation exists as a distributed system. In both installations the armatures are steered using multiple autonomous micro controllers connected to each their part of the strata of soft switches. The movement of the installations is therefore instigated as a set of independent actions and reactions in turn unified by the continuity of the textile membrane. In this way the building skin itself becomes part of the computation, effectuating the emergence of the complete movement score. 

Figure 2: Breathing Room, textile view

The key constituent of the two installations lies with their programming. In both installations interaction happens through material manipulation and state change. This material call and response is the basis for the programming of their responsive behaviours. The code is constructed as finite loops prompting the movement of the motors and armature. As the membranes fold and unfold they instigate changes within the circuitry thereby continually calling forth further movement cycles. Each movement loop is potentially the last. It is only through the material interaction, the actual self-touching of the membranes, that a continuity of movement is established. This continuity, the pulsing of switching and unswitching, creates the emergence of a behavioural patterning defining the architectural quality and presence. 

It is into this cyclical looping that the user enters. In Breathing Room and Slow Furl interaction is physical. Those engaging with the works are encouraged to manipulate the material through touch and movement. As they manipulate the physical arrangement of the membranes, shifting their positions and creating change in the externalised circuitry they affect and change the production of new movement cycles. The inhabitants and the environments become part of the same material interaction, both affecting and changing the movement scores of the structure. The presence of the user is conceived as an intensification, or, conversely, a relaxing, of the continual movement of the installation. 

Figure 3: Slow Furl, textile view (2).

Self production and the auto-poetic system

The installations explore the idea of self-production. Rather than considering interaction as something the user instigates and controls, interactivity is understood as the means by which the installations iteratively self-produce their movement cycles and, through this, their behaviour. It is in this way that the installations propose new ways of understanding the relationship between user and environment, occupant and space. 

In architecture, the thinking of ubiquitous computing has become synonymous with building automation projects such as the Intelligent Building or the Smart House. As such, the embedding of sensors enables the steering of the complex infrastructures of a building, such as heating, ventilation, and air-conditioning. It is now commonplace that buildings switch their heating or lighting on and off in response to its usage, and that more complex systems such as air-conditioning are steered by central computing units (Wiggington and Harris, 2002; Compagno, 2002). Buildings therefore become interfaces that continually control the complex layering of exchanges between outside environmental conditions and the perceived needs of its user. To use the favoured analogy of the cyberneticians, the building becomes an advancement of the simple thermostat, a means by which the conditions of the living environment can be regulated and controlled (Weiner, 1965). 

While this model is useful for the control of complex building infrastructures, it also points to a particular understanding of the role of technology in architecture, a role both challenged and intensified through Breathing Room and Slow Furl. In conventional instances of ubiquitous computing, the subject, or the user model, becomes the centre of an interactive system, where every call demands a response. The environment then remains subservient to the user, creating a relationship of control and determinacy. 

Yet in cybernetics, the causal model of the thermostat has been the cause of much debate. If Norbert Weiner understood the thermostat as a control unit generating the stability of a system, the second order cyberneticians developed a more complex model of the interconnection of system and environment, one in which a circular causality allows for inter-relational actions and reactions (von Glaserfeld, 2000). Here, the continual exchange of conditions between environment and system is understood as a mutual negotiation in which there is no primary cause or instigator. Instead of understanding the system as under a measure of control, environment and system are conceived as mutually dependent, continually affecting each other through their recursive interchanges. 

It is this circular relationship that Breathing Room and Slow Furl seek to engage. In Breathing Room and Slow Furl the aim is to create a position where user and environment are part of the same cycle of material interaction. The condition of self-touching (installation) and touching (user) affects the same part of the circuitry thereby creating equivalence between action and interaction.

While on one level the rethinking of causality challenges the primacy of the subject, it also proposes a more fundamental shift in understanding the central aims for the system. Where the causal model points beyond the system for the instigation of change, the recursive interactions of the interconnected system seeks stability as a result of the dynamics of the interchanges between system and environment. With this model of recursive interchange an idea of ‘liveness’ is introduced. The live system is one that can preserve its interactions and recursive interchanges between system and environment in ongoing self-production. As defined by the cybernetician and biologist Humberto Maturana, an auto-poetic system (both organic and mechanical) is one which continually reconstitutes itself through its iteration of its processes of production. Here, self-production becomes the key objective for the system for which  ‘… everything that takes place in [the] autopoetic system is subordinate to the realisation of its autopoiesis’ (Maturana, 1980: 149). Whereas the system is open to its environment, it is closed in its aim, or, as Ranulph Glanville puts it: ‘… while it is open to information, its organisation remains closed’ (Glanville, 2004: 1381).

Returning to an architectural perspective, this shift of focus fundamentally changes the original percept of the building as subservient to its user into a near converse construct of introspection and self absorption. In this optic, the building becomes concerned with itself, and its occupant changes from “user” to participant, intrinsically linked to the system and part of its cycles of self-production. 

In the making of Breathing Room and Slow Furl the thinking of self-production as a means of interaction has lead to a similar state of introspection. In both installations the focus of interaction is to preserve the continual movement of its membranes. But this does not rupture the programmatic intentions of the installations. Despite existing as speculative probes both installations engage abstract programmes of occupation and use. 

Breathing Room was developed as a probe for an interactive scenography. The scenography was imagined for a dance performance and the folds and pleated gatherings of the fabric are scaled for the human body. As an extended backdrop, Breathing Room creates a shallow space that rhythmically expands and contracts opening up to its user, allowing the dancer to enter its complex skins. Slow Furl is an active wall membrane developed as a site specific installation for the Lighthouse Gallery, UK. The membrane includes scaled deepenings that act as seats, inviting the user into its flowing skins. Both installations engage the user directly through a measure of programmed occupation, and the introspective and in essence autonomous sense of interaction affects the design space of their conception.

Clearly, these shifts imply profound challenges for the practice of architecture. If architecture is traditionally understood as containing its occupant, and architectural design practice is concerned with the conception of this frame, inscribing this process into a cybernetic understanding of interaction and self-production changes architecture and its design from the making of an absolute specification to that of the devising of a dynamically changing systemic relationship. Thinking architecture as live, as motile and as self-sustaining, challenges both the object and the process of architectural making. 

Pulse as a Measure of Self-Production

In Breathing Room and Slow Furl the recursive interchange and self-production have lead to the conceptualisation of pulse as a way of understanding the temporal dimensions of the installations. The spatial intentions of the installations are defined across time, occupying time as an integral part of their performance. But this is not an abstract time of endless extension. Rather it is a time of agency and action: an enacted time that is neither cyclical nor linear, but random and fractured, always potentially, and disastrously, final.

The two installations are different in their way of engaging this temporal extension. Breathing Room was developed first. The installation is lightweight and transparent, its pulsing tuned to the breath of its occupant. In Breathing Room the armature is made from steel rods controlled by a pulley system. Much as with a fishing rod, the pulleys cause the rods to bend in a spline shape. The armature is clad with a translucent organza fabric pattern cut to vary in density as it folds. The material performance of the fabric is further detailed through laser cut patterns that locally changes the translucency, weight and draping of the membrane. Into this building skin conductive fibre is embroidered, creating a complex patterning of soft switches. The switches are connected to the steel armature. As such the armature becomes part of the circuitry passing the state changes to the micro-controllers. The fabric and the armature are understood as parts of the same complex composite material – collapsing, sensing and actuation – in which the motor systems trigger the switching and unswitching of the externalised circuitry, causing a continual oscillation making the membrane contract and expand. 

In Breathing Room the space is highly motile. The lightness of the membrane, variegated by its detailing, gives the installation an ephemeral expression. The movement loops are likewise agile, the pulse of the installation resonating with the pulse of its inhabitant.

Figure 4: Breathing Room, topological view

Figure 5: Breathing Room, topological view (2)

Slow Furl was developed after Breathing Room. As a reaction to the design of the temporal pulsing in Breathing Room, the aim for Slow Furl was to query the temporal dimensionality of a motile architecture. During the design of Breathing Room the tempo of each movement loop was discussed. Referencing former work (Ramsgard Thomsen 2007a, 2007b, 2008), it was noted that the timeframes of these installations had related to the size of the structure. Just like the quick breath of a small bird, the size of the organism relates to its breath or pulse. Critical to that reflection was the consideration of what would happen to this system at the scale of architecture, and how this could be related to the slow pulse of the inorganic. As such, Slow Furl explores a geological time of imperceptible flow. Like a glacier or a frozen river, the installation imagines a space of continual change taking place in times that exceed our immediate perception. 

As a structure, Slow Furl is in many ways similar to the Breathing Room installation. Slow Furl is defined by a thick spacer fabric cladding a dynamic armature which is moved by a series of stepper motors, that in turn are steered by a network of independent micro-controllers. Armature and fabric are understood as part of a complex composite that merge sensation and actuation. But where Breathing Room is ephemeral and agile, Slow Furl is thick-bodied and intransigent, moving as if in a deep pace at the edge of stasis. 

Figure 6: Slow Furl, topological view

In Slow Furl the armature is made of a wood structure with a series of jointed arms pulled vertically by the motors. The installation is heavy-set; the wood armature weighing the structure down and the many joints generating friction and an inherent inflexibility. In contrast the motors are exceptionally weak. Running at 5 volts they move at a quick pace without much torque. In collaboration with an engineering company we developed bespoke gearboxes gearing down the pace of the motors and generating a slower but more forceful output. In this way Slow Furl uses little energy to create great strength at a deeply set slow pace. 

The movement of the installation is differentiated in areas of movement contrasted with areas of stasis. Its thick membranes meet the underlying armature, partly in a tight fit and partly through a voluptuous folding allowing for the movement of the armature. In Slow Furl the triggering of movement happens through the same structure of movement loops as in the Breathing Room installation. The membrane is embroidered with conductive fibres creating an externalised circuitry that as the installation moves triggers new movement cycles. The occupant enters this self-interaction. Invited to touch, or to sit within the deep folds of its pleated membranes the user reconfigures the externalised circuitry. However, because of the slow speed of the movement, users have little means of understanding their effect on its slight pace changes. Users therefore remain almost entirely outside an experience of interaction, relating more to the structure’s material presence than its responsiveness. 

Challenging Material Practices: Looking for the instable 

As practice based probes the two installations query the material-tectonic realisation of a motile architecture as well as its conceptualisation. The idea of embedding computationally steered events into building skins presents profound challenges to the material practices of architecture. As architecture becomes a material for ubiquitous computing it becomes necessary to develop a new material foundation that allows for sensing and actuation. This presents a key shift in architectural practice away from the stable, towards the motile suggesting new structural concepts by which the state-shifting can be integrated as an inherent part of material performance. Breathing Room and Slow Furl thus investigate the making of ‘transitive materials’ that combine the structural performance of traditional material approaches with new computationally-informed properties. Transitive materials are inherently transformative. They are: ‘material organisations containing computation(s) and/or promoting unique digital fabrication methods as elements inherent to their properties and behaviour’ (Coelho et al, 2007: 2)  The interest in structural and material consequences of an architectural configuration of ubiquitous computing is informing a growing number of architects. In the work of practitioners such as Philip Beesley (Beesley, 2007), Omar Kahn (Kahn, 2009) and Neri Oxman (Oxman, 2008), spatial design takes its point of departure in material design investigating the potentials for actuation and movement, whether at the scale of the joint or in the actual compositing of materials themselves.

In Breathing Room and Slow Furl the membranes merge sensing and actuation into one complex skin. Here, textiles are understood as a material approach, a means by which the designed interactivity can be materialised. Conceived as composite materials, the membranes collect skin and substructure, merging the inherent pliability of textiles with the motile performance of the armatures. Rather than building for permanence, the installations seek to engage the material performances of the organza and steel, spacer fabric and wood, so as to create a platform for movement and change. The installations are therefore in very fundamental ways informed by the material presence of their structures. The motility and behaviour of the installations are shaped by the pliability of their cladding fabrics, the flex of the steel rods (Breathing Room) and the weight of the wood structures (Slow Furl) as much as by their programming and motor movement cycles.

Figure 7: Slow Furl, textile view (3)

Conclusions

Breathing Room and Slow Furl explore how ubiquitous computing and the idea of embeddedness pose profound challenges to architectural thinking. Rather than seeing the embedding of computation as an augmentation or extension of existing models of space and inhabitation, these projects ask what happens as computation becomes an integral part of our environment. The installation’s twofold focus on interaction design as well as material design presents a complex weave of exchanges between the computational and the material. In Breathing Room and Slow Furl interaction is uniquely tied to the embodied presence of the material. The interactive system not only relies on outside stimulation, but also on its own material presence for continual invigoration. The externalised circuitry is a network by which the continual process of self-production takes place. The interactive system is always potentially ending, only re-finding its pulse as new connections in its membranes are found. 

Breathing Room and Slow Furl also inquire into the material reality of a ubiquitous computing space. Using textiles as a technology as well as a material, the installations investigate how the design of a material performance can becomes part of spatial design. Rather than understanding the material as an existing strata for the embedding of computation, the installations ask how particular techniques lend themselves towards material specification that can be locally defined both in respect to its structural and its computational properties. 

A further consequence of this thinking is the extension of architecture into the temporal. The installations ask how this temporality can be architecturally stratified. Time is understood through a model of agency, continually enacted and lived. Where Breathing Room seeks to imitate the breath of its user, Slow Furl questions the temporality of interactive experiences. Where computers normalise the instantaneous feedback between the call of the user and the response of the system, Slow Furl asks what happens when this relationship is scaled up. By engaging a geological time, Slow Furl introduces concepts such as resistance and intransigence as ways of understanding the relationship between the moving and the formed. 

Acknowledgements

Breathing Room was developed for the Robo-Art festival and shown at Brandts Klædefabrik in Odense (2007). 

Slow Furl won commission from the InterARCHtive competition supported by the Lighthouse (Brighton), Architecture Centre Network, interactivearchitecture.org and RIBA (Sussex branch). 

The installation was supported by the University of Brighton, School of Architecture and Design.  The installation was shown as part of the Architecture 08 festival in the UK. 

Biographical Note

Mette Ramsgard Thomsen is an architect working with digital technologies. Her research centres on the relationship between computational design, craft and technology framed through “Digital Crafting” as way of questioning how computation, code and digital fabrication challenge the material practices of architecture. Mette Ramsgard Thomsen is Professor at the Royal Academy of Fine Arts, School of Architecture, where she heads the Centre for Information Technology and Architecture [CITA]. 

Karin Bech is an architect working with practice based research at the Centre for IT and Architecture, in Copenhagen. She has been a collaborator on projects as Slow Furl, Thaw and Thicket. Karin Bech is also the co-founder of the architectural office Byen Sover, working with urban ideas and conceptual design projects. The practice of Byen Sover has lead to a series of experimental and inventive projects generated from urban situation and strategies.

References

• Beesley, Philip, Hylozoic Soil: Geotextile Installations : 1995/2007 (Toronto: Riverside Architectural Press, 2007).
• Berzowska, Johanna. ‘Memory Rich Clothing: Second Skins that Communicate Physical Memory’ Proceedings of the 5th conference on Creativity & Cognition (New York: ACM Press, 2005), 32 – 40.
• Coelho, Marcelo , Sadi, S. , Maes, P. , Berzowska, J. and Oxman, N. ‘Transitive Materials: Towards an Integrated Approach to Material Technology’ Ubicomp 2007 Workshop Proceeding, 2007. 
• Compagno Andrea,  Intelligent Glass Facades (Berlin: Birkhauser Publishers, 2002).
• Glanville, Ranulph, ‘The Purpose of Second-Order Cybernetics’ Kybernetes 33.9/10 (2004): 1379 – 1386.
• Glasersfeld, Ernst Von, ‘Reflections on Cybernetics’ Cybernetics And Human Knowing 7.1 (2000): 93–95.
• Khan, Omar, ‘Reflexive Architecture Machines’ CHI 2009 Workshop Proceedings, 2009.
• Maturana, Humberto,  ‘The Organization of the Living: A Theory of the Living Organization’  International Journal of Human-Computer Studies 51.2 (1980): 149-168.
• Oxman, Neri, Acadia, 2008
• Ramsgard Thomsen, Mette, ‘Drawing a Live Section: Explorations into Robotic Membranes’ Kybernetes 36.9/10 (2007a): 1471 – 1485.
• —. ‘Building Liveness, Imagining Architecture as a Robotic Membrane’ Ubiquitous Computing Workshop Proceedings: 9th International Conference, Ubicomp 2007, Innsbruck, Austria, September 16-19, (2007b).
• —. ‘Robotic Membranes’ Exploring a Textile Architecture of Behaviour’ Proto Architecture: Analogue and Digital Hybrids, ed. Sheil, B. (Hoboken, N.J.: John Wiley and Sons Ltd, 2008).
• Wiener, Norbert, Cybernetics or Control and Communication in the Animal and the Machine (Cambridge, Mass.: MIT Press, 1965).
• Weiser, Mark,  ‘Creating the Invisible Interface: (Invited Talk)’ Proceedings of the 7th Annual ACM Symposium on User Interface Software and Technology (Marina del Rey, California, United States: ACM, 1994), 1.
• —. ‘The Computer for the 21st Century’ Scientific American 265.3 (1991): 94-104.
• —. ‘Designing Calm Technology’ PowerGrid Journal 1.01 (1996).
• Wiggington, Michael, and Harris, Jude, Intelligent Skins (Kent: Gray Publishing, 2002).