cast01 // living in mixed reality

[Claudia Westermann]

**************************************************************
R*VIEW OF CAST01 // LIVING IN MIXED REALITIES
presented by Maia ENGELI & Phoebe SENGERS
and discussed with Monika Fleischmann,
Wolfgang Strauss & cast01 Participant
Sunday, Sept 23, 2001, 11 - 13 h, Auditorium, Kunstmuseum, Bonn
http://netzspannung.org/cast01/
**************************************************************

INTRODUCTION
In this document, we review the cast01//living in mixed reality
conference, which took place at Schloss Birlinghoven from Sept 21-22,
2001. This review was developed for the final discussion session of the
conference, which took place at the Kunstmuseum in Bonn on Sept 23. To
the observations and issues that Maia Engeli and Phoebe Sengers
identified for the closing session we have added in this document the
issues, concerns, and proposed actions which the discussion participants
brought up in their responses to the conference at the closing session.
At this last gathering at the end of the conference we wanted to help
figure out what this could mean for the next steps that each one of us
will do individually. The questions we address are as follows: What have
we learned at cast01, what were the inspirations it brought us? Which
connections and possible collaborations were established? What tasks
should we engage in and what may be important duties
Stephen Johnes proposed to introduce the notion of "shared reality" at
the end of the conference. Over the two days of the conference we have
shared many insights into different aspects of "mixed realities". The
individuals that were clustering together during the conference are now
transforming into nodes of a network. This network - and this is very
fortunate - already has as a support structure the Internet platform
"netzspannung.org". http://netzspannung.org/
This resume is not going to be a summary of the individual panels, since
those summaries can be found in the proceedings as introductions to each
panel. For this resume, a number of issues that were brought up during
the conference have been identified. Six are offered as observations.
Four will be presented as issues that should be further discussed,
because they are important for this community and there was not enough
time during the past two days to discuss them at an appropriate depth.
We then describe the issues, concerns, and actions which were brought up
by discussion participants.

OBSERVATIONS
We describe here trends that emerged clearly through the course of the
conference. These themes lay out the current state of work in our field
and suggest where we might like to go next.

MIXED REALITY
The conference revealed the many ways in which the metaphor of "mixed
reality" can be understood. Mixed reality can be thought of in the
relatively technical sense as the combination of real and virtual
spaces, or, as Axel Wirths put it, the "floating border between physical
space and media." At the same time, the conference also dealt with the
mixed realities of art and technology, and some participants, such as
Machiko Kusahara, underscored the importance of the mixed reality
boundary between popular culture and technology, in which the public
picks up on and incorporates technical developments, giving them meaning
in our day-to-day lives. Several people pointed out that mixed reality
has a historical dimension, as we have always combined fantasy and
fiction with the materiality of our 'real' lives. The richness of this
concept seems to have found real resonance, and to describe well the
different conceptual interfaces which overlap and connect to each other
in the constituency of the conference.

THE DISAPPEARING COMPUTER
Many participants underscored the theme of the disappearing computer. In
this view, the computer and attendant technology disappears into
background awareness, so that we can focus directly on the experience of
the user in interaction with the information system. - Hiroshi Ishii,
for example, argued that a major, sometimes forgotten, insight of the
ubiquitous computing movement is that technological encumbrances should
disappear from everyday awareness, surrounding us with systems which are
integrated in our lives.
- Wolfgang Strauss argues that an important goal in mixed reality is to
get rid of data gloves, headsets, and other physical encumbrances that
come between us and the experience we are trying to communicate, and
that one must find nonintrusive technologies, such as vision, which
allow human users to interact with information systems using natural
body senses, unencumbered by wires.
- Didier Stricker presented an augmented reality system for cultural
heritage, using see-through video displays to show how archaeological
sites looked in their heyday. In his system, the user must carry heavy
machinery in a backpack; Stricker stated as a goal that the computer
must disappear.
The notion of the disappearing computer has two major positive
properties. First, it allows one to focus on and design the experience
of interaction from a human perspective, avoiding an the temptation to
focus on the technology itself. Second, successful mixed reality systems
which employ this heuristic are able to create powerful experiences, in
which physical objects seem to come alive and interact with us.
At the same time, there is a danger in the ideology of the disappearing
computer: out of sight means out of mind, and when people can no longer
see the technology which underlies the experience, it may make it more
difficult to understand and question that technology. An interesting
example of this phenomenon came up in Bruce Wyman's description of an
exhibit for the Chicago Museum of Science & Industry employing mixed
reality technologies. The exhibit was intended to teach people about
technology, but the technology which underlies the exhibit itself was
designed to be fully transparent; visitors could learn about technology,
but had no access to understanding the technology of the exhibit itself.
This is not necessarily a problem for this exhibit, but it would be
interesting to think about designing an exhibit for a museum setting
which does not hide, but rather explains its own technology.

MOBILE COMPUTING
The developments in the field of mobile computing are nowadays very
technology driven and often based on very shallow concepts. Numerous
presentations have shown that Mobile Computing is a field that is wide
open for creative exploration and inventions from new applications for
existing devices to the conception of new mobile devices.
- Per Persson presented expressive messaging, an enhancement of SMS messaging.
- Jürgen Enge showed an animated interactive interface to follow soccer
games, an enhancement of current text based information as well as a new
extension course for mobile application design.
- Machiko Kusahara shared insights in Japanese mobile applications, i.e.
successors of the Tamagotchi as well as the tendency towards Java3D
games for handheld devices.
- Bruce Newman showed NetPass, a wireless ID card developed for an
exhibition. "With NetPass the network can suddenly see you".
- Ralf Schreiber presented a population of living particles, that
perform absolutely independent, they collect their energy from solar
cells and behave independently from any human interaction.
- Sara Robert's to project equips humans with dolls connected into a
mobile communication network. The dolls are the voice of this network,
as well as its ear.

BODY AS INTERFACE
The quest for new kinds of interaction and interfaces was prominent in
this conference. In the introductory session Martin Reiser, the director
of the Fraunhofer Institute for Media Communication emphasised the need
for interfaces that integrate all senses. Hiroshi Ishii reminded us in
his keynote speech that in most computational interfaces, "the eyes are
in charge and the hands are underemployed." He presented a number of
sensual tangible interfaces and devices, like inTouch, curlybot, I/O
bulb, musicbottles, triangles or PingPongPlus. Some other projects in
this area include the following:
- Wolfgang Strauss presented the body as an instrument in a physical
room furnished with data.
- Steven Schkolne presentation of a Responsive Workbench System, in
which physical movement drives 3-dimensional virtual object
construction, showed very impressively how working can become a performance.
- Antonio Camurri presented the analysis of the expressive intentions of
human movement and its application to artistic performances.
- Alok Nandi in art.live, where the body is thought of as a joystick,
and Marikki Hakola in "Figure" and other projects showed mixes of
natural and synthetic imagery, letting the human become part of the
virtual space.
- Rebecca Allen explores breathing as an interface to introduce subtle
dynamics in a virtual scene.
- Volker Morawe and Roman Kirschner presented "His Master's Voice", a
board game with chant-sensitive ball robots that move in response to the
user singing the right tone.
While we explore the body as an interface, Martina Leeker reminds us
that "the body is not a tool or instrument that you can fully control."

QUALITY OF EXPERIENCE
A theme which ran through many of the conference contributions, was a
focus on designing the quality of the experience which the user has of
the information system, rather than on optimizing its efficiency or
functionality. Hiroshi Ishii, for example, in his discussion of Tangible
Bits focuses on the poetics of the interface; he is interested in the
aesthetic and emotional value of the interaction, and finds that this
aspect of design is something computer scientists often do not get. Axel
Wirths focused on this question of experience when he asked how we can
use mediatecture to make a new quality of life. Atau Tanaka provided a
useful metaphor for thinking about these issues when he contrasted
tools, which support efficiency and optimality, with instruments such as
the violin, which have an inherent character. This notion of the
character of the interface may prove to be a useful metaphor for
designing rich experiences.
How can we learn to build experiences with character? In describing her
experience with the NextTV project, Maddy Janse argued that
technological tools for creating new experiences need input from artists
to become mature. Similarly, Olivier Avaro argued that the new formats
being developed for expression, such as MPEG-7, need input from art and
the humanities in order to be adequate to the new experiences people
would like to create. It may be that this is an area, which is
particularly ripe for the mixed reality of art and technology.

STRATEGIES
Jill Scott pointed out, that "the boundary between hackers and artists
is being blurred." In the presentations we have observed different ways
of exploring and developing new uses for technology and I would like to
mention the two extremes:
- On the "computer science" side we find Marc Cavazza et al.
implementing AI-driven characters.
- On the "users" side "Parsival Cyberstaging" by Christoph Rodatz, who
deliberately has chosen not to implement or design something new but to
work with existing technology and to appropriate it for unforeseen uses.
These examples exemplify a phenomenon of the mixed community that has
gathered at cast01. There are numerous synergies that can emerge, but
there is also a need for a considerable culture in the discussions to
allow the synergies to emerge. A computer science specialist can
primarily discuss the aspects of computer science and maybe less the
visual appearance, which are paramount to a visually trained person. We
hope that this will lead to

ISSUES
In the following, we identify several key issues, which came up
repeatedly during the course of the conference. Since there is a certain
amount of subjective perspective in the description of these issues, we
have annotated each with the name of the person who summarized the issue.

ART - TECHNOLOGY - SCIENCE (Sengers)
In a conference on art, science, and technology, a major issue will
clearly be the relationship between the disciplines. This relationship
can be understood in many ways; Leon van Noorden talked about an
interface between the disciplines, while Martin Reiser used the term
networking (Vernetzung) to describe their interaction. Many conference
participants, such as Machiko Kusahara, Maurice Benayoun, and Olivier
Avaro asked about the role of the artist and of the technologist in this
constellation. No one, clear answer has emerged; but what is clear is
that this conference is not the only place where these questions are
being asked.
It seems that this is a fertile time for this constellation of
disciplines and approaches. I am working on a committee for the US
National Academy of Sciences on the relationship between information
technology and creative practices, and questions similar to the ones
posed at this conference come up there. For example, many people ask
whether it is better for an artist and a technologist to work together,
or whether it is better for one person to become proficient in both art
and technology. Although Olivier Avaro pointed out that it is
unrealistic to expect that many people will become transdisciplinary, it
is good that there are some people who are. If we wish to encourage this
work, we must ask ourselves what kind of institutional and strategic
support we can offer to those working in this new area between the disciplines.
What is this new area? Hiroshi Ishii preferred not to think of it as
something new, but rather to inhabit the edge of the established
disciplines. But, as a conference like cast01 shows, this new
transdiscipline not only lives on margins; a community is already
forming. Often, this area is defined by a diagram like the following,
which shows art, science, and technology as overlapping areas, where the
interesting regions are where art overlaps with science or technology.
Maybe it is useful to think about the constitution of this area in a
different way, as occupying a triangle between art, science and
technology, and cultural critique [note: in the ensuing discussion,
people pointed out other missing disciplines, such as design,
human-computer interaction, and performance; this diagram is not
intended to exclude other possibilities]. Within this multi-disciplinary
field are various projects, which combine the poles in different ways
and to different extents. Pedro Sepulveda's Digital Shelters project, or
speculative designs for protecting the public from cell phone radiation,
combines an artistic/architectural project with considerations from a
critique of the information saturation of cities and possibilities
supported by new technologies.
In fact, artists are already used to thinking along the dimension
between art and cultural critique. The reason the dimension of cultural
critique should be explicitly mentioned is because it makes clear the
potential role of the humanities and social sciences in this new
intellectual field, rather than leaving them as silent partners in the
background of many of the projects presented at cast01. In addition, in
my experience people doing projects that combine science and technology
with cultural critique find themselves grappling with many of the same
issues as those involved in combining science and technology with art;
thinking these disciplines together makes clear which discussions may be
useful, and who should be involved in them. Finally, it is important to
highlight the role of cultural critique in following Manfred Fassler's
call for the necessity of "software critique," new tools for evaluating
the human meaning of software systems in ways, which are not a part of
traditional computer science.
netzspannung.org, the internet platform launched at cast01, may be one
place where this new discipline can live. netzspannung.org - the media
lab on the Internet - is conceived as a platform for media art
production. A complex information space that is intuitively navigable
thanks to knowledge discovery interfaces. The platform is developed
within the project communication of art & technology (cat) at MARS
Exploratory Media Lab.

ART AND AUDIENCE (Engeli)
The boundaries between creators, performers, and audience are blurring.
The relation between artist and audience is changing and new roles may
be emerging. The question was raised whether or not the artists take the
audience into consideration enough. There are several examples in which
the audience is being thought in a central role to the installation.
- Gernot and Raimund Schatz realized the importance of anticipating as
best as possible the visitors' behaviours and observed audiences in
other museums when they were designing "Sonosphere" an exhibition on
auditory perception.
- Marikki Hakola exemplifies in her pieces (Figure and others) how
through the means of hypermontage, the visitor becomes co-author and
co-maker.
- In Steve Schkolne's presentation of the Responsive Workbench system,
the act of creating and the creation can be distinguished. The
performance and the object coexist, each one with its distinct aesthetic
qualities depending on the user's abilities.
- Manuel Abendroth presented "Space Navigable Music" a fantastic media
for artists to create audio-visual pieces and spaces for an audience.
- Emmanuel Moreno in "Alice at the Tea Party" and Marc Cavazza with the
AI characters showed pieces that allowed for interaction with synthetic 
characters.
- Christoph Rodatz in "Parsifal Cyberstaging" introduces the modern
phenomenon of zapping and multitasking on top of the Parsival performance.
- netzspannung.org collects contributions from a virtual audience. There
is a desire to see this community and to understand what is going on.
The visualisation therefore gives this community a face, a character,
and a dynamically evolving express on.
- Maddy Janse, in her talk about increasing the TV experience, pointed
out that experience is fundamental to human existence. In her talk (and
paper) she listed the basic elements for the perception of interactivity
(feedback, control, participation, production, communication,
interdependencies) of which one or more are necessary to achieve an
immersive experience.
- Martina Leeker demonstrated the power of the perversion of
interaction. In "telematic dreaming" the performers get confused because
the behaviour of the image does not correspond to their expectancies, in
"Tristan and Isolde Don't Touch" the audience through its action creates
action, but also makes it impossible to see it.

SHARED REALITY / REACTIONS TO TERROR ATTACKS (Sengers)
As mentioned earlier, a noticeable trend at the cast01 conference is a
focus on shared reality, i.e. thinking not only of mixing real and
virtual realities, but also of a social networking of different people's
realities. Per Persson, for example, underscored the importance of
supporting the social aspects of networked communications. Manfred
Fassler pointed out that networking in Computer Science is precisely not
about people, and he called for an interdisciplinary perspective on the
human dimensions of networking. An example of such a social shared
reality system is Atau Tanaka's "Global String," a one-string musical
instrument which exists partly in real spaces and partly over the
internet, allowing for distributed duets. Other examples of shared
reality systems include Sara Roberts's distributed, networked doll
project and Mina Hagedorn's installations for remote communication.
Often, projects in shared reality are motivated by a hope for
connection, tolerance, and unity. But an understanding that our
realities are intertwined with others' must be accompanied by a
realization that shared reality also brings us up against conditions
where those realities are incommensurable. This is underscored
dramatically by the recent terror attacks in New York and Washington,
which make clear that American reality is intertwined with hostile and
destructive realities, and that this shared reality has serious
consequences. In talking about shared reality, it is therefore also
essential that we talk in the context of these terror attacks about the
negative aspects of shared reality, and how we as artists -
technologists - scientists can react to them.
Here, we think back to Victoria Vesna's presentation, in which she
described her arrival in Germany on Oct. 11th and ensuing redesign of
her installation in order to react to the terror attacks. She reminded
us that when war happens, the social networks which we have built up may
go down. Victoria Vesna argued that it is essential to be able to react
quickly to such events. We believe that essential attributes are not
only speed of response but also length of time in remembering. In one or
two weeks, or one or two months, we will have forgotten what these
events have done to us, and it is also the artist's role to work against
such convenient forgetting. As Roy Ascott pointed out, the terror
attacks are not a unique, isolated event, but have a long history in two
world views which are unable to fuse and closed to dialogue.
Presenters like Hans Rainer Friedrich, Victoria Vesna, and Jill Scott
argued that we must work towards tolerance and respect. These goals are
good ones, but abstract; in the concrete mess of the actual conflict,
these goals can be difficult to apply. For one thing, the broad goals
such as tolerance and unity which underlie shared reality concepts may
bring with them, as Leon van Noorden pointed out, a destruction of
local, particular cultures; we follow Jill Scott in hoping for ways to
handle the conflicts inherent in shared realities while respecting the
texture of local cultures.
Given the recent horrendous events, it is natural that the cast01
community discuss ways in which it can respond. But between statements
on the responsibilities of artists, technologists, and other knowledge
workers, we must also ask whether it is really possible for us to make
much of a difference. It seems that a symbolic or metaphorical
resistance on the part of artists may not be enough to alter the current
geopolitical situation. Manfred Fassler argued that in a complex network
of technology and people, it is extremely difficult to predict the
global effects of local actions. Similarly, while we can call for
particular forms of action, we may never be able to say for certain what
their effects will be.
Following Jill Scott, it certainly seems possible for educational
efforts to play an important role. But we would like to suggest that an
additional dimension could be that we become more modest. Perhaps we,
the Western intellectual elite, are not able to solve this problem
alone. As Roy Ascott argued, perhaps we can instead learn from the
people we would like to help.

EDUCATION (Engeli)
Education is the ultimate issue in many regards. The theme filled the
last evening of the conference with numerous interesting contributions.
The time was lacking for the necessary discussions around this theme,
which touches upon all of the above mentioned issue and many more.
Manfred Fassler's keynote made the complexity around the issue of
education obvious. Aside from an interesting list of topics that need to
be taught to the students in the field of "mixed realities," he also
raised the important question: How is creativity and culture coming into
existence within the dynamically changing structure of a mixture of
humans and computers?
New kinds of courses were presented:
- Jürgen Enge showed the new extension course of the HGK Zürich on
mobile application design.
- Andrew Vande Moere presented the course and exhibition "Recombinant
Realities" held at the Architecture Department at the ETH Zürich. In
this course many aspects, from conceptual design to programming, were
combined to create an appropriate learning experience.
- Irina Kaldrack presented the CD-ROM "Interfaces - Interaction -
Performance. About the use of digital technology in theatre". She
pointed out the relativity of knowledge and how she designed the CD-ROM
to open space for questions and reflections.
- Ralf Schreiber reported from his "living particles" workshops where
the participants communicate with the particles using very simple sounds.
Student (or ex-student) projects included the following:
- Mina von Hagedorn's "information bodies" at the RCA in London are
telematic installations to overcome physical distance.
- Martin Schneider and Fabian Härle (digital sparks award 2001)
implemented the "Genius Bodybuilder", a tool for the evolution of
virtual characters.
- Sascha Kempe and Michael Wolf (digital sparks award 2001) interpreted
Italo Calvino's invisible cities as artificial worlds in "Stadtwirklichkeit".
- Tamas Szakal (digital sparks award 2001) used telephones and answering
machines for the net audio installation "Dialtone".
It was remarked that students often conduct their projects with very
little or no interaction with the professor. This leads to the several
questions: How is the role of education changing? Who is actually
teaching the students? Is the traditionally strong relation between
culture and education weakening? Are there other structures that promote
the necessary cultural discourse?

DISCUSSION
(Sunday, Sept 23, 2001, Auditorium, Kunstmuseum, Bonn)
In the ensuing discussion, audience participants responsed to the above
issues and brought up additional issues and actions which they
considered important. These represent the reaction of conference
participants to the conference, as follows:
- Issues in the art - technology - science complex:
o There is a need for institutional support for transdisciplinary work.
o Universities do not seem to work well for this, because many have
problems dealing with transdisciplinarity.
o There is a difficulty of reviewing across the disciplines. At a
conference like cast01, scientists may end up reviewing a paper by a
designer, and an artist a paper by an engineer. To deal with this
problem, we need to learn to write better to be able to communicate
outside one's own discipline.
- Issues in education:
o We need to encourage students to write and think about art.
o There is a need for image literacy for the public. This may be a good
role for museums.
o Students in this area tend to be largely on their own, because of the
speed at which technology is changing. There is a need for a new form of
teaching. One possibility is to think of the professor not as a
communicator of age-old knowledge but as a facilitator of student projects.
- Issues in media art / media philosophy
o There is a need for a historical understanding of history of media and
interactivity within the art world itself. This is an educational task
that those involved in interactive media need to take upon themselves.
o There are practical difficulties in media philosophy that many texts
are only available in their original language.
o There is a need for media art not to close in upon itself. It must
maintain openness to contact with other disciplines.
o Are we developing an elite class?
- Methods and strategies for this new area:
o One interesting approach to the problematic of designing for a
specific audience is user-centered design, and particularly the approach
of cultural probes invented by Tony Dunne and Bill Gaver at the Royal
College of Art
o Given the speed at which technology changes, it is essential that we
as a community find a way to focus on long-term issues, which will not
change so rapidly.
o It is important to develop tools for a living, world-wide culture and
dialogue. At the same time, we must acknowledge the fragility of the
tools we already have, as we discovered when we could not telephone New
York or Washington in the days after Sept. 11.
o There are questions about the role of performance, design, HCI,
architecture, and other disciplines in this field.
o There is a need for concrete experiments and practice in addition to theory.
o We must explore the consequences for human relationships to technology
of the disappearance of the computer.
- Issues in the interface to business:
o What happens when artist builds something that can practically change
the world? This adds a new level of responsibility.
o Is it possible for business to provide concrete support for artists?
At the close of the discussion, we began to address concrete actions
which we could take to address these issues. The following list is a start:
- For the problem of translations, institutions must work together to
make this happen.
- We need to find, construct, and find funding for alternatives to
universities as institutional structures to house this field.
- We should not wait for the money to come to support the field, but
rather find ways to do the work now.
We look forward to further discussions and developments in this field,
and encourage people to use the netzspannung.org platform as one area in
which these discussions can occur and developments be documented.
Monika Fleischmann & Wolfgang Strauss
Directors of MARS Exploratory Media Lab


The revised website of http://netzspannung.org and
http://netzspannung.org/cast01 will be publicised within a couple of
days.