background image
"Sousveillance"
Inverse Surveillance in Multimedia Imaging
Steve Mann
Dept. of Electrical and Computer Engineering
University of Toronto
Toronto, Canada
mann@eecg.utoronto.ca
ABSTRACT
This is a personal narrative that began 30 years ago as a
childhood hobby, of wearing and implanting various sensors,
effectors, and multimedia computation in order to re-define
personal space and modify sensory perception computation-
ally. This work involved the creation of various computa-
tional seeing aids that evolved into a new kind of visual art,
using multimedia cyborglogs. Becoming at one with the
machine, the author was able to explore a new humanity
at the nexus of cyberspace and the real world. The author
presents what was discovered accidentally, as a result of fac-
ing "cyborg discrimination". In particular, over the past
30 years, peer discrimination has decreased, while institu-
tional and organized discrimination has intensified. Most
notably, it was discovered that cyborg discrimination was
most intense in establishments having the most surveillance.
Rather than avoid such establishments, the author was able
to explore and capture unique aspects to understand surveil-
lance in new ways. The word sur-veillance denotes a God's
eye view from on high (i.e.
French for "to watch from
above"). An inverse, called sous-veillance (French for "to
watch from below") explores what happens when cameras
move from lamp posts and ceilings down to eye level. Fi-
nally, it is suggested that new personal multimedia tech-
nologies, like mass-produced wearable cameraphones, can be
used as tools for artists to explore "equiveillance" by shifting
this equilibrium between surveillance and sousveillance with
inverse/reverse accountability/recountability/continuability
of continuous sur/sousveillance.
Categories and Subject Descriptors
J.5 [Computer Applications]: ARTS AND HUMANI-
TIES--Fine arts
General Terms
Design, Experimentation, Performance, Theory, Verification
Keywords
surveillance, inverse surveillance, sousveillance, weblog, cy-
borglog, computer mediated reality, eyetap, equiveillance,
terrorism, guerrorism, survey, sousvey, perveillance
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. To copy otherwise, to
republish, to post on servers or to redistribute to lists, requires prior specific
permission and/or a fee.
MM'04, October 10-16, 2004, New York, New York, USA.
Copyright 2004 ACM 1-58113-893-8/04/0010...$5.00.
What is sousveillance?
SURveillance ("eye-in-the-sky") versus SOUSveillance:
bringing cameras from the heavens, "down to earth".
The word "Surveillance" is French for "to watch from above".
It typically describes situations where person(s) of higher
authority (e.g. security guards, department store owners,
or the like) watch over citizens, suspects, or shoppers. The
higher authority has often been said to be "Godlike" rather
than down at the same level as the individual party or
parties under surveillance [Foucault 1977]. In this paper,
surveillance is defined as the capture of multimedia content
(audio, video, or the like), by a higher entity that is not a
peer of, or a party to, the activity being recorded.
The author has suggested "sous-veillance" as French for
"to watch from below".
The term "sousveillance" refers
both to hierarchical sousveillance, e.g. citizens photograph-
ing police, shoppers photographing shopkeepers, and taxi-
cab passengers photographing cab drivers, as well as per-
sonal sousveillance (bringing cameras from the lamp posts
and ceilings, down to eye-level, for human-centered record-
ing of personal experience).
It should be noted that the two aspects of sousveillance
(hierarchy reversal and human-centeredness) often interchange,
e.g. the driver of a cab one day, may be a passenger in some-
one else's cab the next day.
Thus a main feature of "sousveillance" as a tool for multi-
media artists is effortless capture, processing, storage, recall,
620
background image
and transmission of an activity by a participant in the ac-
tivity.
Disclaimer the role of the individual artist and
personal passion outside the traditional academic
laboratory: Because this paper describes the author's own
personal experiences of inventing, designing, building, and
living with a variety of body borne computer-based visual
information capture, processing, and mediation devices in
everyday life, there is a necessary narrative element that
would be diminished if it were forced to conform to the ob-
jectivity usually found in a scholarly article.
The practice beginning in the author's childhood, involved
30 years of bearable (wearable, implantable/dermaplantable,
and body/brain modification) systems and devices. This
practice would outstrip a normal ethics review process, so a
certain element of this work reaches beyond the traditional
manner of scientific explorations, perhaps more into the do-
main traditionally reserved for the Fine Arts. The arts is
one of the few places where there exists an accepted practice
of performance art, body art, body modification (like the
sex change experiment of Professor Sandy Stone, Eduardo
Kac's microchip implanted in the body
1
, the "Cyborgian
Primitives" movement), and the like.
0.1
Computer Mediated Reality
Since the 1970s the author has been exploring electroni-
cally mediated environments using body­borne computers.
These explorations in Computer Mediated Reality were an
attempt at creating a new way of experiencing the percep-
tual world, using a variety of different kinds of sensors, trans-
ducers, and other body­borne devices controlled by a wear-
able computer [7].
0.2
Practical Applications
Early on, the author recognized the utility of computer
mediated perception (computationally modified presenta-
tion of sensory data). For this kind of work, the author in-
vented a device that intercepted rays of eyeward bound light,
and resynthesized (typically with a computer-controlled laser)
substitute rays so that the resynthesized rays could be collinear
with the measured rays. This resulted in a device where
three elements existed at the same point in space: (1) the
effective center of projection of a camera or other sensor; (2)
the convergence point of the above collinear rays of light;
and (3) at least one eye of the wearer. Thus the device is
equivalent to putting both a camera and a display inside the
eye. Such a device, fitted to one or both eyes, is called an
EyeTap device [7].
EyeTap devices can be used for electric seeing aids, or
when used together with a similar device called the EarTap,
for converting the body, in effect, into a camera phone.
0.3
Personal Safety Device
The author's mediated reality devices also included the
capability of lifelong capture and transmission of physiolog-
ical signals together with the EyeTap signal. Capture of
the data can allow such a system to function much like the
"black box" flight recorder in an aircraft that provides evi-
1
Others, such as Kevin Warwick, have also followed in Kac's
footsteps, some for artistic reasons like Kac, and others for
more utilitarian reasons.
dence as to why an accident or deliberate violent act
occurred.
To protect the data of the "black box" life recorder from
accidental or malicious damage, the data has generally been
transmitted and recorded at remote locations. Addition-
ally, for example, transmission of synchronized timestamped
ECG data allows a remote physician to observe not only
the electrical heart activity, but also the visual environment
which may provide clues as to environmental causes of ECG
irregularities such as arrhythmia.
When it is worn continuously (e.g. out of medical neces-
sity to capture valid data) the long-term adaptation to see-
ing through the device also provides a unique opportunity to
capture, process, store, and recall visual memories. Unlike
a mere wearable camera, the EyeTap, because it becomes
a manner of seeing, captures exactly what the bearer does
see. This results in a new kind of EyeTap cinematographic
vision, together with a serendipitously generated logfile that
happens without conscious thought or effort.
A cyborg (in the Manfred Clynes sense of a technological
synergy that doesn't require conscious thought or effort),
can thus generate a lifelong logfile for personal experience
capture. Such a logfile is called a cyborglog
(http://en.wikipedia.org/wiki/CyborgLog).
Later with the advent of the World Wide Web cybor-
glogs also became weblogs [Ito 2004], an example of which
is shown in Fig 1.
Ironically, the coverage of the East Campus fire (Fig 1)
resulted in negative press
Wearable Web Camera Goes Too Far, Anders Hove, Execu-
tive Editor,
www-tech.mit.edu/Issue/V116/N28/mann.28c.html from the
very paper that might have used the pictures captured in the
cyborglog. It is interesting to note that Hove's first main
objection was the strange physical appearance (to use his
words it's "worse than Spandex, tweed, and bell-bottoms
combined"), rather than the privacy issues. This was an
objection also raised when the author had driver's license
pictures and passport pictures taken, and finally succeeded
in making a legal argument as to why self-modification of
physical appearance must be accepted, after which a num-
ber of passports and driver's licenses were issued with the
author's newly created physical appearance.
In particular, living within a permanently installed/instilled
photographic perspective allows the bearer to capture pre-
cious yet serendipitous moments in life, such as the birth of
a newborn, or baby's first steps.
0.4
Related work
Despite the initial negative reactions, a lot of good came
of the explorations in web-based cyborglogs (time-stamped
diaries of serendipitous personal experience recordings made
available to the world). Others are also now proposing sim-
ilar projects. Industry is also recognizing the importance of
inverse surveillance. For example, the Hitachi Design Cen-
ter in Milano recently sponsored an event entitled "Applied
Dreams Workshop 3: 'Surveillance and Sousveillance'
".
Nokia is planning a "life 'blog" (lifelong weblog) prod-
uct similar to the author's life 'glog (lifelong cyborglog)
project. Microsoft's "sensecam" and "MyLifeBits" projects
(http://research.microsoft.com/CARPE2004/) and Hewlett
621
background image
Figure 1: In this cyborglog, the author encountered an event serendip-
itously through ordinary everyday activity. As it turned out later, the
newspapers had very desperately wanted to get this event covered,
but could not reach any of their photojournalists in time to cover the
event. The author, however, was able to offer hundreds of pictures of
the event, wirelessly transmitted, while the event was still happening.
Furthermore, a collaboration with a large number of remote viewers
enabled a new form of Computer Supported Cooperative Journalism.
Packard's "Casual Capture" project also build upon various
concepts of sousveillance.
Sousveillance is related (even if by inverses) to the tra-
dition of surveillance, and to the artistic practice explored
by artists, such as Julie Scher, and the Surveillance Camera
Players, among others, working in the medium of surveil-
lance.
Organizations such as Future Physical are also "stretching
technology a human adventure" and developing "cultural
program exploring boundaries between virtual and physi-
cal", e.g. "How will the human body interact with digital
tools in the future?". See for example, Wearable Computing
Links,
www.futurephysical.org/pages/content/wearable/links.html
In relation to the Fine Arts, the continuous nature of
sousveillance (i.e. continuous archival of personal experi-
ence) is very much like the concept of "living art". Tehch-
ing defined "living art" performances as being of one year
in duration (e.g. Tehching Hsieh and Linda Montano held
opposite ends of a rope but never touched each other for one
year), although other durations are possible (e.g. Montano's
14 year long clothing colour experiments, wearing only one
colour of clothing for each of the 14 years, etc.). The au-
thor's 30 year long exploration and 20 year long actual ex-
periment in bridging the gap between cyberspace and the
real world by living day-to-day life through the electric eye-
glass is thus an example that might also be considered part
of the tradition of "living art".
Moreover, recently there has been a growing sousveillance
industry, with three workshops, organized independently,
but around the same time:
· International Workshop on Inverse Surveillance (IWIS 2004),
April 12th. This workshop is based on 3 years of planning and
previous "inverse conferences" entitled DECONference 2001,
DECONference 2002, and DECONference 2003. See, for ex-
ample,
http://wearcam.org/iwis/ and http://deconference.com
· Memory and Sharing of Experiences, in cooperation with Per-
vasive 2004, April 20th, 2004, Vienna, Austria. See, for exam-
ple,
www.ii.ist.i.kyoto-u.ac.jp/ sumi/pervasive04/
Sumi, for example, makes the distinction between surveillance
(sensors in the environment) and sousveillance (sensors attached
to persons) through the use of "the term 'ubiquitous' to de-
scribe sensors set up around the room and 'wearable' to spec-
ify sensors carried by users"[9]. Some of this work also relates
directly to computer mediated reality [4][2].
· Continuous Archival and Retrieval of Personal Experiences (CARPE
2004), New York, New York, October 15th 2004, held in con-
junction with the conference in which this paper appears (ACM
Multimedia).
The work presented in this paper is distinct from that of
the sousveillance industry which is not focused on art, or
the related philosophical and technosocial issues. Likewise,
much of the existing work in performance art, and body art
is not directly connected to the sousveillance industry, in
terms of tools for art and intervention. Thus there is a
largely unfulfilled need for such tools.
While it is well known that technology influences art,
(e.g. Scher's surveillance-based art is obviously influenced
by surveillance technologies), it is hoped that art will also
influence technology [1], and in particular, it is hoped that
art will influence the growing sousveillance industry as much
as the surveillance industry has influenced art.
1.
COMPUTER MEDIATED REALITY AS
A TOOL FOR TRANSFORMING EVERY-
DAY LIFE INTO VISUAL ART
Stepping beyond the obvious practical uses of Computer
Mediated Reality, there is a more existential motivation re-
garding how we, as humans, are able to choose the manner
in which we define ourselves [10]. The lifelong cyborglog
recorder is more than just a visual memory prosthetic. It is
also a new tool for the visual arts.
One of the author's original goals of Computer Mediated
Reality was to create a body­borne wireless sensory envi-
ronment which, although technically sophisticated, would
function more in the spirit of an artist's personal notes or
a painter's canvas. Thus computer-mediated reality was a
form of artistic exploration.
In the early 1980s the author was asked to exhibit his com-
puter mediated visual experiences in various art galleries,
resulting in a genre of photographic memory characterized
by the computer mediation, capture, sharing, recording, and
processing of everyday visual experiences. See Fig 2.
These images were created using a concept of vector spaces
made from photographic quantities, that the author called
"painting with lightvectors".
622
background image
(a)
(b)
(c)
Figure 2: Living in a computer mediated environment as a new way of seeing the world as visual art (a) A mid 1980s view of a corridor at
McMaster University, and (b) of the Mann residence. (c) Computer mediated view of a television placed on an easel at the base of a commonly
photographed space, Niagara Falls. Reality once mediated through television, is again mediated through the wearable computer, as a form of
social commentary on what is reality.
Briefly summarized, lightvector paintings are made by
combining differently illuminated exposures of the same sub-
ject matter, as illustrated in Fig. 3.
This process of "painting with lightvectors" was also pos-
sible with a group of people wearing computerized seeing
aids that were tuned to the same virtual channel, so that
there was a shared computer-mediated visual reality. In
this way, the team experienced a collectively modified view
of the world, in the production of visual art. Such early ap-
paratus was more cumbersome, however, and thus perhaps
less well suited to widespread use as a tool for multimedia
artists. (See Fig 4(a).)
More recently, versions of this system have been made
available for others to use, with computer programs that
can be downloaded from comparametric.sourceforge.net and
run on less cumbersome systems, easily made from mobile
(small 12 volt automotive) computers, as shown in Fig 4(b).
This new tool for artistic exploration is very easy to use,
and can be taught in just a few minutes, to anyone with no
prior experience. The new hand-held form factor can also
be passed around quickly among a group of individuals, so
that they can all feel like they are participating in the use of
the tool. The grip, similar to the rubber grip of a hammer,
makes the tool easy to pass from one person to another, and
thus it is very suitable for teaching large groups of students.
2.
CYBORG DISCRIMINATION: ACCIDEN-
TAL DISCOVERIES IN SOUSVEILLANCE
By the summer of 1985 the author had built a wearable
computer mediated reality system into a jacket, which he
wore in much of his day-to-day life.
This resulted in two kinds of public reactions:
· peer discrimination from individuals, either to the out-
ward appearance while wearing the entire system, or
the discrimination that remained when the outwardly
visible portions were removed, leaving only the per-
manently attached electrodes, subdermal and derma-
plant
2
portions of the apparatus (e.g. with regards
2
Dermaplants refer to devices such as subdermal electrodes,
to the portions of the apparatus that are permanently
attached to the body being seen by others during com-
munal change of clothes for high school gym class, the
need to wear a full-body bathing suit to cover derma-
plants during swims, or the like);
· official discrimination by representatives of large orga-
nizations, allegedly acting on the wishes of the organi-
zation. This discrimination pertained to both the un-
usual outward appearance of the apparatus, the func-
tionality of the apparatus (evidence capture, live trans-
mission of visual images of the official and the offi-
cials establishment, etc.), as well as the inward ap-
pearance of the body even when the main portion is
removed (permanently attached electrodes, subdermal
and dermaplant portions of the apparatus that might
become visible in an airport stripsearch room).
The author discovered these various elements of discrimi-
nation by accident, simply through the process of living the
bearable (wearable/implantable) computing lifestyle. Of the
various forms of discrimination, the author could forsee the
day when the apparatus would no longer have an unusual
appearance, because miniaturization would some day allow
all of the apparatus to be implanted (and concealed) within
the body. Ten to twenty years later, this vision was to have
been realized simply by the miniaturization of the apparatus
into what appear like ordinary clothing and eyewear (Fig 5).
To achieve such a concealment opportunity, the author
invented a new kind of eyeglass design in which the frames
come right through the center of the visual field. With ma-
terials and assistance provided by Rapp optical, eyeglass
frames were assembled using standard photochromic pre-
scription lenses drilled in two places on the left eye, and
transdermal wound closure, connections on deliberately self-
inflicted wounds for purpose of making better connections,
and other devices permanently attached to, on, or below the
surface of the skin. The author finds that Dermabond (TM)
wound closure material manufactured by Closure Medical
is often useful for making, growing, or maintaining derma-
plants.
623
background image
Figure 3: Painting with lightvectors: A lightvector painting is made
from various exposures to different sources of illumination. In, for
example, v32.jpg, the open basement door under cell block "A" on
Alcatraz Island, is exposed to light from a flash lamp held to the
left. The flash lamp is then moved to the right, to illuminate the
scene from the right, in exposure v36.jpg. Finally, exposure v37.jpg
captures light coming from upstairs, beyond the jail bars above the
door. Each such picture is displayed on the eyepiece of the author's
wearable computer, as the author walks around in the space, illumi-
nating the space from various viewpoints. These pictures are then
converted into lightspace by applying an estimate of the inverse of
the camera's photographic response function[6]. The resulting pho-
tographic quantities are added together, and the combined exposure
is then converted from light space back into a picture. (C) Copyleft,
S. Mann, 1993.
four places on the right eye, to accommodate a break in the
eyeglass frame along the right eye (the right lens being held
on with two miniature bolts on either side of the break).
The author then bonded fiber optic bundles concealed by
the frames, to locate the camera and aremac in back of the
device.
The eyeglasses of Fig 5 were crude and simple. A more
sophisticated design uses a plastic coating to completely con-
ceal all the elements, so that even when examined closely,
evidence of the EyeTap is not visible.
The peer discrimination by the masses was also simply
seen as a matter of education and acceptance.
The au-
thor found that this form of discrimination began to decline
sharply in the mid 1980s (beginning around 1984, amid the
new-wave androgyny where transhumanism began to take
acceptance first in the transgender community and then in
society as a whole).
By the 1990s, such peer discrimination had largely dis-
appeared, yet the organizational discrimination continued
to increase and intensify. For example, recently, the author
was physically assaulted by a number of security guards at
the Art Gallery of Ontario. Rather than asking the author
to leave, the guards simply pushed the author out of the
gallery. The author later asked the Chief Curator as to the
(a)
(b)
Figure 4: (a) Early tools for lightvector paintings: Jacket based
computer system that was completed in the summer of 1985 was used
in conjunction with a 2.4 kJ flashlamp in a 14 inch (356mm) reflector.
Three separate long communications antennas are visible, two from
the backpack and one from the jacket based computer. (b) A user-
friendly mass-produced tool for artists to use provides clear
step-by-step instructions on a TV screen that's attached to a
light source. The TV (a standard NTSC TV) attaches to the bot-
tom of the handgrip, and a standard electronic flash attaches to the
top. A mobile (12 volt automotive) computer at the base station and
the TV on the hand grip eliminate the need for a cumbersome wear-
able computer system typical of the 1970s and early 1980s lightvector
painting systems. Computer programs to make it work are freely
available at comparametric.sourceforge.net
reason for this action. The reason given was a possibility of
copyright infringement.
This raises an important question as to the right to fair
use of one's personal environs, and personal experiences, es-
pecially in view of an acquired dependence on computerized
visual memory. It seemed the author had unwittingly come
to confront, explore, and understand issues concerning the
ownership of space and whether such ownership should pro-
vide an advantage in perpetrating copyright infringement
(i.e. what if their surveillance cameras capture a picture of
art that a patron is wearing, such as a painting on a T-shirt
-- does that justify the patron smashing up their surveil-
lance cameras because of the mere possibility of copyright
infringement?). It also raises questions pertaining to the rel-
ative worth of humans and walls (e.g. a painting hung on a
wall gets more protection tha a painting worn on a T-shirt).
Physical assault in response to a mere potential for copy-
right infringement seems specious at best, given recent Supreme
Court rulings allowing photocopiers in libraries, despite the
fact that they could be used beyond the level permitted by
fair use
.
This response from the gallery, and other similar insti-
tutions has actually increased not decreased the amount of
recording done:
· Before such incidents the author used to make seeing
aids that did not necessarily record;
· now any seeing aid that we make in our lab is equipped
with a retroactive record capability. This helps the
wearer keep an evidence log in case such violence oc-
curs.
The irony, therefore, in such physical assaults is that the
fears of the security guards are coming to fruition by way of
their own actions. Similarly, one would expect that if people
went around smashing up surveillance cameras with base-
ball bats, this would probably cause more to be installed,
624
background image
Figure 5: Fully functional electronic eyeglasses built into wire frames.
Eyeward­bound light is diverted along the right temple by way of a
fiber bundle, into a miniature camera. A laser directs light along
another fiber system on the left temple to redraw the modified reality
onto the retina. The details are provided in [7].
and would cause more detailed recordings of each one to be
made. It is therefore futile to resort to violence as a means
of suppressing evidence gathering technologies.
Thus the fundamentally most difficult element of discrim-
ination appeared to be the official discrimination based on
functionality of the cyborg.
The author began to understand this discrimination through-
out the 1970s and early 1980s, as being correlated to the
degree of surveillance present in an establishment. It ap-
peared, for example, that the establishments where official
discrimination was greatest, were the very same establish-
ments where the use of video surveillance was the greatest.
Therefore the author, through simply a personal desire
to live in a computer mediated world, encountered hostili-
ties from paranoid security guards, seemingly afraid of be-
ing held accountable. It seemed that the very people who
pointed cameras at citizens were the ones who were most
afraid of new inventions and technologies of citizen cameras.
The harsh and sometimes hostile discrimination against
the author, by officials, security guards, and representa-
tives of large organizations led the author to begin thinking
mainly about official discrimination against cyborg func-
tionality. In order to learn from these hostilities, the au-
thor wished to understand this discrimination by applying
the scientific method, within an ethnomethodological sense,
which evolved into using body-borne multimedia computa-
tion as a tool for social inquiry and action research [3][8]
on surveillance as an emergent agenda. However the unique
framework and situation did not conform to a particular aca-
demic discipline (psychology, sociology, science, engineering,
etc.). Therefore this work was often appreciated more within
the arts community, where interdisciplinarity was fully em-
braced even many years ago.
Various places that the author was most strongly prohib-
ited from entering seemed to include places like maffia run
gambling casinos, pawnshops where money laundering
might be taking place, and jewellery stores. Such organi-
zations were ironically the places where surveillance cameras
were abundant.
Along another avenue of discourse, the author began to
undertake a series of explorations in which he unwittingly
became what others referred to as an "artist", despite having
a more science and engineering based background.
This exploration into the Fine Arts arose from a desire to
try to understand the reasoning behind such organizational
discrimination, rather than simply avoiding it.
For example, as a departure from EyeTap eyeglasses as
seeing aids, the author also constructed various forms of
cyborg jewellery, in order to test an hypothesis, namely that
jewellery store owners would welcome and appreciate having
pictures taken by innovative jewellery. Thus the author built
Personal Safety Devices (PSDs) into jewellery (Fig. at top of
first page of paper). The reaction was quite surprising. Even
when blatantly told that the devices contained a camera,
jewellery store and pawnshop owners did not object to the
device in any way. Although the device does not allow the
wearer to live in a computer mediated world, it captures
all the elements of paranoia that the officials most feared,
e.g. primarily a video captured record of their establishment
and activities. Yet they accepted this alternative form of the
device without complaint, largely because it so nicely landed
within their genre. Indeed, many of the jewellery store
owners wanted to commercialize and sell the sousveillance
necklace and other domewear products.
The sousveillance necklace established a more inclusive
narrative that treated the store clerks and security guards
as colleagues. As a sharp departure from 20th century "us
versus them" thinking, the cyborglog jewellery created a
new kind of artistic practice and discourse. By presenting it
as an object that the guards and shopkeepers could try on,
and look at themselves in the mirror wearing, they had no
problem with it being in their store, transmitting images to
the World Wide Web. It is therefore interesting to note that
an inclusionary rather than exclusionary element of sousveil-
lance is possible. While surveillance tends to be exclusion-
ary, and tends to present a very strong "us versus them"
directionality, sousveillance can be made to operate much
more like Peer-to-Peer, in the sense of creating a level play-
ing field. The sousveillance landscape therefore may include
both shoppers and shopkeepers, wearing personal recording
devices. Moreover, a shopkeeper may then assume multiple
roles, e.g. one role is obedience to a store manager, but an-
other role might be the capture of his or her own personal
"day in my life at work" cyborglog to share with friends.
2.1
Schrodinger's Cam: Nonwillful blindness
with the maybecamera
Another aspect of artistic discourse and philosophical ex-
ploration was the reflectionism [5] of uncertainty (Fig 6). A
large number of wireless webcam shirts were made, but only
some of them had cameras in them. They were then shuf-
fled and distributed widely. Honestly not knowing whether
or not one was wearing a camera added a new dimension
to putting the uncertainty principle into artistic practice.
Moreover, consider, for example, the "sousveillance under-
ground" as a probe into New York proposed ban on pho-
625
background image
(a)
(b)
(c)
(d)
(e)
Figure 6: Heisenberg Uncertainty with Schrodinger's Cam: The Maybecamera. A large number of wireless webcam shirts were made,
but only some having cameras. These were shuffled so each wearer did not know whether or not theirs had a camera in it. (a) Closeup picture of
one of many maybecameras showing a detournement, reversalism/reflectionism, and deconstruction of the typical language (text) of surveillance.
(b) A number of people wore these gambling (e.g. Casino Niagara, etc.) without incident. This suggests that perhaps the guards don't read
shirts. The wearer's don't know which shirts contain wearable wireless web cameras and which shirts don't. (c) The author's maybecamera
design is spreading around the world. Dr. S. Pantagis, a physician at a New York hospital, made an initial batch of 25 of these to distribute to
New York poets, followed by a larger production run. (d) as seen in a New York department store's security camera. (e) closeup view seen by
security camera.
Figure 7: The international Workshop on Inverse Surveillance was
an intimate gathering, limited to 30 participants from around the
world, and held in three small rooms like that shown above. There
were 3 simultaneous tracks: Philosophy of souveillance; sousveillance
industry and business opportunites; and existemology (epistemology
of self determination).
tography in subways. An exhibit of subway photographs is
expected to follow.
3.
(DE)CON
CLUSIONS:
THE NEED FOR FURTHER INQUIRY
Computer-mediated reality, with its origins as a tool for
lightvector painting, has been presented as a new tool for
visual artists. In particular, the personal experience capture
of sousveillance is useful both to see the world in a different
light, as well as to challenge our preconceived notions of an
otherwise one-sided surveillance-only society.
However, the concepts of sousveillance raise many ques-
tions that cannot be answered in a single paper. It is there-
fore hoped that it will create a broader intellectual land-
scape, and a basis upon which to build many different re-
search directions.
Arising out of a 2001 event, the author began a collabora-
tion with Samsung on a conference entitled "Sousveillance
Fusion 2002", which eventually led to a series of conferences,
in 2002, 2003, and a smaller workshop in 2004 (Fig 7 and 8).
The goal of this workshop was not so much to solve the
technical problems (many of which have already been solved
many years ago), but to address the problems that an indi-
vidual, working alone, could never solve. These fundamen-
Figure 8: A number of participants were also remotely present from
around the world.
tal problems are the technosocial problems; for this we need
large scale collaboration.
One important result of the discussion was a better un-
derstanding of the shifting equilibrium between surveillance
and sousveillance, as outlined in the Equiveillance Table
below:
Surveillance
Sousveillance
------------
-------------
God's eye view from above.
Human's eye view.
(Authority watching from on-
("Down-to-earth.")
high.)
Cameras usually mounted on high
Cameras down-to-earth (at
poles, up on ceiling, etc..
(ground level), e.g. at human
eye-level.
"Eye in the Sky"
"Eye in the Eye"
(i.e. camera in the sky)
(i.e. eye is the camera)
Sur-veiller is French for "to
Sous-veiller is French for "to
watch from above".
watch from below".
Architecture-centered
Human-centered
(e.g. cameras usually mounted
(e.g. cameras carried or worn
on or in structures).
by, or on, people).
Recordings made by authorities,
Recordings of an activity
remote security staff, etc..
made by a participant in the
activity.
Note that in most states it's
In most states it's legal to
626
background image
illegal to record a phone
record a phone conversation of
conversation of which you are
which you are a party.
Perhaps
not a party.
Perhaps the same
the same would apply to an
would apply to an audiovisual
audiovisual recording of your own
recording of somebody else's
conversations, i.e. conversations
conversation.
in which you are a party.
Recordings are usually kept in
Recordings are often made public
secret.
e.g., on the World Wide Web.
Process usually shrouded in
Process, technology, etc., are
secrecy.
usually public, open source, etc..
Panoptic origins, as described
Community-based origins, e.g.
by Foucault, originally in the
a personal electronic diary,
context of a prison in which
made public on the World Wide
prisoners were isolated from
Web. Sousveillance tends to
each other but visible at all
bring together individuals, e.g.
times by guards.
Surveillance
it tends to make a large city
tends to isolate individuals
function more like a small town,
from one another while setting
with the pitfalls of gossip, but
forth a one-way visibility to
also the benefits of a sense of
authority figures.
community participation.
Privacy violation may go
Privacy violation is usually
un-noticed, or un-checked.
immediately evident.
Tends
Tends to not be self-correcting. to be self-correcting.
It's hard to have a heart-to-
At least there's a chance you
heart conversation with a lamp
can talk to the person behind
post on top of which is
the sousveillance camera.
mounted a surveillance camera.
When combined with computers,
When combined with computers,
we get ubiquitous computing
we get wearable computing.
("ubiqcomp") or pervasive
("wearcomp").
Wearcomp usually
computing ("pervcomp").
doesn't require the cooperation
Perveillance (ubiq./perv. comp)
of any infrastructure in the
tends to rely on cooperation
environments around us.
of the infrastructure in the
environments around us.
With surveillant-computing,
With sousveillant-computing, it
the locus of control tends to
is possible for the locus of
be with the authorities.
control to be more distributed.
and, in particular, to rest
with the individual.
A larger symposium on sousveillance is planned for 2005.
More is written on sousveillance as the paper correspond-
ing to the keynote address at a workshop attached to this
ACM Multimedia conference, entitled "Continuous Archival
and Retrieval of Personal Experiences" (CARPE). In that
20 pape paper, related concepts of sur/sousveillance, equiv-
eillance, and auditor/viditor relationships are described, to-
gether with EyeTap device invention, design, and realiza-
tion.
4.
ACKNOWLEDGEMENTS
I'd like to thank my many past and present students, in
particular, James Fung, Corey Manders, Daniel Chen (dust-
ing), Mark Post (sequencer), Chris Aimone, and Anurag Se-
hgal (keyer), who've tolerated and contributed to the growth
of the artistic practice of sousveillance, as well as Alex Jaimes
who has made many useful suggestions on the original and
revised manuscripts. I'd also like to acknowledge our many
sponsors, including Nikon, for supplying the camera sys-
tems, and Daymen Photo, for supplying the Metz Mecablitz
units used in our lightvectoring art.
5.
GLOSSARY OF NEW TERMS
· C^
oteveillance (also known as coveillance): People watching peo-
ple, i.e. the specific aspect of sousveillance that pertains di-
rectly to peer to peer monitoring and recording of activities by
another at the same level of social hierarchy.
· Equiveillance: The equilibrium (balance) between surveillance
and sousveillance.
· Existemology (portmanteau of Existential Epistemology): An
epistemology of choice and metaphysics of freewill that typi-
cally arises from self-constructed reality intermediaries. Exam-
ples include the learning process that arises from the compu-
tationally modified sensory input of a self-constructed electric
seeing aid.
· Guerrorism (Portmanteau of Guerre and Terrorism): The un-
lawful use of authority, by a security guard, garrison, or watch,
for the purpose of intimidating civilians or citizens into support
for, or compliance with, a war, such as a "war on terrorism",
"war on crime", "war on (the) disease(d)", "war on (some)
drugs", or the like.
· Perveillance: Pervasive surveillance, typically attained through
the use of pervasive or ubiquitous computing.
· Sousveillance (undersight): (1) The recording of an activity by
a participant in the activity, typically by way of a human-borne
camera; (2) Inverse surveillance (also known as reverse surveil-
lance or inverted surveillance), i.e. the recording or monitoring
of a high ranking official by a person of lower authority.
· Sousvival: A sustained existence through low intensity peace-
fare. To actively seek ways of continued living without killing
others.
6.
REFERENCES
[1] The influence of art and design on computer science research
and development. In W. J. Mitchell, A. S. Inouye, and M. S.
Blumenthal, editors, The Influence of Art and Design on
Computer Science Research and Development, chapter 4.
National Academy of Sciences, 2003.
http://bob.nap.edu/html/beyond productivity/ch4.html.
[2] K. Aizawa, T. Hori, S. Kawasaki, and T. Ishikaw. Capture and
efficient retrieval of life log. In Proceedings of the Pervasive
2004 Workshop on Memory and Sharing of Experiences,
Linz/Vienna, Austria, Tuesday, April 20 2004. IEEE Computer
Society.
[3] W. Carr and S. Kemmis. Becoming Critical: Education,
Knowledge and Action Research. The Falmer Press, London,
1986.
[4] A. Frigo. Storing, indexing and retrieving my autobiography. In
Proceedings of the Pervasive 2004 Workshop on Memory and
Sharing of Experiences, Linz/Vienna, Austria, Tuesday, April
20 2004. IEEE Computer Society.
[5] S. Mann. Reflectionism and diffusionism. Leonardo,
http://wearcam.org/leonardo/index.htm, 31(2):93­102, 1998.
[6] S. Mann. Comparametric equations with practical applications
in quantigraphic image processing. IEEE Trans. Image Proc.,
9(8):1389­1406, August 2000. ISSN 1057-7149.
[7] S. Mann. Intelligent Image Processing. John Wiley and Sons,
November 2 2001. ISBN: 0-471-40637-6.
[8] J. Masters. The history of action research. In I. Hughes, editor,
Action Research Electronic Reader.
http://www.behs.cchs.usyd.edu.au/arow/Reader/rmasters.htm.
[9] Y. Sumi, S. Ito, T. Matsuguchi, S. Fels, and K. Mase.
Collaborative capturing and interpretation of interactions. In
Proceedings of the Pervasive 2004 Workshop on Memory and
Sharing of Experiences, Linz/Vienna, Austria, Tuesday, April
20 2004. IEEE Computer Society.
[10] S. M. (with Hal Niedzviecki). Cyborg: Digital Destiny and
Human Possibility in the Age of the Wearable Computer.
Randomhouse (Doubleday), November 6 2001. ISBN:
0-385-65825-7.
627