Louisa-thesis outline
Draft, 10th Feburary 2022. The dreamer wanders through bridging fields.
Glossary
Because I will be notoriously tossing around terms in the following text, that I tucked into my 'nice words' bag throughout my research, I want to explain what they mean (to me) below:
- magic realism: the art of making a fictional event blend into a regular conversation by making talking about as if it was a real occurence
- event horizon: The boundaries of the area in which there are interactions planted which consciously feed into the narrative development.
- pattern recognition
- mirror world
- gateways: agents of a network placed in a specific geographic location, to receive real-time, location-specific information, send it back and react accordingly.
- ambient intelligence:
- sensor networks
- XR
- smart spaces
- ubiquitous computing
- ARG
- pervasive game
- dissociation: a psychological experience of detachment from ones' environment
Research Question
How can I suggest alternate ways of urban behaviour in an industrial playground like Rotterdam, moving people to play instead of be played? How could a player's mod of the smart city look like?
Introduction
[700 words]
The dreamer wanders through bridging fields
I wake up from a loud bang. I am standing in the middle of my room. To my feet, a bottle is spilling water all over the blanket I am wrapped in. I grab the bottle, take it to the sink where I put it down and look at my face in the mirror. It takes me a moment to realise I have been sleep walking again.- October, 2021.
For the first time I am able to remember the dream I had, while moving through my room in parallel. There were blankets in my dream too, and the proportions of the floor plan roughly matched. Everything further away than my hands was rather dark and blurry. I am in the same space in a new timeline. Thanks to the maps roughly matching up, I was able to move around without walking against walls bumping my head, considering that making the bottle fall was an accident. Something clicks and I realise why, in my creative practice, I am obsessed with giving objects, places and people in real spaces a new meaning within another narrative. Subconsciously I felt these sensations for many nights as a child, and now as a strange recurrence as an adult. Do I want to go back? Or do I want to recreate this sensation in others?
There is a difference between a space and a place. A place has a fixed geographic location, but can host different spaces at the same time, in different timelines, for different people, even all at once. The way we experience our own mobility though theses places is dictated by our personal conception of the paths and boundaries within its space. These boundaries can be fluid, moving beyond the simple distinctions between and open or closed door. I was sensing these invisible borders and forces as I was moving through the city around me, but tended to games for concrete examples.
"Well, what's the problem?", "There's some kind of force field!" - Antz, 1998
The moment of reaching the boundaries of a seemingly open map destroys the immersion of the player and reveals the underlying limited grid. Game makers will try to avoid this discovery and confrontation at all costs. Well established methods of concealing the limitations of movement on the map seem to integrate a second, fictional reasoning in line with the overall narrative of the game. There is a clear link between these thought processes and methods and the real life applications made by city planners. Citizens should not feel threatened or locked in, but should be kept content by the constant illusion of an open map.
Try a little thought experiment with me: If you would build a city, or a village for a new video game, using only the places you've visited and the people you have interacted with in the last year, would it feel sufficient? All the other districts, buildings and people would vanish off your map, leaving only the bare bones off the cornerstones and pathways you have moved through.
One could argue that humans are creatures of habit, causing us to revisit the same places over and over again, without straying too far off. While this might be a very valid factor, I am convinced there is more at play. From the very blueprints of the cities to the ubiquitous computing emerging from its structures. Smart objects, surveillance cameras, movement sensors, sound recorders mediating almost every interaction and creating an omnipresence of computed control.
This leads me to consider navigating through these emerging smart spaces quite similar to being on rails in a pre-written storyline of a game, enforced by the layout of its places and spaces. In this research I am taking some steps back into the past, to the very beginning of this century, to understand the different components at play. From handheld GPS devices to sensor networks, I am laying down the bare structure of the dream of ubiquitous computing. From these bare bones, I am constructing a city game highlighting the effect of ubiquitous computing on the invisible borders of our shared map, whilst staging moments where the gameplay breaks these preconceived paths and behavioural scripts. I am interested in unveiling the electric structures interlacing real spaces, like mirror worlds built from the same spatial grid.
[2250 words]
In this chapter I am researching methods video game makers as well as city planners are commonly using to herd user movement to desired areas. How are spaces being used to manipulate user's movement and actions? How is this tactical dissociation from our environment amplified in games and how can I actively confront and challenge this phenomenon?
Living in Rotterdam for a few years, I noticed that I started moving in very strict patterns through the urban landscape. It was as if I was inhabiting a tiny village, spread around different districts. It felt almost impossible to change the paths that I travelled through. As if there was an invisible force holding me back from peeking into a side street I have never been to, or to stop and look closely at the buildings I passed by. But there wasn't, was there? I remember telling a friend that I feel as if I cannot see the corners of the rooms I am in any more. I can only see what I need to see in order to move around and do what's needed. "This is what growing up is like." They said. "I feel like I'm in a video game." I said. In video games, you seem to instinctively know where to go, what object to click on or pick up, who to fight and who to befriend. It seems as is there was an infinite amount of things to explore, places to visit and people to have meaningful dialogue with. If the game makers have done their job, you won't notice that every other room in that building was empty, that only in that main hall you stumbled into, their was an overflow of storyline presented to you. This made me think back about the small village I have lined out for myself within the city of Rotterdam. I have the constant feeling that I could go anywhere, at any time, if only I chose to.
We chase the light, the places where there are people, talking. open doors and wide streets. What makes a place seem closed? What makes it seem open? Could a place be locked for me but open for someone else? What code languages are used to communicate this selective openness? What are the signals we emit and chase after as we move through public space?
Invisible Borders in Video Games
Focussing on video game techniques to hide the map borders, integrating limits of map and unreachable areas as part of storyline.
Have you ever walked into one direction for a long time in a game, just to see if you can reach its borders? You wouldn't be the only one. There are gamers who record themselves for hours and hours on end just to capture the moment they finally cannot go any further, revealing the boundaries of the map. Video game maps are often surrounded by sea, a reason for players not to consider it as part of the playable area. But what happens when a player does decide, against all odds, to go for an extended swim? Well, in some games they might drown, in some a shark might attack, and in some a massive electric eel might penetrate their submarine. Understandably, since the depths of the ocean are not a safe place for a humans to explore, many dangers are waiting under its reflective cover. In some war games, players might get warnings about landmines, when trying to reach a part of the map that's not intended for game play, not adding to the progression of the narrative. Also understandably, apparently this is war and this can mean that there are areas that are unsafe to walk over.
player swam too far
shark circles player
player dies and sinks
.png)
Invisible Borders in Virtual Reality
In XR, geographical markers are translated into fictional environment to make sure players are not walking against walls or outside of playing field, like a wall appearing as a scary abyss.
Last September, during the media art festival RGB XYZ in Amsterdam, my sister put on VR glasses for the first time. There was an immediate cut from her and her previous environment, a rather blank room. She started moving around in excitement, calling out the things she was seeing. All of the sudden I saw her walking closely to the wall. She stopped right in front of it, looking down and saying: Woah!! It goes down so far. What she was seeing instead of the wall was an abyss. Instinctual fear kept her from daring to take any step further.
Ubiquitous Games
"a game only exists when its being played" - Alternate Reality Games and the Cusp of Digital Gameplay
A project I find particularly interesting is the location-based, hybrid catch game "Can You See Me Now?" by Blast Theory. Using GPS sensors in order to map the physical live location and movement of players to a simulation of the same space in a video game, a strange disconnect between on and offline players is being highlighted.
“A crucial feature of Blast Theory projects is the ability to extend user and audience affect outside the game - rather than delimiting our consciousness to the stereotypical and virtual, the gameplay pushes us to understand aspects of ourselves, our communities and social responsibility. This is partially achieved by the very visceral gameplay - in CYSMN? the players and gameplay self-generate affects of pursuers and pursued. ... It was encouraging to see Blast Theory awarded the Prix Ars Electronica Golden Nica, which has in the past lauded some commercial, apolitical projects.[1]”
ubiquitous games/ubiquitous extraction-everything is a potential part of the game
Ubiquitous games are seemingly omnipresent, as they make use of various media, which are part of the player's everyday reality. By blending the storyline seamlessly along multiple channels, the player is in a constant state of wonder. This leads us to the following chapter, in which I will draw parallels between ubiquitous games and ubiquitous computing, not only in terms of applied technologies and structure, but as a psychological experience of omnipresence.
Chapter 2: Take a Leap in the Dark
[2250 words] keywords: playbour, smart city, recursive hierarchies
Encased in concrete and metal armour, lie the keys to the pulsating veins of a mirror world. Whatever makes it inside its bloodstream, will travel fast and eternally through the nexus of time and space it occupies in that instant. When talking about a feeling of constantly being watched and tracked, it is tempting to go down the route of discussing privacy invasion of big tech companies and governments, which give users of mainstream media a constant feeling of paranoia to be caught misbehaving. My focus point lies somewhere in this ballpark, but not quite there, which is why I will avoid going into depth on this well discussed topic. I am after something which I would describe as the potential of mirror world networks for augmenting the perception of real spaces. Not as in the usual vision of augmented reality, through glasses or phones revealing a fictive visible layer on top of real spaces. I am talking about worlds within worlds, recursive hierarchies unravelling narratives which rest on shared geographical markers.
In order to stalk down this phenomenon and pin it down, I will start by tracing the very bricks which hold the city together. I will then chase down the power lines woven amidst, all the way to its the whirring epicenter. By understanding the early visions of ambient spaces, laid out by research groups like the one's being 'Project Oxygen' by MIT, I aim to grasp the organism in its complexity and understand at which point of its evolution we find ourselves today. By laying the bones bare, and identifying the various body parts, I will discuss in what way these pathways lend themselves to be used and abused for ludification.
I am floating
I am moving
I am not inside my body
Blueprints of Control
Subtle methods of herding user's movement is of course not a digital-born phenomenon. Making sure that people stay on designated areas and behave in pre-scripted ways is applied to the very blueprints of the city that its buildings and social encounters emerge from and has so for centuries. In the early past, surveillance and tracking has been able to spill into every corner of the urban landscapes, from parking lots to lamp posts, justified in the name of safety, tolerated only in light of escalation. Video surveillance has first been implemented in Manchester during football games in the 80s, since the hooligans were hard to control. Police was trained to read behaviour always expecting escalation, making it impossible for people to act different than the before mentioned location-based (and time based) scripts.
In this chapter I want to make a connection between ubiquitous games and the surveillance capitalism applied in smart city technology, both creating a sensation of unknowing where the boundaries of interaction with the systems in place start and stop. As Prof. Soshana Zuboff lays out in "The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power", surveillance capitalism is a new phenomenon bred on the fertile grounds of individualization, emerging in a second wave of modernity. The needs and desires which come along with self-determination of digital content streams result in an unprecedented, unhinged wave of a new mutation of capitalism. Surveillance capitalism is supported by widespread digital infrastructures, connected in a web of personal computers, phones but in also increasing network of an internet of things governing and mediating virtually any human interaction. The new stage for this is the city itself, no longer confining the manipulation of user movement and behaviour to online platforms, but the infrastructures of the cities around the screens.
Surveillance Capitalism is spilling into the city streets, leading you to where the algorithm thinks you should be going. How will companies make use of playbour, keeping themselves and the players happy? How can we appropriate these methods for play without labour and make them our own? What could mods of the 'smart city' look like?
"Imagine these supply chains growing and becoming more elaborate as interfaces are invented for nearly every aspect of human experience. This began in the online world, with our searching, our browsing, our social interactions and so forth, but these interfaces now extend to every aspect of human activity. Now we carry a little computer in our pocket. We’re out in the world, subjected to many more devices, sensors and cameras. Every “smart” product and “personalized” service is a supply chain interface extracting everything from our location and movements to our voices, conversations, gaze, gait, posture, faces, who we are with, what we’re buying, what we’re eating, where we’re going, what we’re doing in our homes, what we’re doing in our automobiles — all of these dimensions of human experience are now exposed to these interfaces that claim and data-fy our lives." - https://www.cigionline.org/articles/shoshana-zuboff-undetectable-indecipherable-world-surveillance-capitalism/
"Eventually surveillance capitalists discovered that the best source of predictive data is to actually intervene in people’s behaviour and shape it. This is what I call “economies of action.” It means that surveillance capital must commandeer the increasingly ubiquitous digital infrastructure that saturates our lives, in order to remotely influence, manipulate and modify our behaviour in the direction of its preferred commercial outcomes.[...] Now the same digital architecture used for monitoring becomes the means of behaviour modification with programmed triggers, subliminal cues, rewards, punishments, social comparison dynamics – all of it aimed at tuning and herding human behaviour in the direction that aligns with the commercial goals of business customers."
Players Locked in The Game
"economies of action" "Pokémon Go was spun off from Google as Niantic Labs with Hanke (google mapping expert) as its head. It presented itself to the world as if it was some clever start-up, though Google remained its primary investor. The game brought Google’s experimentation on tuning and herding populations to the real world, to real streets, real towns, real cities, engaging families and friends in a search game without revealing that it was really the participants who were being searched. Game players were actually pawns in a larger, higher-order game aimed at providing “footfall” to establishments that paid Niantic Labs a fee for consumer traffic in exactly the same way that online advertisers pay for click-through behaviour. Niantic used gamification, with its rewards and punishments, to herd people to its real-world business customers, from McDonald’s and Starbucks to Joe’s Pizza." - Soshana Zuboff for https://www.cigionline.org Shoshana Zuboff on the Undetectable, Indecipherable World of Surveillance Capitalism
Ambient Intelligence
"The memory represents space in this world, and the processor time." (David Hillel Gelernter, 1992)
what kind of technology is being used to move people around in the city?
Let's have a look at the technology in place. What structures are covering the city streets, how do they sense changes in the environment, where does the information go and how is it fed back to the reality of the city?
During my research into augmented urban spaces, I started reading about Ambient Intelligence in relation to ubiquitous computing. I found out about "Project Oxygen", which seems to have been written around 2002, which is a project by a research group an MIT in collaboration(?) with google which lays out the concept for a omnipresent, invisible technological system:
"Oxygen enables pervasive, human-centered computing through a combination of specific user and system technologies. Oxygen's user technologies directly address human needs. Speech and vision technologies enable us to communicate with Oxygen as if we're interacting with another person, saving much time and effort. Automation, individualized knowledge access, and collaboration technologies help us perform a wide variety of tasks that we want to do in the ways we like to do them.
Oxygen's device, network, and software technologies dramatically extend our range by delivering user technologies to us at home, at work or on the go. Computational devices, called Enviro21s (E21s), embedded in our homes, offices, and cars sense and affect our immediate environment. Handheld devices, called Handy21s (H21s), empower us to communicate and compute no matter where we are. Dynamic, self-configuring networks (N21s) help our machines locate each other as well as the people, services, and resources we want to reach. Software that adapts to changes in the environment or in user requirements (O2S) help us do what we want when we want to do it."
I found this proposal interesting as it somehow seemed to bring a lot of the scattered mechanics, like internet of things, sensor networks, predictive intelligences, etc together.
"Rotterdam Maasvlakte
SkyLab has expanded the public Internet of Things network in the Port of Rotterdam with a loRaWAN® Gateway on the Uniper Benelux, Rotterdam. The gateway is a Dutch record, with a height of 130 meters from ground level. The gateway is connected to the largest and fastest growing IoT community network in the world, The Things Network & The Things Industries. This gateway can receive sensor information in a radius of 15 km and covers a large area of about 700 km2 or more for outdoor sensors. It provides a large part of the Rotterdam Maasvlakte, North Sea, Hoek van Holland, Zeeland and the “Havenspoorlijn”."
https://www.skylab.nl/gateways/
“The Internet of Things use-cases for this CENT-R network are myriad, ranging from city-light management to asset tracking to measuring climate data,” said Remy de Jong Sr., CEO of SkyLab. “With the right sensors connected to this LoRaWAN network through Kerlink gateways, virtually every IoT use case is possible.”
https://www.thethingsnetwork.org/ LoRaWAN
SEMTECH https://www.semtech.com/ "Semtech is a leading global supplier of high performance analogue and mixed-signal semiconductors and advanced algorithms for infrastructure, high-end consumer and industrial equipment.
Through our advanced technology platforms, Semtech is enabling mission critical applications for three of the industry's fastest growing markets: Internet of Things, Data Centers and Mobility."
https://wyldnetworks.com/what-is-a-lorawan-gateway/ https://www.kerlink.com/product/wanesy-wave/
It seems like the next step in the development of smart city devices is connecting them all in one big network, like The Things Network, where all incoming data can be processed and monitored. I do not mean to paint these developments as a purely negative process, as there are many positive outcomes of this self regulating smart devices, adjusting immediately and efficiently to changes in their environment. Smart light poles can adjust the brightness according to the environment, saving energy. A lot of these tools are intended to foster energy saving and circular solutions. But the privacy concerns of such a vast amount of easily updateable computers, determining the movement flow and civil behaviour are to be taken seriously. Let's take Sidewalk Lab's proposal 'smart city' for Toronto as an example. The google funded company produces devices which can be attached and integrated into public space.
They hired privacy expert Ann Cavoukian to deal with the big privacy questions in this design proposal, who eventually resigned because the company rejected their proposal for privacy by design, stripping any collected footage from personal information, f.e. blurring faces immediately. With this proposal, there would have still been plenty of data to process, but no individual could be traced back. It is very worrying that this was a deal breaker for the company.
https://www.youtube.com/watch?v=1t12UqYl5SA
Sensor Network
Locking/Unlocking
Geofencing
GPS Tracking
It seems inevitable to consider the GPS, or Global Positioning System, which assigns coordinates to its scattered modems, as they move over the shared map. This technology which has initially been built for the U.S. Army turned out to be a helpful navigational tool for everyday uses.
In terms of technological infrastructure, on the one hand there is a space of ubiquitous computing, consisting of ―interlinked capacities for memory and data storage, for perception and environmental sensing, and for the interpretation of contexts and situations‖ (Phillips and Wiegerling, 2007: 05). But on the other hand there is an explosion of mobile and locative media which are now part of a technological network of signs produced and accessed globally that, at the same time, allow a precise and instant positioning within geographical places and territories which, in turn, could lead to a meaningful link with specific locations. - Rodrigo Firmino and Fábio Duarte: Manifestations and implications of an augmented urban life Researching this method and political motivation behind its application.
--
Geofencing is a software feature used to draw geographic boundaries based on GPS or RFID information. Based on whether you are in or outside of these boundaries is decisive for what functions are available and which content is visible within an application or other service. This virtual space is mapped onto the physical landscape. The main motivation for geofencing seems to be security, as its being used in security tags in clothes in shops and for car sharing services. But geofencing is not only used to prevent theft, it is also used in order to target voters in political campaigns. Geofences can be drawn around a single building, a neighbourhood or an entire state in order to target people with relevant content in order to stir them to vote for one party instead of another.
We can no longer trace back if we want something because of smart marketing or because of our own free choice.
--
Geocaching is a worldwide, ever-expanding treasure hunt in which anyone can hide a cache and publish its geo-location via specific coordinates. Players nearby will then search for it and log their name once found. This mechanism is often used by locals to lead strangers to interesting sights of the area.
Chapter 3: The Game
[2250 words]
I was challenged to think of games/events/activities which can exist during times of a rather strict lockdown and closed cultural venues. This led me to work on location-based games, to unlock audio-pieces and later on even a secret concert venue. Thinking of puzzles and progressive narratives and then seeing people interact with them is incredibly rewarding and feels like a powerful tool of capturing peoples attention, even if for just a little while. My aim is to activate people and change their state of mind and experience of their environment. Making use of the group present and the things everyone could contribute to the experience, instead of making them passive spectators of a work, is very valuable to me. During this research I have tried geocaching, an internation treasure-hunt of hidden name logs in public spaces. Looking closely, everything could be a potential hint. In parallel, I am building on my research on interactive fiction and targeted advertisement.
Which elements can i use, what platforms and engines would make sense? In this chapter I will document the process of building a location-based game, reviewing tests and adapting design choices. Am I choosing the right locations and methods in order to support the narrative?
game mechanics:
a series of gateways 1 way mirror hiding means losing predicting players movement using smart objects to trace progress coordinates game is using secret code to hide in plain sight
Forming a progressive storyline
How could geographical markers like coordinates connect alternate, fictive spaces? Playing the game should result in subtle changes in public spaces of city and the game should adapt to the way its being played.
Creating a visual language and communication methods
Play Tests
Documentation and analysis.
Development Week TETEM
March
in-game
Documentation of actual Gameplay.
post-game
Analysis. I am planning to review the different game methods I have tested and applied and in what ways they have supported the progression of the narrative through lived experience. Based on the results of this analysis I will suggest areas of improvement and propose a future set-up of the game.
Conclusion
[700 words]
Based on my research on methods of location-based advertisement, I will formulate my vision of its potential and dangers, picturing a ludified smart city and celebrating ways of dancing on the structures of its system.
Bibliography
Books & Articles
Huizinga, Johan (1955). Homo Ludens: a Study of the Play-Element in Culture. Boston, MA: The Beacon Press.
Gelernter, D. (1992). Mirror worlds, or, The day software puts the universe in a shoebox ... : how it will happen and what it will mean. New York: Oxford University Press.
Turkle, S. (1996). Life on the Screen: Identity in the Age of the Internet. London: Weidenfeld & Nicholson.
Aarseth, E.J. (1997). Cybertext: Perspectives on Ergodic Literature. Baltimore, Md.: Johns Hopkins University Press.
Towell, J. & Towell, E. (1997). Presence in Text-Based Networked Virtual Environments or “MUDS”. Massachusetts: Massachusetts Institute of Technology.
Szulborski, D. (2005). This is Not a Game: A Guide to Alternate Reality Gaming. Pennsylvania: New-Fiction Pub.
von Borries, F. & Walz, S.P & Boettger, M. (2007). Space Time Play: Computer Games, Architecture and Urbanism: The Next Level. Switzerland, Basel: Birkhaeuser Verlag AG.
Eyal, N & Hoover, R. (2014). Hooked: How to Build Habit-Forming Products. Penguin Books Ltd.
Zuboff, S. (2019). The Age of Surveillance Capitalism: The Fight for the Future at the New Frontier of Power. London: Profile Books.
Hoffman, D.D. (2019). The Case Against Reality: How Evolution Hid the Truth From Our Eyes. United States of America: W.W. Norton & Company.
Web Articles
[1] Blast Theory. (n.d.). Can You See Me Now? [online] Available at: https://www.blasttheory.co.uk/projects/can-you-see-me-now/.
from, D. (2019). What is geo-fencing (geofencing)? - Definition from WhatIs.com. [online] WhatIs.com. Available at: https://whatis.techtarget.com/definition/geofencing.
Games
Blast Theory. (2003). Can you see me now?. Rotterdam, The Netherlands.
Kihara, T. (2018). Escape the Smart City. Amsterdam, The Netherlands.
Groothuizen, R. (2021). I want to delete it all, but not now. Enschede, The Netherlands.
Affect Lab (2020). Good Neighbours. Amsterdam, The Netherlands.
Appendix
Game Narrative
I would like to attach the written narrative of the game I built here for further reading.