Martin (XPUB)-project proposal: Difference between revisions

From XPUB & Lens-Based wiki
 
(44 intermediate revisions by the same user not shown)
Line 1: Line 1:
<div style='  
<div style='  
width: 50%;   
width: 100%;   
font-size:14px;
font-size:16px;
background-color: white;
background-color: white;
color:black;
color:black;
Line 9: Line 9:
'>
'>


==<p style="font-family:helvetica">Graduate proposal guidelines</p>==


==<p style="font-family:helvetica">Graduate proposal guidelines</p>==
===What do you want to make?===


===<p style="font-family:helvetica">What do you want to make?</p>===
My project is a data collection installation that monitors people's behaviors in public physical spaces while explicitly encouraging them to help the algorithm collect more information. An overview of how it works is presented here in the project proposal and will be subject to further developments in the practice.
<br><br>
The way the device is designed doesn’t pretend to give any beneficial outcomes for the subject, but only makes visible the benefits that the machine is getting from collecting their data. Yet, the way the device visually or verbally presents this collected data is done in a grateful way, which might be stimulating for the subject. In that sense, the subject, despite knowing that their actions are done solely to satisfy the device, could become intrigued, involved, or even addicted by a mechanism that deliberately uses it as a commodity. In that way, I intend to trigger conflictual feelings in the visitor’s mind, situated between a state of awareness regarding the operating monetization of their physical behaviors, and a state of engagement/entertainment /stimulation regarding the interactive value of the installation.
<br><br>
My first desire is to make the mechanisms by which data collection is carried out, marketized and legitimized both understandable and accessible. The array of sensors, the Arduinos and the screen are the mainly technological components of this installation. Rather than using an already existing and complex tracking algorithm, the program is built from scratch, kept open source and limits itself to the conversion of a restricted range of physical actions into interactions. These include the detection of movements, positions, lapse of time spent standing still or moving, and entry or exit from a specific area of detection. Optionally they may also include the detection of the subject smartphone device or the log on a local Wi-Fi hotspot made by the subject.
<br><br>
In terms of mechanic, the algorithm creates feedback loops starting from: <br>
_the subject behaviors being converted into information; <br>
_the translation of this information into written/visual feedback; <br>
_and the effects of this feedbacks on subject’s behavior; and so on. <br>
By doing so, it tries to shape the visitors as free data providers inside their own physical environment, and stimulate their engagement by converting each piece of collected information into points/money, feeding a user score among a global ranking.
<br><br>
On the screen, displayed events can be:


I want to <b>explore the key roles of digital and physical architectures in the representation and transmission of knowledge</b> by merging them under the form of a <b>physical exhibition interface</b>. To be more specific, I wish to build an exhibition space inspired from the <b>elastic (plastic)</b> display/render of online contents</b> that mostly adapt to the user/viewers perspective. In that sense, I want to build an exhibiting device willing to <b>put the spectator in the position of being the curator and user of it's own physical exhiibition space</b> and <b>allow a single representation/content/artwork to be displayed under a wide range of possible settings</b>. Comparably as a pointer or a cursor, the spectator/user can move inside the space (SpectatorX/SpectatorY = MouseX/MouseY) and resize this same space (SpaceWidth & SpaceHeight = WindowWidth & Windowheight) by pushing or pulling a movable wall fixed on some rails/wheels. The number of spectators, its/their position inside the space, as well as the interactions engaged with it will affect various display factors</b> such as the lighting, sound, projection format, information layout, etc. Such interactions could also <b>give space to some accidents, unforeseen, bugs, deadzones and glitches</b> that might have been intentionaly left there). This is an attempt to <b>speculate on how the cybernetics(?) affects alls aspect of our lifes and could also transform the exhibition space, the curatiorial practice, our experience of art and the nature of representation itself</b>.
[[File:Simu part 01.gif|right|800px|thumb|https://www.dropbox.com/s/zyv3c2ypit8qo1y/Simulation_MartinFoucaut.mp4?dl=0 full HD video]]
[[File:Simu part 02.gif|right|800px|thumb]]
[[File:Simu part 03.gif|right|800px|thumb]]


* I am still questionning if this space should be completely empty of information or not. In my mind, inside the space should be displayed with the help of one or several beamer(s)) (or screens) the live values of the variable exhibition space (width/lenght; spectator position; luminosity; information font; sound frequency, etc), in order to enhance the idea of being inside a exhibiting device. The way these values are displayed (format, font, layout) should also change depending on the values themselves (I will make some sketches about that).
_ “subject [] currently located at [ ]” <br>
[x points earned/given]<br>
_ “subject [] entered the space” <br>
[x points earned/given]<br>
_ “subject [] left the space”<br>
[x points earned/given]<br>
_ “subject [] moving/not moving”<br>
[x points earned/given]<br>
_ “subject [] distance to screen: [ ] cm” <br>
[x points earned/given]<br>
_ “subject [] stayed at [ ] since [ ] seconds” <br>
[x points earned/given]<br>
_ “subject [] device detected <br>
[x points earned/given] (optional)<br>
_ “subject logged onto local Wi-Fi<br>
[x points earned/given] (optional)<br>
<br>
Added to that comes the instructions and comments from the devices in reaction to the subject’s behaviors:<br>
<br>
_ “Congratulations, you have now given the monitor 25 % of all possible data to collect!” <br>
[when 25-50-75-90-95-96-97-98-99% of the total array of events has been detected at least once]<br>
_ “Are you sure you don’t want to move to the left? The monitor has only collected data from 3 visitors so far in this spot!”<br>
[if the subject stands still in a specific location]<br>
_ “Congratulations, the monitor has reached 1000 pieces of information from you!”<br>
[unlocked at x points earned/given]<br>
_ “If you stay there for two more minutes, there is a 99% chance you will be in the top 100 of ALL TIME data-givers!”<br>
[if the subject stand still in a specific location]<br>
_ “Leaving all ready? The monitor has yet to collect 304759 crucial pieces of information from you!”<br>
[if the subject is a the edge of the detection range]<br>
_ “You are only 93860 pieces of information away from being the top one data-giver!”<br>
[unlocked at x points earned/given]<br>
_ “Statistics show that people staying for more than 5 minutes average will benefit me on average 10 times more!”<br>
[randomly appears]<br>
_ “The longer you stay on this spot, the more chance you have to win a “Lazy data-giver” badge”<br>
[if the subject stands still for a long time any location]<br>
<br>
Responding positively to the monitors instructions unlocks special achievement and extra points<br>
<br>
—Accidental data-giver badge <br>
[unlocked if the subject has passed the facility without deliberately wishing to interact with it] + [x points earned/given]<br>
—Lazy data-giver badge <br>
[unlocked if the subject has been standing still for at least one minute] + [x points earned/given]<br>
—Novice data-giver badge <br>
[unlocked if the subject has been successfully completing 5 missions from the monitor] + [x points earned/given]<br>
—Hyperactive data-giver badge <br>
[unlocked if the subject has never been standing still for 10 seconds within 2 minutes lapse time] + [x points earned/given]<br>
—Expert data-giver badge <br>
[unlocked if the subject has been successfully completing 10 missions from the monitor within 10 minutes] + [x points earned/given]<br>
—Master data-giver badge <br>
[unlocked if the subject has been successfully logging on the local Wi-Fi hotspot] + [x points earned/given] (optional)<br>
<br>
On the top left side of the screen, a user score displays the number of points generated by the collected pieces of information, and the unlocking of special achievements instructed by the monitor.<br>
<br>
—Given information: 298 pieces <br>
[displays number of collected events]<br>
——$ collected from your data: 1.312$ <br>
[conversion of collected events and achievement into points]<br>
<br>
On the top right of the screen, the user is ranked among x number of previous visitors and the prestigious badge recently earned is displayed bellow<br>
<br>
—subject global ranking: 3/42 <br>
[compares subject’s score to all final scores from previous subjects]<br>
—subject status:  expert data-giver<br>
[display the most valuable reward unlocked by the subject]<br>
<br>
When leaving the detection range, the subject gets a warning message and a countdown starts, and encouraging it to take the quick decision to come back<br>
<br>
—“Are you sure you want to leave? You have 5-4-3-2-1-0 seconds to come back within the detection range”<br>
[displayed as long as the subject remains completely undetected]<br>
<br>
If the subject definitely stands out of the detection range for more than 5 seconds, the monitor will also address a thankful message and the amount of money gathered, achievements, ranking, complete list of collected information and a qr code will be printed as a receipt with the help of a thermal printer. The QR will be a link to my thesis.<br>
<br>
—* “Thank you for helping today, don’t forget to take your receipt in order to collect and resume your achievements”<br>
[displayed after 5 seconds being undetected]<br>
<br>
In order to collect, read or/and use that piece of information, the visitor will inevitably have to come back within the range of detection, and intentionally, or not, reactivate the data tracking game. It is therefore impossible to leave the area of detection without leaving at least one piece of your own information printed in the space. Because of this, the physical space should gradually be invaded by tickets scattered on the floor. As in archaeology, these tickets give a precise trace of the behavior and actions of previous subjects for future subjects. <br>


===<p style="font-family:helvetica">How do you plan to make it?</p>===
===Why do you want to make it?===


While working with <b>Arduino Mega</b> and <b>Rasperry Pi</b>, my aim is to start <b>from the smallest and most simple prototype</b>, <b>gradually increase its scale/technicality until reaching human scale</b> and getting closer from emulating the properties of a Web window (see: [[XPUB2_Research_Board_/_Martin_Foucaut#Prototyping|prototyping]]). Once an <b>exhibtion space will be found/determined</b> for the graduation, I will  <b>build a custom mobile wall</b>, or use a preexisting one, add some handles to it, and fix it on a rail system that will allow to reduce or expend the size of a space from in between a maximum and a minimum range. This wall will include on on the back at least one sensor that will help to determine the size of the room in real time. (see [[XPUB2_Research_Board_/_Martin_Foucaut#Creating_an_elastic_exhibition_space|schema]]). With the help of an array of sensors putted on the 3 static walls of the exhibition space, the space will be mapped into a grid, and will allow to know the real-time position of the spectator(s) inside of it.  
When browsing online or/and using connected devices in the physical world, even the most innocent action/information can be invisibly recorded, valued and translated into informational units, subsequently generating profit for monopolistic companies. While social platforms, brands, public institutions and governments explicitly promote the use of monitoring practices in order to better serve or protect us, we could also consider these techniques as implicitly political, playing around some dynamics of visibility and invisibility in order to assert new forms of power over targeted audiences.  
<br><br>
<br><br>
In order to better <b>define my audience, the issues, and the direction of this work, I will take advantages of the different venues organized by XPUB</b> to involve people into <b>testing and reflecting on various prototypes of this work in progress.</b> (see [[XPUB2_Research_Board_/_Martin_Foucaut#Venues|venue1]])
In the last decade, a strong mistrust of new technologies has formed in the public opinion, fueled by events such as the revelations of Edward Snowden, the Cambridge Analytica scandal or the proliferation of fake news on social networks. We have also seen many artists take up the subject, sometimes with activist purposes. But even if a small number of citizens have begun to consider the social and political issues related to mass surveillance, and some individuals/groups/governments/associations have taken legal actions, surveillance capitalism still remains generally accepted, often because ignored or/and misunderstood.
(see: [[XPUB2_Research_Board_/_Martin_Foucaut#Sketch_3:_Arduino_Uno_.2B_Sensor_.2B_LCD_.2B_2_LED.29_.3D_Physical_vs_Digital_Range_detector|simulation]])
<br><br>
Thanks to the huge profits generated by the data that we freely provide every day, big tech companies have been earning billions of dollars over the sale of our personal information. With that money, they could also further develop deep machine learning programs, powerful recommendation systems, and to broadly expand their range of services in order to track us in all circumstances and secure their monopolistic status. Even if we might consider this realm specific to the online world, we have seen a gradual involvement from the same companies to monitor the physical world and our human existences in a wide array of contexts. For example, with satellite and street photography (Google Earth, Street View), geo localization systems, simulated three-dimensional environments (augmented reality, virtual reality or metaverse) or extensions of our brains and bodies (vocal assistance and wearable devices). Ultimately, this reality has seen the emergence of not only a culture of surveillance but also of self-surveillance, as evidenced by the popularity of self-tracking and data sharing apps, which legitimize and encourage the datafication of the body for capitalistic purposes.
<br><br>
For the last 15 years, self-tracking tools have made their way to consumers. I believe that this trend is showing how ambiguous our relationship can be with tools that allow such practices. Through my work, I do not wish to position myself as a whistleblower, a teacher or activist. Indeed, to adopt such positions would be hypocritical, given my daily use of tools and platforms that resort to mass surveillance. Instead, I wish to propose an experience that highlights the contradictions in which you and I, internet users and human beings, can find ourselves. This contradiction is characterized by a paradox between our state of concern about the intrusive surveillance practices operated by the Web giants (and their effects on societies and humans) and a state of entertainment or even active engagement with the tools/platforms through which this surveillance is operated/allowed. By doing so, I want to ask how do these companies still manage to get our consent and what human biases do they exploit in order to do so. That’s is how my graduation work and my thesis will investigate the effect of gamification, gambling or reward systems as well as a the esthetization of data/self-data as means to hook our attention, create always more interactions and orientate our behaviors.


[[File:SensorSpace.gif|200px|thumb|left|Sensor Test VS Elastic Space<br><b>Click to watch</b>]]
[[File:Sensor Wall 01.png|200px|thumb|right|Sensor Wall 01]]
[[File:SensorMediaQueries 01.gif|200px|thumb|center|SensorMediaQueries<br><b>Click to watch</b>]]
<br><br><br>


===<p style="font-family:helvetica">What is your timetable?</p>===
===How to you plan to make it and on what timetable?===


* <b>1st semester</b> Prototyping with Arduino all long, getting started with Raspery, and finding a space to set up
I am developing this project with Arduino Uno/Mega boards, an array of ultrasonic sensor, P5.js and screens.<br><br>
# 1st prototype: mini arduio + light sensor (<b>understanding arduino basics</b>) [[https://pzwiki.wdka.nl/mediadesign/XPUB2_Research_Board_/_Martin_Foucaut#Prototype_1_:_Arduino_.2B_Resistor|link]]
# 2nd prototype: arduino uno + utlrasonic sensor (<b>working with ultrasonic sensors</b>) [[https://pzwiki.wdka.nl/mediadesign/XPUB2_Research_Board_/_Martin_Foucaut#Prototype_2:_Arduino_.2B_Ultrasonic_sensor|link]]
# 3rd prototype: arduino uno + utlrasonic sensor + LCD screen (<b>working with values display</b>) [[https://pzwiki.wdka.nl/mediadesign/XPUB2_Research_Board_/_Martin_Foucaut#Prototype_3:_Arduino_Uno_.2B_Sensor_.2B_LCD_.28.2B_LED.29|link]]
# 4th prototype: arduino uno + utlrasonic sensor + 2 LEDS (<b>working with in-between distance range values detection</b>) [[https://pzwiki.wdka.nl/mediadesign/XPUB2_Research_Board_/_Martin_Foucaut#Prototype_4:_Arduino_Uno_.2B_Sensor_.2B_LCD_.2B_2_LED_.3D_Physical_vs_Digital_Range_detector|link]]
# 5th prototype: arduino uno + 3 utlrasonic sensor + 3 LEDS (<b>mapping range values detection in a grid and attributing signals with LEDS</b>) [[https://pzwiki.wdka.nl/mediadesign/XPUB2_Research_Board_/_Martin_Foucaut#Prototype_5:_Arduino_Uno_.2B_3_Sensor_.2B_3_LEDS|link]]
# 6th prototype: arduino uno + 3 utlrasonic sensor + 12 LEDS (<b>mapping range values detection in a grid and attributing signals with more LEDS</b>) [[https://pzwiki.wdka.nl/mediadesign/XPUB2_Research_Board_/_Martin_Foucaut#Prototype_6:_Arduino_Uno_.2B_3_Sensor_.2B_12_LEDS|link]]
# 7th prototype: arduino uno + 3 utlrasonic sensor + 1 buzzer + 1 LCD + 1 Potentiometer (<b>adding audio signals to the range value detection</b>) [[https://pzwiki.wdka.nl/mediadesign/XPUB2_Research_Board_/_Martin_Foucaut#Prototype_7:_Arduino_Uno_.2B_12_LEDS_.2B_3_Sensor_.2B_Buzzer_.2B_Potentiometer_.2B_LCDS|link]]
# 8th prototype: arduino uno + 3 utlrasonic sensor + 1 buzzer + 1 LCD + 1 Potentiometer + mini breadboard (<b>separating sensors from each others</b>) [[https://pzwiki.wdka.nl/mediadesign/XPUB2_Research_Board_/_Martin_Foucaut#Prototype_8:_Arduino_Uno_.2B_12_LEDS_.2B_3_Sensor_on_mini_breadboards_.2B_Buzzer_.2B_Potentiometer_.2B_LCD|link]]
# 9th prototype: arduino Uno + 21 LEDS + 7 Sensor + Buzzer + Potentiometer + LCD <b>expending the prototype to human scale</b>
——————————— NOW —————————————————————————————————————————
# Upcoming: arduino uno + 3 utlrasonic sensor + 5V Relay + Lamp (<b>controlling a lamp with arduino</b>)
# Upcoming: arduino uno + 3 utlrasonic sensor + ESP8266 (WIFI) + Rasperry Pi (Self hosted website) (<b>transmit or/and control value from arduino to computer and vice versa via WIFI transmitter</b>)
# Upcoming: small room + arduino uno + 8 utlrasonic sensor + ESP8266 (WIFI) + Rasperry Pi (connect to a Web page)


 
How does it work?
* <b>2nd semester</b>: Find what will be graduation space, build the mobile wall, and translate the setup installation to human/spectator scale.  
<br><br>
# Show prototype and schemas of the wall to wood and metal workshops in order to get advices until final validation to build (<b>starting to build physical elements</b>)
The ultrasonic sensors can detect obstacles in a physical space and know the distance between the sensor and obstacle(s) by sending and receiving back an ultrasound. The Arduino Uno/Mega boards are microcontrollers which can receive this information, run it in a program in order to convert these values into a mm/cm/m but also map the space into an invisible grid. Ultimately, values collected on the Arduino’s serial monitor can be sent to P5.js through P5.serialcontrol. P5.js will then allow a greater freedom in way the information can be displayed on the screens.
# Search, find and validate what will be the space used for the installation during the graduation.
<br><br>
# Start building of the movable wall by considering the characteristic of the space used for graduation.
Process:
# Implement the sensors inside the movable wall, and the other devices in the fixed space
<br><br>
 
1st semester: Building a monitoring device, converting human actions into events, and events into visual feedbacks
===<p style="font-family:helvetica">Why do you want to make it?</p>===
<br><br>
 
During the first semester, I am focused on exploring monitoring tools that can be used in the physical world, with a specific attention to ultrasonic sensors. Being new to Arduino programming, my way of working is to start from the smallest and most simple prototype and gradually increase its scale/technicality until reaching human/architectural scale. Prototypes are subject to testing, documentation and comments helping to define which direction to follow. The first semester also allows to experiment with different kind of screen (LCD screens, Touch screens, computer screens, TV screens) until finding the most adequate screen monitor(s) for the final installation. Before building the installation, the project is subject to several sketching and animated simulations in 3 dimensions, exploring different scenarios and narrations. At the end of the semester, the goal is to be able to convert a specific range of human actions into events and visual feedback creating a feedback loop from the human behaviors being converted into information; the translation of this information into written/visual feedbacks; and the effects of this feedbacks on human behavior; and so on.
In opposition to the physical exhibition space, <b>the Web offers to each of its user/visitors a custom point of view</b> based on an <b>innumerable and everchanging array of technological factors</b>. I like to call this the technological context. Among these, we could list: the browser, the device, the explotation system, the screensize, the resolution, the user configuration and defaults settings, the updates, the IP adress, etc.. This technological complexity <b>diffracts the possible renders of a same Web page in an almost infinite array of user perspectives</b>. Therefore, <b>this is the nature and meaning of representation itself that is redifined by the Web</b>. Web representations are sort of <b>plastic/elastic</b>, they <b>demultiplies</b> and <b>transforms themselves</b> as much as needed in order to be <b>rendered in an optimal way through our own user perspective/interface</b>. By implementing these notions and properties into the phyiscal exhibition spaces, I would like to <b>put a step in the curatorial practice</b>.
<br><br>
2nd semester: Implementing gamification with the help of collaborative filtering, point system and ranking.
<br><br>
During the second semester, it is all about building and implementing a narration with the help of gaming mechanics that will encourage humans to feed the data gathering device with their own help. An overview of how it works is presented here in the project proposal and will be subject to further developments in the practice.
<br><br>
To summarize the storyline, the subject being positioned in the detection zone finds herself/himself unwillingly embodied as the main actor of a data collection game. Her/His mere presence generates a number of points/dollars displayed on a screen, growing as she/he stays within the area. The goal is simple: to get a higher score/rank and unlock achievements by acting as recommended by a data-collector. This can be done by setting clear goals/rewards to the subject, and putting its own performance in comparison to all the previous visitors, giving unexpected messages/rewards, and give an aesthetic value to the displayed informations.  
<br><br>
The mechanism is based on a sample of physical events that have been already explored in the first semester of prototyping (detection of movements, positions, lapse of time spent standing still or moving, and entry or exit from a specific area of detection). Every single detected event in this installation is stored in a data bank, and with the help of collaborative filtering, will  allow to the display of custom recommendations such as:
<br><br>
_ “Congratulations, you have now given the monitor 12 % of all possible data to collect”<br>
_ “Are you sure you don’t want to move to the left? The monitor has only collected data from 3 visitors so far in this spot”<br>
_ “Congratulations, the monitor has reached 1000 pieces of information from you!”<br>
_ “If you stay there for two more minutes, there is a 99% chance you will be in the top 100 of ALL TIME data-givers”<br>
_ “Leaving all-ready? The monitor has yet 304759 crucial pieces of information to collect from you”<br>
_ “You are only 93860 actions away from being the top one data-giver”<br>
_ “Statistics are showing that people staying for more than 5 minutes average will be 10 times more benefitting for me”<br>
_ “The longer you stay on this spot, the more chance you have to win a “Lazy data-giver” badge”<br>
<br><br>
<br><br>
From our own user perspective/point of view, behind our own screen, <b>this technological complexity and the infinite spectrum of perspectives that it leads to can hardly be considered</b> (expect [http://whatyouseeiswhatyouget.net/ here] for example). This brings us to <b>uncounsioulsy forget about the singularity and fragility of what we is being seen/experienced/interpretated</b>. By creating a physical interface conceived on the model of a responsive Web page, I want to give to the visitors the power to <b>manipulate and diffract this spectrum of perspectives by their own hands</b> and <b>to consider the role and effects of these mediating technologies on the visitor's behaviors and perception of an artwork/content.</b>
The guideline is set out here, but will be constantly updated with the help of experiments and the results observed during the various moments of interaction between the students and the algorithm. For this purpose, the installation under construction will be left active and autonomous in its place of conception (studio) and will allow anyone who deliberately wishes to interact with it to do so. Beyond the voluntary interactions, my interest is also to see what can be extracted from people simply passing in front of this installation. In addition to this, some of the mechanics of the installation will be further explored by collaborating with other students, and setting up more ephemeral and organized experiences with the participants. (ex: 15 February 2022 with Louisa)
<br><br>
<br><br>
On the other hand, I am attached to the idea of <b>reversing the desktop metaphor</b>. The desktop metaphors refers to the terms and objects that the Web borrowed from the physical world in order to make its own concepts more familiar and intelligible to its users. Now, largely democratized and widely spread in modern society, <b>people may have now more concrete experiences of the digital/Web interface than the physical space</b>. Museums, hotels, houses, cars interiors, restaurants are themselves becoming more and more comparable to digital interface where everything is optimized, and where our <b>behaviours, actions and even inactions are being detected and converted into commands in order to offer a more customized (and lucrative) experience to each of us</b>. In that sense, we are <b>getting closer from becoming users of our own interfaced/augmented physical realities</b>. By creating a exhibition spaces explicitly inspired from a desktop Web interface, I wish to question <b>what could be the future of exhibition space, and what are the vulnerabilities of such technologies</b>.  
This semester will also include the creation of a definite set of illustrations participating to engage the participants of the installation in a more emotional way, the illustrations will be made by an illustrator/designer, with whom I usually collaborate.  
<br><br>
<br><br>
Conceiving the exhibition space as a Web interface, and the spectator as a user is also about <b>puting together two layers of reality that are too often clearly opposed/seperated(IRL VS Online)</b>. This is about <b>experiencing their ambiguities, similarities, and differences</b>. It is about <b>reconsidering their modalities by making them reflect on each others, and <b>making the user/spectator reflect on its own agencies inside of them</b>. (see: [[Martin_(XPUB)-thesis_outline#II._Reversing_the_desktop_metaphor|Reversing the desktop metaphor]])
3rd semester: Build the final installation of final assessment and graduation show. Test runs, debug and final touchs.
<br><br>
<br><br>
More generally, it is about <b>reflecting on media itself</b>, and deal with a paradox that I've always found interesting: The better a medium mediates, the more it becomes invisible and unconsidered. </b>  see: [[Martin_(XPUB)-thesis_outline#2._MEDIATIZING_THE_MEDIA_.28Optional.29|Mediatizing the media]]). This observation makes me want to mediate the media and to <b>give spectators more occasions to focus on what is containing, surrounding, holding or hosting a representation</b> instead of giving all our focus on the representation itself.
During the third semester, the installation should be settled in the school, in the alumni area, next to XPUB studio for the final assessment, and ultimately settled again at WORM for the graduation show. I am interested in putting this installation into privileged spaces of human circulation, (such as hallways) that would more easily involve the detection of people, and highlight the intrusive aspect of such technologies. The narration, the mechanics, illustrations and graphic aspect should be finalized at the beginning of the 3rd semester, and subject to intense test runs during all that period until meeting the deadline.


===<p style="font-family:helvetica">Who can help you?</p>===


*About the <b>overall project</b>
===Relation to larger context===


# Stephane Pichard, ex-teacher and ex production-tutor <b>for advices and help about scenography</b>
As GAFAM companies are facing more and more legal issues, and held accountable in growing numbers of social and political issues around the world, the pandemic context has greatly contributed to make all of us more dependent than ever on the online services provided by these companies and to somehow force our consent. While two decades of counter-terrorism measures legitimized domestic and public surveillance techniques such as online and video monitoring, the current public health crisis made even more necessary the use of new technologies for regulating the access to public spaces and services, but also for socializing, working together, accessing to culture, etc. In a lot of countries, from a day to another, and for an undetermined time, it has become necessary to carry a smartphone (or a printed QR code) in order to get access transport, entertainment, cultural and catering services, but also in order to do simple things such as to have a look at the menu in a bar/restaurant or to make an order.. Thus, this project takes place in a context where techno-surveillance has definitely taken a determining place in the way we can access spaces and services related to the physical world. <br><br>
# Emmanuel Cyriaque: my ex-teacher and writting-tutor <b>for advices and help contextualize my work</b>


*About <b>Arduino</b>
Data Marketisation / Self Data: Quantified Self / Attention Economy / Public Health Surveillance / Cybernetics 


# XPUB Arduino Group
# Dennis de Bel
# Aymeric Mansoux


*About <b>Rasperry Pi</b>
===Who can help you?===


# XPUB2 students (Jacopo, Camillo, Federico, etc)
<b>Prototyping</b><br>
# Michael Murtaugh
Manetta Berends<br>
Michael Murtaugh<br>
<br>
<b>Writing</b> <br>
Luke Williams<br>
<br>
<b>Arduino</b><br>
XPUB Arduino knowledge sharing group<br>
Dennis de Bel <br>
Aymeric Mansoux<br>
<br>
<b>Installation building</b><br>
Wood station<br>
Interaction station <br>
<br>
<b>Illustrations</b><br>
Adrien Jacquemet (illustrator/graphic designer)<br>
<br>
<b>Installation location</b> <br>
Leslie Robbins<br>
<br>
<b>Extra help for game narratives</b><br>
Rosa Zangenberg (writer/artist)<br>


*About <b>creating the physical elements</b>:
===Relation to previous practice?===


# Wood station (for movable walls)
During my previous studies in graphic design, I started being engaged with the new media by making a small online reissue of Raymond Queneau’s book called Exercices de Style. In this issue called Incidences Médiatiques (2017), the user/reader was encouraged to explore the 99 different versions of a same story written by the author in a less-linear way. The idea was to consider each user graphic user interface as a unique reading context. It would determine which story could be read, depending on the device used by the reader, and the user could navigate through these stories by resizing the Web window, by changing browser or by using on a different device.
# Metal station (for rails)
<br><br>
# Interaction station (for arduino/rasperyPi assistance)
As part of my graduation project called Media Spaces (2019), I wanted to reflect on the status of networked writing and reading, by programming my thesis in the form of Web to Print website. Subsequently, this website became translated in the physical space as a printed book, and a series of installations displayed in an exhibition space that was following the online structure of my thesis (home page, index, part 1-2-3-4). In that way, I was interested to inviting to visitors to make a physical experience some aspects of the Web
<br><br>
As a first-year student of Experimental Publishing, I continued to work in that direction by eventually creating a meta-website called Tense (2020) willing to display the invisible html <meta> tags inside of an essay in order to affect our interpretation of the text. In 2021, I worked on a geocaching pinball game highlighting invisible Web event, and a Web oscillator, which amplitude and frequency range were directly related to the user’s cursor position and browser screen-size.
<br><br>
While it has always been clear to me that these works were motivated by the desire to define media as context, subject or/and content, the projects presented here have often made use of surveillance tools to detect and translate user information into feedbacks, participating in the construction of an individualized narrative or/and a unique viewing/listening context (interaction, screen size, browser, mouse position). The current work aims to take a critical look at the effect of these practices in the context of techno surveillance.
<br><br>
Similarly, projects such as Media Spaces have sought to explore the growing confusion between human and web user, physical and virtual space or online and offline spaces. This project will demonstrate that these growing confusions will eventually lead us to be tracked in all circumstances, even in our most innocuous daily human activities/actions.


*About <b>theory/writting practice</b>:  
[[File:Incidences médiatiques .gif|400px|thumb|left|Incidences Médiatiques]]
[[File:Martin3.gif|400px|thumb|center|Media Spaces]]
[[File:MFworks2.gif|400px|thumb|right|Browser Pinball]]
[[File:OscillatorGifWiki.gif|400px|thumb|left|Web Oscillator]]
[[File:Tense Button Display 2.png|400px|thumb|center|Tense]]
[[File:CROQUIS EXPRESS PLANCHES IMPRESSION.jpg|800px|thumb|left|Media Spaces sketches]]
[[File:ESPACESMEDITATIQUE MARTINFOUCAUT 01.jpg|400px|thumb|center|Media Spaces thesis]]


# Rosa Zangenberg: ex-student in history art and media at Leiden Universtity.
<br><br><br><br><br><br><br><br>
# Yael: ex-student in philosophy, getting started with curatorial practice and writtings about the challenges/modalities of the exhibition space. Philosophy of the media (?)


*About <b>finding an exhibiting space</b>:
===Selected References===


# Leslie Robbins
<b>Works:</b>


===<p style="font-family:helvetica">Relation to previous practice</p>===
* M. DARKE, fairlyintelligent.tech (2021) https://fairlyintelligent.tech/
 
« invites us to take on the role of an auditor, tasked with addressing the biases in a speculative AI »Alternatives to techno-surveillance
During the first part of my previous studies, I really started being engaged into questioning the media by making a small online reissue of Raymond Queneau's book <i>Exercices de Style</i>. In this work called <i>Incidences Médiatiques</i>, the user/reader was encouraged to explore the 99 differents ways to tell a same story from Queneau, by putting itself in at least 99 different reading contexts. In order to suggest a more <b>non-linear reading experience, reflecting on the notion of context, perpective and point of view</b>, the user could unlock these stories by zooming-in or out the Web window, resizing it, changing the browser, going on a different device, etc. As part of my previous graduation project, I wanted to <b>reflect on the status of networked writing and reading</b>, by programming my thesis in the form of Web to Print Website. Subsequently, this website became <b>translated in the physical space</b> as a printed book, a set of meta flags, and a series of installations displayed in <b>a set of exhibition rooms that was following the online structure of thesis</b> (home page, index, part 1-2-3-4) [https://martinfoucaut.com/ESPACES-MEDIATIQUES-EN Project link]. It was my first attempt to create a physical interface inside an exhibition space, but it was focused on the structure and non-linear navigation rather than the elastic property of Web Interfaces. As a first year student of Experimental Publishing, I continued to work in that direction by eventually <b>creating a [https://issue.xpub.nl/13/TENSE/ meta-website]</b> making visible html <meta> tags in the middle of an essay. I also worked a [[SPECIAL_ISSUE_14_MARTIN_BOARD#GAME_PROTOTYPING|geocaching pinball]] <b>highligting invisible Web events</b> as well as a [[User:Martin#XPUB_Festival:_window.open_.2B_Civil_Entertainment_Sirens_.28audiovisual_performance.29|Web oscillator]] inspired from analog instruments's body size, and which amplitude and frequency range were directly related <b> to the user's device screen-size </b>.
<br>
 
* MANUEL BELTRAN, Data Production Labor (2018) https://v2.nl/archive/works/data-production-labour/
[[File:Incidences médiatiques .gif|250px|thumb|left|Incidences Médiatiques <br><b>click to watch GIF</b>]]
Expose humans as producers of useful intellectual labor that is benefiting to the tech giants and the use than can be made out of that labor.
[[File:Martin Foucaut Tense V3.gif|250px|thumb|right|Special issue 13 - Wor(l)ds for the Future<br> Tense screen recording montage of <i>Tense</i><br><b>click to watch GIF</b>]]
<br>
[[File:ESPACESMEDITATIQUE MARTINFOUCAUT 07.jpg|250px||thumb|left|Media Spaces - graduation project]]
* TEGA BRAIN and SURYA MATTU, Unfit-bits (2016) http://tegabrain.com/Unfit-Bits
[[File:ESPACESMEDITATIQUE MARTINFOUCAUT 06.jpg|250px|thumb|right|Media Spaces - graduation project]]
Claims that that technological devices can be manipulated easily and hence, that they are fallible and subjective. They do this by simply placing a self-tracker (connected bracelet) in another context, such as on some other objects, in order to confuse these devices.
 
<br>
[[File:ESPACESMEDITATIQUE MARTINFOUCAUT 09.jpg|250px|thumb|left|Media Spaces - graduation project|center]]
* JACOB APPELBAUM, Autonomy Cube (2014), https://www.e-flux.com/announcements/2916/trevor-paglen-and-jacob-appelbaumautonomy-cube/
[[File:ESPACESMEDITATIQUE MARTINFOUCAUT 04.jpg|250px|thumb|right|Media Spaces - graduation project|center]]
Allows galleries to enjoy encrypted internet access and communications, through a Tor Network
 
<br>  
<br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br>
* STUDIO MONIKER, Clickclickclick.click (2016) https://clickclickclick.click/
 
You are rewarded for exploring all the interactive possibilities of your mouse, revealing how our online behaviors can be monitored and interpretated by machines.
===<p style="font-family:helvetica">Relation to a larger context</p>===
<br>
 
* RAFAEL LOZANO-HEMMER, Third Person (2006) https://www.lozano-hemmer.com/third_person.php
<b>Curatorial Practice</b> / <b>New Media Art</b> / <b>Information Visualization</b> / <b>Software Art</b> / <b>Institutional Critique</b> / <b>Human Sciences</b> / <b>Cybernetics</b>
Portrait of the viewer is drawn in real time by active words, which appear automatically to fill his or her silhouette https://www.lozano-hemmer.com/third_person.php
 
<br>
===<p style="font-family:helvetica">Key References</p>===
* JONAL LUND, What you see is what you get (2012) http://whatyouseeiswhatyouget.net/
 
«Every visitor to the website’s browser size, collected, and played back sequentially, ending with your own.»
* [https://clickclickclick.click/ Clickclickclick.click] - VPRO Medialab & Moniker
<br>
 
* USMAN HAQUE, Mischievous Museum (1997) https://haque.co.uk/work/mischievous-museum/
*Stéphanie Moser, 2010. [[THE_DEVIL_IS_IN_THE_DETAILS:_MUSEUM_-_Displays_and_the_Creation_of_Knowledge|THE DEVIL IS IN THE DETAILS: MUSEUM - Displays and the Creation of Knowledge]]. 1st ed. Southampton, England
Readings of the building and its contents are therefore always unique -- no two visitors share the same experience. https://haque.co.uk/work/mischievous-museum/
 
<br><br>
*Alexander R. Galloway - [[Alexander_R._Galloway_-_The_Interface_Effect|The Interface Effect]] 1st ed. Malden, USA: Polity Press.
<b>Books & Articles:</b>
 
*Jonas Lund, 2012. [http://whatyouseeiswhatyouget.net/ What you see is what you get]
 
*Shilpa Gupta, 2009 - 2010. [http://shilpagupta.com/speaking-wall/| Speaking Wall]


*Frederick Kiesler, 1925, [https://thecharnelhouse.org/2013/11/19/frederick-kiesler-city-of-space-1925/ City of space]
* SHOSHANA ZUBOFF, The Age of Surveillance Capitalism (2020)
Warns against this shift towards a «surveillance capitalism». Her thesis argues that, by appropriating our personal data, the digital giants are manipulating us and modifying our behavior, attacking our free will and threatening our freedoms and personal sovereignty.
<br>
* EVGENY MOROZOV, Capitalism’s New Clothes (2019)
Extensive analysis and critic of Shoshana Zuboff research and publications.
<br>
* BYRON REEVES AND CLIFFORD NASS, The Media Equation, How People Treat Computers, Television, and New Media Like Real People and Places (1996)
Precursor study of the relation between humans and machine, and how do you human relate to them.
<br>
* OMAR KHOLEIF, Goodbye, World! — Looking at Art in the digital Age (2018)
Authors shares it’s own data as a journal in a part of the book, while on another part, question how the Internet has changed the way we perceive and relate, and interact with/to images.
<br>
* KATRIN FRITSCH, Towards an emancipatory understanding of widespread datafication (2018)
Suggests that in response to our society of surveillance, artists can suggest activist response that doesn’t necessarily involve technological literacy, but instead can promote strong counter metaphors or/and counter use of these intrusive technologies.


*Brendan Howell, 2017(?) - [http://screenl.es/ The screenless office]
</div>

Latest revision as of 12:47, 26 January 2022

Graduate proposal guidelines

What do you want to make?

My project is a data collection installation that monitors people's behaviors in public physical spaces while explicitly encouraging them to help the algorithm collect more information. An overview of how it works is presented here in the project proposal and will be subject to further developments in the practice.

The way the device is designed doesn’t pretend to give any beneficial outcomes for the subject, but only makes visible the benefits that the machine is getting from collecting their data. Yet, the way the device visually or verbally presents this collected data is done in a grateful way, which might be stimulating for the subject. In that sense, the subject, despite knowing that their actions are done solely to satisfy the device, could become intrigued, involved, or even addicted by a mechanism that deliberately uses it as a commodity. In that way, I intend to trigger conflictual feelings in the visitor’s mind, situated between a state of awareness regarding the operating monetization of their physical behaviors, and a state of engagement/entertainment /stimulation regarding the interactive value of the installation.

My first desire is to make the mechanisms by which data collection is carried out, marketized and legitimized both understandable and accessible. The array of sensors, the Arduinos and the screen are the mainly technological components of this installation. Rather than using an already existing and complex tracking algorithm, the program is built from scratch, kept open source and limits itself to the conversion of a restricted range of physical actions into interactions. These include the detection of movements, positions, lapse of time spent standing still or moving, and entry or exit from a specific area of detection. Optionally they may also include the detection of the subject smartphone device or the log on a local Wi-Fi hotspot made by the subject.

In terms of mechanic, the algorithm creates feedback loops starting from:
_the subject behaviors being converted into information;
_the translation of this information into written/visual feedback;
_and the effects of this feedbacks on subject’s behavior; and so on.
By doing so, it tries to shape the visitors as free data providers inside their own physical environment, and stimulate their engagement by converting each piece of collected information into points/money, feeding a user score among a global ranking.

On the screen, displayed events can be:

Simu part 02.gif
Simu part 03.gif

_ “subject [] currently located at [ ]”
[x points earned/given]
_ “subject [] entered the space”
[x points earned/given]
_ “subject [] left the space”
[x points earned/given]
_ “subject [] moving/not moving”
[x points earned/given]
_ “subject [] distance to screen: [ ] cm”
[x points earned/given]
_ “subject [] stayed at [ ] since [ ] seconds”
[x points earned/given]
_ “subject [] device detected
[x points earned/given] (optional)
_ “subject logged onto local Wi-Fi
[x points earned/given] (optional)

Added to that comes the instructions and comments from the devices in reaction to the subject’s behaviors:

_ “Congratulations, you have now given the monitor 25 % of all possible data to collect!”
[when 25-50-75-90-95-96-97-98-99% of the total array of events has been detected at least once]
_ “Are you sure you don’t want to move to the left? The monitor has only collected data from 3 visitors so far in this spot!”
[if the subject stands still in a specific location]
_ “Congratulations, the monitor has reached 1000 pieces of information from you!”
[unlocked at x points earned/given]
_ “If you stay there for two more minutes, there is a 99% chance you will be in the top 100 of ALL TIME data-givers!”
[if the subject stand still in a specific location]
_ “Leaving all ready? The monitor has yet to collect 304759 crucial pieces of information from you!”
[if the subject is a the edge of the detection range]
_ “You are only 93860 pieces of information away from being the top one data-giver!”
[unlocked at x points earned/given]
_ “Statistics show that people staying for more than 5 minutes average will benefit me on average 10 times more!”
[randomly appears]
_ “The longer you stay on this spot, the more chance you have to win a “Lazy data-giver” badge”
[if the subject stands still for a long time any location]

Responding positively to the monitors instructions unlocks special achievement and extra points

—Accidental data-giver badge
[unlocked if the subject has passed the facility without deliberately wishing to interact with it] + [x points earned/given]
—Lazy data-giver badge
[unlocked if the subject has been standing still for at least one minute] + [x points earned/given]
—Novice data-giver badge
[unlocked if the subject has been successfully completing 5 missions from the monitor] + [x points earned/given]
—Hyperactive data-giver badge
[unlocked if the subject has never been standing still for 10 seconds within 2 minutes lapse time] + [x points earned/given]
—Expert data-giver badge
[unlocked if the subject has been successfully completing 10 missions from the monitor within 10 minutes] + [x points earned/given]
—Master data-giver badge
[unlocked if the subject has been successfully logging on the local Wi-Fi hotspot] + [x points earned/given] (optional)

On the top left side of the screen, a user score displays the number of points generated by the collected pieces of information, and the unlocking of special achievements instructed by the monitor.

—Given information: 298 pieces
[displays number of collected events]
——$ collected from your data: 1.312$
[conversion of collected events and achievement into points]

On the top right of the screen, the user is ranked among x number of previous visitors and the prestigious badge recently earned is displayed bellow

—subject global ranking: 3/42
[compares subject’s score to all final scores from previous subjects]
—subject status: expert data-giver
[display the most valuable reward unlocked by the subject]

When leaving the detection range, the subject gets a warning message and a countdown starts, and encouraging it to take the quick decision to come back

—“Are you sure you want to leave? You have 5-4-3-2-1-0 seconds to come back within the detection range”
[displayed as long as the subject remains completely undetected]

If the subject definitely stands out of the detection range for more than 5 seconds, the monitor will also address a thankful message and the amount of money gathered, achievements, ranking, complete list of collected information and a qr code will be printed as a receipt with the help of a thermal printer. The QR will be a link to my thesis.

—* “Thank you for helping today, don’t forget to take your receipt in order to collect and resume your achievements”
[displayed after 5 seconds being undetected]

In order to collect, read or/and use that piece of information, the visitor will inevitably have to come back within the range of detection, and intentionally, or not, reactivate the data tracking game. It is therefore impossible to leave the area of detection without leaving at least one piece of your own information printed in the space. Because of this, the physical space should gradually be invaded by tickets scattered on the floor. As in archaeology, these tickets give a precise trace of the behavior and actions of previous subjects for future subjects.

Why do you want to make it?

When browsing online or/and using connected devices in the physical world, even the most innocent action/information can be invisibly recorded, valued and translated into informational units, subsequently generating profit for monopolistic companies. While social platforms, brands, public institutions and governments explicitly promote the use of monitoring practices in order to better serve or protect us, we could also consider these techniques as implicitly political, playing around some dynamics of visibility and invisibility in order to assert new forms of power over targeted audiences.

In the last decade, a strong mistrust of new technologies has formed in the public opinion, fueled by events such as the revelations of Edward Snowden, the Cambridge Analytica scandal or the proliferation of fake news on social networks. We have also seen many artists take up the subject, sometimes with activist purposes. But even if a small number of citizens have begun to consider the social and political issues related to mass surveillance, and some individuals/groups/governments/associations have taken legal actions, surveillance capitalism still remains generally accepted, often because ignored or/and misunderstood.

Thanks to the huge profits generated by the data that we freely provide every day, big tech companies have been earning billions of dollars over the sale of our personal information. With that money, they could also further develop deep machine learning programs, powerful recommendation systems, and to broadly expand their range of services in order to track us in all circumstances and secure their monopolistic status. Even if we might consider this realm specific to the online world, we have seen a gradual involvement from the same companies to monitor the physical world and our human existences in a wide array of contexts. For example, with satellite and street photography (Google Earth, Street View), geo localization systems, simulated three-dimensional environments (augmented reality, virtual reality or metaverse) or extensions of our brains and bodies (vocal assistance and wearable devices). Ultimately, this reality has seen the emergence of not only a culture of surveillance but also of self-surveillance, as evidenced by the popularity of self-tracking and data sharing apps, which legitimize and encourage the datafication of the body for capitalistic purposes.

For the last 15 years, self-tracking tools have made their way to consumers. I believe that this trend is showing how ambiguous our relationship can be with tools that allow such practices. Through my work, I do not wish to position myself as a whistleblower, a teacher or activist. Indeed, to adopt such positions would be hypocritical, given my daily use of tools and platforms that resort to mass surveillance. Instead, I wish to propose an experience that highlights the contradictions in which you and I, internet users and human beings, can find ourselves. This contradiction is characterized by a paradox between our state of concern about the intrusive surveillance practices operated by the Web giants (and their effects on societies and humans) and a state of entertainment or even active engagement with the tools/platforms through which this surveillance is operated/allowed. By doing so, I want to ask how do these companies still manage to get our consent and what human biases do they exploit in order to do so. That’s is how my graduation work and my thesis will investigate the effect of gamification, gambling or reward systems as well as a the esthetization of data/self-data as means to hook our attention, create always more interactions and orientate our behaviors.


How to you plan to make it and on what timetable?

I am developing this project with Arduino Uno/Mega boards, an array of ultrasonic sensor, P5.js and screens.

How does it work?

The ultrasonic sensors can detect obstacles in a physical space and know the distance between the sensor and obstacle(s) by sending and receiving back an ultrasound. The Arduino Uno/Mega boards are microcontrollers which can receive this information, run it in a program in order to convert these values into a mm/cm/m but also map the space into an invisible grid. Ultimately, values collected on the Arduino’s serial monitor can be sent to P5.js through P5.serialcontrol. P5.js will then allow a greater freedom in way the information can be displayed on the screens.

Process:

1st semester: Building a monitoring device, converting human actions into events, and events into visual feedbacks

During the first semester, I am focused on exploring monitoring tools that can be used in the physical world, with a specific attention to ultrasonic sensors. Being new to Arduino programming, my way of working is to start from the smallest and most simple prototype and gradually increase its scale/technicality until reaching human/architectural scale. Prototypes are subject to testing, documentation and comments helping to define which direction to follow. The first semester also allows to experiment with different kind of screen (LCD screens, Touch screens, computer screens, TV screens) until finding the most adequate screen monitor(s) for the final installation. Before building the installation, the project is subject to several sketching and animated simulations in 3 dimensions, exploring different scenarios and narrations. At the end of the semester, the goal is to be able to convert a specific range of human actions into events and visual feedback creating a feedback loop from the human behaviors being converted into information; the translation of this information into written/visual feedbacks; and the effects of this feedbacks on human behavior; and so on.

2nd semester: Implementing gamification with the help of collaborative filtering, point system and ranking.

During the second semester, it is all about building and implementing a narration with the help of gaming mechanics that will encourage humans to feed the data gathering device with their own help. An overview of how it works is presented here in the project proposal and will be subject to further developments in the practice.

To summarize the storyline, the subject being positioned in the detection zone finds herself/himself unwillingly embodied as the main actor of a data collection game. Her/His mere presence generates a number of points/dollars displayed on a screen, growing as she/he stays within the area. The goal is simple: to get a higher score/rank and unlock achievements by acting as recommended by a data-collector. This can be done by setting clear goals/rewards to the subject, and putting its own performance in comparison to all the previous visitors, giving unexpected messages/rewards, and give an aesthetic value to the displayed informations.

The mechanism is based on a sample of physical events that have been already explored in the first semester of prototyping (detection of movements, positions, lapse of time spent standing still or moving, and entry or exit from a specific area of detection). Every single detected event in this installation is stored in a data bank, and with the help of collaborative filtering, will allow to the display of custom recommendations such as:

_ “Congratulations, you have now given the monitor 12 % of all possible data to collect”
_ “Are you sure you don’t want to move to the left? The monitor has only collected data from 3 visitors so far in this spot”
_ “Congratulations, the monitor has reached 1000 pieces of information from you!”
_ “If you stay there for two more minutes, there is a 99% chance you will be in the top 100 of ALL TIME data-givers”
_ “Leaving all-ready? The monitor has yet 304759 crucial pieces of information to collect from you”
_ “You are only 93860 actions away from being the top one data-giver”
_ “Statistics are showing that people staying for more than 5 minutes average will be 10 times more benefitting for me”
_ “The longer you stay on this spot, the more chance you have to win a “Lazy data-giver” badge”


The guideline is set out here, but will be constantly updated with the help of experiments and the results observed during the various moments of interaction between the students and the algorithm. For this purpose, the installation under construction will be left active and autonomous in its place of conception (studio) and will allow anyone who deliberately wishes to interact with it to do so. Beyond the voluntary interactions, my interest is also to see what can be extracted from people simply passing in front of this installation. In addition to this, some of the mechanics of the installation will be further explored by collaborating with other students, and setting up more ephemeral and organized experiences with the participants. (ex: 15 February 2022 with Louisa)

This semester will also include the creation of a definite set of illustrations participating to engage the participants of the installation in a more emotional way, the illustrations will be made by an illustrator/designer, with whom I usually collaborate.

3rd semester: Build the final installation of final assessment and graduation show. Test runs, debug and final touchs.

During the third semester, the installation should be settled in the school, in the alumni area, next to XPUB studio for the final assessment, and ultimately settled again at WORM for the graduation show. I am interested in putting this installation into privileged spaces of human circulation, (such as hallways) that would more easily involve the detection of people, and highlight the intrusive aspect of such technologies. The narration, the mechanics, illustrations and graphic aspect should be finalized at the beginning of the 3rd semester, and subject to intense test runs during all that period until meeting the deadline.


Relation to larger context

As GAFAM companies are facing more and more legal issues, and held accountable in growing numbers of social and political issues around the world, the pandemic context has greatly contributed to make all of us more dependent than ever on the online services provided by these companies and to somehow force our consent. While two decades of counter-terrorism measures legitimized domestic and public surveillance techniques such as online and video monitoring, the current public health crisis made even more necessary the use of new technologies for regulating the access to public spaces and services, but also for socializing, working together, accessing to culture, etc. In a lot of countries, from a day to another, and for an undetermined time, it has become necessary to carry a smartphone (or a printed QR code) in order to get access transport, entertainment, cultural and catering services, but also in order to do simple things such as to have a look at the menu in a bar/restaurant or to make an order.. Thus, this project takes place in a context where techno-surveillance has definitely taken a determining place in the way we can access spaces and services related to the physical world.

Data Marketisation / Self Data: Quantified Self / Attention Economy / Public Health Surveillance / Cybernetics


Who can help you?

Prototyping
Manetta Berends
Michael Murtaugh

Writing
Luke Williams

Arduino
XPUB Arduino knowledge sharing group
Dennis de Bel
Aymeric Mansoux

Installation building
Wood station
Interaction station

Illustrations
Adrien Jacquemet (illustrator/graphic designer)

Installation location
Leslie Robbins

Extra help for game narratives
Rosa Zangenberg (writer/artist)

Relation to previous practice?

During my previous studies in graphic design, I started being engaged with the new media by making a small online reissue of Raymond Queneau’s book called Exercices de Style. In this issue called Incidences Médiatiques (2017), the user/reader was encouraged to explore the 99 different versions of a same story written by the author in a less-linear way. The idea was to consider each user graphic user interface as a unique reading context. It would determine which story could be read, depending on the device used by the reader, and the user could navigate through these stories by resizing the Web window, by changing browser or by using on a different device.

As part of my graduation project called Media Spaces (2019), I wanted to reflect on the status of networked writing and reading, by programming my thesis in the form of Web to Print website. Subsequently, this website became translated in the physical space as a printed book, and a series of installations displayed in an exhibition space that was following the online structure of my thesis (home page, index, part 1-2-3-4). In that way, I was interested to inviting to visitors to make a physical experience some aspects of the Web

As a first-year student of Experimental Publishing, I continued to work in that direction by eventually creating a meta-website called Tense (2020) willing to display the invisible html <meta> tags inside of an essay in order to affect our interpretation of the text. In 2021, I worked on a geocaching pinball game highlighting invisible Web event, and a Web oscillator, which amplitude and frequency range were directly related to the user’s cursor position and browser screen-size.

While it has always been clear to me that these works were motivated by the desire to define media as context, subject or/and content, the projects presented here have often made use of surveillance tools to detect and translate user information into feedbacks, participating in the construction of an individualized narrative or/and a unique viewing/listening context (interaction, screen size, browser, mouse position). The current work aims to take a critical look at the effect of these practices in the context of techno surveillance.

Similarly, projects such as Media Spaces have sought to explore the growing confusion between human and web user, physical and virtual space or online and offline spaces. This project will demonstrate that these growing confusions will eventually lead us to be tracked in all circumstances, even in our most innocuous daily human activities/actions.

Incidences Médiatiques
Media Spaces
Browser Pinball
Web Oscillator
Tense
Media Spaces sketches
Media Spaces thesis









Selected References

Works:

« invites us to take on the role of an auditor, tasked with addressing the biases in a speculative AI »Alternatives to techno-surveillance

Expose humans as producers of useful intellectual labor that is benefiting to the tech giants and the use than can be made out of that labor.

Claims that that technological devices can be manipulated easily and hence, that they are fallible and subjective. They do this by simply placing a self-tracker (connected bracelet) in another context, such as on some other objects, in order to confuse these devices.

Allows galleries to enjoy encrypted internet access and communications, through a Tor Network

You are rewarded for exploring all the interactive possibilities of your mouse, revealing how our online behaviors can be monitored and interpretated by machines.

Portrait of the viewer is drawn in real time by active words, which appear automatically to fill his or her silhouette https://www.lozano-hemmer.com/third_person.php

«Every visitor to the website’s browser size, collected, and played back sequentially, ending with your own.»

Readings of the building and its contents are therefore always unique -- no two visitors share the same experience. https://haque.co.uk/work/mischievous-museum/

Books & Articles:

  • SHOSHANA ZUBOFF, The Age of Surveillance Capitalism (2020)

Warns against this shift towards a «surveillance capitalism». Her thesis argues that, by appropriating our personal data, the digital giants are manipulating us and modifying our behavior, attacking our free will and threatening our freedoms and personal sovereignty.

  • EVGENY MOROZOV, Capitalism’s New Clothes (2019)

Extensive analysis and critic of Shoshana Zuboff research and publications.

  • BYRON REEVES AND CLIFFORD NASS, The Media Equation, How People Treat Computers, Television, and New Media Like Real People and Places (1996)

Precursor study of the relation between humans and machine, and how do you human relate to them.

  • OMAR KHOLEIF, Goodbye, World! — Looking at Art in the digital Age (2018)

Authors shares it’s own data as a journal in a part of the book, while on another part, question how the Internet has changed the way we perceive and relate, and interact with/to images.

  • KATRIN FRITSCH, Towards an emancipatory understanding of widespread datafication (2018)

Suggests that in response to our society of surveillance, artists can suggest activist response that doesn’t necessarily involve technological literacy, but instead can promote strong counter metaphors or/and counter use of these intrusive technologies.