XPUB2 Research Board / Martin Foucaut: Difference between revisions

From XPUB & Lens-Based wiki
(36 intermediate revisions by the same user not shown)
Line 156: Line 156:
==<p style="font-family:helvetica">Graduate proposal guidelines</p>==
==<p style="font-family:helvetica">Graduate proposal guidelines</p>==


===<p style="font-family:helvetica">What do you want to make?</p>===
===What do you want to make?===


I want to <b>explore the key roles of digital and physical architectures in the representation and transmission of knowledge</b> by merging them under the form of a <b>physical exhibition interface</b>. To be more specific, I wish to build an exhibition space inspired from the <b>elastic (plastic)</b> display/render of online contents</b> that mostly adapt to the user/viewers perspective. In that sense, I want to build an exhibiting device willing to <b>put the spectator in the position of being the curator and user of it's own physical exhiibition space</b> and <b>allow a single representation/content/artwork to be displayed under a wide range of possible settings</b>. Comparably as a pointer or a cursor, the spectator/user can move inside the space (SpectatorX/SpectatorY = MouseX/MouseY) and resize this same space (SpaceWidth & SpaceHeight = WindowWidth & Windowheight) by pushing or pulling a movable wall fixed on some rails/wheels. The number of spectators, its/their position inside the space, as well as the interactions engaged with it will affect various display factors</b> such as the lighting, sound, projection format, information layout, etc. Such interactions could also <b>give space to some accidents, unforeseen, bugs, deadzones and glitches</b> that might have been intentionaly left there). This is an attempt to <b>speculate on how the cybernetics(?) affects alls aspect of our lifes and could also transform the exhibition space, the curatiorial practice, our experience of art and the nature of representation itself</b>.
My project is a data collection installation that monitors people's behaviors in public physical spaces while explicitly encouraging them to help the algorithm collect more information. An overview of how it works is presented here in the project proposal and will be subject to further developments in the practice.
<br><br>
The way the device is designed doesn’t pretend to give any beneficial outcomes for the subject, but only makes visible the benefits that the machine is getting from collecting their data. Yet, the way the device visually or verbally presents this collected data is done in a grateful way, which might be stimulating for the subject. In that sense, the subject, despite knowing that their actions are done solely to satisfy the device, could become intrigued, involved, or even addicted by a mechanism that deliberately uses it as a commodity. In that way, I intend to trigger conflictual feelings in the visitor’s mind, situated between a state of awareness regarding the operating monetization of their physical behaviors, and a state of engagement/entertainment /stimulation regarding the interactive value of the installation.
<br><br>
My first desire is to make the mechanisms by which data collection is carried out, marketized and legitimized both understandable and accessible. The array of sensors, the Arduinos and the screen are the mainly technological components of this installation. Rather than using an already existing and complex tracking algorithm, the program is built from scratch, kept open source and limits itself to the conversion of a restricted range of physical actions into interactions. These include the detection of movements, positions, lapse of time spent standing still or moving, and entry or exit from a specific area of detection. Optionally they may also include the detection of the subject smartphone device or the log on a local Wi-Fi hotspot made by the subject.
<br><br>
In terms of mechanic, the algorithm creates feedback loops starting from: <br>
_the subject behaviors being converted into information; <br>
_the translation of this information into written/visual feedback; <br>
_and the effects of this feedbacks on subject’s behavior; and so on. <br>
By doing so, it tries to shape the visitors as free data providers inside their own physical environment, and stimulate their engagement by converting each piece of collected information into points/money, feeding a user score among a global ranking.
<br><br>
On the screen, displayed events can be:


* I am still questionning if this space should be completely empty of information or not. In my mind, inside the space should be displayed with the help of one or several beamer(s)) (or screens) the live values of the variable exhibition space (width/lenght; spectator position; luminosity; information font; sound frequency, etc), in order to enhance the idea of being inside a exhibiting device. The way these values are displayed (format, font, layout) should also change depending on the values themselves (I will make some sketches about that).
[[File:Simu part 01.gif|right|800px|thumb|https://www.dropbox.com/s/zyv3c2ypit8qo1y/Simulation_MartinFoucaut.mp4?dl=0 full HD video]]
[[File:Simu part 02.gif|right|800px|thumb]]
[[File:Simu part 03.gif|right|800px|thumb]]
 
_ “subject [] currently located at [ ]” <br>
[x points earned/given]<br>
_ “subject [] entered the space” <br>
[x points earned/given]<br>
_ “subject [] left the space”<br>
[x points earned/given]<br>
_ “subject [] moving/not moving”<br>
[x points earned/given]<br>
_ “subject [] distance to screen: [ ] cm” <br>
[x points earned/given]<br>
_ “subject [] stayed at [ ] since [ ] seconds” <br>
[x points earned/given]<br>
_ “subject [] device detected <br>
[x points earned/given] (optional)<br>
_ “subject logged onto local Wi-Fi<br>
[x points earned/given] (optional)<br>
<br>
Added to that comes the instructions and comments from the devices in reaction to the subject’s behaviors:<br>
<br>
_ “Congratulations, you have now given the monitor 25 % of all possible data to collect!” <br>
[when 25-50-75-90-95-96-97-98-99% of the total array of events has been detected at least once]<br>
_ “Are you sure you don’t want to move to the left? The monitor has only collected data from 3 visitors so far in this spot!”<br>
[if the subject stands still in a specific location]<br>
_ “Congratulations, the monitor has reached 1000 pieces of information from you!”<br>
[unlocked at x points earned/given]<br>
_ “If you stay there for two more minutes, there is a 99% chance you will be in the top 100 of ALL TIME data-givers!”<br>
[if the subject stand still in a specific location]<br>
_ “Leaving all ready? The monitor has yet to collect 304759 crucial pieces of information from you!”<br>
[if the subject is a the edge of the detection range]<br>
_ “You are only 93860 pieces of information away from being the top one data-giver!”<br>
[unlocked at x points earned/given]<br>
_ “Statistics show that people staying for more than 5 minutes average will benefit me on average 10 times more!”<br>
[randomly appears]<br>
_ “The longer you stay on this spot, the more chance you have to win a “Lazy data-giver” badge”<br>
[if the subject stands still for a long time any location]<br>
<br>
Responding positively to the monitors instructions unlocks special achievement and extra points<br>
<br>
—Accidental data-giver badge <br>
[unlocked if the subject has passed the facility without deliberately wishing to interact with it] + [x points earned/given]<br>
—Lazy data-giver badge <br>
[unlocked if the subject has been standing still for at least one minute] + [x points earned/given]<br>
—Novice data-giver badge <br>
[unlocked if the subject has been successfully completing 5 missions from the monitor] + [x points earned/given]<br>
—Hyperactive data-giver badge <br>
[unlocked if the subject has never been standing still for 10 seconds within 2 minutes lapse time] + [x points earned/given]<br>
—Expert data-giver badge <br>
[unlocked if the subject has been successfully completing 10 missions from the monitor within 10 minutes] + [x points earned/given]<br>
—Master data-giver badge <br>
[unlocked if the subject has been successfully logging on the local Wi-Fi hotspot] + [x points earned/given] (optional)<br>
<br>
On the top left side of the screen, a user score displays the number of points generated by the collected pieces of information, and the unlocking of special achievements instructed by the monitor.<br>
<br>
—Given information: 298 pieces <br>
[displays number of collected events]<br>
—Points: 312000 <br>
[conversion of collected events and achievement into points]<br>
<br>
On the top right of the screen, the user is ranked among x number of previous visitors and the prestigious badge recently earned is displayed bellow<br>
<br>
—subject global ranking: 3/42 <br>
[compares subject’s score to all final scores from previous subjects]<br>
—subject status:  expert data-giver<br>
[display the most valuable reward unlocked by the subject]<br>
<br>
When leaving the detection range, the subject gets a warning message and a countdown starts, and encouraging it to take the quick decision to come back<br>
<br>
—“Are you sure you want to leave? You have 5-4-3-2-1-0 seconds to come back within the detection range”<br>
[displayed as long as the subject remains completely undetected]<br>
<br>
If the subject definitely stands out of the detection range for more than 5 seconds, the monitor will also address a thankful message and the amount of money gathered, achievements, ranking, complete list of collected information and a qr code will be printed as a receipt with the help of a thermal printer. The QR will be a link to my thesis.<br>
<br>
—* “Thank you for helping today, don’t forget to take your receipt in order to collect and resume your achievements”<br>
[displayed after 5 seconds being undetected]<br>
<br>
In order to collect, read or/and use that piece of information, the visitor will inevitably have to come back within the range of detection, and intentionally, or not, reactivate the data tracking game. It is therefore impossible to leave the area of detection without leaving at least one piece of your own information printed in the space. Because of this, the physical space should gradually be invaded by tickets scattered on the floor. As in archaeology, these tickets give a precise trace of the behavior and actions of previous subjects for future subjects. <br>


===<p style="font-family:helvetica">How do you plan to make it?</p>===
===Why do you want to make it?===


While working with <b>Arduino Mega</b> and <b>Rasperry Pi</b>, my aim is to start <b>from the smallest and most simple prototype</b>, <b>gradually increase its scale/technicality until reaching human scale</b> and getting closer from emulating the properties of a Web window (see: [[XPUB2_Research_Board_/_Martin_Foucaut#Prototyping|prototyping]]). Once an <b>exhibtion space will be found/determined</b> for the graduation, I will  <b>build a custom mobile wall</b>, or use a preexisting one, add some handles to it, and fix it on a rail system that will allow to reduce or expend the size of a space from in between a maximum and a minimum range. This wall will include on on the back at least one sensor that will help to determine the size of the room in real time. (see [[XPUB2_Research_Board_/_Martin_Foucaut#Creating_an_elastic_exhibition_space|schema]]). With the help of an array of sensors putted on the 3 static walls of the exhibition space, the space will be mapped into a grid, and will allow to know the real-time position of the spectator(s) inside of it.  
When browsing online or/and using connected devices in the physical world, even the most innocent action/information can be invisibly recorded, valued and translated into informational units, subsequently generating profit for monopolistic companies. While social platforms, brands, public institutions and governments explicitly promote the use of monitoring practices in order to better serve or protect us, we could also consider these techniques as implicitly political, playing around some dynamics of visibility and invisibility in order to assert new forms of power over targeted audiences.  
<br><br>
<br><br>
In order to better <b>define my audience, the issues, and the direction of this work, I will take advantages of the different venues organized by XPUB</b> to involve people into <b>testing and reflecting on various prototypes of this work in progress.</b> (see [[XPUB2_Research_Board_/_Martin_Foucaut#Venues|venue1]])
In the last decade, a strong mistrust of new technologies has formed in the public opinion, fueled by events such as the revelations of Edward Snowden, the Cambridge Analytica scandal or the proliferation of fake news on social networks. We have also seen many artists take up the subject, sometimes with activist purposes. But even if a small number of citizens have begun to consider the social and political issues related to mass surveillance, and some individuals/groups/governments/associations have taken legal actions, surveillance capitalism still remains generally accepted, often because ignored or/and misunderstood.
(see: [[XPUB2_Research_Board_/_Martin_Foucaut#Sketch_3:_Arduino_Uno_.2B_Sensor_.2B_LCD_.2B_2_LED.29_.3D_Physical_vs_Digital_Range_detector|simulation]])
<br><br>
Thanks to the huge profits generated by the data that we freely provide every day, big tech companies have been earning billions of dollars over the sale of our personal information. With that money, they could also further develop deep machine learning programs, powerful recommendation systems, and to broadly expand their range of services in order to track us in all circumstances and secure their monopolistic status. Even if we might consider this realm specific to the online world, we have seen a gradual involvement from the same companies to monitor the physical world and our human existences in a wide array of contexts. For example, with satellite and street photography (Google Earth, Street View), geo localization systems, simulated three-dimensional environments (augmented reality, virtual reality or metaverse) or extensions of our brains and bodies (vocal assistance and wearable devices). Ultimately, this reality has seen the emergence of not only a culture of surveillance but also of self-surveillance, as evidenced by the popularity of self-tracking and data sharing apps, which legitimize and encourage the datafication of the body for capitalistic purposes.
<br><br>
For the last 15 years, self-tracking tools have made their way to consumers. I believe that this trend is showing how ambiguous our relationship can be with tools that allow such practices. Through my work, I do not wish to position myself as a whistleblower, a teacher or activist. Indeed, to adopt such positions would be hypocritical, given my daily use of tools and platforms that resort to mass surveillance. Instead, I wish to propose an experience that highlights the contradictions in which you and I, internet users and human beings, can find ourselves. This contradiction is characterized by a paradox between our state of concern about the intrusive surveillance practices operated by the Web giants (and their effects on societies and humans) and a state of entertainment or even active engagement with the tools/platforms through which this surveillance is operated/allowed. By doing so, I want to ask how do these companies still manage to get our consent and what human biases do they exploit in order to do so. That’s is how my graduation work and my thesis will investigate the effect of gamification, gambling or reward systems as well as a the esthetization of data/self-data as means to hook our attention, create always more interactions and orientate our behaviors.


[[File:SensorSpace.gif|200px|thumb|left|Sensor Test VS Elastic Space<br><b>Click to watch</b>]]
[[File:Sensor Wall 01.png|200px|thumb|right|Sensor Wall 01]]
[[File:SensorMediaQueries 01.gif|200px|thumb|center|SensorMediaQueries<br><b>Click to watch</b>]]
<br><br><br>


===<p style="font-family:helvetica">What is your timetable?</p>===
===How to you plan to make it and on what timetable?===


<b>1st semester</b> Prototyping mainly with Arduino, connecting Arduino to P5.js , finding a space to set up the installation for final assesment
I am developing this project with Arduino Uno/Mega boards, an array of ultrasonic sensor, P5.js and screens.<br><br>
* 1st prototype: mini arduio + light sensor (<b>understanding arduino basics</b>) [[https://pzwiki.wdka.nl/mediadesign/XPUB2_Research_Board_/_Martin_Foucaut#Prototype_1_:_Arduino_.2B_Resistor|link]]
* 2nd prototype: Arduino uno + utlrasonic sensor (<b>working with ultrasonic sensors</b>) [[https://pzwiki.wdka.nl/mediadesign/XPUB2_Research_Board_/_Martin_Foucaut#Prototype_2:_Arduino_.2B_Ultrasonic_sensor|link]]
* 3rd prototype: Arduino uno + utlrasonic sensor + LCD screen (<b>working with values display</b>) [[https://pzwiki.wdka.nl/mediadesign/XPUB2_Research_Board_/_Martin_Foucaut#Prototype_3:_Arduino_Uno_.2B_Sensor_.2B_LCD_.28.2B_LED.29|link]]
* 4th prototype: Arduino uno + utlrasonic sensor + 2 LEDS (<b>working with in-between distance range values detection</b>) [[https://pzwiki.wdka.nl/mediadesign/XPUB2_Research_Board_/_Martin_Foucaut#Prototype_4:_Arduino_Uno_.2B_Sensor_.2B_LCD_.2B_2_LED_.3D_Physical_vs_Digital_Range_detector|link]]
* 5th prototype: Arduino uno + 3 utlrasonic sensor + 3 LEDS (<b>mapping range values detection in a grid and attributing signals with LEDS</b>) [[https://pzwiki.wdka.nl/mediadesign/XPUB2_Research_Board_/_Martin_Foucaut#Prototype_5:_Arduino_Uno_.2B_3_Sensor_.2B_3_LEDS|link]]
* 6th prototype: Arduino uno + 3 utlrasonic sensor + 12 LEDS (<b>mapping range values detection in a grid and attributing signals with more LEDS</b>) [[https://pzwiki.wdka.nl/mediadesign/XPUB2_Research_Board_/_Martin_Foucaut#Prototype_6:_Arduino_Uno_.2B_3_Sensor_.2B_12_LEDS|link]]
* 7th prototype: Arduino uno + 3 utlrasonic sensor + 1 buzzer + 1 LCD + 1 Potentiometer (<b>adding audio signals to the range value detection</b>) [[https://pzwiki.wdka.nl/mediadesign/XPUB2_Research_Board_/_Martin_Foucaut#Prototype_7:_Arduino_Uno_.2B_12_LEDS_.2B_3_Sensor_.2B_Buzzer_.2B_Potentiometer_.2B_LCDS|link]]
* 8th prototype: Arduino uno + 3 utlrasonic sensor + 1 buzzer + 1 LCD + 1 Potentiometer + mini breadboard (<b>separating sensors from each others</b>) [[https://pzwiki.wdka.nl/mediadesign/XPUB2_Research_Board_/_Martin_Foucaut#Prototype_8:_Arduino_Uno_.2B_12_LEDS_.2B_3_Sensor_on_mini_breadboards_.2B_Buzzer_.2B_Potentiometer_.2B_LCD|link]]
* 9th prototype: Arduino Mega + 21 LEDS + 7 Sensor + Buzzer + Potentiometer + LCD (<b>expending the prototype to human scale</b>)
* 10th prototype: Arduino Mega + 7 Sensors + LCD + 3 buzzers + P5.js  (<b>connecting the prototoype to a Web page</b>)
——————————— NOW —————————————————————————————————————————
* Upcoming - 11th prototype: Arduino Mega + 7 Sensors + LCD + P5.js  (change the size of a Web page while moving the sensor wall)
* Upcoming - 12th prototype: Arduino Mega + 7 Sensors + LCD + P5.js (play sounds and affect pitch/tone depending on position one Web page)
* Upcoming: arduino uno + 3 utlrasonic sensor + ESP8266 (WIFI) + Rasperry Pi (Self hosted website) (<b>transmit or/and control value from arduino to computer and vice versa via WIFI transmitter</b>)
* Upcoming: small room + arduino uno + 8 utlrasonic sensor + ESP8266 (WIFI) + Rasperry Pi (connect to a Web page)


 
How does it work?
* <b>2nd semester</b>: Find what will be graduation space, build the mobile wall, and translate the setup installation to human/spectator scale.  
<br><br>
# Show prototype and schemas of the wall to wood and metal workshops in order to get advices until final validation to build (<b>starting to build physical elements</b>)
The ultrasonic sensors can detect obstacles in a physical space and know the distance between the sensor and obstacle(s) by sending and receiving back an ultrasound. The Arduino Uno/Mega boards are microcontrollers which can receive this information, run it in a program in order to convert these values into a mm/cm/m but also map the space into an invisible grid. Ultimately, values collected on the Arduino’s serial monitor can be sent to P5.js through P5.serialcontrol. P5.js will then allow a greater freedom in way the information can be displayed on the screens.
# Search, find and validate what will be the space used for the installation during the graduation.
<br><br>
# Start building of the movable wall by considering the characteristic of the space used for graduation.
Process:
# Implement the sensors inside the movable wall, and the other devices in the fixed space
<br><br>
 
1st semester: Building a monitoring device, converting human actions into events, and events into visual feedbacks
===<p style="font-family:helvetica">Why do you want to make it?</p>===
<br><br>
 
During the first semester, I am focused on exploring monitoring tools that can be used in the physical world, with a specific attention to ultrasonic sensors. Being new to Arduino programming, my way of working is to start from the smallest and most simple prototype and gradually increase its scale/technicality until reaching human/architectural scale. Prototypes are subject to testing, documentation and comments helping to define which direction to follow. The first semester also allows to experiment with different kind of screen (LCD screens, Touch screens, computer screens, TV screens) until finding the most adequate screen monitor(s) for the final installation. Before building the installation, the project is subject to several sketching and animated simulations in 3 dimensions, exploring different scenarios and narrations. At the end of the semester, the goal is to be able to convert a specific range of human actions into events and visual feedback creating a feedback loop from the human behaviors being converted into information; the translation of this information into written/visual feedbacks; and the effects of this feedbacks on human behavior; and so on.
In opposition to the physical exhibition space, <b>the Web offers to each of its user/visitors a custom point of view</b> based on an <b>innumerable and everchanging array of technological factors</b>. I like to call this the technological context. Among these, we could list: the browser, the device, the explotation system, the screensize, the resolution, the user configuration and defaults settings, the updates, the IP adress, etc.. This technological complexity <b>diffracts the possible renders of a same Web page in an almost infinite array of user perspectives</b>. Therefore, <b>this is the nature and meaning of representation itself that is redifined by the Web</b>. Web representations are sort of <b>plastic/elastic</b>, they <b>demultiplies</b> and <b>transforms themselves</b> as much as needed in order to be <b>rendered in an optimal way through our own user perspective/interface</b>. By implementing these notions and properties into the phyiscal exhibition spaces, I would like to <b>put a step in the curatorial practice</b>.
<br><br>
2nd semester: Implementing gamification with the help of collaborative filtering, point system and ranking.
<br><br>
During the second semester, it is all about building and implementing a narration with the help of gaming mechanics that will encourage humans to feed the data gathering device with their own help. An overview of how it works is presented here in the project proposal and will be subject to further developments in the practice.
<br><br>
To summarize the storyline, the subject being positioned in the detection zone finds herself/himself unwillingly embodied as the main actor of a data collection game. Her/His mere presence generates a number of points/dollars displayed on a screen, growing as she/he stays within the area. The goal is simple: to get a higher score/rank and unlock achievements by acting as recommended by a data-collector. This can be done by setting clear goals/rewards to the subject, and putting its own performance in comparison to all the previous visitors, giving unexpected messages/rewards, and give an aesthetic value to the displayed informations.  
<br><br>
The mechanism is based on a sample of physical events that have been already explored in the first semester of prototyping (detection of movements, positions, lapse of time spent standing still or moving, and entry or exit from a specific area of detection). Every single detected event in this installation is stored in a data bank, and with the help of collaborative filtering, will  allow to the display of custom recommendations such as:
<br><br>
_ “Congratulations, you have now given the monitor 12 % of all possible data to collect”<br>
_ “Are you sure you don’t want to move to the left? The monitor has only collected data from 3 visitors so far in this spot”<br>
_ “Congratulations, the monitor has reached 1000 pieces of information from you!”<br>
_ “If you stay there for two more minutes, there is a 99% chance you will be in the top 100 of ALL TIME data-givers”<br>
_ “Leaving all-ready? The monitor has yet 304759 crucial pieces of information to collect from you”<br>
_ “You are only 93860 actions away from being the top one data-giver”<br>
_ “Statistics are showing that people staying for more than 5 minutes average will be 10 times more benefitting for me”<br>
_ “The longer you stay on this spot, the more chance you have to win a “Lazy data-giver” badge”<br>
<br><br>
<br><br>
From our own user perspective/point of view, behind our own screen, <b>this technological complexity and the infinite spectrum of perspectives that it leads to can hardly be considered</b> (expect [http://whatyouseeiswhatyouget.net/ here] for example). This brings us to <b>uncounsioulsy forget about the singularity and fragility of what we is being seen/experienced/interpretated</b>. By creating a physical interface conceived on the model of a responsive Web page, I want to give to the visitors the power to <b>manipulate and diffract this spectrum of perspectives by their own hands</b> and <b>to consider the role and effects of these mediating technologies on the visitor's behaviors and perception of an artwork/content.</b>
The guideline is set out here, but will be constantly updated with the help of experiments and the results observed during the various moments of interaction between the students and the algorithm. For this purpose, the installation under construction will be left active and autonomous in its place of conception (studio) and will allow anyone who deliberately wishes to interact with it to do so. Beyond the voluntary interactions, my interest is also to see what can be extracted from people simply passing in front of this installation. In addition to this, some of the mechanics of the installation will be further explored by collaborating with other students, and setting up more ephemeral and organized experiences with the participants. (ex: 15 February 2022 with Louisa)
<br><br>
<br><br>
On the other hand, I am attached to the idea of <b>reversing the desktop metaphor</b>. The desktop metaphors refers to the terms and objects that the Web borrowed from the physical world in order to make its own concepts more familiar and intelligible to its users. Now, largely democratized and widely spread in modern society, <b>people may have now more concrete experiences of the digital/Web interface than the physical space</b>. Museums, hotels, houses, cars interiors, restaurants are themselves becoming more and more comparable to digital interface where everything is optimized, and where our <b>behaviours, actions and even inactions are being detected and converted into commands in order to offer a more customized (and lucrative) experience to each of us</b>. In that sense, we are <b>getting closer from becoming users of our own interfaced/augmented physical realities</b>. By creating a exhibition spaces explicitly inspired from a desktop Web interface, I wish to question <b>what could be the future of exhibition space, and what are the vulnerabilities of such technologies</b>.  
This semester will also include the creation of a definite set of illustrations participating to engage the participants of the installation in a more emotional way, the illustrations will be made by an illustrator/designer, with whom I usually collaborate.  
<br><br>
<br><br>
Conceiving the exhibition space as a Web interface, and the spectator as a user is also about <b>puting together two layers of reality that are too often clearly opposed/seperated(IRL VS Online)</b>. This is about <b>experiencing their ambiguities, similarities, and differences</b>. It is about <b>reconsidering their modalities by making them reflect on each others, and <b>making the user/spectator reflect on its own agencies inside of them</b>. (see: [[Martin_(XPUB)-thesis_outline#II._Reversing_the_desktop_metaphor|Reversing the desktop metaphor]])
3rd semester: Build the final installation of final assessment and graduation show. Test runs, debug and final touchs.
<br><br>
<br><br>
More generally, it is about <b>reflecting on media itself</b>, and deal with a paradox that I've always found interesting: The better a medium mediates, the more it becomes invisible and unconsidered. </b>  see: [[Martin_(XPUB)-thesis_outline#2._MEDIATIZING_THE_MEDIA_.28Optional.29|Mediatizing the media]]). This observation makes me want to mediate the media and to <b>give spectators more occasions to focus on what is containing, surrounding, holding or hosting a representation</b> instead of giving all our focus on the representation itself.
During the third semester, the installation should be settled in the school, in the alumni area, next to XPUB studio for the final assessment, and ultimately settled again at WORM for the graduation show. I am interested in putting this installation into privileged spaces of human circulation, (such as hallways) that would more easily involve the detection of people, and highlight the intrusive aspect of such technologies. The narration, the mechanics, illustrations and graphic aspect should be finalized at the beginning of the 3rd semester, and subject to intense test runs during all that period until meeting the deadline.


===<p style="font-family:helvetica">Who can help you?</p>===


*About the <b>overall project</b>
===Relation to larger context===


# Stephane Pichard, ex-teacher and ex production-tutor <b>for advices and help about scenography</b>
As GAFAM companies are facing more and more legal issues, and held accountable in growing numbers of social and political issues around the world, the pandemic context has greatly contributed to make all of us more dependent than ever on the online services provided by these companies and to somehow force our consent. While two decades of counter-terrorism measures legitimized domestic and public surveillance techniques such as online and video monitoring, the current public health crisis made even more necessary the use of new technologies for regulating the access to public spaces and services, but also for socializing, working together, accessing to culture, etc. In a lot of countries, from a day to another, and for an undetermined time, it has become necessary to carry a smartphone (or a printed QR code) in order to get access transport, entertainment, cultural and catering services, but also in order to do simple things such as to have a look at the menu in a bar/restaurant or to make an order.. Thus, this project takes place in a context where techno-surveillance has definitely taken a determining place in the way we can access spaces and services related to the physical world. <br><br>
# Emmanuel Cyriaque: my ex-teacher and writting-tutor <b>for advices and help contextualize my work</b>


*About <b>Arduino</b>
Data Marketisation / Self Data: Quantified Self / Attention Economy / Public Health Surveillance / Cybernetics 


# XPUB Arduino Group
# Dennis de Bel
# Aymeric Mansoux


*About <b>Rasperry Pi</b>
===Who can help you?===


# XPUB2 students (Jacopo, Camillo, Federico, etc)
<b>Prototyping</b><br>
# Michael Murtaugh
Manetta Berends<br>
Michael Murtaugh<br>
<br>
<b>Writing</b> <br>
Luke Williams<br>
<br>
<b>Arduino</b><br>
XPUB Arduino knowledge sharing group<br>
Dennis de Bel <br>
Aymeric Mansoux<br>
<br>
<b>Installation building</b><br>
Wood station<br>
Interaction station <br>
<br>
<b>Illustrations</b><br>
Adrien Jacquemet (illustrator/graphic designer)<br>
<br>
<b>Installation location</b> <br>
Leslie Robbins<br>
<br>
<b>Extra help for game narratives</b><br>
Rosa Zangenberg (writer/artist)<br>


*About <b>creating the physical elements</b>:
===Relation to previous practice?===


# Wood station (for movable walls)
During my previous studies in graphic design, I started being engaged with the new media by making a small online reissue of Raymond Queneau’s book called Exercices de Style. In this issue called Incidences Médiatiques (2017), the user/reader was encouraged to explore the 99 different versions of a same story written by the author in a less-linear way. The idea was to consider each user graphic user interface as a unique reading context. It would determine which story could be read, depending on the device used by the reader, and the user could navigate through these stories by resizing the Web window, by changing browser or by using on a different device.
# Metal station (for rails)
<br><br>
# Interaction station (for arduino/rasperyPi assistance)
As part of my graduation project called Media Spaces (2019), I wanted to reflect on the status of networked writing and reading, by programming my thesis in the form of Web to Print website. Subsequently, this website became translated in the physical space as a printed book, and a series of installations displayed in an exhibition space that was following the online structure of my thesis (home page, index, part 1-2-3-4). In that way, I was interested to inviting to visitors to make a physical experience some aspects of the Web
<br><br>
As a first-year student of Experimental Publishing, I continued to work in that direction by eventually creating a meta-website called Tense (2020) willing to display the invisible html <meta> tags inside of an essay in order to affect our interpretation of the text. In 2021, I worked on a geocaching pinball game highlighting invisible Web event, and a Web oscillator, which amplitude and frequency range were directly related to the user’s cursor position and browser screen-size.
<br><br>
While it has always been clear to me that these works were motivated by the desire to define media as context, subject or/and content, the projects presented here have often made use of surveillance tools to detect and translate user information into feedbacks, participating in the construction of an individualized narrative or/and a unique viewing/listening context (interaction, screen size, browser, mouse position). The current work aims to take a critical look at the effect of these practices in the context of techno surveillance.
<br><br>
Similarly, projects such as Media Spaces have sought to explore the growing confusion between human and web user, physical and virtual space or online and offline spaces. This project will demonstrate that these growing confusions will eventually lead us to be tracked in all circumstances, even in our most innocuous daily human activities/actions.


*About <b>theory/writting practice</b>:


# Rosa Zangenberg: ex-student in history art and media at Leiden Universtity.
===Selected References===
# Yael: ex-student in philosophy, getting started with curatorial practice and writtings about the challenges/modalities of the exhibition space. Philosophy of the media (?)


*About <b>finding an exhibiting space</b>:
<b>Works:</b>


# Leslie Robbins
* M. DARKE, fairlyintelligent.tech (2021) https://fairlyintelligent.tech/
« invites us to take on the role of an auditor, tasked with addressing the biases in a speculative AI »Alternatives to techno-surveillance
<br>
* MANUEL BELTRAN, Data Production Labor (2018) https://v2.nl/archive/works/data-production-labour/
Expose humans as producers of useful intellectual labor that is benefiting to the tech giants and the use than can be made out of that labor.
<br>
* TEGA BRAIN and SURYA MATTU, Unfit-bits (2016) http://tegabrain.com/Unfit-Bits
Claims that that technological devices can be manipulated easily and hence, that they are fallible and subjective. They do this by simply placing a self-tracker (connected bracelet) in another context, such as on some other objects, in order to confuse these devices.
<br>
* JACOB APPELBAUM, Autonomy Cube (2014), https://www.e-flux.com/announcements/2916/trevor-paglen-and-jacob-appelbaumautonomy-cube/
Allows galleries to enjoy encrypted internet access and communications, through a Tor Network
<br>
* STUDIO MONIKER, Clickclickclick.click (2016) https://clickclickclick.click/
You are rewarded for exploring all the interactive possibilities of your mouse, revealing how our online behaviors can be monitored and interpretated by machines.
<br>
* RAFAEL LOZANO-HEMMER, Third Person (2006) https://www.lozano-hemmer.com/third_person.php
Portrait of the viewer is drawn in real time by active words, which appear automatically to fill his or her silhouette https://www.lozano-hemmer.com/third_person.php
<br>
* JONAL LUND, What you see is what you get (2012) http://whatyouseeiswhatyouget.net/
«Every visitor to the website’s browser size, collected, and played back sequentially, ending with your own.»
<br>
* USMAN HAQUE, Mischievous Museum (1997) https://haque.co.uk/work/mischievous-museum/
Readings of the building and its contents are therefore always unique -- no two visitors share the same experience. https://haque.co.uk/work/mischievous-museum/
<br><br>
<b>Books & Articles:</b>


===<p style="font-family:helvetica">Relation to previous practice</p>===
* SHOSHANA ZUBOFF, The Age of Surveillance Capitalism (2020)
 
Warns against this shift towards a «surveillance capitalism». Her thesis argues that, by appropriating our personal data, the digital giants are manipulating us and modifying our behavior, attacking our free will and threatening our freedoms and personal sovereignty.
During the first part of my previous studies, I really started being engaged into questioning the media by making a small online reissue of Raymond Queneau's book <i>Exercices de Style</i>. In this work called <i>Incidences Médiatiques</i>, the user/reader was encouraged to explore the 99 differents ways to tell a same story from Queneau, by putting itself in at least 99 different reading contexts. In order to suggest a more <b>non-linear reading experience, reflecting on the notion of context, perpective and point of view</b>, the user could unlock these stories by zooming-in or out the Web window, resizing it, changing the browser, going on a different device, etc. As part of my previous graduation project, I wanted to <b>reflect on the status of networked writing and reading</b>, by programming my thesis in the form of Web to Print Website. Subsequently, this website became <b>translated in the physical space</b> as a printed book, a set of meta flags, and a series of installations displayed in <b>a set of exhibition rooms that was following the online structure of thesis</b> (home page, index, part 1-2-3-4) [https://martinfoucaut.com/ESPACES-MEDIATIQUES-EN Project link]. It was my first attempt to create a physical interface inside an exhibition space, but it was focused on the structure and non-linear navigation rather than the elastic property of Web Interfaces. As a first year student of Experimental Publishing, I continued to work in that direction by eventually <b>creating a [https://issue.xpub.nl/13/TENSE/ meta-website]</b> making visible html <meta> tags in the middle of an essay. I also worked a [[SPECIAL_ISSUE_14_MARTIN_BOARD#GAME_PROTOTYPING|geocaching pinball]] <b>highligting invisible Web events</b> as well as a [[User:Martin#XPUB_Festival:_window.open_.2B_Civil_Entertainment_Sirens_.28audiovisual_performance.29|Web oscillator]] inspired from analog instruments's body size, and which amplitude and frequency range were directly related <b> to the user's device screen-size </b>.
<br>
 
* EVGENY MOROZOV, Capitalism’s New Clothes (2019)
[[File:Incidences médiatiques .gif|250px|thumb|left|Incidences Médiatiques <br><b>click to watch GIF</b>]]
Extensive analysis and critic of Shoshana Zuboff research and publications.
[[File:Martin Foucaut Tense V3.gif|250px|thumb|right|Special issue 13 - Wor(l)ds for the Future<br> Tense screen recording montage of <i>Tense</i><br><b>click to watch GIF</b>]]
<br>
[[File:ESPACESMEDITATIQUE MARTINFOUCAUT 07.jpg|250px||thumb|left|Media Spaces - graduation project]]
* BYRON REEVES AND CLIFFORD NASS, The Media Equation, How People Treat Computers, Television, and New Media Like Real People and Places (1996)
[[File:ESPACESMEDITATIQUE MARTINFOUCAUT 06.jpg|250px|thumb|right|Media Spaces - graduation project]]
Precursor study of the relation between humans and machine, and how do you human relate to them.
 
<br>
[[File:ESPACESMEDITATIQUE MARTINFOUCAUT 09.jpg|250px|thumb|left|Media Spaces - graduation project|center]]
* OMAR KHOLEIF, Goodbye, World! — Looking at Art in the digital Age (2018)  
[[File:ESPACESMEDITATIQUE MARTINFOUCAUT 04.jpg|250px|thumb|right|Media Spaces - graduation project|center]]
Authors shares it’s own data as a journal in a part of the book, while on another part, question how the Internet has changed the way we perceive and relate, and interact with/to images.
 
<br>
<br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br>
* KATRIN FRITSCH, Towards an emancipatory understanding of widespread datafication (2018)
 
Suggests that in response to our society of surveillance, artists can suggest activist response that doesn’t necessarily involve technological literacy, but instead can promote strong counter metaphors or/and counter use of these intrusive technologies.
===<p style="font-family:helvetica">Relation to a larger context</p>===
 
With the growing presence of digital tools in all aspects of our lives, the distinctions between the physical and virtual worlds are being blurred as they tend to affect each other, create interdependencies, and transform our environments into cybernetic spaces. where our behaviours tend to be translated into informational units. In that context, amy aim is to observe and speculate on the ways cybernetics could considerably change and affect modes of representation inside exhibition spaces, and the agencies or status of the visitors themselves.
 
<b>Curatorial Practice</b> / <b>New Media Art</b> / <b>Information Visualization</b> / <b>Software Art</b> / <b>Institutional Critique</b> / <b>Human Sciences</b> / <b>Cybernetics</b>
 
===<p style="font-family:helvetica">Key References</p>===
 
* [https://clickclickclick.click/ Clickclickclick.click] -  VPRO Medialab & Moniker
 
*Stéphanie Moser, 2010. [[THE_DEVIL_IS_IN_THE_DETAILS:_MUSEUM_-_Displays_and_the_Creation_of_Knowledge|THE DEVIL IS IN THE DETAILS: MUSEUM - Displays and the Creation of Knowledge]]. 1st ed. Southampton, England
 
*Alexander R. Galloway - [[Alexander_R._Galloway_-_The_Interface_Effect|The Interface Effect]] 1st ed. Malden, USA: Polity Press.
 
*Jonas Lund, 2012. [http://whatyouseeiswhatyouget.net/ What you see is what you get]
 
*Shilpa Gupta, 2009 - 2010. [http://shilpagupta.com/speaking-wall/| Speaking Wall]
 
*Frederick Kiesler, 1925, [https://thecharnelhouse.org/2013/11/19/frederick-kiesler-city-of-space-1925/ City of space]
 
*Brendan Howell, 2017(?) - [http://screenl.es/ The screenless office]


==<p style="font-family:helvetica"> Reading Sources</p> ==
==<p style="font-family:helvetica"> Reading Sources</p> ==
Line 299: Line 414:
=<p style="font-family:helvetica">Draft Thesis</p>=
=<p style="font-family:helvetica">Draft Thesis</p>=


Notes:
===What do you want to make?===


*For what concerns the form/share of the thesis itself, I am considering to create a WebtoPrint thesis that would display a different amount/arrangement of content (text and images), depending on the user's device (context). If this thesis has to be printed, each user should also get a different physical book (format, layout, etc).  
My project is a data collection installation that monitors people's behaviors in public physical spaces while explicitly encouraging them to help the algorithm collect more information. An overview of how it works is presented here in the project proposal and will be subject to further developments in the practice.
* I know what I want to talk about, and how to connect it to my production, but the structure (parts, sub-parts, and order) is definitly not fixed.
<br><br>
* Reasons why I connect the Web digital interfaces to specificaly the physical exhibition space should be clarified
The way the device is designed doesn’t pretend to give any beneficial outcomes for the subject, but only makes visible the benefits that the machine is getting from collecting their data. Yet, the way the device visually or verbally presents this collected data is done in a grateful way, which might be stimulating for the subject. In that sense, the subject, despite knowing that their actions are done solely to satisfy the device, could become intrigued, involved, or even addicted by a mechanism that deliberately uses it as a commodity. In that way, I intend to trigger conflictual feelings in the visitor’s mind, situated between a state of awareness regarding the operating monetization of their physical behaviors, and a state of engagement/entertainment /stimulation regarding the interactive value of the installation.
* Focus on the cybernetic empire, and its influence on different aspects of society (beyond the exhibition space)
<br><br>
* Dystopian space?
My first desire is to make the mechanisms by which data collection is carried out, marketized and legitimized both understandable and accessible. The array of sensors, the Arduinos and the screen are the mainly technological components of this installation. Rather than using an already existing and complex tracking algorithm, the program is built from scratch, kept open source and limits itself to the conversion of a restricted range of physical actions into interactions. These include the detection of movements, positions, lapse of time spent standing still or moving, and entry or exit from a specific area of detection. Optionally they may also include the detection of the subject smartphone device or the log on a local Wi-Fi hotspot made by the subject.
* How systems works, how do they use you, how do they structure you as you
<br><br>
* Where are these sensors are used outside of these exhibition space.
In terms of mechanic, the algorithm creates feedback loops starting from: <br>
_the subject behaviors being converted into information; <br>
_the translation of this information into written/visual feedback; <br>
_and the effects of this feedbacks on subject’s behavior; and so on. <br>
By doing so, it tries to shape the visitors as free data providers inside their own physical environment, and stimulate their engagement by converting each piece of collected information into points/money, feeding a user score among a global ranking.
<br><br>
On the screen, displayed events can be:


==<p style="font-family:helvetica">Introduction</p>==
_ “subject [] currently located at [ ]” <br>
[x points earned/given]<br>
_ “subject [] entered the space” <br>
[x points earned/given]<br>
_ “subject [] left the space”<br>
[x points earned/given]<br>
_ “subject [] moving/not moving”<br>
[x points earned/given]<br>
_ “subject [] distance to screen: [ ] cm” <br>
[x points earned/given]<br>
_ “subject [] stayed at [ ] since [ ] seconds” <br>
[x points earned/given]<br>
_ “subject [] device detected <br>
[x points earned/given] (optional)<br>
_ “subject logged onto local Wi-Fi<br>
[x points earned/given] (optional)<br>
<br>
Added to that comes the instructions and comments from the devices in reaction to the subject’s behaviors:<br>
<br>
_ “Congratulations, you have now given the monitor 25 % of all possible data to collect!” <br>
[when 25-50-75-90-95-96-97-98-99% of the total array of events has been detected at least once]<br>
_ “Are you sure you don’t want to move to the left? The monitor has only collected data from 3 visitors so far in this spot!”<br>
[if the subject stands still in a specific location]<br>
_ “Congratulations, the monitor has reached 1000 pieces of information from you!”<br>
[unlocked at x points earned/given]<br>
_ “If you stay there for two more minutes, there is a 99% chance you will be in the top 100 of ALL TIME data-givers!”<br>
[if the subject stand still in a specific location]<br>
_ “Leaving all ready? The monitor has yet to collect 304759 crucial pieces of information from you!”<br>
[if the subject is a the edge of the detection range]<br>
_ “You are only 93860 pieces of information away from being the top one data-giver!”<br>
[unlocked at x points earned/given]<br>
_ “Statistics show that people staying for more than 5 minutes average will benefit me on average 10 times more!”<br>
[randomly appears]<br>
_ “The longer you stay on this spot, the more chance you have to win a “Lazy data-giver” badge”<br>
[if the subject stands still for a long time any location]<br>
<br>
Responding positively to the monitors instructions unlocks special achievement and extra points<br>
<br>
—Accidental data-giver badge <br>
[unlocked if the subject has passed the facility without deliberately wishing to interact with it] + [x points earned/given]<br>
—Lazy data-giver badge <br>
[unlocked if the subject has been standing still for at least one minute] + [x points earned/given]<br>
—Novice data-giver badge <br>
[unlocked if the subject has been successfully completing 5 missions from the monitor] + [x points earned/given]<br>
—Hyperactive data-giver badge <br>
[unlocked if the subject has never been standing still for 10 seconds within 2 minutes lapse time] + [x points earned/given]<br>
—Expert data-giver badge <br>
[unlocked if the subject has been successfully completing 10 missions from the monitor within 10 minutes] + [x points earned/given]<br>
—Master data-giver badge <br>
[unlocked if the subject has been successfully logging on the local Wi-Fi hotspot] + [x points earned/given] (optional)<br>
<br>
On the top left side of the screen, a user score displays the number of points generated by the collected pieces of information, and the unlocking of special achievements instructed by the monitor.<br>
<br>
—Given information: 298 pieces <br>
[displays number of collected events]<br>
—Points: 312000 <br>
[conversion of collected events and achievement into points]<br>
<br>
On the top right of the screen, the user is ranked among x number of previous visitors and the prestigious badge recently earned is displayed bellow<br>
<br>
—subject global ranking: 3/42 <br>
[compares subject’s score to all final scores from previous subjects]<br>
—subject status:  expert data-giver<br>
[display the most valuable reward unlocked by the subject]<br>
<br>
When leaving the detection range, the subject gets a warning message and a countdown starts, and encouraging it to take the quick decision to come back<br>
<br>
<br>
[...]
—“Are you sure you want to leave? You have 5-4-3-2-1-0 seconds to come back within the detection range”<br>
[displayed as long as the subject remains completely undetected]<br>
<br>
<br>
People may have now <b>more concrete experiences of the digital/Web interfaces than the physical space</b>. Museums, galleries, restaurants, hotels, houses or cars are themselves commonly taking the shape of <b>interfaced physical environments</b>. Both online and in the physical reality, our (past and present) behaviours (and non-behaviours) can be detected and used by the use of various technologies in order to provide us an <b>individualized/customizable perception/experience of a same environment</b>. With the embodiement of user interfaces into many layers of the physical reality</b>, the so called "reality" becomes not only augmented but also optimized to our very own perception. In that sense, "what you see is what you get" may also define new modalities to the spaces of representation (exhibition spaces), the agencies of its visitors, and the experience of art itself.  
If the subject definitely stands out of the detection range for more than 5 seconds, the monitor will also address a thankful message and the amount of money gathered, achievements, ranking, complete list of collected information and a qr code will be printed as a receipt with the help of a thermal printer. The QR will be a link to my thesis.<br>
<br>
—* “Thank you for helping today, don’t forget to take your receipt in order to collect and resume your achievements”<br>
[displayed after 5 seconds being undetected]<br>
<br>
In order to collect, read or/and use that piece of information, the visitor will inevitably have to come back within the range of detection, and intentionally, or not, reactivate the data tracking game. It is therefore impossible to leave the area of detection without leaving at least one piece of your own information printed in the space. Because of this, the physical space should gradually be invaded by tickets scattered on the floor. As in archaeology, these tickets give a precise trace of the behavior and actions of previous subjects for future subjects. <br>


Through <b>digital and analogical comparisons</b> between Web interfaces and Exhibition space, I wish to find out what are the elements defining, communicating or giving stucture to contents inside both of these spaces, how do they relate to each others, and what could be the effects of mergings their concepts/properties.
===Why do you want to make it?===


=<p style="font-family:helvetica">I.   Agencies and factors within the spaces of representation</p>=
When browsing online or/and using connected devices in the physical world, even the most innocent action/information can be invisibly recorded, valued and translated into informational units, subsequently generating profit for monopolistic companies. While social platforms, brands, public institutions and governments explicitly promote the use of monitoring practices in order to better serve or protect us, we could also consider these techniques as implicitly political, playing around some dynamics of visibility and invisibility in order to assert new forms of power over targeted audiences.
<br><br>
In the last decade, a strong mistrust of new technologies has formed in the public opinion, fueled by events such as the revelations of Edward Snowden, the Cambridge Analytica scandal or the proliferation of fake news on social networks. We have also seen many artists take up the subject, sometimes with activist purposes. But even if a small number of citizens have begun to consider the social and political issues related to mass surveillance, and some individuals/groups/governments/associations have taken legal actions, surveillance capitalism still remains generally accepted, often because ignored or/and misunderstood.
<br><br>
Thanks to the huge profits generated by the data that we freely provide every day, big tech companies have been earning billions of dollars over the sale of our personal information. With that money, they could also further develop deep machine learning programs, powerful recommendation systems, and to broadly expand their range of services in order to track us in all circumstances and secure their monopolistic status. Even if we might consider this realm specific to the online world, we have seen a gradual involvement from the same companies to monitor the physical world and our human existences in a wide array of contexts. For example, with satellite and street photography (Google Earth, Street View), geo localization systems, simulated three-dimensional environments (augmented reality, virtual reality or metaverse) or extensions of our brains and bodies (vocal assistance and wearable devices). Ultimately, this reality has seen the emergence of not only a culture of surveillance but also of self-surveillance, as evidenced by the popularity of self-tracking and data sharing apps, which legitimize and encourage the datafication of the body for capitalistic purposes.
<br><br>
For the last 15 years, self-tracking tools have made their way to consumers. I believe that this trend is showing how ambiguous our relationship can be with tools that allow such practices. Through my work, I do not wish to position myself as a whistleblower, a teacher or activist. Indeed, to adopt such positions would be hypocritical, given my daily use of tools and platforms that resort to mass surveillance. Instead, I wish to propose an experience that highlights the contradictions in which you and I, internet users and human beings, can find ourselves. This contradiction is characterized by a paradox between our state of concern about the intrusive surveillance practices operated by the Web giants (and their effects on societies and humans) and a state of entertainment or even active engagement with the tools/platforms through which this surveillance is operated/allowed. By doing so, I want to ask how do these companies still manage to get our consent and what human biases do they exploit in order to do so. That’s is how my graduation work and my thesis will investigate the effect of gamification, gambling or reward systems as well as a the esthetization of data/self-data as means to hook our attention, create always more interactions and orientate our behaviors.




==<p style="font-family:helvetica">1.        THE AGENCIES OF USERS & SPECTATORS</p>==
===How to you plan to make it and on what timetable?===


What are users and spectators allowed or expected to do, what should they agree on, what is the meaning of being a spectator or a user, what is the purpose of an exhibition space or an Web interface, how could both differs and relate to each others on all these aspect.
I am developing this project with Arduino Uno/Mega boards, an array of ultrasonic sensor, P5.js and screens.<br><br>


===1.1            The user agency through the Web interfaces===
How does it work?
<br><br>
The ultrasonic sensors can detect obstacles in a physical space and know the distance between the sensor and obstacle(s) by sending and receiving back an ultrasound. The Arduino Uno/Mega boards are microcontrollers which can receive this information, run it in a program in order to convert these values into a mm/cm/m but also map the space into an invisible grid. Ultimately, values collected on the Arduino’s serial monitor can be sent to P5.js through P5.serialcontrol. P5.js will then allow a greater freedom in way the information can be displayed on the screens.
<br><br>
Process:
<br><br>
1st semester: Building a monitoring device, converting human actions into events, and events into visual feedbacks
<br><br>
During the first semester, I am focused on exploring monitoring tools that can be used in the physical world, with a specific attention to ultrasonic sensors. Being new to Arduino programming, my way of working is to start from the smallest and most simple prototype and gradually increase its scale/technicality until reaching human/architectural scale. Prototypes are subject to testing, documentation and comments helping to define which direction to follow. The first semester also allows to experiment with different kind of screen (LCD screens, Touch screens, computer screens, TV screens) until finding the most adequate screen monitor(s) for the final installation. Before building the installation, the project is subject to several sketching and animated simulations in 3 dimensions, exploring different scenarios and narrations. At the end of the semester, the goal is to be able to convert a specific range of human actions into events and visual feedback creating a feedback loop from the human behaviors being converted into information; the translation of this information into written/visual feedbacks; and the effects of this feedbacks on human behavior; and so on.
<br><br>
2nd semester: Implementing gamification with the help of collaborative filtering, point system and ranking.
<br><br>
During the second semester, it is all about building and implementing a narration with the help of gaming mechanics that will encourage humans to feed the data gathering device with their own help. An overview of how it works is presented here in the project proposal and will be subject to further developments in the practice.
<br><br>
To summarize the storyline, the subject being positioned in the detection zone finds herself/himself unwillingly embodied as the main actor of a data collection game. Her/His mere presence generates a number of points/dollars displayed on a screen, growing as she/he stays within the area. The goal is simple: to get a higher score/rank and unlock achievements by acting as recommended by a data-collector.  This can be done by setting clear goals/rewards to the subject, and putting its own performance in comparison to all the previous visitors, giving unexpected messages/rewards, and give an aesthetic value to the displayed informations.
<br><br>
The mechanism is based on a sample of physical events that have been already explored in the first semester of prototyping (detection of movements, positions, lapse of time spent standing still or moving, and entry or exit from a specific area of detection). Every single detected event in this installation is stored in a data bank, and with the help of collaborative filtering, will  allow to the display of custom recommendations such as:
<br><br>
_ “Congratulations, you have now given the monitor 12 % of all possible data to collect”<br>
_ “Are you sure you don’t want to move to the left? The monitor has only collected data from 3 visitors so far in this spot”<br>
_ “Congratulations, the monitor has reached 1000 pieces of information from you!”<br>
_ “If you stay there for two more minutes, there is a 99% chance you will be in the top 100 of ALL TIME data-givers”<br>
_ “Leaving all-ready? The monitor has yet 304759 crucial pieces of information to collect from you”<br>
_ “You are only 93860 actions away from being the top one data-giver”<br>
_ “Statistics are showing that people staying for more than 5 minutes average will be 10 times more benefitting for me”<br>
_ “The longer you stay on this spot, the more chance you have to win a “Lazy data-giver” badge”<br>
<br><br>
The guideline is set out here, but will be constantly updated with the help of experiments and the results observed during the various moments of interaction between the students and the algorithm. For this purpose, the installation under construction will be left active and autonomous in its place of conception (studio) and will allow anyone who deliberately wishes to interact with it to do so. Beyond the voluntary interactions, my interest is also to see what can be extracted from people simply passing in front of this installation. In addition to this, some of the mechanics of the installation will be further explored by collaborating with other students, and setting up more ephemeral and organized experiences with the participants. (ex: 15 February 2022 with Louisa)
<br><br>
This semester will also include the creation of a definite set of illustrations participating to engage the participants of the installation in a more emotional way, the illustrations will be made by an illustrator/designer, with whom I usually collaborate.
<br><br>
3rd semester: Build the final installation of final assessment and graduation show. Test runs, debug and final touchs.
<br><br>
During the third semester, the installation should be settled in the school, in the alumni area, next to XPUB studio for the final assessment, and ultimately settled again at WORM for the graduation show. I am interested in putting this installation into privileged spaces of human circulation, (such as hallways) that would more easily involve the detection of people, and highlight the intrusive aspect of such technologies. The narration, the mechanics, illustrations and graphic aspect should be finalized at the beginning of the 3rd semester, and subject to intense test runs during all that period until meeting the deadline.


====1.1.1            Terms, conditions, agreements====


Cookies, privacy, legal uses, advertisment, copyrights, etc
===Relation to larger context===


===1.2            The spectator agency through the Exhibition Spaces/Museums/Galleries===
As GAFAM companies are facing more and more legal issues, and held accountable in growing numbers of social and political issues around the world, the pandemic context has greatly contributed to make all of us more dependent than ever on the online services provided by these companies and to somehow force our consent. While two decades of counter-terrorism measures legitimized domestic and public surveillance techniques such as online and video monitoring, the current public health crisis made even more necessary the use of new technologies for regulating the access to public spaces and services, but also for socializing, working together, accessing to culture, etc. In a lot of countries, from a day to another, and for an undetermined time, it has become necessary to carry a smartphone (or a printed QR code) in order to get access transport, entertainment, cultural and catering services, but also in order to do simple things such as to have a look at the menu in a bar/restaurant or to make an order.. Thus, this project takes place in a context where techno-surveillance has definitely taken a determining place in the way we can access spaces and services related to the physical world. <br><br>


====1.2.1            Rules, safety, regulations====
Data Marketisation / Self Data: Quantified Self / Attention Economy / Public Health Surveillance / Cybernetics 


Artwork safety, public safety, prohibed items, public speaking, photography, equipments, behavior, circulation, etc.
Maybe even more than on the Web, being a gallery/museum visitor implies to agree on terms and conditions.


* example: Louvres Visitors rules: https://www.louvre.fr/en/visit/museum-rules
===Who can help you?===


==<p style="font-family:helvetica">2.  SPATIAL/TECHNOLOGICAL FACTORS</p>==
<b>Prototyping</b><br>
Manetta Berends<br>
Michael Murtaugh<br>
<br>
<b>Writing</b> <br>
Luke Williams<br>
<br>
<b>Arduino</b><br>
XPUB Arduino knowledge sharing group<br>
Dennis de Bel <br>
Aymeric Mansoux<br>
<br>
<b>Installation building</b><br>
Wood station<br>
Interaction station <br>
<br>
<b>Illustrations</b><br>
Adrien Jacquemet (illustrator/graphic designer)<br>
<br>
<b>Installation location</b> <br>
Leslie Robbins<br>
<br>
<b>Extra help for game narratives</b><br>
Rosa Zangenberg (writer/artist)<br>


What are the spatial, technological, political factors defining the context/situation in which the representation is being displayed and experienced.
===Relation to previous practice?===


===<p style="font-family:helvetica">2.1            Technological context of the Web</p>===
During my previous studies in graphic design, I started being engaged with the new media by making a small online reissue of Raymond Queneau’s book called Exercices de Style. In this issue called Incidences Médiatiques (2017), the user/reader was encouraged to explore the 99 different versions of a same story written by the author in a less-linear way. The idea was to consider each user graphic user interface as a unique reading context. It would determine which story could be read, depending on the device used by the reader, and the user could navigate through these stories by resizing the Web window, by changing browser or by using on a different device.
<br><br>
As part of my graduation project called Media Spaces (2019), I wanted to reflect on the status of networked writing and reading, by programming my thesis in the form of Web to Print website. Subsequently, this website became translated in the physical space as a printed book, and a series of installations displayed in an exhibition space that was following the online structure of my thesis (home page, index, part 1-2-3-4). In that way, I was interested to inviting to visitors to make a physical experience some aspects of the Web
<br><br>
As a first-year student of Experimental Publishing, I continued to work in that direction by eventually creating a meta-website called Tense (2020) willing to display the invisible html <meta> tags inside of an essay in order to affect our interpretation of the text. In 2021, I worked on a geocaching pinball game highlighting invisible Web event, and a Web oscillator, which amplitude and frequency range were directly related to the user’s cursor position and browser screen-size.
<br><br>
While it has always been clear to me that these works were motivated by the desire to define media as context, subject or/and content, the projects presented here have often made use of surveillance tools to detect and translate user information into feedbacks, participating in the construction of an individualized narrative or/and a unique viewing/listening context (interaction, screen size, browser, mouse position). The current work aims to take a critical look at the effect of these practices in the context of techno surveillance.
<br><br>
Similarly, projects such as Media Spaces have sought to explore the growing confusion between human and web user, physical and virtual space or online and offline spaces. This project will demonstrate that these growing confusions will eventually lead us to be tracked in all circumstances, even in our most innocuous daily human activities/actions.


====2.1.1                An infinite array of individualized/customized perpectives ====


In opposition to the physical exhibition space, <b>the Web offers to each of its user/visitors a custom point of view</b> based on an <b>innumerable and everchanging array of technological factors</b>. I like to call this array of factors: a (technological) context. Among these factors, we could list: the browser, the device, the explotation system, the screensize, the resolution, the user configuration and defaults settings, the updates, the IP adress, etc.. This technological complexity <b>diffracts the possible renders of a same Web page in an almost infinite array of user perspectives</b>.  From our own user perspective/point of view, behind our own screen, <b>this technological complexity and the infinite spectrum of perspectives that it leads to can hardly be considered</b> (expect [http://whatyouseeiswhatyouget.net/ here] for example). This brings us to <b>uncounsioulsy forget about the singularity and fragility of what we is being seen/experience/interpretated</b>
===Selected References===


Ref:
<b>Works:</b>
* [http://whatyouseeiswhatyouget.net/ What you see is what you get — Jonas Lund] (2012)


====  2.1.2              Elasticity, obsolescence and unpredictability / Responsive technology====
* M. DARKE, fairlyintelligent.tech (2021) https://fairlyintelligent.tech/
 
« invites us to take on the role of an auditor, tasked with addressing the biases in a speculative AI »Alternatives to techno-surveillance
Web representations are sort of  <b>plastic/elastic</b>, they <b>demultiplies</b> and <b>transforms themselves</b> as much as needed in order to be <b>rendered in an optimal way through our own user perspective/interface</b>. Added to that, the display/render of a website are also affected by the constant evolution of the Web fitself, with <b>patches, updates, expired and added elements that contribute to the ephemerality and unpredictability of what can be seen</b>. In order to overcome the impredictability of rendering online interfaces among the incredible diversity of connected devices, a <b>technology of flexibility/responsiveness/elasticity</b> has been developped, improved and  democratised on the Web, and willing to offer an optimal render in most technological contexts.
<br>
 
* MANUEL BELTRAN, Data Production Labor (2018) https://v2.nl/archive/works/data-production-labour/
Ref:  
Expose humans as producers of useful intellectual labor that is benefiting to the tech giants and the use than can be made out of that labor.
*[[Plasticity of User Interfaces:A Revised Reference Framework]] NOTES INSIDE<br>
<br>
Gaëlle Calvary, Joëlle Coutaz, David Thevenin
* TEGA BRAIN and SURYA MATTU, Unfit-bits (2016) http://tegabrain.com/Unfit-Bits
Quentin Limbourg, Nathalie Souchon, Laurent Bouillon, Murielle Florins, Jean Vanderdonckt
Claims that that technological devices can be manipulated easily and hence, that they are fallible and subjective. They do this by simply placing a self-tracker (connected bracelet) in another context, such as on some other objects, in order to confuse these devices.
<br>
* JACOB APPELBAUM, Autonomy Cube (2014), https://www.e-flux.com/announcements/2916/trevor-paglen-and-jacob-appelbaumautonomy-cube/
Allows galleries to enjoy encrypted internet access and communications, through a Tor Network
<br>  
* STUDIO MONIKER, Clickclickclick.click (2016) https://clickclickclick.click/
You are rewarded for exploring all the interactive possibilities of your mouse, revealing how our online behaviors can be monitored and interpretated by machines.
<br>
* RAFAEL LOZANO-HEMMER, Third Person (2006) https://www.lozano-hemmer.com/third_person.php
Portrait of the viewer is drawn in real time by active words, which appear automatically to fill his or her silhouette https://www.lozano-hemmer.com/third_person.php
<br>
* JONAL LUND, What you see is what you get (2012) http://whatyouseeiswhatyouget.net/
«Every visitor to the website’s browser size, collected, and played back sequentially, ending with your own.»
<br>
* USMAN HAQUE, Mischievous Museum (1997) https://haque.co.uk/work/mischievous-museum/
Readings of the building and its contents are therefore always unique -- no two visitors share the same experience. https://haque.co.uk/work/mischievous-museum/
<br><br>
<br><br>
See more:
<b>Books & Articles:</b>
* Lopez, J.F., Szekely, P., Web page adaptation for Universal Access, in Proc. of Conf. on Universal Access in HCI UAHCI’ 2001<br>
(New Orleans, August 5-10, 2001), Lawrence Erlbaum Associates, Mahwah, 2001,
 
===<p style="font-family:helvetica">2.2            Technological contexts in the museum/exhibition space</p>===
 
====  2.2.1                Public space and agents of the production of knowledge====
 
Architecture, scale, size, interior design, colors, layout, writing, arrangement, lighting, display, etc
 
* Stéphanie Moser, 2010. [[THE  DEVIL  IS IN THE  DETAILS: MUSEUM - Displays  and the Creation of Knowledge]] [https://pzwiki.wdka.nl/mw-mediadesign/images/5/57/The_Devil_is_in_the_details-_DETAIT-_MUSEUM_Displays_and_the_Creation_of_Knowledge.pdf Doc]. 1st ed. Southampton, England
 
====2.2.2              Institutional critique (optional)====
 
Questioning and redifining the exhibition spaces and the heritage from the White Cube by the institutional critique practice (?)
 
* [[From the Critique of Institutions to an Institution of Critique - Andrea Fraser]][https://aaaaarg.fail/upload/andrea-fraser-from-the-critique-of-institutions-to-an-institution-of-critique-1.pdf Doc]
* [http://nt2.uqam.ca/fr/biblio/after-white-cube [[After the White Cube.]]] [https://www.lrb.co.uk/the-paper/v37/n06/hal-foster/after-the-white-cube ref]
 
=<p style="font-family:helvetica">II.    Reversing the desktop metaphor</p>=
 
The desktop metaphor was invented in the early ages of computers for facilitating the use and understanding of the digital interfaces, by making mental associations related to domains from the physical world.  Now democratised, widely used and quiet often replacing our needs to converge in physical spaces, I would like to reverse the process by getting inspired by the concepts of the Web interfaces in order to suggest a singular experience and understanding of the physical exhibition space who is awell another space of representation.
 
==<p style="font-family:helvetica">1. CONCEPTS OF THE INTERFACED REALITY</p>==
 
Conceiving the exhibition space as a digital Web interface and exploring concepts that bring together notions from both digital and physical world.
 
*Ref: The screenless office - Brendan Howell (http://screenl.es/)
 
===1.1            "Architectural Device" ===
 
Conceiving the architecture as a technological and political device made of a set of factors and parameters that can be configured
 
===1.2            "Physical Events" ===
 
On the Web, our actions and inactions can be converted into (silent and invisible) events that can give activate things and be converted into valuable informations for advertisers, algorythms, etc.
How could such thing be conceptualized inside an exhibition space.
 
===1.3            "Programmed physical space" ===
 
Comparing the programming of an interface with the curation of a exhbibition space. Could an exhibition space be programmed?
 
===1.4            "Exhibition User" ===
 
Conceiving the Spectator as a User of the physical space
 
===1.5            "Variable Display" ===
 
Conceiving the physical space as an elastic/variable and potentially unpredicatable display; in order to diffract the range of viewing contexts offered by the Web.
 
=<p style="font-family:helvetica">Conclusion</p>=
 
[...]
 
==<p style="font-family:helvetica">References</p>==
   
 
* [https://clickclickclick.click/ Clickclickclick.click] -  VPRO Medialab & Moniker
 
*Stéphanie Moser, 2010. [[THE_DEVIL_IS_IN_THE_DETAILS:_MUSEUM_-_Displays_and_the_Creation_of_Knowledge|THE DEVIL IS IN THE DETAILS: MUSEUM - Displays and the Creation of Knowledge]]. 1st ed. Southampton, England
 
*Alexander R. Galloway - [[Alexander_R._Galloway_-_The_Interface_Effect|The Interface Effect]] 1st ed. Malden, USA: Polity Press.
 
*Jonas Lund, 2012. [http://whatyouseeiswhatyouget.net/ What you see is what you get]
 
*Shilpa Gupta, 2009 - 2010. [http://shilpagupta.com/speaking-wall/| Speaking Wall]
 
*Frederick Kiesler, 1925, [https://thecharnelhouse.org/2013/11/19/frederick-kiesler-city-of-space-1925/ City of space]
 
*Brendan Howell, 2017(?) - [http://screenl.es/ The screenless office]


* SHOSHANA ZUBOFF, The Age of Surveillance Capitalism (2020)
Warns against this shift towards a «surveillance capitalism». Her thesis argues that, by appropriating our personal data, the digital giants are manipulating us and modifying our behavior, attacking our free will and threatening our freedoms and personal sovereignty.
<br>
* EVGENY MOROZOV, Capitalism’s New Clothes (2019)
Extensive analysis and critic of Shoshana Zuboff research and publications.
<br>
* BYRON REEVES AND CLIFFORD NASS, The Media Equation, How People Treat Computers, Television, and New Media Like Real People and Places (1996)
Precursor study of the relation between humans and machine, and how do you human relate to them.
<br>
* OMAR KHOLEIF, Goodbye, World! — Looking at Art in the digital Age (2018)
Authors shares it’s own data as a journal in a part of the book, while on another part, question how the Internet has changed the way we perceive and relate, and interact with/to images.
<br>
<br>
More [[XPUB2_Research_Board_/_Martin_Foucaut#Readings_.28new.29.28english.29.28with_notes_in_english.29|here]]
* KATRIN FRITSCH, Towards an emancipatory understanding of widespread datafication (2018)
Suggests that in response to our society of surveillance, artists can suggest activist response that doesn’t necessarily involve technological literacy, but instead can promote strong counter metaphors or/and counter use of these intrusive technologies.


=<p style="font-family:helvetica">What is my work, What do I want to tell, What is my position</p>=
=<p style="font-family:helvetica">What is my work, What do I want to tell, What is my position</p>=
Line 857: Line 1,072:
[[File:Sensor Wall 01.png|300px|thumb|left|Sensor Wall 01]]
[[File:Sensor Wall 01.png|300px|thumb|left|Sensor Wall 01]]
[[File:PhysicalMapping3.png|500px|thumb|right|PhysicalMapping2]]
[[File:PhysicalMapping3.png|500px|thumb|right|PhysicalMapping2]]
<br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br>
<br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br>
  //LIBRARIES
 
   
===<p style="font-family:helvetica">Sketch 10:  Arduino Mega + 7 Sensors + LCD + 3 buzzers + P5.js </p> ===
    #include "pitches.h" //PITCH
 
    #include <LiquidCrystal.h> //LCD
[[File:P5.js sensor.gif|300px|thumb|left|P5.js and ultrasonic sensor]]
    //#include <LiquidCrystal_I2C.h> //LCD
 
    #include <LcdBarGraph.h> //LCD
The goal here was to create a first communication between the physical setup and a P5.js web page
    #include <Wire.h> // LCD
 
    #include <NewPing.h> //SENSOR ACCURACY
<br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br>
   
 
    LiquidCrystal lcd(34, 35, 32, 33, 30, 31);
===<p style="font-family:helvetica">Sketch 11:  Arduino Mega + UltraSonicSensor + LCD TouchScreen </p> ===
   
 
    //LCD  
[[File:LCDArduinoVariableposter.gif|300px|thumb|left|LCD Arduino Variable poster]]
    const int SDAPin = A4; //Data pin
 
    const int SCLPin = A5; //Clock pin
<br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br>
   
 
    //BUZZER
 
    const int BUZZER = 24;
  #include <TFT_HX8357.h>
    
    
    //A
  TFT_HX8357 tft = TFT_HX8357();  
   
    int trigPinA= 2;
    int EchoPinA= 3;
    
    
    //B
   
    int trigPinB= 4;
    int EchoPinB= 5;
    
    
    //C
  unsigned long targetTime = 0;
   
  byte red = 0;
    int trigPinC= 6;
  byte green = 0;
    int EchoPinC= 7;
  byte blue = 255;
   
  byte state = 0;
    //D
  unsigned int colour = blue << 255;
   
    int trigPinD = 8;                                
    int EchoPinD= 9;    
    
    
                             
  #define TFT_GREY 0x5AEB // New colour
    //D
   
    int trigPinE = 10;                                 
    int EchoPinE= 11; 
    
    
    //F
  //Ultrasonic Sensor
   
  int VCC = 13;
    int trigPinF = 12;                                 
  int trigPin = 11;
    int EchoPinF= 13;  
  int echoPin = 12;
     
    //G
   
    int trigPinG= 22;
    int EchoPinG= 23;
   
    //A RANGG OF LEDS
   
    int LED_A1_ping= 25;
    int LED_A2_ping= 26;
    int LED_A3_ping= 27;
   
    //B RANGG OF LEDS
   
    int LED_B1_ping= 28; 
    int LED_B2_ping= 29;
    int LED_B3_ping= 36;
   
    //C RANGG OF LEDS
   
    int LED_C1_ping= 37; 
    int LED_C2_ping= 38;
    int LED_C3_ping= 39;
   
    //D RANGG OF LEDS
   
    int LED_D1_ping= 40; 
    int LED_D2_ping= 41;
    int LED_D3_ping= 42;
    
    
    //G RANGG OF LEDS
  long duration, distance, UltraSensor;  
   
  char data;
    int LED_E1_ping= 43 ;
  String SerialData="";
    int LED_E2_ping= 44;
    int LED_E3_ping= 45;
    
    
    //F RANGG OF LEDS
  void setup(void) {
      
     tft.init();
     int LED_F1_ping= 46;
     tft.setRotation(2);
     int LED_F2_ping= 47;
     tft.fillScreen(TFT_BLACK);
     int LED_F3_ping= 48;
     targetTime = millis() + 100;
    
    
     //G RANGG OF LEDS
    Serial.begin (9600);
     //define pin modes
    pinMode(VCC, OUTPUT); 
    pinMode(trigPin, OUTPUT); //sets the trigPin as an Output
    pinMode(echoPin, INPUT);  //sets the echoPin as an Input
      
      
     int LED_G1_ping= 49;   
     digitalWrite(VCC, HIGH)// +5V - Pin 13 as VCC
    int LED_G2_ping= 50;
  }
    int LED_G3_ping= 51;
  void rainbow_fill()
  {
      
      
     //LCD DISPLAY
     for (int i = 45; i < 421; i++) {
    //LiquidCrystal_I2C lcd = LiquidCrystal_I2C (0x3F,16,2);
      tft.drawFastHLine(1, i, tft.width(), colour); // in this example tft.width() returns the pixel width of the display
   
      switch (state) {
    long duration, distance, UltraSensorA, UltraSensorB, UltraSensorC, UltraSensorD, UltraSensorE, UltraSensorF,  UltraSensorG;  
        case 0:
    char data;
          blue ++;
    String SerialData="";
          if (blue == 0) {
   
            blue = 1;
    void setup()
            state = 1;
    {// START SGTUP FUNCTION
          }
   
          break;
    lcd.begin(16, 2);
        case 1:
    //lcd.init();
          green--;
      
          if (green == 255) {
    Serial.begin (9600);
            green = 254;
                             
            state = 2;
     pinMode(BUZZER, OUTPUT);
          }
          break;
        case 2:
          blue ++;
          if (blue == 0) {
            blue = 1;
            state = 3;
          }
        case 5:
          green --;
          if (blue == 255) {
            blue = 254;
            state = 0;
          }
          break;
      }
      colour = red << 0 | green << 255 | blue << 0;
     }
  }
 
  void loop() {
     SonarSensor(trigPin,echoPin);             
    UltraSensor = distance;  
    
    
    //setup pins sensor A
    Serial.print("A: ");
    pinMode(trigPinA, OUTPUT);
    Serial.print(UltraSensor);
    pinMode(EchoPinA, INPUT);
    Serial.println(" cm");
    pinMode(LED_A1_ping, OUTPUT);
    tft.setCursor(0, 0, 2);
    pinMode(LED_A2_ping, OUTPUT);
    tft.setTextColor(TFT_WHITE);  tft.setTextSize(1.8);
    pinMode(LED_A3_ping, OUTPUT);
    tft.setTextFont(4);
    tft.println("AQUARIUM 2.0");
    tft.println("29.11.2021");  
    
    
    //setup pins sensor B
    tft.setCursor(215, 0, 2);
    pinMode(trigPinB, OUTPUT);
    tft.setTextColor(TFT_WHITE); tft.setTextSize(1.8);
    pinMode(EchoPinB, INPUT);
    tft.setTextFont(4);
    pinMode(LED_B1_ping, OUTPUT);
    tft.println(" by XPUB");
    pinMode(LED_B2_ping, OUTPUT);
    pinMode(LED_B3_ping, OUTPUT);
    
    
    //setup pins sensor C
    tft.setCursor(215, 24, 2);
    pinMode(trigPinC, OUTPUT);
    tft.setTextColor(TFT_WHITE); tft.setTextSize(1.8);
    pinMode(EchoPinC, INPUT);
    tft.setTextFont(4);  
    pinMode(LED_C1_ping, OUTPUT);
    tft.println(" students");
    pinMode(LED_C2_ping, OUTPUT);
    pinMode(LED_C3_ping, OUTPUT);
   
    // setup pins sensor D
    pinMode(trigPinD, OUTPUT);                      
    pinMode(EchoPinD, INPUT);                        
    pinMode(LED_D1_ping, OUTPUT);                
    pinMode(LED_D2_ping, OUTPUT);                
    pinMode(LED_D3_ping, OUTPUT);                
    
    
    //setup pins sensor E
    tft.setCursor(0, 426, 2);
    pinMode(trigPinE, OUTPUT);
    tft.setTextColor(TFT_WHITE); tft.setTextSize(1.8);
    pinMode(EchoPinE, INPUT);
    tft.setTextFont(4);
    pinMode(LED_E1_ping, OUTPUT);                 
    tft.println("We are");
    pinMode(LED_E2_ping, OUTPUT);                 
    pinMode(LED_E3_ping, OUTPUT);                 
   
    //setup pins sensor F
    pinMode(trigPinF, OUTPUT);
    pinMode(EchoPinF, INPUT);
    pinMode(LED_F1_ping, OUTPUT);                 
    pinMode(LED_F2_ping, OUTPUT);                 
    pinMode(LED_F3_ping, OUTPUT);                 
     
    //setup pins sensor G
    pinMode(trigPinG, OUTPUT);
    pinMode(EchoPinG, INPUT);
    pinMode(LED_G1_ping, OUTPUT);                 
    pinMode(LED_G2_ping, OUTPUT);                 
    pinMode(LED_G3_ping, OUTPUT);                 
   
    //inisializG LED status
    digitalWrite(LED_A1_ping,LOW);
    digitalWrite(LED_A2_ping,LOW);
    digitalWrite(LED_A3_ping,LOW);
   
    digitalWrite(LED_B1_ping,LOW);
    digitalWrite(LED_B2_ping,LOW);
    digitalWrite(LED_B3_ping,LOW);
   
    digitalWrite(LED_C1_ping,LOW);
    digitalWrite(LED_C2_ping,LOW);
    digitalWrite(LED_C3_ping,LOW);
   
    digitalWrite(LED_D1_ping,LOW);
    digitalWrite(LED_D2_ping,LOW);
    digitalWrite(LED_D3_ping,LOW);
    
    
    digitalWrite(LED_E1_ping,LOW);
    tft.setCursor(0, 454, 2);
    digitalWrite(LED_E2_ping,LOW);
    tft.setTextColor(TFT_WHITE);  tft.setTextSize(1.8);
    digitalWrite(LED_E3_ping,LOW);
    tft.setTextFont(4);
    tft.println("Open!");
   
    tft.setCursor(135, 426, 2);
    tft.setTextColor(TFT_WHITE);  tft.setTextSize(1.8);
    tft.setTextFont(4);
    tft.println("you are");
   
    tft.setCursor(222, 426, 2);
    tft.setTextColor(TFT_WHITE, TFT_BLACK);  tft.setTextSize(1.8);
    tft.setTextFont(4);
    tft.println(UltraSensor);  
    
    
    digitalWrite(LED_F1_ping,LOW);
    tft.setCursor(284, 426, 2);
    digitalWrite(LED_F2_ping,LOW);
    tft.setTextColor(TFT_WHITE);  tft.setTextSize(1.8);
    digitalWrite(LED_F3_ping,LOW);
    tft.setTextFont(4);
    tft.println("cm");
   
    tft.setCursor(135, 454, 2);
    tft.setTextColor(TFT_WHITE);  tft.setTextSize(1.8);
    tft.setTextFont(4);
    tft.println("away from event");  
    
    
    digitalWrite(LED_G1_ping,LOW);
    if (targetTime < millis()) {
    digitalWrite(LED_G2_ping,LOW);
      targetTime = millis() + 10;
    digitalWrite(LED_G3_ping,LOW);
      rainbow_fill(); // Fill the screen with rainbow colours
 
    }
    }
      
      
     void loop()  
     
    {
    // Fill screen with random colour so we can see the effect of printing with and without
     // a background colour defined
    //tft.fillScreen(255,255,255);
      
      
     
     // Set "cursor" at top left corner of display (0,0) and select font 2
     // START THG LOOP FUNCTION
     // (cursor will move to next line automatically during printing with 'tft.println'
    SonarSensor(trigPinA,EchoPinA);             
     //  or stay on the line is there is room for the text with tft.print)
     UltraSensorA = distance;
    SonarSensor(trigPinB,EchoPinB);             
    UltraSensorB = distance;
    SonarSensor(trigPinC,EchoPinC);             
    UltraSensorC = distance;
    SonarSensor(trigPinD, EchoPinD);             
    UltraSensorD = distance;                                         
    SonarSensor(trigPinE,EchoPinE);             
    UltraSensorE = distance;
    SonarSensor(trigPinF,EchoPinF);             
    UltraSensorF = distance;
    SonarSensor(trigPinG,EchoPinG);             
    UltraSensorG = distance;
     
    Serial.print("A: ");
     Serial.print(UltraSensorA);
    Serial.println(" cm");
    
    
    Serial.print("B: ");
    Serial.print(UltraSensorB);
    Serial.println(" cm");
    
    
    Serial.print("C: ");
    Serial.print(UltraSensorC);
    Serial.println(" cm");
 
    Serial.print("D: ");
    Serial.print(UltraSensorD);
    Serial.println(" cm");
 
    Serial.print("E: ");
    Serial.print(UltraSensorE);
    Serial.println(" cm");
 
    Serial.print("F: ");
    Serial.print(UltraSensorF);
    Serial.println(" cm");
   
    Serial.print("G: ");
    Serial.print(UltraSensorG);
    Serial.println(" cm");
   
   
   
    lcd.setCursor(2,2);
    lcd.print(UltraSensorD);
   
    lcd.setCursor(7,0);
    lcd.print(UltraSensorA);
   
    lcd.setCursor(11,2);
    lcd.print(UltraSensorC);
       
     // A SENSOR ———————————————————————————————————————————————————————————————————————————————————————————————
     // A SENSOR ———————————————————————————————————————————————————————————————————————————————————————————————
    // A1———————————————————————————————————————————————————————————————————————————————————————————————
      // A1———————————————————————————————————————————————————————————————————————————————————————————————
      if(UltraSensorA <=10)// if distance is lGss than 10 Cm turn thG LED ON
        if(UltraSensor <=30)// if distance is lGss than 10 Cm turn thG LED ON
      {
        {
         lcd.setCursor(0,1);
         tft.setCursor(0, 200, 2);
         lcd.print("1:");
         tft.setTextColor(TFT_WHITE); tft.setTextSize(1);
        digitalWrite(LED_A1_ping,HIGH);
         tft.setTextFont(4);
         digitalWrite(BUZZER, HIGH);
         //tft.println("Message1");
        delay(10);
         digitalWrite(LED_A1_ping,LOW);
        digitalWrite(BUZZER, LOW);
         delay(10);   
         delay(10);   
      }
         }
      else                // else turn thG LED OFF
         else                // else turn thG LED OFF
      {
         {
         digitalWrite(LED_A1_ping,LOW);
         }
        digitalWrite(BUZZER, LOW);
         // A2———————————————————————————————————————————————————————————————————————————————————————————————
      }
        if((UltraSensor >=31) && (UltraSensor <=60))
      // A2———————————————————————————————————————————————————————————————————————————————————————————————
        {
      if((UltraSensorA >=11) && (UltraSensorA <=20))
        tft.setCursor(0, 200, 2);
      {
         tft.setTextColor(TFT_WHITE); tft.setTextSize(1);
        lcd.setCursor(0,1);
         tft.setTextFont(4);
        lcd.print("2:");
         //tft.println("Message2");
        digitalWrite(LED_A2_ping,HIGH);
         delay(10);  
        digitalWrite(BUZZER, HIGH);
         }
        delay(50);
         else                // else turn thG LED OFF
         digitalWrite(LED_A2_ping,LOW);
        {
        digitalWrite(BUZZER, LOW);
         }
        delay(50);
        // A3———————————————————————————————————————————————————————————————————————————————————————————————
      }
        if((UltraSensor >=61) && (UltraSensor <=300))
      else                // else turn thG LED OFF
        {
      {
         tft.setCursor(0, 200, 2);
         digitalWrite(LED_A2_ping,LOW);
         tft.setTextColor(TFT_WHITE); tft.setTextSize(1);
      }
         tft.setTextFont(4);
      // A3———————————————————————————————————————————————————————————————————————————————————————————————
        //tft.println("Message3");
      if((UltraSensorA >=21) && (UltraSensorA <=30))
      {
         lcd.setCursor(0,1);
        lcd.print("3:");
        digitalWrite(LED_A3_ping,HIGH);
        digitalWrite(BUZZER, HIGH);
        delay(100);
        digitalWrite(LED_A3_ping,LOW);
        digitalWrite(BUZZER, LOW);
        delay(100);
      }
      else             
      {
         digitalWrite(LED_A3_ping,LOW);
      }
      // A4———————————————————————————————————————————————————————————————————————————————————————————————
  //    if((UltraSensorA >=31) && (UltraSensorA <=10000))  
  //    {
  //      digitalWrite(LED_A4_ping,HIGH);
  //      lcd.setCursor(0,1);
  //      lcd.print("4:");
  //    }
  //    else               
  //    {
  //      digitalWrite(LED_A4_ping,LOW);
  //    }
      // B SENSOR ———————————————————————————————————————————————————————————————————————————————————————————————
      // B1———————————————————————————————————————————————————————————————————————————————————————————————
      if(UltraSensorB <=10)
      {
         lcd.setCursor(10,2);
        lcd.print("1:");
         digitalWrite(LED_B1_ping,HIGH);
        digitalWrite(BUZZER, HIGH);
        delay(10);
         digitalWrite(LED_B1_ping,LOW);
        digitalWrite(BUZZER, LOW);
        delay(10); 
      }
      else
      {
        digitalWrite(LED_B1_ping,LOW);
      }
      // B2———————————————————————————————————————————————————————————————————————————————————————————————
      if((UltraSensorB >=11) && (UltraSensorB <=20))
      {
        lcd.setCursor(10,2);
        lcd.print("2:");
        digitalWrite(LED_B2_ping,HIGH);
        digitalWrite(BUZZER, HIGH);
         delay(50);
         digitalWrite(LED_B2_ping,LOW);
         digitalWrite(BUZZER, LOW);
        delay(50);  }
      else                // else turn thG LED OFF
      {
         digitalWrite(LED_B2_ping,LOW);
      }
      // B3———————————————————————————————————————————————————————————————————————————————————————————————
      if((UltraSensorB >=21) && (UltraSensorB <=30))  
      {
         lcd.setCursor(10,2);
         lcd.print("3:");
        digitalWrite(LED_B3_ping,HIGH);
        digitalWrite(BUZZER, HIGH);
        delay(100);
        digitalWrite(LED_B3_ping,LOW);
        digitalWrite(BUZZER, LOW);
        delay(100);    
      }
      else                // else turn thG LED OFF
      {
         digitalWrite(LED_B3_ping,LOW);
      }
      // B4———————————————————————————————————————————————————————————————————————————————————————————————
  //    if((UltraSensorB >=31) && (UltraSensorB <=10000))
  //    {
  //    digitalWrite(LED_B4_ping,HIGH);
  //    lcd.setCursor(10,2);
  //    lcd.print("4:");
  //    }
  //   else                // else turn thG LED OFF
  //    {
  //      digitalWrite(LED_B4_ping,LOW);
  //    }
      // C SENSOR ———————————————————————————————————————————————————————————————————————————————————————————————
      // C1———————————————————————————————————————————————————————————————————————————————————————————————
      if(UltraSensorC <=10)
      {
        lcd.setCursor(5,0);
        lcd.print("1:");
        digitalWrite(LED_C1_ping,HIGH);
        digitalWrite(BUZZER, HIGH);
        delay(10);
        digitalWrite(LED_C1_ping,LOW);
        digitalWrite(BUZZER, LOW);
         delay(10);  
         delay(10);  
      }
        }
      else
        else              
      {
        {
         digitalWrite(LED_A3_ping,LOW);
         }
      }
        if(UltraSensor >=301)
      // C2———————————————————————————————————————————————————————————————————————————————————————————————
        {
      if((UltraSensorC >=11) && (UltraSensorC <=20)) // if distance is lGss than 10 Cm turn thG LED ON
         tft.setCursor(0, 200, 2);
      {
         tft.setTextColor(TFT_WHITE);  tft.setTextSize(1);
         lcd.setCursor(5,0);
         tft.setTextFont(4);
         lcd.print("2:");
         //tft.println("Message4");
         digitalWrite(LED_C2_ping,HIGH);
         digitalWrite(BUZZER, HIGH);
         delay(10);
         delay(10);
         digitalWrite(LED_C2_ping,LOW);
         }
        digitalWrite(BUZZER, LOW);
         else              
         delay(10);  }
         {
      else                // else turn thG LED OFF
         }
      {
   }
        digitalWrite(LED_C2_ping,LOW);
      }
      // C3———————————————————————————————————————————————————————————————————————————————————————————————
      if((UltraSensorC >=21) && (UltraSensorC <=30)) // if distance is lGss than 10 Cm turn thG LED ON
      {
        lcd.setCursor(5,0);
        lcd.print("3:");
        digitalWrite(LED_C3_ping,HIGH);
        digitalWrite(BUZZER, HIGH);
        delay(10);
        digitalWrite(LED_C3_ping,LOW);
        digitalWrite(BUZZER, LOW);
         delay(10);  }
      else                // else turn thG LED OFF
      {
         digitalWrite(LED_C3_ping,LOW);
      }
      // C4———————————————————————————————————————————————————————————————————————————————————————————————
  //    if((UltraSensorC >=31) && (UltraSensorC <=10000)) // if distance is lGss than 10 Cm turn thG LED ON
  //    {
  //      digitalWrite(LED_C4_ping,HIGH);
  //      lcd.setCursor(5,0);
  //      lcd.print("4:");
  //    }
  //    else                // else turn thG LED OFF
  //    {
  //      digitalWrite(LED_C4_ping,LOW);
   //    }
    
    
    // D SENSOR ———————————————————————————————————————————————————————————————————————————————————————————————
  void SonarSensor(int trigPinSensor,int echoPinSensor)//it takes the trigPIN and the echoPIN
  // D1———————————————————————————————————————————————————————————————————————————————————————————————
      if(UltraSensorD <=10)// if distance is lGss than 10 Cm turn thG LED ON
       {
       {
         digitalWrite(LED_D1_ping,HIGH);
         //generate the ultrasonic wave
        digitalWrite(BUZZER, HIGH);
       digitalWrite(trigPinSensor, LOW);// put trigpin LOW  
        delay(10);
       delayMicroseconds(2);// wait 2 microseconds
        digitalWrite(LED_D1_ping,LOW);
       digitalWrite(trigPinSensor, HIGH);// switch trigpin HIGH
        digitalWrite(BUZZER, LOW);
       delayMicroseconds(10); // wait 10 microseconds
        delay(10); 
       digitalWrite(trigPinSensor, LOW);// turn it LOW again
      }
        
      else                // else turn thG LED OFF
       //read the distance
       {
       duration = pulseIn(echoPinSensor, HIGH);//pulseIn funtion will return the time on how much the configured pin remain the level HIGH or LOW; in this case it will return how much time echoPinSensor stay HIGH
        digitalWrite(LED_D1_ping,LOW);
       distance= (duration/2) / 29.1; // A1 we have to divide the duration by two 
        digitalWrite(BUZZER, LOW);
      }
      // D2———————————————————————————————————————————————————————————————————————————————————————————————
      if((UltraSensorD >=11) && (UltraSensorD <=20))
      {
        digitalWrite(LED_D2_ping,HIGH);
        digitalWrite(BUZZER, HIGH);
        delay(50);
        digitalWrite(LED_D2_ping,LOW);
        digitalWrite(BUZZER, LOW);
        delay(50);
      }
       else                // else turn thG LED OFF
      {
        digitalWrite(LED_D2_ping,LOW);
      }
      // D3———————————————————————————————————————————————————————————————————————————————————————————————
       if((UltraSensorD >=21) && (UltraSensorD <=30))
      {
        digitalWrite(LED_D3_ping,HIGH);
        digitalWrite(BUZZER, HIGH);
        delay(100);
        digitalWrite(LED_D3_ping,LOW);
        digitalWrite(BUZZER, LOW);
        delay(100);
      }
      else             
      {
        digitalWrite(LED_D3_ping,LOW);
      }
      // E SENSOR ———————————————————————————————————————————————————————————————————————————————————————————————
    // E1———————————————————————————————————————————————————————————————————————————————————————————————
       if(UltraSensorE <=10)// if distance is lGss than 10 Cm turn thG LED ON
       {
        digitalWrite(LED_E1_ping,HIGH);
        digitalWrite(BUZZER, HIGH);
        delay(10);
        digitalWrite(LED_E1_ping,LOW);
        digitalWrite(BUZZER, LOW);
        delay(10); 
      }
      else                // else turn thG LED OGG
      {
        digitalWrite(LED_E1_ping,LOW);
        digitalWrite(BUZZER, LOW);
       }
       // E2———————————————————————————————————————————————————————————————————————————————————————————————
       if((UltraSensorE >=11) && (UltraSensorE <=20))
      {
        digitalWrite(LED_E2_ping,HIGH);
        digitalWrite(BUZZER, HIGH);
        delay(50);
        digitalWrite(LED_E2_ping,LOW);
        digitalWrite(BUZZER, LOW);
        delay(50);
      }
      else                // else turn thG LED OGG
      {
        digitalWrite(LED_E2_ping,LOW);
      }
      // E3———————————————————————————————————————————————————————————————————————————————————————————————
      if((UltraSensorE >=21) && (UltraSensorE <=30))
      {
        digitalWrite(LED_E3_ping,HIGH);
        digitalWrite(BUZZER, HIGH);
        delay(100);
        digitalWrite(LED_E3_ping,LOW);
        digitalWrite(BUZZER, LOW);
        delay(100);
       }
      else             
      {
        digitalWrite(LED_E3_ping,LOW);
      }
      // F SENSOR ———————————————————————————————————————————————————————————————————————————————————————————————
    // F1———————————————————————————————————————————————————————————————————————————————————————————————
      if(UltraSensorF <=10)// if distance is lGss than 10 Cm turn thG LED ON
      {
        digitalWrite(LED_F1_ping,HIGH);
        digitalWrite(BUZZER, HIGH);
        delay(10);
        digitalWrite(LED_F1_ping,LOW);
        digitalWrite(BUZZER, LOW);
        delay(10); 
      }
      else                // else turn the LEF OFF
      {
        digitalWrite(LED_F1_ping,LOW);
        digitalWrite(BUZZER, LOW);
      }
      // F2———————————————————————————————————————————————————————————————————————————————————————————————
      if((UltraSensorF >=11) && (UltraSensorF <=20))
      {
        digitalWrite(LED_F2_ping,HIGH);
        digitalWrite(BUZZER, HIGH);
        delay(50);
        digitalWrite(LED_F2_ping,LOW);
        digitalWrite(BUZZER, LOW);
        delay(50);  
      }
      else                // else turn the LEF OFF
      {
        digitalWrite(LED_F2_ping,LOW);
      }
      // F3———————————————————————————————————————————————————————————————————————————————————————————————
      if((UltraSensorF >=21) && (UltraSensorF <=30))
      {
        digitalWrite(LED_F3_ping,HIGH);
        digitalWrite(BUZZER, HIGH);
        delay(100);
        digitalWrite(LED_F3_ping,LOW);
        digitalWrite(BUZZER, LOW);
        delay(100);
       }
       }
      else             
      {
        digitalWrite(LED_F3_ping,LOW);
      }
      // G SENSOR ———————————————————————————————————————————————————————————————————————————————————————————————
    // G1———————————————————————————————————————————————————————————————————————————————————————————————
      if(UltraSensorG <=10)// if distance is lGss than 10 Cm turn thG LED ON
      {
        digitalWrite(LED_G1_ping,HIGH);
        digitalWrite(BUZZER, HIGH);
        delay(10);
        digitalWrite(LED_G1_ping,LOW);
        digitalWrite(BUZZER, LOW);
        delay(10); 
      }
      else                // else turn the LEG OGG
      {
        digitalWrite(LED_G1_ping,LOW);
        digitalWrite(BUZZER, LOW);
      }
      // G2———————————————————————————————————————————————————————————————————————————————————————————————
      if((UltraSensorG >=11) && (UltraSensorG <=20))
      {
        digitalWrite(LED_G2_ping,HIGH);
        digitalWrite(BUZZER, HIGH);
        delay(50);
        digitalWrite(LED_G2_ping,LOW);
        digitalWrite(BUZZER, LOW);
        delay(50);
      }
      else                // else turn the LEG OGG
      {
        digitalWrite(LED_G2_ping,LOW);
      }
      // G3———————————————————————————————————————————————————————————————————————————————————————————————
      if((UltraSensorG >=21) && (UltraSensorG <=30))
      {
        digitalWrite(LED_G3_ping,HIGH);
        digitalWrite(BUZZER, HIGH);
        delay(100);
        digitalWrite(LED_G3_ping,LOW);
        digitalWrite(BUZZER, LOW);
        delay(100);
      }
      else             
      {
        digitalWrite(LED_G3_ping,LOW);
      }
    }
   
    void SonarSensor(int trigPinSensor,int echoPinSensor)//it takes the trigPIN and the echoPIN
    {
      //generate the ultrasonic wave
    digitalWrite(trigPinSensor, LOW);// put trigpin LOW
    delayMicroseconds(2);// wait 2 microseconds
    digitalWrite(trigPinSensor, HIGH);// switch trigpin HIGH
    delayMicroseconds(10); // wait 10 microseconds
    digitalWrite(trigPinSensor, LOW);// turn it LOW again
   
    //read the distance
    duration = pulseIn(echoPinSensor, HIGH);//pulseIn funtion will return the time on how much the configured pin remain the level HIGH or LOW; in this case it will return how much time echoPinSensor stay HIGH
    distance= (duration/2) / 29.1; // A1 we have to divide the duration by two 
    }


===<p style="font-family:helvetica">Sketch 10:  Arduino Mega + 7 Sensors + LCD + 3 buzzers + P5.js </p> ===
 


The goal here was to create a first communication between the physical setup and a P5.js web page


====<p style="font-family:helvetica"> About the ESP8266 module</p>  ====
====<p style="font-family:helvetica"> About the ESP8266 module</p>  ====
Line 1,511: Line 1,325:
* Connect multiple arduinos (if necessary): https://www.youtube.com/watch?v=tU6jIoQ6M_E
* Connect multiple arduinos (if necessary): https://www.youtube.com/watch?v=tU6jIoQ6M_E
* Connect lamp to arduino: https://www.youtube.com/watch?v=F-yk4Tyc44g
* Connect lamp to arduino: https://www.youtube.com/watch?v=F-yk4Tyc44g
===<p style="font-family:helvetica">How to connect Arduino to P5.js</p> ===
[[File:SerialControl PortID.png|300px|thumb|right|Serial Control Port ID through P5.serialcontrol app]]
1. Download and install [https://github.com/p5-serial/p5.serialcontrol/releases P5.serialcontrol app]<br>
2. Get your serial port number through the app (see thumbnail)<br>
3. Connect to https://editor.p5js.org/ for less issues with libraries, and preferably on Chrome<br>
4. Reference to try [https://medium.com/@yyyyyyyuan/tutorial-serial-communication-with-arduino-and-p5-js-cd39b3ac10ce test 1];[https://itp.nyu.edu/physcomp/labs/labs-serial-communication/lab-serial-input-to-the-p5-js-ide/ test2]
<br><br><br><br><br><br><br><br><br><br><br>


==<p style="font-family:helvetica"> Prototyping Ressources</p> ==
==<p style="font-family:helvetica"> Prototyping Ressources</p> ==
Line 1,617: Line 1,420:


=<p style="font-family:helvetica">Venues</p> =
=<p style="font-family:helvetica">Venues</p> =
==<p style="font-family:helvetica"> Introduction </p>==
* [https://pad.xpub.nl/p/20210928_xpub2 PAD]
We will organize 2 moments of shared work in progress: one in October, one in November.
It is not necessarily a presentation, but more conversation-based in order to practice the "making public act".
Speak about our work, conversations with people about it, and people talking about our work.


==<p style="font-family:helvetica">Venue 1: Aquarium </p>==
==<p style="font-family:helvetica">Venue 1: Aquarium </p>==
Line 1,702: Line 1,496:


<br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br>
<br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br>
=<p style="font-family:helvetica">Venues</p> =
==<p style="font-family:helvetica">Venue 2: Aquarium 2.0 </p>==
===<p style="font-family:helvetica">Description</p>===
<br>
Date 29th Nov — 4th Dec 2021 <br>
Time 15h — 18h <br>
29th Nov — 4th Dec 2021 (all day)<br>
Location: De Buitenboel, Rosier Faassenstraat 22 3025 GN Rotterdam, NL<br>
<br><br>
AQUARIUM 2.0 <br>
<br>
An ongoing window exhibition with Clara Gradel, Floor van Meeuwen, Martin Foucaut, Camilo Garcia, Federico Poni, Nami Kim, Euna Lee, Kendal Beynon, Jacopo Lega and Louisa Teichmann<br>
<br>
Tap upon the glass and peer into the research projects we are currently working on.
From Monday 29th of November until Saturday 4th of December we put ourselves on display in the window of De Buitenboel as an entry point into our think tank. Navigating between a range of technologies, such as wireless radio waves, virtual realities, sensors, ecological and diffractive forms of publishing, web design frameworks, language games, and an ultra-territorial residency; we invite you to gaze inside the tank and float with us. Welcome back to the ecosystem of living thoughts.<br>
==<p style="font-family:helvetica">Aquarium LCD Portal (29 Nov – 4th Dec)</p>==
This interactive micro-installation composed of a LCD screen and sensor(s) invites users/visitors to change the color of the screen and displayed messages by getting more or less close from the window.
[https://www.pzwart.nl/blog/2021/11/30/aquarium-2-0/ Link]
[[File:ScreenPortalFront.jpg|300px|thumb|left|ScreenPortalFront]]
[[File:ScreenPortalback.jpg|300px|thumb|right|ScreenPortalback]]
[[File:LCDScreenTest.gif|600px|thumb|center|LCDScreenTest]]
<br><br><br><br><br><br><br><br><br><br><br><br><br>


=<p style="font-family:helvetica">Readings (new)(english)(with notes in english) </p>=
=<p style="font-family:helvetica">Readings (new)(english)(with notes in english) </p>=
Line 1,791: Line 1,614:
* Library vs Exhibition Space = Use vs Display
* Library vs Exhibition Space = Use vs Display
* Book-theme exhibitions
* Book-theme exhibitions
==<p style="font-family:helvetica">About User vs Visitor, or user in exhibition space</p>==
[[Designing the user experience in exhibition spaces - Elisa Rubegni, Caporali Maurizio, Antonio Rizzo, Erik Grönvall]]
* What are the GUI intentions
* What is the WIMP interaction model
* What are the post-Wimp models
* About Memex


==<p style="font-family:helvetica">About User Interface</p>==
==<p style="font-family:helvetica">About User Interface</p>==
Line 1,889: Line 1,720:
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Grenoble, 21 December 2001.
Grenoble, 21 December 2001.
→&nbsp;9.&nbsp;[[Graspable interfaces (Fitzmaurice et al., 1995)]] [https://www.dgp.toronto.edu/~gf/papers/PhD%20-%20Graspable%20UIs/Thesis.gf.html link]


==<p style="font-family:helvetica">About User Condition</p>==
==<p style="font-family:helvetica">About User Condition</p>==

Revision as of 12:29, 26 January 2022

Pads

Manetta / Michael

Steve / Marloes

Eleanor Greenhalgh

Links

Seminars (source)

Key Dates and Deadlines

These are the key dates for 2021-22

  • 19 November - Graduate Proposal Deadline

Last year's Graduate Proposals UPLOAD YOUR PROPOSAL HERE!

  • 19 November - Thesis Outline Deadline

Last year's Thesis Outlines UPLOAD YOUR THESIS OUTLINE HERE!

  • 3 Dec - Deadline First Chapter
  • 18 Feb - Deadline First Draft Thesis
  • 18 March - Deadline Second Draft thesis (texts to 2nd readers)
  • 1 April - Deadlines Second readers' comments
  • 14 April - DEADLINE THESIS

Guides and Guidelines

LB Code link (in progress)

About thesis

Thesis criteria

  1. Intelligibly express your ideas, thoughts and reflections in written English.
  2. Articulate in writing a clear direction of your graduate project by being able to identify complex and coherent questions, concepts and appropriate forms.
  3. Clearly structure and analyse an argument.
  4. Use relevant source material and references.
  5. Research texts and practices and reflect upon them analytically.
  6. Synthesize different forms of knowledge in a coherent, imaginative and distinctive way.
  7. Position one's own views within a broader context.
  8. Recognize and perform the appropriate mode of address within a given context.
  9. Engage in active dialogue about your written work with others.

Thesis format

  1. A report on your research and practice.
  1. An analytical essay exploring related artistic, theoretical, historical and critical issues and practices that inform your practice, without necessarily referring to your work directly.
  1. The presentation of a text as a body of creative written work.

Thesis Outline (guideline)

Don't make it more than 1500 words

What is your question?

Break the proposed text down into parts.  Think of the separate sections as "containers" (this may change as you progress with the text but try to make a clear plan with a word count in place)

Thesis Outline (consider the following before writing the outline. Include all these points in the intro to the outline)

Conceptual Outline (what is your question? Try to be a specific as possible. More specific than identifying a subject or general interest. It helps to ask: "what questions does the work I make generate?")

Why do you want to write this text?

Outline of Methodology  (for example: " I would like to structure my thesis in relation to the a series of interviews I will conduct for my proposed project"  OR  "I will make a 'close reading' of three of my past projects" 

Time line (how will you plan your time between now and April)


  • Introduction- overview 

[500 words]

  • Chapter 1 

[2000 words]

  • Chapter 2 

[2000 words]

  • Chapter 3 

[2000 words]

  • Conclusion [500 words] 

=7000

Bibliography

Annotated bibliography (five texts max). Make a synopsis of 5 texts  that will be central to your thesis.

  • Example of annotated bibliography 

https://pzwiki.wdka.nl/mediadesign/Mia/Thesis

  • Example of a thesis outline:

    #)     https://pzwiki.wdka.nl/mw-mediadesign/images/f/f3/Thesis_outline_final_Yuching.pdf     #1)     https://pzwiki.wdka.nl/mediadesign/User:Zpalomagar/THESIS_OUTLINE/FIFTH_DRAFT

Referencing System

  • Harvard Referencing system PDF


Graduate proposal guidelines

What do you want to make?

My project is a data collection installation that monitors people's behaviors in public physical spaces while explicitly encouraging them to help the algorithm collect more information. An overview of how it works is presented here in the project proposal and will be subject to further developments in the practice.

The way the device is designed doesn’t pretend to give any beneficial outcomes for the subject, but only makes visible the benefits that the machine is getting from collecting their data. Yet, the way the device visually or verbally presents this collected data is done in a grateful way, which might be stimulating for the subject. In that sense, the subject, despite knowing that their actions are done solely to satisfy the device, could become intrigued, involved, or even addicted by a mechanism that deliberately uses it as a commodity. In that way, I intend to trigger conflictual feelings in the visitor’s mind, situated between a state of awareness regarding the operating monetization of their physical behaviors, and a state of engagement/entertainment /stimulation regarding the interactive value of the installation.

My first desire is to make the mechanisms by which data collection is carried out, marketized and legitimized both understandable and accessible. The array of sensors, the Arduinos and the screen are the mainly technological components of this installation. Rather than using an already existing and complex tracking algorithm, the program is built from scratch, kept open source and limits itself to the conversion of a restricted range of physical actions into interactions. These include the detection of movements, positions, lapse of time spent standing still or moving, and entry or exit from a specific area of detection. Optionally they may also include the detection of the subject smartphone device or the log on a local Wi-Fi hotspot made by the subject.

In terms of mechanic, the algorithm creates feedback loops starting from:
_the subject behaviors being converted into information;
_the translation of this information into written/visual feedback;
_and the effects of this feedbacks on subject’s behavior; and so on.
By doing so, it tries to shape the visitors as free data providers inside their own physical environment, and stimulate their engagement by converting each piece of collected information into points/money, feeding a user score among a global ranking.

On the screen, displayed events can be:

Simu part 02.gif
Simu part 03.gif

_ “subject [] currently located at [ ]”
[x points earned/given]
_ “subject [] entered the space”
[x points earned/given]
_ “subject [] left the space”
[x points earned/given]
_ “subject [] moving/not moving”
[x points earned/given]
_ “subject [] distance to screen: [ ] cm”
[x points earned/given]
_ “subject [] stayed at [ ] since [ ] seconds”
[x points earned/given]
_ “subject [] device detected
[x points earned/given] (optional)
_ “subject logged onto local Wi-Fi
[x points earned/given] (optional)

Added to that comes the instructions and comments from the devices in reaction to the subject’s behaviors:

_ “Congratulations, you have now given the monitor 25 % of all possible data to collect!”
[when 25-50-75-90-95-96-97-98-99% of the total array of events has been detected at least once]
_ “Are you sure you don’t want to move to the left? The monitor has only collected data from 3 visitors so far in this spot!”
[if the subject stands still in a specific location]
_ “Congratulations, the monitor has reached 1000 pieces of information from you!”
[unlocked at x points earned/given]
_ “If you stay there for two more minutes, there is a 99% chance you will be in the top 100 of ALL TIME data-givers!”
[if the subject stand still in a specific location]
_ “Leaving all ready? The monitor has yet to collect 304759 crucial pieces of information from you!”
[if the subject is a the edge of the detection range]
_ “You are only 93860 pieces of information away from being the top one data-giver!”
[unlocked at x points earned/given]
_ “Statistics show that people staying for more than 5 minutes average will benefit me on average 10 times more!”
[randomly appears]
_ “The longer you stay on this spot, the more chance you have to win a “Lazy data-giver” badge”
[if the subject stands still for a long time any location]

Responding positively to the monitors instructions unlocks special achievement and extra points

—Accidental data-giver badge
[unlocked if the subject has passed the facility without deliberately wishing to interact with it] + [x points earned/given]
—Lazy data-giver badge
[unlocked if the subject has been standing still for at least one minute] + [x points earned/given]
—Novice data-giver badge
[unlocked if the subject has been successfully completing 5 missions from the monitor] + [x points earned/given]
—Hyperactive data-giver badge
[unlocked if the subject has never been standing still for 10 seconds within 2 minutes lapse time] + [x points earned/given]
—Expert data-giver badge
[unlocked if the subject has been successfully completing 10 missions from the monitor within 10 minutes] + [x points earned/given]
—Master data-giver badge
[unlocked if the subject has been successfully logging on the local Wi-Fi hotspot] + [x points earned/given] (optional)

On the top left side of the screen, a user score displays the number of points generated by the collected pieces of information, and the unlocking of special achievements instructed by the monitor.

—Given information: 298 pieces
[displays number of collected events]
—Points: 312000
[conversion of collected events and achievement into points]

On the top right of the screen, the user is ranked among x number of previous visitors and the prestigious badge recently earned is displayed bellow

—subject global ranking: 3/42
[compares subject’s score to all final scores from previous subjects]
—subject status: expert data-giver
[display the most valuable reward unlocked by the subject]

When leaving the detection range, the subject gets a warning message and a countdown starts, and encouraging it to take the quick decision to come back

—“Are you sure you want to leave? You have 5-4-3-2-1-0 seconds to come back within the detection range”
[displayed as long as the subject remains completely undetected]

If the subject definitely stands out of the detection range for more than 5 seconds, the monitor will also address a thankful message and the amount of money gathered, achievements, ranking, complete list of collected information and a qr code will be printed as a receipt with the help of a thermal printer. The QR will be a link to my thesis.

—* “Thank you for helping today, don’t forget to take your receipt in order to collect and resume your achievements”
[displayed after 5 seconds being undetected]

In order to collect, read or/and use that piece of information, the visitor will inevitably have to come back within the range of detection, and intentionally, or not, reactivate the data tracking game. It is therefore impossible to leave the area of detection without leaving at least one piece of your own information printed in the space. Because of this, the physical space should gradually be invaded by tickets scattered on the floor. As in archaeology, these tickets give a precise trace of the behavior and actions of previous subjects for future subjects.

Why do you want to make it?

When browsing online or/and using connected devices in the physical world, even the most innocent action/information can be invisibly recorded, valued and translated into informational units, subsequently generating profit for monopolistic companies. While social platforms, brands, public institutions and governments explicitly promote the use of monitoring practices in order to better serve or protect us, we could also consider these techniques as implicitly political, playing around some dynamics of visibility and invisibility in order to assert new forms of power over targeted audiences.

In the last decade, a strong mistrust of new technologies has formed in the public opinion, fueled by events such as the revelations of Edward Snowden, the Cambridge Analytica scandal or the proliferation of fake news on social networks. We have also seen many artists take up the subject, sometimes with activist purposes. But even if a small number of citizens have begun to consider the social and political issues related to mass surveillance, and some individuals/groups/governments/associations have taken legal actions, surveillance capitalism still remains generally accepted, often because ignored or/and misunderstood.

Thanks to the huge profits generated by the data that we freely provide every day, big tech companies have been earning billions of dollars over the sale of our personal information. With that money, they could also further develop deep machine learning programs, powerful recommendation systems, and to broadly expand their range of services in order to track us in all circumstances and secure their monopolistic status. Even if we might consider this realm specific to the online world, we have seen a gradual involvement from the same companies to monitor the physical world and our human existences in a wide array of contexts. For example, with satellite and street photography (Google Earth, Street View), geo localization systems, simulated three-dimensional environments (augmented reality, virtual reality or metaverse) or extensions of our brains and bodies (vocal assistance and wearable devices). Ultimately, this reality has seen the emergence of not only a culture of surveillance but also of self-surveillance, as evidenced by the popularity of self-tracking and data sharing apps, which legitimize and encourage the datafication of the body for capitalistic purposes.

For the last 15 years, self-tracking tools have made their way to consumers. I believe that this trend is showing how ambiguous our relationship can be with tools that allow such practices. Through my work, I do not wish to position myself as a whistleblower, a teacher or activist. Indeed, to adopt such positions would be hypocritical, given my daily use of tools and platforms that resort to mass surveillance. Instead, I wish to propose an experience that highlights the contradictions in which you and I, internet users and human beings, can find ourselves. This contradiction is characterized by a paradox between our state of concern about the intrusive surveillance practices operated by the Web giants (and their effects on societies and humans) and a state of entertainment or even active engagement with the tools/platforms through which this surveillance is operated/allowed. By doing so, I want to ask how do these companies still manage to get our consent and what human biases do they exploit in order to do so. That’s is how my graduation work and my thesis will investigate the effect of gamification, gambling or reward systems as well as a the esthetization of data/self-data as means to hook our attention, create always more interactions and orientate our behaviors.


How to you plan to make it and on what timetable?

I am developing this project with Arduino Uno/Mega boards, an array of ultrasonic sensor, P5.js and screens.

How does it work?

The ultrasonic sensors can detect obstacles in a physical space and know the distance between the sensor and obstacle(s) by sending and receiving back an ultrasound. The Arduino Uno/Mega boards are microcontrollers which can receive this information, run it in a program in order to convert these values into a mm/cm/m but also map the space into an invisible grid. Ultimately, values collected on the Arduino’s serial monitor can be sent to P5.js through P5.serialcontrol. P5.js will then allow a greater freedom in way the information can be displayed on the screens.

Process:

1st semester: Building a monitoring device, converting human actions into events, and events into visual feedbacks

During the first semester, I am focused on exploring monitoring tools that can be used in the physical world, with a specific attention to ultrasonic sensors. Being new to Arduino programming, my way of working is to start from the smallest and most simple prototype and gradually increase its scale/technicality until reaching human/architectural scale. Prototypes are subject to testing, documentation and comments helping to define which direction to follow. The first semester also allows to experiment with different kind of screen (LCD screens, Touch screens, computer screens, TV screens) until finding the most adequate screen monitor(s) for the final installation. Before building the installation, the project is subject to several sketching and animated simulations in 3 dimensions, exploring different scenarios and narrations. At the end of the semester, the goal is to be able to convert a specific range of human actions into events and visual feedback creating a feedback loop from the human behaviors being converted into information; the translation of this information into written/visual feedbacks; and the effects of this feedbacks on human behavior; and so on.

2nd semester: Implementing gamification with the help of collaborative filtering, point system and ranking.

During the second semester, it is all about building and implementing a narration with the help of gaming mechanics that will encourage humans to feed the data gathering device with their own help. An overview of how it works is presented here in the project proposal and will be subject to further developments in the practice.

To summarize the storyline, the subject being positioned in the detection zone finds herself/himself unwillingly embodied as the main actor of a data collection game. Her/His mere presence generates a number of points/dollars displayed on a screen, growing as she/he stays within the area. The goal is simple: to get a higher score/rank and unlock achievements by acting as recommended by a data-collector. This can be done by setting clear goals/rewards to the subject, and putting its own performance in comparison to all the previous visitors, giving unexpected messages/rewards, and give an aesthetic value to the displayed informations.

The mechanism is based on a sample of physical events that have been already explored in the first semester of prototyping (detection of movements, positions, lapse of time spent standing still or moving, and entry or exit from a specific area of detection). Every single detected event in this installation is stored in a data bank, and with the help of collaborative filtering, will allow to the display of custom recommendations such as:

_ “Congratulations, you have now given the monitor 12 % of all possible data to collect”
_ “Are you sure you don’t want to move to the left? The monitor has only collected data from 3 visitors so far in this spot”
_ “Congratulations, the monitor has reached 1000 pieces of information from you!”
_ “If you stay there for two more minutes, there is a 99% chance you will be in the top 100 of ALL TIME data-givers”
_ “Leaving all-ready? The monitor has yet 304759 crucial pieces of information to collect from you”
_ “You are only 93860 actions away from being the top one data-giver”
_ “Statistics are showing that people staying for more than 5 minutes average will be 10 times more benefitting for me”
_ “The longer you stay on this spot, the more chance you have to win a “Lazy data-giver” badge”


The guideline is set out here, but will be constantly updated with the help of experiments and the results observed during the various moments of interaction between the students and the algorithm. For this purpose, the installation under construction will be left active and autonomous in its place of conception (studio) and will allow anyone who deliberately wishes to interact with it to do so. Beyond the voluntary interactions, my interest is also to see what can be extracted from people simply passing in front of this installation. In addition to this, some of the mechanics of the installation will be further explored by collaborating with other students, and setting up more ephemeral and organized experiences with the participants. (ex: 15 February 2022 with Louisa)

This semester will also include the creation of a definite set of illustrations participating to engage the participants of the installation in a more emotional way, the illustrations will be made by an illustrator/designer, with whom I usually collaborate.

3rd semester: Build the final installation of final assessment and graduation show. Test runs, debug and final touchs.

During the third semester, the installation should be settled in the school, in the alumni area, next to XPUB studio for the final assessment, and ultimately settled again at WORM for the graduation show. I am interested in putting this installation into privileged spaces of human circulation, (such as hallways) that would more easily involve the detection of people, and highlight the intrusive aspect of such technologies. The narration, the mechanics, illustrations and graphic aspect should be finalized at the beginning of the 3rd semester, and subject to intense test runs during all that period until meeting the deadline.


Relation to larger context

As GAFAM companies are facing more and more legal issues, and held accountable in growing numbers of social and political issues around the world, the pandemic context has greatly contributed to make all of us more dependent than ever on the online services provided by these companies and to somehow force our consent. While two decades of counter-terrorism measures legitimized domestic and public surveillance techniques such as online and video monitoring, the current public health crisis made even more necessary the use of new technologies for regulating the access to public spaces and services, but also for socializing, working together, accessing to culture, etc. In a lot of countries, from a day to another, and for an undetermined time, it has become necessary to carry a smartphone (or a printed QR code) in order to get access transport, entertainment, cultural and catering services, but also in order to do simple things such as to have a look at the menu in a bar/restaurant or to make an order.. Thus, this project takes place in a context where techno-surveillance has definitely taken a determining place in the way we can access spaces and services related to the physical world.

Data Marketisation / Self Data: Quantified Self / Attention Economy / Public Health Surveillance / Cybernetics


Who can help you?

Prototyping
Manetta Berends
Michael Murtaugh

Writing
Luke Williams

Arduino
XPUB Arduino knowledge sharing group
Dennis de Bel
Aymeric Mansoux

Installation building
Wood station
Interaction station

Illustrations
Adrien Jacquemet (illustrator/graphic designer)

Installation location
Leslie Robbins

Extra help for game narratives
Rosa Zangenberg (writer/artist)

Relation to previous practice?

During my previous studies in graphic design, I started being engaged with the new media by making a small online reissue of Raymond Queneau’s book called Exercices de Style. In this issue called Incidences Médiatiques (2017), the user/reader was encouraged to explore the 99 different versions of a same story written by the author in a less-linear way. The idea was to consider each user graphic user interface as a unique reading context. It would determine which story could be read, depending on the device used by the reader, and the user could navigate through these stories by resizing the Web window, by changing browser or by using on a different device.

As part of my graduation project called Media Spaces (2019), I wanted to reflect on the status of networked writing and reading, by programming my thesis in the form of Web to Print website. Subsequently, this website became translated in the physical space as a printed book, and a series of installations displayed in an exhibition space that was following the online structure of my thesis (home page, index, part 1-2-3-4). In that way, I was interested to inviting to visitors to make a physical experience some aspects of the Web

As a first-year student of Experimental Publishing, I continued to work in that direction by eventually creating a meta-website called Tense (2020) willing to display the invisible html <meta> tags inside of an essay in order to affect our interpretation of the text. In 2021, I worked on a geocaching pinball game highlighting invisible Web event, and a Web oscillator, which amplitude and frequency range were directly related to the user’s cursor position and browser screen-size.

While it has always been clear to me that these works were motivated by the desire to define media as context, subject or/and content, the projects presented here have often made use of surveillance tools to detect and translate user information into feedbacks, participating in the construction of an individualized narrative or/and a unique viewing/listening context (interaction, screen size, browser, mouse position). The current work aims to take a critical look at the effect of these practices in the context of techno surveillance.

Similarly, projects such as Media Spaces have sought to explore the growing confusion between human and web user, physical and virtual space or online and offline spaces. This project will demonstrate that these growing confusions will eventually lead us to be tracked in all circumstances, even in our most innocuous daily human activities/actions.


Selected References

Works:

« invites us to take on the role of an auditor, tasked with addressing the biases in a speculative AI »Alternatives to techno-surveillance

Expose humans as producers of useful intellectual labor that is benefiting to the tech giants and the use than can be made out of that labor.

Claims that that technological devices can be manipulated easily and hence, that they are fallible and subjective. They do this by simply placing a self-tracker (connected bracelet) in another context, such as on some other objects, in order to confuse these devices.

Allows galleries to enjoy encrypted internet access and communications, through a Tor Network

You are rewarded for exploring all the interactive possibilities of your mouse, revealing how our online behaviors can be monitored and interpretated by machines.

Portrait of the viewer is drawn in real time by active words, which appear automatically to fill his or her silhouette https://www.lozano-hemmer.com/third_person.php

«Every visitor to the website’s browser size, collected, and played back sequentially, ending with your own.»

Readings of the building and its contents are therefore always unique -- no two visitors share the same experience. https://haque.co.uk/work/mischievous-museum/

Books & Articles:

  • SHOSHANA ZUBOFF, The Age of Surveillance Capitalism (2020)

Warns against this shift towards a «surveillance capitalism». Her thesis argues that, by appropriating our personal data, the digital giants are manipulating us and modifying our behavior, attacking our free will and threatening our freedoms and personal sovereignty.

  • EVGENY MOROZOV, Capitalism’s New Clothes (2019)

Extensive analysis and critic of Shoshana Zuboff research and publications.

  • BYRON REEVES AND CLIFFORD NASS, The Media Equation, How People Treat Computers, Television, and New Media Like Real People and Places (1996)

Precursor study of the relation between humans and machine, and how do you human relate to them.

  • OMAR KHOLEIF, Goodbye, World! — Looking at Art in the digital Age (2018)

Authors shares it’s own data as a journal in a part of the book, while on another part, question how the Internet has changed the way we perceive and relate, and interact with/to images.

  • KATRIN FRITSCH, Towards an emancipatory understanding of widespread datafication (2018)

Suggests that in response to our society of surveillance, artists can suggest activist response that doesn’t necessarily involve technological literacy, but instead can promote strong counter metaphors or/and counter use of these intrusive technologies.

Reading Sources

Themes (keywords)

  • Interfaced Reality
  • Museum Display vs Screen display
  • Exhibition space vs User interface
  • Web Elasticy vs Physical Rigidity
  • Museology / Curation / Gallery and Museum display
  • Technological context
  • Mediatization of Media / Meta Art

Draft Thesis

What do you want to make?

My project is a data collection installation that monitors people's behaviors in public physical spaces while explicitly encouraging them to help the algorithm collect more information. An overview of how it works is presented here in the project proposal and will be subject to further developments in the practice.

The way the device is designed doesn’t pretend to give any beneficial outcomes for the subject, but only makes visible the benefits that the machine is getting from collecting their data. Yet, the way the device visually or verbally presents this collected data is done in a grateful way, which might be stimulating for the subject. In that sense, the subject, despite knowing that their actions are done solely to satisfy the device, could become intrigued, involved, or even addicted by a mechanism that deliberately uses it as a commodity. In that way, I intend to trigger conflictual feelings in the visitor’s mind, situated between a state of awareness regarding the operating monetization of their physical behaviors, and a state of engagement/entertainment /stimulation regarding the interactive value of the installation.

My first desire is to make the mechanisms by which data collection is carried out, marketized and legitimized both understandable and accessible. The array of sensors, the Arduinos and the screen are the mainly technological components of this installation. Rather than using an already existing and complex tracking algorithm, the program is built from scratch, kept open source and limits itself to the conversion of a restricted range of physical actions into interactions. These include the detection of movements, positions, lapse of time spent standing still or moving, and entry or exit from a specific area of detection. Optionally they may also include the detection of the subject smartphone device or the log on a local Wi-Fi hotspot made by the subject.

In terms of mechanic, the algorithm creates feedback loops starting from:
_the subject behaviors being converted into information;
_the translation of this information into written/visual feedback;
_and the effects of this feedbacks on subject’s behavior; and so on.
By doing so, it tries to shape the visitors as free data providers inside their own physical environment, and stimulate their engagement by converting each piece of collected information into points/money, feeding a user score among a global ranking.

On the screen, displayed events can be:

_ “subject [] currently located at [ ]”
[x points earned/given]
_ “subject [] entered the space”
[x points earned/given]
_ “subject [] left the space”
[x points earned/given]
_ “subject [] moving/not moving”
[x points earned/given]
_ “subject [] distance to screen: [ ] cm”
[x points earned/given]
_ “subject [] stayed at [ ] since [ ] seconds”
[x points earned/given]
_ “subject [] device detected
[x points earned/given] (optional)
_ “subject logged onto local Wi-Fi
[x points earned/given] (optional)

Added to that comes the instructions and comments from the devices in reaction to the subject’s behaviors:

_ “Congratulations, you have now given the monitor 25 % of all possible data to collect!”
[when 25-50-75-90-95-96-97-98-99% of the total array of events has been detected at least once]
_ “Are you sure you don’t want to move to the left? The monitor has only collected data from 3 visitors so far in this spot!”
[if the subject stands still in a specific location]
_ “Congratulations, the monitor has reached 1000 pieces of information from you!”
[unlocked at x points earned/given]
_ “If you stay there for two more minutes, there is a 99% chance you will be in the top 100 of ALL TIME data-givers!”
[if the subject stand still in a specific location]
_ “Leaving all ready? The monitor has yet to collect 304759 crucial pieces of information from you!”
[if the subject is a the edge of the detection range]
_ “You are only 93860 pieces of information away from being the top one data-giver!”
[unlocked at x points earned/given]
_ “Statistics show that people staying for more than 5 minutes average will benefit me on average 10 times more!”
[randomly appears]
_ “The longer you stay on this spot, the more chance you have to win a “Lazy data-giver” badge”
[if the subject stands still for a long time any location]

Responding positively to the monitors instructions unlocks special achievement and extra points

—Accidental data-giver badge
[unlocked if the subject has passed the facility without deliberately wishing to interact with it] + [x points earned/given]
—Lazy data-giver badge
[unlocked if the subject has been standing still for at least one minute] + [x points earned/given]
—Novice data-giver badge
[unlocked if the subject has been successfully completing 5 missions from the monitor] + [x points earned/given]
—Hyperactive data-giver badge
[unlocked if the subject has never been standing still for 10 seconds within 2 minutes lapse time] + [x points earned/given]
—Expert data-giver badge
[unlocked if the subject has been successfully completing 10 missions from the monitor within 10 minutes] + [x points earned/given]
—Master data-giver badge
[unlocked if the subject has been successfully logging on the local Wi-Fi hotspot] + [x points earned/given] (optional)

On the top left side of the screen, a user score displays the number of points generated by the collected pieces of information, and the unlocking of special achievements instructed by the monitor.

—Given information: 298 pieces
[displays number of collected events]
—Points: 312000
[conversion of collected events and achievement into points]

On the top right of the screen, the user is ranked among x number of previous visitors and the prestigious badge recently earned is displayed bellow

—subject global ranking: 3/42
[compares subject’s score to all final scores from previous subjects]
—subject status: expert data-giver
[display the most valuable reward unlocked by the subject]

When leaving the detection range, the subject gets a warning message and a countdown starts, and encouraging it to take the quick decision to come back

—“Are you sure you want to leave? You have 5-4-3-2-1-0 seconds to come back within the detection range”
[displayed as long as the subject remains completely undetected]

If the subject definitely stands out of the detection range for more than 5 seconds, the monitor will also address a thankful message and the amount of money gathered, achievements, ranking, complete list of collected information and a qr code will be printed as a receipt with the help of a thermal printer. The QR will be a link to my thesis.

—* “Thank you for helping today, don’t forget to take your receipt in order to collect and resume your achievements”
[displayed after 5 seconds being undetected]

In order to collect, read or/and use that piece of information, the visitor will inevitably have to come back within the range of detection, and intentionally, or not, reactivate the data tracking game. It is therefore impossible to leave the area of detection without leaving at least one piece of your own information printed in the space. Because of this, the physical space should gradually be invaded by tickets scattered on the floor. As in archaeology, these tickets give a precise trace of the behavior and actions of previous subjects for future subjects.

Why do you want to make it?

When browsing online or/and using connected devices in the physical world, even the most innocent action/information can be invisibly recorded, valued and translated into informational units, subsequently generating profit for monopolistic companies. While social platforms, brands, public institutions and governments explicitly promote the use of monitoring practices in order to better serve or protect us, we could also consider these techniques as implicitly political, playing around some dynamics of visibility and invisibility in order to assert new forms of power over targeted audiences.

In the last decade, a strong mistrust of new technologies has formed in the public opinion, fueled by events such as the revelations of Edward Snowden, the Cambridge Analytica scandal or the proliferation of fake news on social networks. We have also seen many artists take up the subject, sometimes with activist purposes. But even if a small number of citizens have begun to consider the social and political issues related to mass surveillance, and some individuals/groups/governments/associations have taken legal actions, surveillance capitalism still remains generally accepted, often because ignored or/and misunderstood.

Thanks to the huge profits generated by the data that we freely provide every day, big tech companies have been earning billions of dollars over the sale of our personal information. With that money, they could also further develop deep machine learning programs, powerful recommendation systems, and to broadly expand their range of services in order to track us in all circumstances and secure their monopolistic status. Even if we might consider this realm specific to the online world, we have seen a gradual involvement from the same companies to monitor the physical world and our human existences in a wide array of contexts. For example, with satellite and street photography (Google Earth, Street View), geo localization systems, simulated three-dimensional environments (augmented reality, virtual reality or metaverse) or extensions of our brains and bodies (vocal assistance and wearable devices). Ultimately, this reality has seen the emergence of not only a culture of surveillance but also of self-surveillance, as evidenced by the popularity of self-tracking and data sharing apps, which legitimize and encourage the datafication of the body for capitalistic purposes.

For the last 15 years, self-tracking tools have made their way to consumers. I believe that this trend is showing how ambiguous our relationship can be with tools that allow such practices. Through my work, I do not wish to position myself as a whistleblower, a teacher or activist. Indeed, to adopt such positions would be hypocritical, given my daily use of tools and platforms that resort to mass surveillance. Instead, I wish to propose an experience that highlights the contradictions in which you and I, internet users and human beings, can find ourselves. This contradiction is characterized by a paradox between our state of concern about the intrusive surveillance practices operated by the Web giants (and their effects on societies and humans) and a state of entertainment or even active engagement with the tools/platforms through which this surveillance is operated/allowed. By doing so, I want to ask how do these companies still manage to get our consent and what human biases do they exploit in order to do so. That’s is how my graduation work and my thesis will investigate the effect of gamification, gambling or reward systems as well as a the esthetization of data/self-data as means to hook our attention, create always more interactions and orientate our behaviors.


How to you plan to make it and on what timetable?

I am developing this project with Arduino Uno/Mega boards, an array of ultrasonic sensor, P5.js and screens.

How does it work?

The ultrasonic sensors can detect obstacles in a physical space and know the distance between the sensor and obstacle(s) by sending and receiving back an ultrasound. The Arduino Uno/Mega boards are microcontrollers which can receive this information, run it in a program in order to convert these values into a mm/cm/m but also map the space into an invisible grid. Ultimately, values collected on the Arduino’s serial monitor can be sent to P5.js through P5.serialcontrol. P5.js will then allow a greater freedom in way the information can be displayed on the screens.

Process:

1st semester: Building a monitoring device, converting human actions into events, and events into visual feedbacks

During the first semester, I am focused on exploring monitoring tools that can be used in the physical world, with a specific attention to ultrasonic sensors. Being new to Arduino programming, my way of working is to start from the smallest and most simple prototype and gradually increase its scale/technicality until reaching human/architectural scale. Prototypes are subject to testing, documentation and comments helping to define which direction to follow. The first semester also allows to experiment with different kind of screen (LCD screens, Touch screens, computer screens, TV screens) until finding the most adequate screen monitor(s) for the final installation. Before building the installation, the project is subject to several sketching and animated simulations in 3 dimensions, exploring different scenarios and narrations. At the end of the semester, the goal is to be able to convert a specific range of human actions into events and visual feedback creating a feedback loop from the human behaviors being converted into information; the translation of this information into written/visual feedbacks; and the effects of this feedbacks on human behavior; and so on.

2nd semester: Implementing gamification with the help of collaborative filtering, point system and ranking.

During the second semester, it is all about building and implementing a narration with the help of gaming mechanics that will encourage humans to feed the data gathering device with their own help. An overview of how it works is presented here in the project proposal and will be subject to further developments in the practice.

To summarize the storyline, the subject being positioned in the detection zone finds herself/himself unwillingly embodied as the main actor of a data collection game. Her/His mere presence generates a number of points/dollars displayed on a screen, growing as she/he stays within the area. The goal is simple: to get a higher score/rank and unlock achievements by acting as recommended by a data-collector. This can be done by setting clear goals/rewards to the subject, and putting its own performance in comparison to all the previous visitors, giving unexpected messages/rewards, and give an aesthetic value to the displayed informations.

The mechanism is based on a sample of physical events that have been already explored in the first semester of prototyping (detection of movements, positions, lapse of time spent standing still or moving, and entry or exit from a specific area of detection). Every single detected event in this installation is stored in a data bank, and with the help of collaborative filtering, will allow to the display of custom recommendations such as:

_ “Congratulations, you have now given the monitor 12 % of all possible data to collect”
_ “Are you sure you don’t want to move to the left? The monitor has only collected data from 3 visitors so far in this spot”
_ “Congratulations, the monitor has reached 1000 pieces of information from you!”
_ “If you stay there for two more minutes, there is a 99% chance you will be in the top 100 of ALL TIME data-givers”
_ “Leaving all-ready? The monitor has yet 304759 crucial pieces of information to collect from you”
_ “You are only 93860 actions away from being the top one data-giver”
_ “Statistics are showing that people staying for more than 5 minutes average will be 10 times more benefitting for me”
_ “The longer you stay on this spot, the more chance you have to win a “Lazy data-giver” badge”


The guideline is set out here, but will be constantly updated with the help of experiments and the results observed during the various moments of interaction between the students and the algorithm. For this purpose, the installation under construction will be left active and autonomous in its place of conception (studio) and will allow anyone who deliberately wishes to interact with it to do so. Beyond the voluntary interactions, my interest is also to see what can be extracted from people simply passing in front of this installation. In addition to this, some of the mechanics of the installation will be further explored by collaborating with other students, and setting up more ephemeral and organized experiences with the participants. (ex: 15 February 2022 with Louisa)

This semester will also include the creation of a definite set of illustrations participating to engage the participants of the installation in a more emotional way, the illustrations will be made by an illustrator/designer, with whom I usually collaborate.

3rd semester: Build the final installation of final assessment and graduation show. Test runs, debug and final touchs.

During the third semester, the installation should be settled in the school, in the alumni area, next to XPUB studio for the final assessment, and ultimately settled again at WORM for the graduation show. I am interested in putting this installation into privileged spaces of human circulation, (such as hallways) that would more easily involve the detection of people, and highlight the intrusive aspect of such technologies. The narration, the mechanics, illustrations and graphic aspect should be finalized at the beginning of the 3rd semester, and subject to intense test runs during all that period until meeting the deadline.


Relation to larger context

As GAFAM companies are facing more and more legal issues, and held accountable in growing numbers of social and political issues around the world, the pandemic context has greatly contributed to make all of us more dependent than ever on the online services provided by these companies and to somehow force our consent. While two decades of counter-terrorism measures legitimized domestic and public surveillance techniques such as online and video monitoring, the current public health crisis made even more necessary the use of new technologies for regulating the access to public spaces and services, but also for socializing, working together, accessing to culture, etc. In a lot of countries, from a day to another, and for an undetermined time, it has become necessary to carry a smartphone (or a printed QR code) in order to get access transport, entertainment, cultural and catering services, but also in order to do simple things such as to have a look at the menu in a bar/restaurant or to make an order.. Thus, this project takes place in a context where techno-surveillance has definitely taken a determining place in the way we can access spaces and services related to the physical world.

Data Marketisation / Self Data: Quantified Self / Attention Economy / Public Health Surveillance / Cybernetics


Who can help you?

Prototyping
Manetta Berends
Michael Murtaugh

Writing
Luke Williams

Arduino
XPUB Arduino knowledge sharing group
Dennis de Bel
Aymeric Mansoux

Installation building
Wood station
Interaction station

Illustrations
Adrien Jacquemet (illustrator/graphic designer)

Installation location
Leslie Robbins

Extra help for game narratives
Rosa Zangenberg (writer/artist)

Relation to previous practice?

During my previous studies in graphic design, I started being engaged with the new media by making a small online reissue of Raymond Queneau’s book called Exercices de Style. In this issue called Incidences Médiatiques (2017), the user/reader was encouraged to explore the 99 different versions of a same story written by the author in a less-linear way. The idea was to consider each user graphic user interface as a unique reading context. It would determine which story could be read, depending on the device used by the reader, and the user could navigate through these stories by resizing the Web window, by changing browser or by using on a different device.

As part of my graduation project called Media Spaces (2019), I wanted to reflect on the status of networked writing and reading, by programming my thesis in the form of Web to Print website. Subsequently, this website became translated in the physical space as a printed book, and a series of installations displayed in an exhibition space that was following the online structure of my thesis (home page, index, part 1-2-3-4). In that way, I was interested to inviting to visitors to make a physical experience some aspects of the Web

As a first-year student of Experimental Publishing, I continued to work in that direction by eventually creating a meta-website called Tense (2020) willing to display the invisible html <meta> tags inside of an essay in order to affect our interpretation of the text. In 2021, I worked on a geocaching pinball game highlighting invisible Web event, and a Web oscillator, which amplitude and frequency range were directly related to the user’s cursor position and browser screen-size.

While it has always been clear to me that these works were motivated by the desire to define media as context, subject or/and content, the projects presented here have often made use of surveillance tools to detect and translate user information into feedbacks, participating in the construction of an individualized narrative or/and a unique viewing/listening context (interaction, screen size, browser, mouse position). The current work aims to take a critical look at the effect of these practices in the context of techno surveillance.

Similarly, projects such as Media Spaces have sought to explore the growing confusion between human and web user, physical and virtual space or online and offline spaces. This project will demonstrate that these growing confusions will eventually lead us to be tracked in all circumstances, even in our most innocuous daily human activities/actions.


Selected References

Works:

« invites us to take on the role of an auditor, tasked with addressing the biases in a speculative AI »Alternatives to techno-surveillance

Expose humans as producers of useful intellectual labor that is benefiting to the tech giants and the use than can be made out of that labor.

Claims that that technological devices can be manipulated easily and hence, that they are fallible and subjective. They do this by simply placing a self-tracker (connected bracelet) in another context, such as on some other objects, in order to confuse these devices.

Allows galleries to enjoy encrypted internet access and communications, through a Tor Network

You are rewarded for exploring all the interactive possibilities of your mouse, revealing how our online behaviors can be monitored and interpretated by machines.

Portrait of the viewer is drawn in real time by active words, which appear automatically to fill his or her silhouette https://www.lozano-hemmer.com/third_person.php

«Every visitor to the website’s browser size, collected, and played back sequentially, ending with your own.»

Readings of the building and its contents are therefore always unique -- no two visitors share the same experience. https://haque.co.uk/work/mischievous-museum/

Books & Articles:

  • SHOSHANA ZUBOFF, The Age of Surveillance Capitalism (2020)

Warns against this shift towards a «surveillance capitalism». Her thesis argues that, by appropriating our personal data, the digital giants are manipulating us and modifying our behavior, attacking our free will and threatening our freedoms and personal sovereignty.

  • EVGENY MOROZOV, Capitalism’s New Clothes (2019)

Extensive analysis and critic of Shoshana Zuboff research and publications.

  • BYRON REEVES AND CLIFFORD NASS, The Media Equation, How People Treat Computers, Television, and New Media Like Real People and Places (1996)

Precursor study of the relation between humans and machine, and how do you human relate to them.

  • OMAR KHOLEIF, Goodbye, World! — Looking at Art in the digital Age (2018)

Authors shares it’s own data as a journal in a part of the book, while on another part, question how the Internet has changed the way we perceive and relate, and interact with/to images.

  • KATRIN FRITSCH, Towards an emancipatory understanding of widespread datafication (2018)

Suggests that in response to our society of surveillance, artists can suggest activist response that doesn’t necessarily involve technological literacy, but instead can promote strong counter metaphors or/and counter use of these intrusive technologies.

What is my work, What do I want to tell, What is my position

Translated from discussion with Michael

People have now more concrete experiences of the digital/Web interface than the physical space. Museums, hotels, houses, cars interiors, restaurants are themselves becoming more and more comparable to digital interface where everything is optimized, and where our behaviours, actions and even inactions are being detected and converted into commands in order to offer a more customized (and profitable) experience to each of us. In that sense, we are getting closer from becoming users of our own interfaced physical reality. By creating a exhibition spaces explicitly inspired from a desktop Web interface, I wish to question what could be the future of exhibition space, what are the limits of this interfaced and individualized reality and how could it affect our own experience and understanding of art.

What could we learn from interface design? What could be the future of exhibition space?

"Bring attention to the systems underlying artistic productions" both on the Web and the physical world
"reversal of the desktop metaphor" (using the virtual as "concrete" to metaphorically understand a physical exhibition space), what will be the future of an exhibition space... (is already working exhibition spaces working with sensors) scary and fascinating at the same time...
"my embracing/use of sensors isn't about proposing these techniques as a solution / ideal / about control... interfaces requiring less and less from us but paradoxically extracting more and more from us"
every small unconsidered behaviour is being used (trying to used)...
there is unpredictable.... because of all the factors, want unexpected things to happen...
the reality of digital isn't all about precision and control, this notion of surprise is key for an experience.
Exploring the fullness of digital / programmed / computational media, including those "edge" cases / the "glitch" ... the surprise...
Examples from museums: (for instance Brussels has the MIM Museum Instrument Museum, sadly the old now retired interface was a system with headphones that were activated in the space, so as you approached vitrines with a violin you would here a performance of that instrument)...
How a mistake can create something else. / Bugs / Glitch Letting an accident/surprise/unexpect exist, exploring the fullness of digital programming
My position seems to fit with Researcher/Designer
Digital is not precise and omnipotent, it has so faults, flows and vulnerabilities.


To check:


Software Art

Software creation or use of software concepts for artworks representation. Commonly put the spectator in the role of a user.

Internet Art

Elements from the Internet bringed outside of the Internet and promoting the Internet as part of both virtual and physical realities.

  • John Ippolito

Post-Internet Art vs Internet 2.0

Post-Internet Art: Litteraly Art after the internet. Can consists of using online material for later use in offline works or can relate on the effect of the Internet in various aspects of the culture, aesthetic and society.

  • Olia Lialina

VS
Internet 2.0: Assuming that a world Internet doesn't exist anymore

  • Zach Blas

Net Art

Started in late 70's and nowadays associated with a outdated era of the Internet (1.0?)
Closely related to Network Art

  • Olia Lialina, My Boyfriend Came Back From the War, 1996

New Aesthetics

Confronting/merging virtual and physical, or humans and machine, etc

  • James Bridle

Funware

Gamification of non-game platforms in order to encourage some actions, behaviors, transactions with the help of various rewarding systems.

Connections to XPUB1

User viewing contexts (on the Web) from special issue 13

Description

Create motion from the diffraction of the user interface which offers flexible and almost infinite possible renders of a same Web page. The sensible variety of user viewing contexts tells about the placiticy of the user interface. This is the result from the wide range user devices, window or screen sizes, Web browsers, (as part of many other parameters). A first of movement capture and montage of the user interface placticity can be as part of the post-production of my interpretation of the esssay "Tense", part of the Special Issue 13.

Capturing and puting into motion the User interface placticity

Trying to play around with the Browser window resizing in order to create a playful animation decidaced be a thumbnail of the project. The two first screen capture will be the basis of the upcoming motion. I will first try to smooth the window movement and make the two screen capture fit togehter before synchronizing them and looping them.


TENSE Motion Rectangle Format Loop in the loop
TENSE MOTION Initial Screen Capture 1




























Notes: Add Web Oscillator

Prototyping

Arduino

Early sketch that is about comparing and questioning our Spectator experience of a physical exhibition space (where everything is often fixed and institutionalized), with our User experience of a Web space (where everything is way more elastic, unpredictable and obsolete). I’m interested about how slighly different can be rendered a same Web page to all different users depending on technological contexts (device nature, browser, IP address, screen size, zoom level, default settings, updates, luminosity, add-ons, restrictions, etc). I would like to try to create a physical exhibition space/installation that would be inspired from the technology of a Web user window interface in order then to play with exhbitions parameters such as the distance between the spectator and the artwork, the circulation in space, the luminosity/lighting of the artwork(s), the sound/acoustics, etc etc etc.

Distance between wall behind the spectator and the artwork has to be translated into a variable that can affect sound or light in the room. Wall position could be connected to the dimensions of a user interface in real time with arduino and a motor.

Create a connected telemeter with an Arduino, a ultrasonic Sensor (HC-SR04) and a ESP8266 module connected to Internet

It seems possible to create your own telemeter with a arduino by implementing an ultrasonic Sensor HC-SR04
By doing so, the values capted by the sensor could potentaialy be directly translated as a variable.
Then with the ESP8266 module, the values could be translated on a database on the internet. Then I could enter that website and see the values from anywhere and use them to control light, sound or anything else I wish.

Tool/Material list:

  • Telemeter (user to get the distance between the device and an obstacle)
  • Rails
  • Handles
  • Wheels
  • Movable light wall
  • Fixed walls
  • USB Cable
  • Connexion cables
  • Arduino
  • ESP8266
Connexion cables (Arduino)
USB Cable
Arduino
HC-SR04 Ultrasonic Sensor
Plywood x 3
Handle
ESP8266
Rail









About the ultrasonic Sensor (HC-SR04)

Characteristics

Here are a few of it's technical characteristic of the HC-SR04 ultrasonic sensor :

  • Power supply: 5v.
  • Consumption in use: 15 mA.
  • Distance range: 2 cm to 5 m.
  • Resolution or accuracy: 3 mm.
  • Measuring angle: < 15°.

Ref More infos about the sensor here and here

Where to buy the ultrasonic Sensor (HC-SR04)

Prototype 1 : Arduino + Resistor

During a workshop, we started with a very basic fake arduino kit, a led, a motor, and a sensor. After making a few connections, we got to understand a bit how it works.


   #include <Servo.h>
   Servo myservo;  // create servo object to control a servo
   int pos = 0;    // variable to store the servo position
   int ldr = 0;    // vairable to store light intensity
   void setup() {
   Serial.begin(9600); // begin serial communication, NOTE:set the same baudrate in the serial monitor/plotter
   myservo.attach(D7);  // attaches the servo on pin 9 to the servo object
   }
   void loop() {
   //lets put the LDR value in a variable we can reuse
   ldr = analogRead(A0);
   
   //the value of the LDR is between 400-900 at the moment 
   //the servo can only go from 0-180
   //so we need to translate 400-900 to 0-180
   //also the LDR value might change depending on the light of day
   //so we need to 'contrain' the value to a certain range
   ldr = constrain(ldr, 400, 900); 
   //now we can translate
   ldr = map(ldr, 400, 900, 0, 180);
   //lets print the LDR value to serial monitor to see if we did a good job
   Serial.println(ldr); // read voltage on analog pin 0, print the value to serial monitor
   //now we can move the sensor accoring to the light/our hand!
   myservo.write(ldr);      // tell servo to go to position in variable 'pos'
   delay(15);    
   }


How to make a engine work
credits: Dennis de Bel
How to make a sensor work
Credits: Dennis de Bel
How to make both sensor and engine works together
Credits: Dennis de Bel
Sensortest during workshop

Split Screen Arduino + Sensor + Serial Plotter + Responsive Space

Trying here to show the simutaneous responses between the sensor, the values, and the simualtion.

Splitscreen Arduino + Sensor + Serial Plotter + Responsive Space




















Prototype 2: Arduino + Ultrasonic sensor

For this very simple first sketch and for later, I will include newPing library that improves a lot the ultrasonic sensor capacities.

Sketch 1: Arduino Uno + Sensor






















 #include <NewPing.h> 
 int echoPin = 10;
 int trigPin = 9;
 
 NewPing MySensor(trigPin, echoPin); //This defines a new variable
 
 void setup() {
   // put your setup code here, to run once:
   Serial.begin(9600);
 }
 
 void loop() {
   // put your main code here, to run repeatedly:
  int duration = MySensor.ping_median(); 
  int distance = MySensor.convert_in(duration);
 
  Serial.print(distance);
  Serial.println("cm");
  delay(250);
 }

Prototype 3: Arduino Uno + Sensor + LCD (+ LED)

All together from https://www.youtube.com/watch?v=GOwB57UilhQ

Sketch 2: Arduino Uno + Sensor + LCD
Sketch 3: Arduino Uno + Sensor + LCD + LED






















  #include <LiquidCrystal.h>
  
  LiquidCrystal lcd(10,9,5,4,3,2);
 
 const int trigPin = 11;
 const int echoPin = 12;
 
 long duration;
 int distance;
 
 void setup() {
   // put your setup code here, to run once:
     analogWrite(6,100);
     lcd.begin(16,2);
     pinMode(trigPin, OUTPUT); // Sets the trigPin as an Output
 pinMode(echoPin, INPUT); // Sets the echoPin as an Input
 Serial.begin(9600); // Starts the serial communication
 
     
 }
 
 void loop() {
 long duration, distance;
   digitalWrite(trigPin,HIGH);
   delayMicroseconds(1000);
   digitalWrite(trigPin, LOW);
   duration=pulseIn(echoPin, HIGH);
   distance =(duration/2)/29.1;
   Serial.print(distance);
   Serial.println("CM");
   delay(10);
 // Prints the distance on the Serial Monitor
 Serial.print("Distance: ");
 Serial.println(distance);
 
     lcd.clear();
     lcd.setCursor(0,0);
     lcd.print("Distance = ");
     lcd.setCursor(11,0);
     lcd.print(distance);
     lcd.setCursor(14,0);
     lcd.print("CM");
     
     delay(500);
     
 }

From this sketch, I start considering that the distance value could be directly sent to a computer and render a Web page depending on its value.
Note: It looks like this sensor max range is 119cm, which is almost 4 times less than the 4 meters max range stated in component description.

Prototype 4: Arduino Uno + Sensor + LCD + 2 LED = Physical vs Digital Range detector

Using in-between values to activate the green LED
Once again, puting together the simulation and the device in use.

Sensor Test VS Elastic Space




































 #include <LiquidCrystal.h>
 #include <LcdBarGraph.h> 
 #include <NewPing.h> 
 
   LiquidCrystal lcd(10,9,5,4,3,2);
 
 const int LED1 = 13; 
 const int LED2 = 8;   
 const int trigPin = 11;
 const int echoPin = 12;
 
 long duration; //travel time
 int distance;
 int screensize;
 
 void setup() {
   // put your setup code here, to run once:
     analogWrite(6,100);
     lcd.begin(16,2);
     pinMode(trigPin, OUTPUT); // Sets the trigPin as an Output
     pinMode(echoPin, INPUT); // Sets the echoPin as an Input
     Serial.begin(9600); // Starts the serial communication
 
     pinMode(LED1, OUTPUT);
     pinMode(LED2, OUTPUT);
 }
 
 void loop() {
 long duration, distance;
   digitalWrite(trigPin,HIGH);
   delayMicroseconds(1000);
   digitalWrite(trigPin, LOW);
   duration=pulseIn(echoPin, HIGH);
   distance =(duration/2)/29.1; //convert to centimers
   screensize = distance*85;
   Serial.print(distance);
   Serial.println("CM");
   Serial.print(screensize);
   delay(10);
 
   if ((distance >= 15) && (distance<=20))
   {
      digitalWrite(LED2, HIGH);
      digitalWrite(LED1, LOW);
   }
   else
   {
      digitalWrite(LED1, HIGH);
      digitalWrite(LED2, LOW);    
   }
 
 // Prints the distance on the Serial Monitor
 Serial.print("Distance: ");
 Serial.println(distance);
 
     lcd.clear();
     lcd.setCursor(0,0);
     lcd.print("ROOM");
     lcd.setCursor(6,0);
     lcd.print(distance);
     lcd.setCursor(9,0);
     lcd.print("cm");    
     lcd.setCursor(0,2);
     lcd.print("SCR");
     lcd.setCursor(6,2);
     lcd.print(screensize);
     lcd.setCursor(9,2);
     lcd.print("x1080px");
         
     delay(500);
     
 }


I brought a second arduino, 2 long breadboards, black cables, another LCD screen, and remade the setup on this format. For some reasons the new LCD screen is not going in the breadboard, and I need more male to female cables in order to connect it correctly. With this longer breadboard, I want to extend the range value system, and make it visible with leds and sounds.

Upgrade























How to get more digital pins [not working]

I tried 4 different tutorials but still didn't find a way to make the thing work, that's very weird, so I will just give up and take a arduino mega =*(

ArduinoExtraDigitalPins





























Prototype 5: Arduino Uno + 3 Sensor + 3 LEDS

With a larger breadboard, connecting 3 sensors all together. Next step will be to define different ranges of inbetween values for each sensor in order to make a grid. To accomplish this grid I will make a second row of sensors such as this, in order to get x and y values in space

Prototype 6: Arduino Uno + 3 Sensor + 12 LEDS

With 3 sensors, added on 2 long breadboads, and with a different set of range values, we can start mapping a space.

SensorMediaQueries
Physical Space Mapping








Prototype 7: Arduino Uno + 12 LEDS + 3 Sensor + Buzzer + Potentiometer + LCD

For this prototype, I implement a buzzer that will emit a specific sound depending on the distance of the obstacle detected by the sensor. I also puted back a LCD displaying the 3 sensors values. The screen luminosity can be changed via a potentiometer.
Ressources:

ArduinoMegaSensorBuzzerLCD





























Prototype 8: Arduino Uno + 12 LEDS + 3 Sensor on mini breadboards + Buzzer + Potentiometer + LCD

Same code, but new setup detaching each sensor from each others and allowing to place them anywhere.

ArduinoMegaSensorBuzzerLCDMinibreadboard.jpg





























Prototype 9: Arduino Uno + 21 LEDS + 7 Sensor + Buzzer + Potentiometer + LCD

Sensor Wall 01
PhysicalMapping2





























Sketch 10: Arduino Mega + 7 Sensors + LCD + 3 buzzers + P5.js

P5.js and ultrasonic sensor

The goal here was to create a first communication between the physical setup and a P5.js web page






































Sketch 11: Arduino Mega + UltraSonicSensor + LCD TouchScreen

LCD Arduino Variable poster






























 #include <TFT_HX8357.h>
 
 TFT_HX8357 tft = TFT_HX8357(); 
 
 
 unsigned long targetTime = 0;
 byte red = 0;
 byte green = 0;
 byte blue = 255;
 byte state = 0;
 unsigned int colour = blue << 255;
 
 #define TFT_GREY 0x5AEB // New colour
 
 //Ultrasonic Sensor
  int VCC = 13;
  int trigPin = 11;
  int echoPin = 12;
 
 long duration, distance, UltraSensor; 
 char data;
 String SerialData="";
 
 void setup(void) {
   tft.init();
   tft.setRotation(2);
   tft.fillScreen(TFT_BLACK);
   targetTime = millis() + 100;
 
   Serial.begin (9600); 
   //define pin modes
   pinMode(VCC, OUTPUT);   
   pinMode(trigPin, OUTPUT); //sets the trigPin as an Output
   pinMode(echoPin, INPUT);  //sets the echoPin as an Input
   
   digitalWrite(VCC, HIGH);  // +5V - Pin 13 as VCC
 }
 void rainbow_fill()
 {
   
   for (int i = 45; i < 421; i++) {
     tft.drawFastHLine(1, i, tft.width(), colour); // in this example tft.width() returns the pixel width of the display
     switch (state) {
       case 0:
         blue ++;
         if (blue == 0) {
           blue = 1;
           state = 1;
         }
         break;
       case 1:
         green--;
         if (green == 255) {
           green = 254;
           state = 2;
         }
         break;
       case 2:
         blue ++;
         if (blue == 0) {
           blue = 1;
           state = 3;
         }
       case 5:
         green --;
         if (blue == 255) {
           blue = 254;
           state = 0;
         }
         break;
     }
     colour = red << 0 | green << 255 | blue << 0;
   }
 }
 
 void loop() {
   SonarSensor(trigPin,echoPin);              
   UltraSensor = distance; 
 
    Serial.print("A: ");
    Serial.print(UltraSensor);
    Serial.println(" cm");
    tft.setCursor(0, 0, 2);
    tft.setTextColor(TFT_WHITE);  tft.setTextSize(1.8);
    tft.setTextFont(4);
    tft.println("AQUARIUM 2.0");
    tft.println("29.11.2021"); 
 
    tft.setCursor(215, 0, 2);
    tft.setTextColor(TFT_WHITE);  tft.setTextSize(1.8);
    tft.setTextFont(4);
    tft.println(" by XPUB");
 
    tft.setCursor(215, 24, 2);
    tft.setTextColor(TFT_WHITE);  tft.setTextSize(1.8);
    tft.setTextFont(4);   
    tft.println(" students");
 
    tft.setCursor(0, 426, 2);
    tft.setTextColor(TFT_WHITE);  tft.setTextSize(1.8);
    tft.setTextFont(4);
    tft.println("We are");
 
    tft.setCursor(0, 454, 2);
    tft.setTextColor(TFT_WHITE);  tft.setTextSize(1.8);
    tft.setTextFont(4);
    tft.println("Open!");
    
    tft.setCursor(135, 426, 2);
    tft.setTextColor(TFT_WHITE);  tft.setTextSize(1.8);
    tft.setTextFont(4);
    tft.println("you are");
    
    tft.setCursor(222, 426, 2);
    tft.setTextColor(TFT_WHITE, TFT_BLACK);  tft.setTextSize(1.8);
    tft.setTextFont(4);
    tft.println(UltraSensor); 
 
    tft.setCursor(284, 426, 2);
    tft.setTextColor(TFT_WHITE);  tft.setTextSize(1.8);
    tft.setTextFont(4);
    tft.println("cm");
    
    tft.setCursor(135, 454, 2);
    tft.setTextColor(TFT_WHITE);  tft.setTextSize(1.8);
    tft.setTextFont(4);
    tft.println("away from event"); 
 
    if (targetTime < millis()) {
     targetTime = millis() + 10;
     rainbow_fill(); // Fill the screen with rainbow colours
    }
   
     
   // Fill screen with random colour so we can see the effect of printing with and without 
   // a background colour defined
   //tft.fillScreen(255,255,255);
   
   // Set "cursor" at top left corner of display (0,0) and select font 2
   // (cursor will move to next line automatically during printing with 'tft.println'
   //  or stay on the line is there is room for the text with tft.print)
 
 
   // A SENSOR ———————————————————————————————————————————————————————————————————————————————————————————————
     // A1———————————————————————————————————————————————————————————————————————————————————————————————
       if(UltraSensor <=30)// if distance is lGss than 10 Cm turn thG LED ON
       {
       tft.setCursor(0, 200, 2);
       tft.setTextColor(TFT_WHITE);  tft.setTextSize(1);
       tft.setTextFont(4);
       //tft.println("Message1");
       delay(10);  
       }
       else                // else turn thG LED OFF
       {
       }
       // A2———————————————————————————————————————————————————————————————————————————————————————————————
       if((UltraSensor >=31) && (UltraSensor <=60))
       {
       tft.setCursor(0, 200, 2);
       tft.setTextColor(TFT_WHITE);  tft.setTextSize(1);
       tft.setTextFont(4);
       //tft.println("Message2");
       delay(10); 
       }
       else                // else turn thG LED OFF
       {
       }
       // A3———————————————————————————————————————————————————————————————————————————————————————————————
       if((UltraSensor >=61) && (UltraSensor <=300))
       {
       tft.setCursor(0, 200, 2);
       tft.setTextColor(TFT_WHITE);  tft.setTextSize(1);
       tft.setTextFont(4);
       //tft.println("Message3");
       delay(10); 
       }
       else               
       {
       }
        if(UltraSensor >=301)
       {
       tft.setCursor(0, 200, 2);
       tft.setTextColor(TFT_WHITE);  tft.setTextSize(1);
       tft.setTextFont(4);
       //tft.println("Message4");
       delay(10);
       }
       else               
       {
       }
 }
 
  void SonarSensor(int trigPinSensor,int echoPinSensor)//it takes the trigPIN and the echoPIN 
     {
       //generate the ultrasonic wave
     digitalWrite(trigPinSensor, LOW);// put trigpin LOW 
     delayMicroseconds(2);// wait 2 microseconds
     digitalWrite(trigPinSensor, HIGH);// switch trigpin HIGH
     delayMicroseconds(10); // wait 10 microseconds
     digitalWrite(trigPinSensor, LOW);// turn it LOW again
     
     //read the distance
     duration = pulseIn(echoPinSensor, HIGH);//pulseIn funtion will return the time on how much the configured pin remain the level HIGH or LOW; in this case it will return how much time echoPinSensor stay HIGH
     distance= (duration/2) / 29.1; // A1 we have to divide the duration by two  
     }



About the ESP8266 module

The ESP8266 is a microcontroller IC with Wi-Fi connection, it will allow us to connect the arduino to the internet so we can get the values obtained from sensors received directly on a self-hosted webpage. From this same web page, it would also be possible to control LESs, motors, LCD screens, etc.

Ressources about ESP8266 module

Kindly fowarded by Lousia:

Which ESP8266 to buy

Things to try

Prototyping Ressources

Do it Yourself Ressources (from Dennis de Bel)

  • Instructables is a huge source of (written) tutorials on all kinds of topics. Keep in mind it's more quantity than quality. Interesting for you might be 'diy sensors'
  • Hand Made Electronic (Music): Great resource for cheap, diy electronics project focussing on

sound/music (pdf findable online)

  • Make: Electronics: Amazing, complete guide to everything 'electronics' (Warning, HUGE pdf)
  • Thingiverse: The place to find 3d printable mechanics, enclosures, parts etc.

Electronic Shops (physical)

LIST OF SHOPS (also more physical NL ones)

Electronic Webshops (NL)

Electronic Webshops (Rest)

PCB making EU (Expensive)

PCB making China (Cheap but import tax)

  • JLCPCB (1 week from design upload to in your hands, low quality solder mask)
  • PCBWAY (1 week from design upload to in your hands)
  • ALLPCB (1 week from design upload to in your hands)

Arduino and Sensors

Sensor only Kit

  • 45-in-1 (aliexpress) Example sensor you will find in such a kit documented here

Arduino Starter Projects

or slightly more complex:

or in videos:

or just many different ideas:

or - of course - on Instructables if you want to have a complete course:

or this course:

ARDUINO + PROCESSING (visualizing sensors)

MISCELANIOUS KEYWORDS and LINKS

Installation

Creating an elastic exhibition space

Responsive Space Installation Simulation
Responsive Space (detail)
Spectator friendly physical exhibition space.png
Moving Wall Structure Shema 1





Ressources

  • Movable walls build out for Art Museum of West Virginia University link
  • Gallery Wall System (GWS) link
  • CASE-REAL installs movable walls inside a basement art gallery in tokyo link

Venues

Venue 1: Aquarium

Description


AQUARIUM 1.0


A Small Ecosystem for Living Thoughts

Monday, 11th October
19:30 – 21:30
Leeszaal Rotterdam West
Rijnhoutplein 3, 3014 TZ Rotterdam

with Clara Gradel, Floor van Meeuwen, Martin Foucaut, Camilo Garcia, Federico Poni, Nami Kim, Euna Lee, Kendal Beynon, Jacopo Lega and Louisa Teichmann

It’s oh-fish-ial! Students of the Experimental Publishing Master invite you to dive into their small ecosystem of living thoughts. Join us for an evening of conversation, discussion and new view points. If you look closely, you might even see some early thesis ideas hatching. Let's leave no rock unturned.

Observation questionnaire

This exercice is a very small, humble and almost 100% analog exercice questioning representation in two small steps.

1st step

photo of a brick












  • 1st step: I give a sheet of paper to people during the venue and ask them to answer a series of questions concerning the object (brick) that is being displayed in the middle of the room on a podium. It is specified to them that they can be anywhere while observing this brick and in any position. Here are the quesitons:


  • Please write down your first name:


  • Describe your position (sitting/standing/other):


  • Describe your location in the room:


  • Describe what you are seeing while looking at the screen:


  • Describe how you feel mentaly/emotionaly:



2nd step

photo of brick displayed inside a computer screen












  • 2nd step: I take the answers, wait a round, and then give back a new sheet of paper to the same people with the exact same questions concerning the respresentation of the object (brick) that is being displayed in the middle of the room on a computer screen on the same podium.

Answer Samples

1.0 Object on a podium

  • 1.1 Sitting on corner stairs —> Want to see it from different angles —> Feeling trapped, frustrated
  • 1.2 Sitting on stairs —> a rock looking dead —> Feeling sad
  • 1.3 Sitting on the left close from columns —> rational observation —> Nostalgic memories because participated to the creation of the object as it looks right now
  • 1.4 Sitting in front of object —> Calm and slighly confused
  • 1.5 Sitting on the floor next to stairs in between the side and the middle —> Looking at the object from the bottom —> Feeling a bit confused and inspired



2.0 Photo of the object displayed on a computer screen placed on a podium

  • 2.1 Sitting on a chair seeing the brick from a bird perspective -> Feeling more control of the situation
  • 2.2 Sitting very close from the brick —> Seeing a flat and almost abstract picture —> Feeling drawn to the picture, aesthetically pleasing, feeling less sad about the object
  • 2.3 Sitting under a table very far way —> Looking abstract but identifiable —> Exited about the unusual and childish observation position
  • 2.4 Sitting on stairs —> and seeing the brick in 2D —> Feeling fine
  • 2.5 Sittiing on the stairs —> Seeing a side of the screen with a top view photo of the object —> Feeling confortable



Answers1_RepresentationQuestionnaire
Answers2_RepresentationQuestionnaire
Answers3_RepresentationQuestionnaire
Answers4_RepresentationQuestionnaire
Answers5_RepresentationQuestionnaire


























Venues

Venue 2: Aquarium 2.0

Description


Date 29th Nov — 4th Dec 2021
Time 15h — 18h
29th Nov — 4th Dec 2021 (all day)
Location: De Buitenboel, Rosier Faassenstraat 22 3025 GN Rotterdam, NL


AQUARIUM 2.0

An ongoing window exhibition with Clara Gradel, Floor van Meeuwen, Martin Foucaut, Camilo Garcia, Federico Poni, Nami Kim, Euna Lee, Kendal Beynon, Jacopo Lega and Louisa Teichmann

Tap upon the glass and peer into the research projects we are currently working on. From Monday 29th of November until Saturday 4th of December we put ourselves on display in the window of De Buitenboel as an entry point into our think tank. Navigating between a range of technologies, such as wireless radio waves, virtual realities, sensors, ecological and diffractive forms of publishing, web design frameworks, language games, and an ultra-territorial residency; we invite you to gaze inside the tank and float with us. Welcome back to the ecosystem of living thoughts.

Aquarium LCD Portal (29 Nov – 4th Dec)

This interactive micro-installation composed of a LCD screen and sensor(s) invites users/visitors to change the color of the screen and displayed messages by getting more or less close from the window. Link

ScreenPortalFront
ScreenPortalback
LCDScreenTest














Readings (new)(english)(with notes in english)

About Institutional Critique

To read

→ 1. Art and Contemporary Critical Practice: Reinventing Institutional CritiqueDoc
→ 2. From the Critique of Institutions to an Institution of Critique - Andrea FraserDoc
→ 3. Institutional critique, an anthology of artists writings - Alexander AlberroDoc

About Techno-Solutionism

To read

→ 1. The Folly of Technological Solutionism: An Interview with Evgeny Morozov - Natasha Dow Schüll

About Meta

To read

→ 1.  The meta as an aesthetic category Bruno Trentini (2014)
→ 2.  File:RMZ ARTIST WRITING(2).pdf The eye tells the story by Rosa Maria Zangenberg (2017)
→ 3.  Leonardo Da Vinci - Paragone by Louise Farago

About exhibition space

To read

→ 2. Kluitenberg, Eric, ed. Book of imaginary media. Excavating the dream of the ultimate communication medium. Rotterdam: NAi Publishers, 2006.
→ 3. The wall and the canvas: Lissitzky’s spatial experiments and the White Cube
→ 6. Decorative Arts: Billy Al Bengston and Frank Gehry discuss their 1968 collaboration at LACMA by Aram Moshayedi
→ 8.  File:Resonance and Wonder STEPHEN GREENBLATT.pdf Resonance and Wonder - STEPHEN GREENBLATT
→ 9.  A Canon of Exhibitions - Bruce Altshuler File:A Canon of Exhibitions - Bruce Altshuler.pdf
→ 10. Documenta - File:A BRIEF HISTORY OF AN EXHIBITION AND ITS CONTEXTS - Klaus Siebenhaar.pdf A BRIEF HISTORY OF AN EXHIBITION AND ITS CONTEXTS - Klaus Siebenhaar
→ 11. Pallasmaa - The Eyes of the Skin File:Pallasmaa - The Eyes of the Skin.pdf
→ 12. Venturi - Learning from Las Vegas File:Venturi - Learning from Las Vegas.pdf
→ 13. Preserving and Exhibiting Media Art: Challenges and Perspectives - JULIA NOORDEGRAAF, COSETTA G. SABA; BARBARA LE MAÎTRE; VINZENZ HEDIGER Copyright: 2013 - Publisher: Amsterdam University Press Series: Framing Film

Reading/Notes

→ 1. After the White Cube. ref 2015 NOTES INSIDE

  • How and why White Cube rised and became democratized
  • White Cube // Consumerism = Art Consumerism?
  • Exhibition Space > Artworks
  • Experience of interpretation = Entertainment of Art?
  • Museum vs Mausoleum


→ 2. Spaces of Experience: Art Gallery Interiors from 1800 – 2000 ref NOTES INSIDE

  • Art vs 50's consumerism / Choregraphy of desire?
  • Check theorists Hermann von Helmholtz and Wilhelm Wundt


→ 3. Colour Critique A Symposium on Colour as an Agent for Taste, Experience and Value in the Exhibition Space NOTES INSIDE
May 24, 2019 - Noise! Frans Hals, Otherwise, Frans Hals Museum
→ 4.  Noise! Frans Hals, Otherwise NOTES INSIDE

  • Role of colours in the viewer's experience of an exhibition
  • Institutional Critique
  • Institutionalised Space / White cube


→ 5. Mental Spaces - Joost Rekveld/Michael van Hoogenhuyze NOTES INSIDE
(course for Artscience 2007/8) doc

  • About perspective
  • About Space time
  • About Cyber Space


→ 6.  THE DEVIL IS IN THE DETAILS: MUSEUM - Displays and the Creation of Knowledge Doc NOTES INSIDE
Stephanie Moser SOUTHAMPTON UNIVERSITY (MUSEUM ANTHROPOLOGY) 2010

  • Architecture (Neoclassical buildings)
  • Big vs Small exhibition Space
  • Lined up objects vs non systematic display
  • Architecture/Design
  • Gallery interiors (Ceiling/Interior Design elements/Furniture
  • Colors
  • Individual lighting of objects vs global lighting
  • Dark vs Bright lighting
  • Chronological vs Thematic arrangement
  • Academic vs Journalistic writting
  • Busy layout vs Minimal Layout
  • Exibition seen vs other exhibitions
  • Themed/idea-oriented vs objectled exhibitions
  • Didactic vs discovery exhibition
  • Contextual, immersive, or atmospheric exhibitions
  • Audience vs Reception


→ 7. Fantasies of the Library - Etienne Turpin (ed.), Anne-Sophie Springer (ed.) Ref; Editeur: The MIT Press; Date de publication: 1 sept. 2018

  • How the a physical organization influence of a bookshelf can influence it's digital version
  • The book as a minitaure gallery/exhibition space
  • The library as a public place of reading
  • Library vs Exhibition Space = Use vs Display
  • Book-theme exhibitions

About User vs Visitor, or user in exhibition space

Designing the user experience in exhibition spaces - Elisa Rubegni, Caporali Maurizio, Antonio Rizzo, Erik Grönvall

  • What are the GUI intentions
  • What is the WIMP interaction model
  • What are the post-Wimp models
  • About Memex

About User Interface

Readings/Notes

→ 1. bootlegAlexander R. Galloway - The Interface Effect 1st ed. Malden, USA: Polity Press.

  • The interface paradox
  • The less they do, the more they achieve and the more they become invisible & unconsidered
  • The interface as a "significant surface"
  • The interface as a gateway
  • The interface as "the place where information moves from one entity to another"
  • The interface as the media itself
  • The interface as "agitation or generative friction between different formats"
  • The interface as "an area" that "separates and mixes the two worlds that meet together there"


→ 2. bootleg Nick Srnicek - Navigating Neoliberalism: Political Aesthetics in an Age of Crisis NOTES INSIDE
Editeur: medium.com, Date de publication: 20 oct. 2016

  • From an aesthetic of sublime into an aesthetics of the interface
  • Cognitive mapping


→ 3. bootleg Program Or Be Programmed - Ten Commands For A Digital Age Douglas Rushkoff NOTES INSIDE
Douglas Rushkoff, A., 2010. Program Or Be Programmed - Ten Commands For A Digital Age Douglas Rushkoff. 1st ed. Minneapolis, USA: OR Books.

  • "Instead of learning about our technology, we opt for a world in which our technology learns about us."
  • Programmed by the interfaces
  • From a transparent to an opaque medium


→ 4. bootlegThe Best Interface Is No Interface - Golden Krishna NOTES INSIDE
Krishna, G., 2015. The Best Interface Is No Interface: The simple path to brilliant technology (Voices That Matter). 1st ed. unknown: New Riders Publishing.

  • "Screen Obsessed Approach to Design"
  • UI vs UX


→ 5. Plasticity of User Interfaces:A Revised Reference Framework NOTES INSIDE
Gaëlle Calvary, Joëlle Coutaz, David Thevenin Quentin Limbourg, Nathalie Souchon, Laurent Bouillon, Murielle Florins, Jean Vanderdonckt

  • About the term 'Placticity'


→ 6. Interface Critique- Beyond UX - FLORIAN HADLER, ALICE SOINÉ; DANIEL IRRGANG DOC Florian Hadler, Alice Soiné, Daniel Irrgang

  • The interface as an "historical artifact", a "space of power"
  • The interface as human -machine boudary
  • What is interface critique
  • Interface in computer science
  • The screen for Lev Manovitch



More to read/see

→ 1. Bickmore, T.W., Schilit, B.N., Digestor: Device- Independent Access To The World Wide Web, in Proc. of 6th Int. World Wide Web Conf. WWW’6
         (Santa Clara, April 1997)

→ 2. Bouillon, L., Vanderdonckt, J., Souchon, N., Recovering Alternative Presentation Models of a Web Page with VAQUITA, Chapter 27, in Proc. of 4th Int. Conf. on Computer- Aided Design of User Interfaces CADUI’2002
         (Valenciennes, May 15-17, 2002)

→ 3. Calvary, G., Coutaz, J., Thevenin, D., Supporting Context Changes for Plastic User Interfaces: a Process and a Mechanism, in “People and Computers XV –
         Interaction without Frontiers”, Joint Proceedings of AFIHM-BCS Conference on Human-Computer Interaction IHM-HCI’2001(Lille, 10-14 September 2001)

→ 4. Cockton, G., Clarke S., Gray, P., Johnson, C., Literate Development: Weaving Human Context into Design Specifications, in “Critical Issues in User Interface Engineering”,
         P. Palanque & D. Benyon (eds), Springer-Verlag, London, 1995.

→ 5. Graham, T.C.N., Watts, L., Calvary, G., Coutaz, J., Dubois, E., Nigay, L., A Dimension Space for the Design of Interactive Systems within their Physical Environments, in Proc. of Conf. on Designing Interactive Systems DIS’2000
          (New York, August 17-19, 2000,), ACM Press, New York, 2000,

→ 6. Lopez, J.F., Szekely, P., Web page adaptation for Universal Access, in Proc. of Conf. on Universal Access in HCI UAHCI’ 2001
         (New Orleans, August 5-10, 2001), Lawrence Erlbaum Associates, Mahwah, 2001,

→ 7. Thevenin, D., Coutaz, J., Plasticity of User Interfaces: Framework and Research Agenda, in Proc. of 7th IFIP International Conference on Human-Computer Interaction Interact' 99
         (Edinburgh, August 30 - September 3, 1999), Chapman & Hall, London, pp. 110-117.

→ 8. Thevenin, D., Adaptation en Interaction Homme-Machine: Le cas de la Plasticité, Ph.D. thesis, Université Joseph Fourier,
          Grenoble, 21 December 2001.

→ 9. Graspable interfaces (Fitzmaurice et al., 1995) link

About User Condition

Readings

→ 1. The User Condition 04: A Mobile First World - Silvio Lorusso Doc

  • Most web user are smarphone users
  • How "mobile's first" affect global web design
  • How "mobile's first" affect the way we use computers

Readings (old)(mostly french)(with notes in french)

Books (old)


→ 1.  L'art comme expérience — John Dewey (french) ⚠️(yet to be filled)⚠️
         publisher: Gallimard (1934)
→ 2.  L'œuvre d'art à l'époque de sa reproductibilité technique — Walter Benjamin (french
         publisher: Alia (1939)
→ 3.  La Galaxie Gutemberg — Marshall McLuhan (french)
         publisher: University of Toronto Press (1962)
→ 3.  Pour comprendre les médias — Marshall McLuhan (french)
         publisher: McGraw-Hill Education (1964)
→ 4.  Dispositif — Jean-Louis Baudry (french)
         publisher: Raymond Bellour, Thierry Kuntzel et Christian Metz (1975)
→ 5.  L’Originalité de l’avant-garde et autres mythes modernistes — Rosalind Krauss (french) ⚠️(yet to be filled)⚠️
         publisher: Macula (1993)
→ 6.  L'art de l'observateur: vision et modernité au XIXe siècle — Jonathan Crary (french)
         publisher: Jacqueline Chambon (Editions) (1994)
→ 7.  Inside the White Cube, the Ideology of Gallery Space — Brian O'Doherty (english) ⚠️(yet to be filled)⚠️
         publisher: Les presses du réel (2008)
→ 8.  Préçis de sémiotique générale — Jean-Marie Klinkenbeg (french) ⚠️(yet to be filled)⚠️
         publisher: Point (2000)
→ 9.  Langage des nouveaux médias — Lev Manovitch (french) ⚠️(yet to be filled)⚠️
         publisher: Presses du Réel (2001)
→ 10. L'empire cybernétique — Cécile Lafontaine (french)
         publisher: Seuil (2004)
→ 11.  La relation comme forme — Jean Louis Boissier (french)
         publisher: Genève, MAMCO(2004)
→ 12.  Le Net Art au musée — Anne Laforêt (french)
         publisher: Questions Théoriques(2011)
→ 13.  Narrative comprehension and Film communication — Edward Branigan (english)
         publisher: Routledge (2013)
→ 14. Statement and counter statement / Notes on experimental Jetset — Experimental Jetset (english)
          publisher: Roma (2015)
→ 15. Post Digital Print — Alessandro Ludovico (french) ≈
          publisher: B42 (2016)
→ 16. L'écran comme mobile — Jean Louis Boissier (french)
          publisher: Presses du réel (2016)
→ 17. Design tactile — Josh Clark (french)
          publisher: Eyrolles (2016)
→ 18. Espaces de l'œuvre, espaces de l'exposition — Pamela Bianchi (french)
          publisher: Eyrolles (2016)
→ 19. Imprimer le monde (french)
          publisher: Éditions HYX et les Éditions du Centre Pompidou (2017)
→ 20. Version 0 - Notes sur le livre numérique (french)
          publisher: ECRIDIL (2018)

Articles (old)

→ 1. Frederick Kiesler — artiste- architecte ⚠️(yet to be filled)⚠️
        (communiqué de presse) Centre pompidou; source : centrepompidou.fr (1996)
→ 2. Oublier l'exposition ⚠️(yet to be filled)⚠️
        Artpress special numéro 21 (2000)
→ 3. Composer avec l’imprévisible: Le questionnaire sur les médias variables ⚠️(yet to be filled)⚠️
        Jon Ippolito; source : variablemedia.net/pdf/Permanence (2003)
→ 4. Esthétique du numérique : rupture et continuité
        Fred Forest; source : archives.icom.museum (2010)
→ 5. La narration interactive ⚠️(yet to be filled)⚠️
        Dragana Trgovčević source : ensci.com/file_intranet/mastere_ctc/etude_Dragana_Trgovcevic.pdf (2011)
→ 6. Des dispositifs aux appareils - L'Espacement d'un calcul
        Anthony Masure source :  anthonymasure.com (2013)
→ 7. Le musée n'est pas un dispositif - Jean-Louis Déotte p.9 - 22 (2011)
→ 8. Apogée et périgée du White Cube Loosli, Alban

References

Exhibition space

→  Prouns Spaces — El lissitzky (1920)
→  City in Space — Frederick Kiesler (1920)
→  The air conditionning Show — Terry Atkinson & Michael Baldwin(1966-67)
→  Sans titre — Michael Asher (1973)
→  Serra Corner prop n°7 (for Nathalie) Richard Serra (1983)
→  Speaking Wall (2009 - 2010)

Nothingness with Media

→  4’’33’ — John Cage (1952)
→  Untitled - A Curse — Tom Friedman (1965)
→  The air conditionning Show — Terry Atkinson & Michael Baldwin(1966-67)
→  Sans titre — Michael Asher (1973)

Mediatization of Media

→  4’’33’ — John Cage (1952)
→  TV Garden — Nam June Paik (1974)
→  Presents — Michael Snow (soon to be translated)
→  Lost Formats Preservation Society — Experimental Jetset (2000)
→  Lost Formats Winterthur — Experimental Jetset (2000)
→  L’atlas critique d’Internet Louise Drulhe (2014-2015)

Flags

→  Netflag — Mark Napier (2002)
→  019 - Flag show (2015)

User perspective

→  What you see is what you get — Jonas Lund (2012)

Media Time perception

→  Present Continuous Past — Dan Graham's (1974)

Experimental cinema

→  Presents — Michael Snow (soon to be translated)
→  Displacements — Michael Naimark (1980)
→  BE NOW HERE — Michael Naimark (1995)

CSS composition

→  Sebastianly Serena
→  Scrollbar Composition
→  into time .com - Rafael Rozendaal
→  Ridge 11 - Nicolas Sassoon
→  Rectangulaire - Claude Closky
→  Jacksonpollock.org - Miltos Manetas
→  Moving Paintings - Annie Abrahams

Media deterioration

→  Img214270417
→  William Basinski - The Disintegration Loops

Undefined

→  Untitled Sans

User friendliness and anti-user friendliness

→  Web-Safe - Juha van Ingen

Media Art conservation

→  The Variable Media Initiative 1999
→  EAI Online Resource Guide forExhibiting, Collecting & Preserving Media Art
→  Matters in Media Art
→  The International Network for the Preservation of Contemporary Art (INCCA)
→  Archiving complex digital artworks - Dušan Barok

Emulation

→  Seeing Double: Emulation in Theory and Practice

Technological Timeline

→  Technological Timeline

Media Art Online Archive

→  ACM SIGGRAPH Art Show Archives
→  Archive of Digital Art (ADA)
→  Ars Electronica Archive
→  Digital Art Web Archive (collected by Cornell)
→  Monoskop
→  The Rhizome ArtBase

Music/Sound

→  The end of music

HTML Quines

→  https://hugohil.github.io/dedans/
→  https://secretgeek.github.io/html_wysiwyg/html.html
→  http://all-html.net/?

More references to check (from THE DEVIL IS IN THE DETAILS: MUSEUM - Displays and the Creation of Knowledge)

  • Alexander, Edward P.

1997 The Museum in America. Walnut Creek,
CA: AltaMira Press.

  • Ambrose, Timoth¡ and Crispin Paine

2006 Museum Basics. 2nd edition. London:
Routledge.

  • Ames, Kenneth L., Barbard Franco, and L. Thomas Frye

1997 Ideas and Images: Developing Interpretive History Exhibits. Walnut Creek,
CA: AltaMira Press.

  • Ames, Michael

1992 Cannibal Tours and Glass Boxes: The Anthropology of Museums. 2nd edition.
Vancouver: University of British Columbia Press.

  • Barringer, Tim, and Tom Fþn, eds.

1997 Colonialism and the Object: Empire,
Material Culture and the Museum. London:
Routledge.

  • Belcher, Michael

199I Exhibitions in Museums.
Leicester: Leicester Museum Studies.

  • Bennett, Tony

1995 The Birth of the Museum.
London: Routledge.

  • Black, Graham

2005 The Engaging Museum. London: Routledge.

  • Bouquet, Mar¡ed.

2001 Academic Anthropology and the Museum.
New York Berghahn Books.

  • Caulton, Tim

1998 Hands on Exhibitions: Managing Interactive Museums and Science Centres.
London: Routledge.

  • Coombes, Annie

1994 Reinventing Africa: Museums, Material Culture and Popular Imagination in Late Victorian and Edwardian England.
New Haven: Yale University Press.

  • Dean, David

1997 Museum Exhibition: Theory and Practice.
London: Routledge.

  • Dubin, Steven

1999 Displays of Power: Memory and Amnesia in the American Museum. New York New York University Press.

  • 2006 Transforming Museums: Mounting Queen Victoria in a Democratic South Africa.

New York Palgrave Macmillan.

  • Falk, John H., and Lynn Dierking

2000 Learning from Museums: Visitor Experiences and the Making of Meaning.
Walnut Creek, CA: AltaMira Press.

  • Fienup-Riordan, Anne

2005 Yup'ik Elders at the Ethnologisches Museum
Berlin: Fieldwork Tumed on Its Head. Seattle: University of Washington Press.

  • Hein, George

1998 Learning in the Museum. London: Routledge.

  • Henderson, Am¡ and Adrienne Kaeppler

1997 Exhibiting Dilemmas: Issues of Representation at the Smithsonian.
Washington, DC: Smithsonian Institution Press.
"In twelve essays on such diverse Smithsonian Institution holdings as the Hope Diamond, the Wright Flyer, wooden Zuni carvings, and the Greensboro, North Carolina Woolworth lunch counter that became a symbol of the Civil Rights movement, Exhibiting Dilemmas explores a wide range of social, political, and ethical questions faced by museum curators in their roles as custodians of culture."

  • Hooper-Greenhill, Eileen

1991 Museum and Gallery Education.
Leicester:Leicester University Press.

  • 1992 Museums and the Shaping of Knowledge.

London: Routledge.

  • 1994 Museums and Their Visitors.

London: Routledge.

  • 2001 Cultural Diversity: Developing Museum Audiences in Britain.

Leicester: Leicester University Press.

  • Kaplan, Flora E. S.

1995 Museums and the Making of 'Ourselves.'
Leicester: Leicester University Press.

  • Karp, Ivan, and Steven D. Lavine

1991 Exhibiting Cultures: The Poetics and Politics of Museum Display.
Washington, DC: Smithsonian Institution Press.

  • Kreps, Christina F.

2003 Liberating Culture: Cross-Cultural Perpectives on Museums, Curation, and heritage Preservation.
London: Routledge.

  • Lindauer, Margaret

2006 The Critical Museum Visitor.
New Museum Theory and Practice: An Introduction. f. Marstine, ed. Pp. 203-225.
Malden: Wiley-Blackwell.

  • Lord, Barr¡ and Gail Lord, eds.

2002 The Manual of Museum Exhibitions.
Wal-nut Creek, CA:AltaMira Press.

  • Macdonald, Sharon, ed.

1998 The Politics of Display. London: Routledge.

  • Macdonald, Sharon, and Gordon Fyfe

1996 Theorizing Museums. Oxford: Blackwell.

  • MacGregor, Arthur


2007 Curiosity and Enlightenment: Collecting and Collections from the Sixteenth to the Nineteenth century.
New Haven: Yale University Press.

  • Macleod, Suzanne, ed.

2005 Reshaping Museum Space: Architecture,Design, Exhibitions. London: Routledge.

  • Mcloughlin, Moira

1999 Museums and the Representation of Native Canadians.
New York Garland Publishing.

  • Metzler, Sally

2008 Theatres of Nature: Dioramas at the Field Museum.
Chicago: Field Museum of Natural History.

  • Moore, Kevin

1997 Museums and Popular Culture.
London: Cassell.

  • Moser, Stephanie

1999 The Dilemma of Didactic Displays: Habitat Dioramas, Life-Groups and Reconstruc- tions of the Past. In Making Early Histories in Museums.
N. Merriman, ed. Pp. 65-116.
London: Cassell/Leicester University Press.

  • 2001 Archaeological Representation: TheVisual Conventions for Constructing Knowledge about the Past. In Archaeological Theory Today.


I. Hodder, ed. Pp. 262-283.
Cambridge: Polity Press.

  • 2003 Representing Human Origins: Constructing Knowledge in Museums and Dismantling the Display Canon.

public Archaeology 3(t):I-17.

  • 2006 Wondrous Curiosities: Ancient Egypt at the British Museum. Chicago: Chicago University Press.


  • 2008 Archaeological Representation: The Consumption and Creation of the Past.

In Oxford Handbook of Archaeology. B. Cunliffe and C. Gosden, eds. pp. 1048- 1077.
Oxford: Oxford University press. Pearce,

  • Susan M., ed.

1994 Interpreting Objects and Collections.
Routledge: Leicester Readers in Museum Studies.

  • Pearce, Susan M.

1998 Museums, Objects and Collections.
Leicester: Leicester University Press.

  • Peers, Laura, and Alison K. Brown, eds.


2003 Museums and Source Communities: A Routledge Reader. London: Routledge.

  • Quinn, Stephen C.

2006 Windows on Nature: The Great Habitat
Dioramas of the American Museum of Natural History. New York Harry N. Abrams,

  • Roberts, Lisa C.

1997 From Knowledge to Narrative: Educators and the Changing Museum. Washington,
DC: Smithsonian Institute Press.

  • Sandell, Richard, ed.

2002 Museums, Societ¡ Inequality.
London: Routledge.

  • Scott, Monique

2007 Rethinking Evolution in the Museum: Envisioning African Origins.
London: Routledge.

  • Serrell, Barbara

1996 Exhibit Labels: An Interpretive Approach.
Walnut Creek, CA: AltaMira press.

2006 fudging Exhibitions: A Framework for Excellence. Walnut Creek,
CA: Left Coast Press.

  • Sheets-Pyenson, Susan

1988 Cathedrals of Science: The Development ofColonial Natural History Museums During the Late Nineteenth Century.
Ontario: McGill-Queen's lJniversity Press.

  • Simpson, Moira

1996 Making Representations: Museums in the Post-Colonial Era.
London: Routledge.

  • Spalding,Iulian

2002 The Poetic Museum: Reviving Historic Collections.
London: Prestel.

  • Swain, Hedley

2007 An Introduction to Museum Archaeology.
Cambridge: Cambridge University press.

  • Vergo, Petet ed.

1990 The New Museology. London: Reaktion Books.

  • Walsh, Kevin

1992 The Representation of the Past: Museums and Heritage in the Post-Modern World.
London: Routledge.

  • Witcomb, Andrea

2003 Re-Imagining the Museum: Beyond the Mausoleum.
London: Routledge.

  • Yanni, Carla

2005 Nature's Museum; Victorian Science and the Architecture of Display.
Princeton: Princeton Architectural Press