Projects SI 28

Writing Machine Ensemble

Participants: Ema, Daan, Robin, Thijs

Description

Writing Machine Ensemble

Explorations in Potential Musicality

With the Writing Machine Ensemble, rythm and repetition will be experienced in a radically new form. Explorations in Potential Musicality is a musical journey through the sonic qualities of printers—with a strong emphasis on improvisation, yet promising dramatic intensity.

Writing Machine Ensemble

The Writing Machine Ensemble, co-founded by it's current members, is an international group of experimental musicians from around the XPUB Studio. Independent and idealistic, they focus on blending the role of 'instrument' and 'musician' in an assemblage of artistic creation, striving to find the essential musical quality in tools of writing. Since its debut in September 2025, the ensemble has performed once on WORM Radio, for the 5th edition of XPUB's Special Issue Radio show, where the group's unique live scoring method was born.

Members

Emanuele Arrigoni, Daan Princen, Robin de Haan,Thijs van Loenhout, Monica Olympia, Erika Model, N.E.C. (Dottie) Parson, Taxan X.Y. Plotter

Medium/Media

Performance (not naming the printers as medium is part of the concept)

(or, if possible, change formatting to that of a concert announcement, so no mention of medium)

Conversations with the curators

Keywords: Technical Assemblage, Live Scoring, Scores, Notation, Sound, Performance, Writing Machines, tools shape practice practice shape tool

-> Input -> Output ->

INPUT: Finger movements that originate symbols and letters onto electronic devices.

OUTPUT[S]: the same input text BUT on long continuous sheets of paper, thanks to the use of ink that is both being sprayed onto the paper thanks to the marvellous technical engineering of a head that moves horizontally, and imprinted, thanks to the mechanical movements of engraved metal rods hitting the paper. --> This sums up into Printing, and those types of printing techniques produces noises. --> Together, noises and symbols on paper, leads people to wander into stuff called semiotics, thoughts and imagination through the use of interpretation.

Gesture Machine

Participants: Valentin

Currently working on:

Experimentin with opensource motion capture datasets in touchdesigner.

Description

N. Katherine Hayles' Cognitive Assemblages: People and technology made by people performing cognitive functions together. Only certain technologies considered as cognitive machines: The ones are capable of interpreting information of its' own environment and act on the changes of the environment therefore changing its own context which creates meaning.

Volumetric Regimes are the industries that keep the technology 3D representation in a close circulation, iterating the same technology over and over again.

As a designer and amateur animator I take the position to make a cognitive animation table and attempting to explore free softwares provided by the above mentioned industry through the lenses of Hayles' Cognitive Assemblages.

more info

Medium/Media

hologram, stop-motion armature, human body

Keywords: gesture, non-verbal communication, body language, animation, stop-motion

-> Input -> Output ->

Input: hand gestures
Output: stop-motion animation

Experiments

00 AMASS/MANO modell

Failed to align python dependencies on mac. A windows trial could be beneficial.

01 Mediapipe - One Hand

Libraries used:

| OpnecCV:

"OpenCV is the largest Computer Vision library in the world. It's used worldwide: in the world's biggest companies, underwater, and in space.

OpenCV was started in 1998 and played the key role in the development of Computer Vision and Artificial Intelligence. It is released under Apache 2.0 license and is open-source and free for everyone."

| Matplotlib: I used Matplotlib to plot the mediapipe data on a '3d visualiser'.

| Mediapipe:

I used media pipe to detect handshape and get 3D data of the skeleton.

| NumPy:

I used numpy to store 3d data in an array which is then used by open3D.

"NumPy is an open source project that enables numerical computing with Python. It was created in 2005 building on the early work of the Numeric and Numarray libraries. NumPy will always be 100% open source software and free for all to use. It is released under the liberal terms of the modified BSD license.

NumPy is developed in the open on GitHub, through the consensus of the NumPy and wider scientific Python community. For more information on our governance approach, please see our Governance Document."

| Pandas:

I used Pandas to export the recorded motion data into a timpestamped CSV.

""

02 Mediapipe - Two Handed - No 3D plot

After 3 days of trying to set up a project for hand tracking using MANO and AMASS but it failed in aliging dependecies. SO i decided to use mediapipe and my webcam for handtracking and exporting it into a timestamped .csv so later i can convert the data into C3D or BVH.

Also tried to convert the CSV into BVH but it failed many times.

Steve:

• look how people react on seeing themselves represented as 3D point cloud
• feedback is essential for a self-sustaining system

03 Blender - Audio Reacting Hand Armature

Thinking about the event in UBIK, started experimenting with animating | a free rigged hand armature with audio input.

Each finger is reacting on different frequecies.

04 Blender - Webcam Reacting Hand Armature

Then i wanted to try to use my webcam and the medipipe hand-tracking to animate the armature but it was very hard to figure out the alignement.

05 Mediapipe - Open3D - Thinking on the event

Anyway, using Blender opened up a ton of new possibilites and i felt that i am drifiting far from my original goal which is building a armature, and i reminded myself that this is only the hand gesture mocap phase so i decided to use the event to collect hand gestures from the performances and the audience and i started thinking on how to make it fun.

So i went back to the first trials but i used open3D to visualize the 3D hand data.

On the event i would like to use wireless webcams attached to caps so people can try them on and the hands will be seen from the view of the person who is wearing it. This could be developed further and aso used in many life situations which is kind of a big thing in the world of mocap.

On the event i would like to still visualise the data in a hologram format but also show the traces of the movement.

And ofcourse have conversations about that i found something beautiful experimenting on the edge of biology and technology. Need a good title....

  Feedback:

  Alexia: 
  * First person view for mocap handgestures. In gaming first person view has a particular function beside beign the easiest depiciton of the human body. This particularfunction is to interact with other characters and often aiming weapon. The esaiest because only the hands need to be animated instead of a whole body. 
  * It is the performance of the hand that is important rather then what gesture it is altough some instructions could help to narrow down possibilites
  * Mirroring effect, how is it different to look in the mirror and see my hands reflected and represented as a skeleton on a digital devices's screen? 

  Lea: 
  *Some instructions would be nice, and the delocalised idea of capturing is definetly something to explore. 
  * Suggests to bring in an pre recorded audio where a voice is talking about cognitve assemblages and performative maachines to bring in context.
  * We are usually not aware of our hands moving and perfoming gestures and this particular demo does makes her more aware and thinks of situations where this could be an important instrument of learning (eg. learning to draw, surgery )
  * If i think of the project as a cognitive machine it helps me to think about what cognitive functions could be performed in animation when my hand movement is captured and projected on an armature hand. :) I can imagine that it is like a surgery ... tbc.
  * She suggests to capture snapshots so it can be documented and shared later. GOOOD ONE!!! 

  Manetta:
  * What is the original contexts of the model and the dependencies im using in my project, is important to see a bit of the history of these models in terms of research!! CHECK Mediapipe and open3D
  * A manual or steps to show the data collected can be used in animation processes 
  * Be clear about personal fascination in stop-motion -> personal contex VERY IMPORTANT to clear intentions and to communicate the idea of the project
  * Motion Capture > Steps > Stop-motion Animation
  * For now it is only a demo of mediapipe's model of handtracking visualized in open3D, so communicating the fascination in making animation + adding context to it (cognitive assemblages) will make it a layaered project.

A story in a story in a story

Participants: Youjung Barbs Kiki

Description

"A story in a story in a story" invites you into an interactive narrative that lets you explore, gather and plot. Wander around in the virtual environment, pick up what you resonate with and that is what you take with you at the end of the day and becomes a tiny part of you.

Medium/Media

terminal, python, pen plotter, text adventure, fiction, paper & ink

Marginalia

Participants: Luca

Currently working on:

Scripting pipeline to convert texts to graphs; Editorial research on specific textual archives

Description

Writing does something when published. Since the invention of text, it has functioned to flatten out a plurality of worlds from history, performing domination, differentiation and justifying violence. Marginalia takes selected texts in different historical contexts, and transforms them in experimental annotations engraved in canonical books. The resulting visual language communicates semiotics of displacement and a cartography manufactured by hegemonic world views. As a performance, layers of text, speech and distortion presents a writing machine as sonic warfare.

Medium/Media:

Laser engraving on books, laser projection, live sound

Keywords: Territory, Cartography, Text, Colonialism

-> Input -> Output ->

text->cartography

Water Vessel: Dwelling in Temporality

Participants: Soda

Description

This work begins with my personal experience of living in Rotterdam — a city that is structured and rhythmically defined by water. Surrounded by ponds, canals, and rainwater systems, I constantly sense a condition of floating dwelling: a way of living that feels both anchored and adrift, stable yet impermanent.

In response to this environment, I started collecting abandoned wooden materials around the city. These wooden materials came from the furniture and wooden waste thrown out by the people living here, construction sites, and branches sawn off by gardeners. This act of gathering is not only a process of reclaiming the city’s discarded fragments, but also a gesture of re-dwelling — an attempt to reconnect with the material traces of the place. Each piece of wood, marked by time, humidity, and human touch, carries the city’s sedimented memory. Its textures and scars speak of erosion, adaptation, and survival — revealing what I call the temporality of dwelling: the sense of being-in-time within unstable conditions.

Through joining, sanding, and bending these fragments, I assembled them into the form of a water vessel — a wooden pool suspended in mid-air. The vessel, as an object, embodies both containment and permeability. Water gives the vessel its meaning, yet constantly transforms its surface and temperature. Their coexistence suggests a reciprocal relationship between dwelling and being dwelled: to inhabit and to be shaped by what we inhabit.

For me, this vessel reflects the experience of living as an immigrant — of existing between lands, languages, and temporalities. In Rotterdam, a city of water and migration, this wooden container becomes a metaphor for temporary belonging, for the fragile balance between stability and drift. It is a space where time gathers, where the act of dwelling is never complete but always in the process of becoming.

Medium/Media

Sculpture / Installation

Keywords: Fluidity, Temporality of Dwelling, Nonlinear Memory, Liminality, Fragmentation

Love Machine

Participants: Phillip

Currently working on:

Picture storybook , comes with slides different images using AI and using my own word to tell the story of my characters such as Clara.

Description

I use AI to write my novel, and then I use AI again to create images for a picture storybook. The prompts are heavily supervised by me, so it is based on my own plot, story, and characters. It includes original storyline as well, and some scenes in the picture storybook are not in my original novel.

The Metathinking Metamachine

Participants: Tommi

Currently working on:

• From Zettelkasten to obsessing over personal knowledge management
• Software escatology

Description

Since the very moment I started XPUB, I have been (over)thinking, possibly obsessing, about the making of Pöuli, and of the (currently very incomplete) website for the Special Issue 28. Wholly absorbed by my tooling hyperfocus, I lost track of time. All collective conversations passed through me. Before I could realize it, people were forming groups and started working on their project for the main event at Ubik.

So Tommi, what are you going to do?
Ehm… … … … …
 …I will build The Metathinking Metamachine!

I genuinely had no idea about what that meant. In my head, I could not picture anything beyond the face of Niklas Luhmann, the inventor of the prodigious Zettelkasten system.

The intention was to address the theme of the 28^th special issue in relation to my own obsessions:

• A Metamachine: a machine about machines, a software made to solve problems, and interact with, other software.
• Metathinking: the process of thinking about thinking, better explained in the linked article.

At that point, conceptualizing, preparing, and enacting anything that would actually do something useful, or produce a meaningful outcome was unfeasible and idealistic. So what then?

Thanks to Manetta’s and Marloes’ guidance and suggestions, I decided to base everything on irony and satire.

I was kindly allowed to light some tiny candles during the performance, to enhance the ironic spiritual factor.

The one thing I was sure of was that I intended on focusing on a pseudo-spiritual and grotesque setting. I based all the performance on a spiritual seance to connect with the Gods of Tooling. Furthermore, while sharing my anxiety and my doubts about the performance with Martina and Claudio, I got to the conclusion that the whole performance should have been fun: I got very excited of performing some sort of nerd stand-up comedy.

Albeit the path leading to the performance has been extremely stressful because I was terribly confused and overwhelmed by the novelty of my first Special Issue and its many inputs, I am incredibly proud of it, for a series of reasons.

• It was my first performance. I never realised that I would actually define myself as an artist and this performance marked a watershed moment in my self-identification as one. Actually, the way I comfortably call myself is pseudo-artist.
• Even though I learned it the hard way, I deeply understood the collaborative nature of XPUB. I have been doing many projects with others, but I did not commit to any collective project to develop with others. I found out that I must actively pursue an intertwining of practices with my fellow students.

Medium/Media

Installation made by a series of drawers for archives + monologue on tools hyperfocus.

Keywords: Metathinking, Knowledge management, tools shape practices shape tools…