Wang/YEAR 2
Head
ThesisDraft
Assesment
User:Wang ziheng/xxxDays
GraduationProject
Research
Regarding the research of the graduation project, I think the project should start with sensors, which is also the most important part of this project. In the third semester(SI24), I have solved most of the technical problems of midi.
Sound Approach
In terms of sound, I plan to try several experimental methods.
MIDI control via sensor data
A single MIDI device sends data through a sensor to control the corresponding software or parameters on the computer.
OSC-controlled waveform changes
An OSC device controls the waveform change of the sound through the current change of the sensor.
Through a simple model, I connected the cd40106 to a fan and a potentiometer, the potentiometer controls the frequnecry of the OSC, the rotation of the fan caused the OSC to be constantly turned on and off.
Arduino-based digital sound device
Based on the digital sound of Arduino, make a digital sound device through the sensor.
Physical pickups with electronic effects
Through physical pickup methods such as piezo sensors, obtain the original physical vibration sound and add electronic components to create reverberation, distortion and other effects.
https://folktek.com/collections/electro-acoustic/products/resonant-garden
The Resonant Garden is an electronic-acoustic hybrid instrument designed to create anything from beats and oddities to dense sound scapes.
The garden utilizes three Alter circuits each equipped with a mic pre-amp, and 4 sprout (stringed) panels for generating acoustics. By plucking, rubbing, tapping or even bowing, those micro-acoustic sounds become amplified and affected by the Alter in any number of ways. In essence, the garden is a large microphone designed to pick up vibrations.
Challenges
Portability and outdoor use
Portability and usage scenarios, whether it must be used outdoors, if it must be used outdoors, whether it must be connected to a computer, (and whether the storm will take away the device, which reminds me of an important challenge of rain receivers, whether it will encounter lightning in a rainstorm when worn, because rain receivers look like lightning rods in a sense.)
https://www.designboom.com/art/taiyi-yu-windmill-beach-rotates-sand-wip-in-play-10-16-2023/
Windmill In Play - Taiyi Yu
Taiyi Yu’s inspiration for W.I.P. struck him upon arriving in the Netherlands, gathering an appreciation for the country’s robust winds and iconic windmills. ‘How have winds been industrialized?’ the designer questioned. This thought became the foundation of his research in which he explored the historical and contemporary appropriation of winds through the lenses of modernity and coloniality.
The windmill, both a concrete and metaphorical symbol of industrialization, underscores the dichotomy between humans and nature. It embodies a mechanistic worldview that views the natural world as a resource, shaping human-nature relationships, influencing daily life, and promoting resource exploitation through functional objects and infrastructure. Through a playful manipulation of this machine, the installation prompts a reflection on whether we can move away from the objectification of winds to foster a renewed and more harmonious relationship with the natural world.
Turbulence - Jiří Suchánek
https://www.jiri-suchanek.net/en/project/turbulence/
Turbulence is outdoor interactive sound installation that reveals possible interaction of wind – sound – electronics and human. Human acts as an active observer and adjuster for this real-time sonification of natural chaotic turbulent air dynamics in dedicated area. Each windmill consists of one MCU, magnetic encoder that measure rotation of the vane, amplifier, speaker and wireless RF chip for communication with central unit and control panel. Sound is digitally generated in Mozzi library and is controlled by the speed and direction of wind.
Connectivity and mobility
If it is a very small device that can be held in the hand, how to connect it to the computer? How to keep it connected to the computer during movement, if it is transmitted through WiFi signals, how to solve technical problems.
http://www.crewdson.net/the-concertronica.html
https://www.vice.com/en/article/eccentric-handmade-instruments-marry-folk-tech-and-electronics/
Concertronica - Hugh Jones
The Concertronica is a controller instrument based around the design of a traditional Concertina.
Each of the two ends has 10 momentary push buttons, whilst the ‘bellows’ action – rather than using air – uses strings on a pulley system that have been hacked out of some old gametrak playstation controllers.
The 4 strings each give a distance measurement as well as an X/Y position, so 3 parameter readings for each string. The base of each string has a pair of RGB LEDs which makes for a really spacey light show when the instrument is being played!
The analog signal is carried from one end to the other using a 26 D-sub connection which I have soldered a bespoke cable for, not something I would recommend doing to anyone else! The instrument is powered via a USB connection to an Arduino which is housed in one of the ends. I am using Max/Msp to convert the signal into midi which I am then sending to Ableton Live.
https://www.dezeen.com/2012/07/14/mouth-tools-by-cheng-guo/
A Wearable device - Mouth Factory by Cheng Guo
This piece features a small windmill mounted in front of the mouth, powered by the breath. As the user exhales, blows, or speaks, the airflow drives the windmill’s rotation, turning a basic bodily function into mechanical energy. The work playfully explores the intersection of the human body and machinery, reimagining the mouth as a tool for production and challenging traditional notions of labor and energy generation.
How to catch the wind
When it comes to obtaining wind energy, the first thing that comes to my mind is windmills, which can be used to generate electricity, water conservancy and other purposes. Unlike windmills, I found two types of windmills in the corresponding devices for wind speed testing. One is horizontal and the other is vertical. Fans in both directions need to be fixed with a rod, but this has a very decisive significance for my equipment design, because it determines whether my equipment is flat or tower-shaped.
I investigated how to affect the change of values through the speed of wind. There are two more reasonable ways. One is to connect the motor. The rotation of the fan will drive the current change of the motor; install the hall effect sensor. Through the change of the magnet, every time the fan rotates a circle, it will be recorded in the sensor.
It also includes different materials, such as metal, wood, cloth, paper and so on.
During further research, I realized that the problem that limits creativity is the fan. It could be more creative in the angle of the fan. Each fan blade can be a different shape, different material, and different orientation. On a whole fan, it will only affect/control one parameter. If I separate the fan blades, I can use each fan blade to control a parameter, which creates more randomness.
References
Flight of Whirligigs - Chuck Dunbar
https://cdunbar.blogspot.com/2009/10/flight-of-whirligigs.html?m=1
Arnhemse Bomen Vertellen - Solitary Oak
https://arnhemsebomenvertellen.nl/T/uKFBL/data/7
https://arnhemsebomenvertellen.nl/
Resonant Garden - folktek
https://folktek.com/collections/electro-acoustic/products/resonant-garden
Concertronica - Hugh Jones
http://www.crewdson.net/the-concertronica.html
https://www.vice.com/en/article/eccentric-handmade-instruments-marry-folk-tech-and-electronics/
Mouth Factory - Cheng Guo
https://www.dezeen.com/2012/07/14/mouth-tools-by-cheng-guo/
Windmill In Play - Taiyi Yu
https://www.designboom.com/art/taiyi-yu-windmill-beach-rotates-sand-wip-in-play-10-16-2023/
Ten Rotors With Malachite - George Rickey
Wind-Driven Drawing Machine - Cheng-Guo
https://concretewheels.com/drawingmachines/sculpture_wddm.htm
The Solar Do-Nothing Machine - Charles and Ray Eames
https://www.eamesoffice.com/the-work/solar-do-nothing-machine/
Parade - Eames
https://www.youtube.com/watch?v=UeELTmFG4oc
Godfried-Willem Raes
Singing Bicycles
https://www.logosfoundation.org/scores_gwr/singbikes.html
Web Audio Conference
https://webaudioconf.com/proceedings/
Turbulence- Jiří Suchánek
https://www.jiri-suchanek.net/en/project/turbulence/
Process
1st Round
555 timer chips
At the first round, I decided to build a simple oscillator circuit using a 555 timer IC to explore interactions with wind by connecting it to a fan. By adding external resistors and capacitors, the circuit generates a waveform.
The 555 timer, with its 8 pins serving various functions, can be configured using this schematic to produce a basic oscillating sound.
The 555 timer can also be connected in combination with more timers to create complex effects. By linking the pin 3 (output) of one timer to the pin 5 (control voltage) of another, you can create a simple low-frequency oscillator (LFO) configuration. Based on this principle, I built a circuit using two 555 timers. When I expanded it to four timers, the sound became very unusual, possibly due to wiring issues as the connections became overly complicated with more chips.
Reference:
https://www.learningaboutelectronics.com/Articles/Voltage-controlled-oscillator-VCO-circuit-with-a-555-timer.php
https://www.fluxmonkey.com/electronoize/40106Oscillator.htm
https://samvssound.com/2017/12/13/555-based-drone-synthesizer-lfos/
Motor and Fan
After building the oscillator circuit, I experimented with generating electricity using a motor. I installed a fan on the motor and added a small LED to test if any electricity was being produced. Despite trying various ways to rotate the fan, including manually spinning the motor, the output was consistently too low to light the LED. Switching to a larger DC motor provided some success, but the LED only lit briefly before the output dropped again.
Through further research, I made a test to confirme that the motor could generate electricity but required higher rotational speed. To achieve this, I connected two motors face-to-face: one powered by a battery to drive the other, which was connected to the LED. This setup produced sufficient speed to light the LED.
https://www.youtube.com/watch?v=qf_m7DP34Fg
I learned that each motor has a specific RPM (rotations per minute) that determines its speed and efficiency as a generator. To continue experimenting in this direction, I would need a lower-RPM motor or use a gear system to increase the motor's rotational speed.
To upgrade the motor and generate electricity, I started looking for inspiration in simple mechanisms, such as those used in bicycles. One example is the bicycle dynamo, which is commonly found in traditional bike lighting systems.
Bicycle Dynamo/Bottle Dynamo:
A bicycle dynamo, also called a bottle dynamo, generates electricity by pressing against the bicycle wheel. As the wheel rotates, the dynamo spins and produces electricity through a basic electromagnetic process.
https://www.logosfoundation.org/scores_gwr/singbikes.html
An interesting example of creative use of bicycle-generated electricity is the singing bicycles, designed by Godfried-Willem Raes for his second symphony in 1976. These bicycles not only generate electricity but also produce sounds as part of a musical performance.
To make the electricity from the dynamo more stable and usable, a voltage regulator would be needed. This helps ensure that the power output remains steady and prevents any damage to connected devices.
As a starting point, the system could detect wind speed by measuring the rotation speed of the fan. Two types of sensors could be used for this purpose:
Optical Encoder:
This sensor uses light and a slotted wheel or gear to detect how many times the light beam has been interrupted. For example, an optical encoder can be used to create a slider wheel by counting the interruptions caused by the slots in the wheel.
https://dartmobo.com/dart-optical-encoder/
Hall Sensor:
This sensor detects magnetic fields. By attaching a magnet to the fan, the Hall sensor can count how many times the magnet passes by the sensor, allowing it to calculate the fan's rotation speed. This data can then be used to estimate the wind speed.
AngleTest
RPI-1031 Angle Sensor
Taking a different approach, I decided to explore angles, orientation, and direction for my next experiment. Using an Arduino, I want to create a MIDI controller or a digital sound device. For this, I used an RPI-1031 angle sensor, which measures two axes (X and Y) to indicate front, back, left and right.
I mapped the X and Y values from the sensor to MIDI signals and integrated them into a DAW, where the mapping worked well, providing a visual representation of the changes in X and Y.
However, mobility remains a significant challenge in this setup. The device needs to be held and rotated for operation, making the USB cable awkward to manage during movement. Achieving wireless MIDI transmission also presents difficulties.
RPI-1031 Example:
http://adam-meyer.com/arduino/RPI-1031
Sundial
For future iterations, I plan to switch to another sensor specifically designed for detecting precise angles and combine with the sunlight, like a sundial/compass, since the current sensor primarily measures tilt. I will begin drafting designs for the next version, as this initial test has been mostly successful.
I found inspiration from the traditional sundial/compass box, which not only shows orientation but also uses a pin and the sun's shadow to indicate time. I can incorporate this concept into my project by adding LDR (light-dependent resistors) or other light sensors to detect sunlight and trigger notes, pitch, or frequency. Meanwhile, the orientation sensor can map to other sound parameters.
With more research into the patterns on the sundial, I was inspired by its unique layout. The sundial has numerous tick marks to represent time, like XII, I, II, with each time segment having a different length. This creates a distinctive and intriguing pattern. Using an experimental approach to structure the scales and segment lengths, I want to reconstruct this design by replacing the time marks with musical scales and assign different notes or pitches to each section. This approach could also result in a distinctive experimental composition.
How to make a sundial?
https://www.sundials.info/projects#edial
Experimental Composition and Spatial Connection with Nature
Asking users to participate together builds a shared connection, not just with nature but with each other. This could extend to performances where audiences co-create a soundscape. With the sound design on each sundial box, it could also make a experimental composition by asking users playing together by towards to different directions. By placing users in different locations, we could explore how distance, orientation, and movement affect the soundscape., like wind through trees or echoes across a valley.
GPS sensor
The device could take the form of a box with a space on top to display a map or area. Users could hold this mobile device to interact with sound in a specific environment, using sunlight and orientation changes to alter the audio output dynamically. This setup merges spatial navigation with sound creation, creating an immersive and interactive experience.
Issues
DAC ADC PWM
At the 1st round, I bought an ESP32-S2 to build a digital synth using the Mozzi library. However, every time I tried to compile the code, it threw the error: 'I2S_MODE_DAC_BUILT_IN' was not declared in this scope. I found that the ESP32-S2 lacks a built-in DAC, which is essential for Mozzi's functionality. I tried to modify the Mozzi library to work with the ESP32-S2, but all attempts were unsuccessful.
DAC (Digital-to-Analog Converter), converts digital values (numbers) into a continuous analog voltage.
ADC (Analog-to-Digital Converter), converts analog voltages (e.g., sensor outputs) into digital values.
PWM (Pulse Width Modulation), outputs a digital signal that simulates an analog value by varying the duty cycle of a square wave.
| Feature | ESP32 | ESP32-S2 | Arduino Leonardo/Uno |
|---|---|---|---|
| WIFI | Yes | Yes | No |
| Bluetooth | Yes | No | No |
| DAC | 2 pins | None | None |
| ADC | Yes | Yes | Yes |
| PWM | Yes | Yes | Yes |