Sonification
Sound and Listening
What is sound
We experience sound thought displacement of air molecules from a source to a the receiver.
When a source emits a sound, it makes the air molecules compress and decompress, traveling away from the source.
The rate of this compression-decompression variation is what is called the frequency of a sound. And the amount of air molecules displace give us the amplitude or volume of a sound.
Sound wave properties
- frequency = pitch
- measured in Hertz (Hz) or cycle per second. 1Hz = 1cycle/sec
- amplitude = volume
- measured in decibel (db)
Sound waves on liquid
audible sound
Human audible sound, or the hearing spectrum, is located between 20Hz and 20 000Hz of of the acoustic spectrum
How do we listen?
How does the movement of air molecules gets perceived as sound?
Spacial Perception Sound
How do we detect the position where sound is coming from?
- arrival time difference
- amplitude difference
- filtering, by the outer ear
Janet Cardiff - binaural mics
Hearing as composing
Recording and listening exercise.
- record 1 to 2 minutes of sound
- write down all you here during those minutes, either with or without headphones
4'33
4'33 (1952) is a probably the most famous John Cage composition, where the performance stands in front of the audience without playing a single note, for 4 minutes and 33 seconds.
http://hyperallergic.com/85779/the-original-john-cage-433-in-proportional-notation-19521953/
- hearing as composing
- all sound can become music
Expanded listening
contact mics
- listen to how solid objects respond to vibration
- due to its physical properties each object will respond - resonate - differently to vibration
- The resonant frequencies are the frequencies that are emphasized by the object
David Tudor - Rainforest (1968)
- http://composers-inside-electronics.net/rainforest/rainforest/RAINFOREST.html
- https://www.youtube.com/embed/iT9HP48sHTg
- http://www.getty.edu/research/tools/guides_bibliographies/david_tudor/av/rainforest.html
- Holzer, Derek, and Mads BEch Paulszewski. n.d. “Rainforest 2012.” http://macumbista.net/files/rainforest_overview.pdf
Electromagnetic pickups
Instead of capturing acoustic waves, like acoustic microphones, electromagnetic pickups capture electromagnetic fields.
Christina Kubisch Electrical Walks explore cities electromagnetic fields.
- http://www.christinakubisch.de/en/works/electrical_walks
- http://cabinetmagazine.org/issues/21/cox.php
- https://www.youtube.com/watch?v=5tCphr8pbFk
Digital audio
Analog audio is stored in a continuous analogue to its source.
http://www.synthgear.com/2014/audio-gear/record-grooves-electron-microscope
More grooves: http://sessionville.com/articles/vinyl-through-the-microscope-looking-glass
Digital audio on the other hand is stored discretely (in segments or samples).
sampling rate and bit rate
Two parameters are essential to digital audio:sampling rate and bit rate
sampling rate: how frequently the analogue signal is sampled File:Discrete-signal.png
- The sampling rate of CDs is 44.1KHz: discrete samples of a signal are stored 44100 times each second.
- Digital video cameras tend to store audio at 48KHz.
bit rate: accuracy of sampling - how many values are used to represent the amplitude of a signal
- The bit rate of a CD is 16-bit: 2¹⁶ (65536) values are used to encode the audio digitally
- 8 bit 2⁸ (256) values
audio formats
- uncompressed: aiff, wav, flac
- compressed: mp3, ogg
Editing
With Audacity http://www.audacityteam.org/
“Audacity.” 2017. FLOSS Manuals. Accessed January 11. http://write.flossmanuals.net/audacity. http://archive.flossmanuals.net/audacity/_booki/audacity/audacity.pdf http://archive.flossmanuals.net/audacity/_booki/audacity/audacity.epub
Bibliography
- Collins, Nicolas. 2006. Handmade Electronic Music. London: Routledge.
- “Sound:An Interactive eBook on the Physics of Sound.” 2017. Sound: An Interactive eBook on the Physics of Sound. Accessed January 11. https://soundphysics.ius.edu/.
BitMaps
bitmap images
You probably have heard before that bitmap images are grids of pixels, where
- resolution: the number of horizontal and vertical pixels. E.g. 1920x1080 is 1920 pixels wide and 1080 high
- color depth: the number of bits used to indicate the color of a single pixel
for a good introduction to bit maps see: See Bourke, Paul. n.d. ‘A Beginners Guide to Bitmaps’. Accessed 2 January 2018. http://paulbourke.net/dataformats/bitmaps/.
Color depth
- 1-bit color (21 = 2 colors): monochrome, often black and white, compact Macintoshes, Atari ST.
- 2-bit color (22 = 4 colors): CGA, gray-scale early NeXTstation, color Macintoshes, Atari ST.
- 3-bit color (23 = 8 colors): many early home computers with TV displays, including the ZX Spectrum and BBC Micro
- 4-bit color (24 = 16 colors): as used by EGA and by the least common denominator VGA standard at higher resolution, color Macintoshes, Atari ST, Commodore 64, Amstrad CPC.
- 5-bit color (25 = 32 colors): Original Amiga chipset
- 6-bit color (26 = 64 colors): Original Amiga chipset
- 8-bit color (28 = 256 colors): most early color Unix workstations, VGA at low resolution, Super VGA, color Macintoshes, Atari TT, Amiga AGA chipset, Falcon030, Acorn Archimedes.
- 12-bit color (212 = 4096 colors): some Silicon Graphics systems, Color NeXTstation systems, and Amiga systems in HAM mode.
- 8-bit color: A very limited but true direct color system, there are 3 bits (8 possible levels) for each of the R and G components, and the two remaining bits in the byte pixel to the B component (four levels), enabling 256 (8 × 8 × 4) different colors.
- Grey scale (8-bit)
- True color (24-bit): Usually, true color is defined to mean 256 shades of red, green, and blue, for a total of 224, or alternately 2563, or 16,777,216 color variations. The human eye can discriminate up to ten million colors
converting resolution
with Imagemagick
to 100px wide
convert myimage.jpg -resize 100x output.jpg
to 100px hight
convert myimage.jpg -resize x100 output.jpg
to 100x100px (image will loose its proportion; hence the "!" so imagemagick is sure that you wanna do that:)
convert myimage.jpg -resize 100x100! output.jpg
converting color (bit depth)
convert an image into monochrome (1 bit depth) image: 2color values (b/w)
convert myimage.jpg -colorspace gray -depth 1 out.png
(2 bit depth) image: 4 color values
convert myimage.jpg -colorspace gray -depth 2 out.png
(3 bit depth) image: 8 color values
convert myimage.jpg -colorspace gray -depth 3 out.png
(4 bit depth) image: 16 color values
convert myimage.jpg -colorspace gray -depth 4 out.png
(8 bit depth) image: 256 color values
convert myimage.jpg -colorspace gray -depth 4 out.png
How can we be certain about this? How do can we see the grid that forms a bitmap image?
- magnify any digital screen
- a Plain-text bitmap: where you can read each pixel: X PixMap (XPM)
X PixMap (XPM) is a plain-text bitmap image format, where each color is assigned a character, which is mapped onto the text space.
convert myimage.jpg out.xpm
/* XPM */
static char *output[] = {
/* columns rows colors chars-per-pixel */
"50 73 2 1 ",
" c black",
". c white",
/* pixels */
" ",
" ",
" ",
" ",
" ................ ",
" ...... . .. . . ....... ",
" .. . . ... ",
" . . .. . . .. ",
" .. . . . . . . . ",
" . . . . . . . .. ",
" . . . . . . . ",
" ... .. . .. ",
" ... .. . ... ",
" .. . ",
" . .. ",
" .. . ",
" . .. ",
" ... . . ",
" . . . ",
" .. . ",
" .. . . . . ",
" . . . ",
" .. . ..... . ",
" . .. .. . ",
" . . .. . . ",
" . .. . .. ",
" .. . . . ..... ",
" . ... . . . ",
" . . .... .. . .. ",
" . .. .... . . ",
" . . . . .... ",
" . . . .. ... ",
" . . . .. ",
" .. . . . . ",
" . . .. . . ",
" . .. .. .. .. ",
" .. ... ... . ",
" .. .. ",
" . . . . . ",
" . . .. . ",
" . . .... . . ",
" . .. ... . .. .. ",
" .. . .. ..... . .. . ... . ",
" . ... .. .... .... . ",
" . ..... ...... ...... . ",
" .. . ..... . ......... ... . ",
" .. ......... .. . ...... . ",
" . .. .................... ",
" . .. . ............... . ",
" . .. .. ............. . ",
" . .. . . ..... . ",
" ... . ... . .. ...... . ",
" ... . ............. . ",
" .. . .. . ",
" ... .. ",
" ... . . .. ",
" .. . . ",
" .. . ",
" .... .. ",
" ..... .... ",
" ..... ",
" ",
" . ",
" .. . ... ",
" ... .. .. ... .. .. ...... ... ",
" ............................ .. ",
" ...... ................. ... . ",
" ...... ... ... ..... .... .. . ",
" .. ",
" . ",
" ",
" ",
" "
};