User:Joca/raw-audio-bitmap

Sonification

Raw audio to data

In a session with Andre we worked on ways to turn raw images into audio, and turn raw audio into images. Building upon that, I got interested in finding what characteristics of the sound influence certain aspects of the image.

During the session we used music and recorded sounds. For my further tests I however decided to use generated tones in audacity, to get full control over parameters like the type of soundwave and the frequency.

How to

Audacity

• Generate a tone
• Export uncompressed -> format raw -> unsigned 8 bit

Imagemagick

convert -depth 8 -size 2102x2799 rgb:image.dat image2.png

Based on the recipe in Sonification.

The software draws the image based on the raw data starting from the upper left pixel, then it will go to the right. After drawing the most right pixel on the first line, it will draw the next line of pixels.

Frequency

The frequency influences the amount of times that the pattern repeats. The higher the frequency, the smaller and more detailed the pattern.

• 

  1Hz Sawtooth

• 

  5 Hz Sawtooth

types of tones

The shape of the soundwaves influences the appearance of the pattern.

• 

  5hz block tone

• 

  5 Hz Sine

Multiple tones

If you have multiple tones of different frequencies, you will hear interference. This interference is also visible in the pattern when you convert the audio to an image. Based on the frequencies the appearance of the pattern will change.

Length

The lengt of the audio file influences the amount of raw data that is generated. To fill a canvas of 1000x1000 pixels, you need about 50 seconds of audio. If you have to little audio, part of the image will turn black. If the sound file is longer, the current pipeline will result in multiple images.

Volume

The volume of the sound (or the amplitude of the soundwaves) controls the contrast of the pattern. By fading out a tone, it's pattern will have a gradient going from opaque to transparent. Fading two different tones into each other, will also blend their respective patterns in the image.

Bitrate

The bitrate controls the amount of raw data generated per second of sound. With a higher bitrate, a sound can fill a bigger image than a sound with the same length but with a lower bitrate.

Next

Using generated tones makes it easier to identify which parameters of the sound influences specific paramets of the resulting image. This can help in getting an idea how 'real world sounds' would look as an image.

In the current pipeline I have to generate the sounds in Audacity. The downside of this approach is that Audacity doesn't offer a command line interface, which makes generating a big number of images time consuming. A next step might be to use sox to generate the sound, so I can generate a big number of different sounds and images using a bash script.

Another thing I need to find out is how to add color to the images. It is possible using noise as an input, but I haven't found out yet how to do it with tones generated in audacity. Adding a header with a color encoding indicator in imagemagick didn't work.

I used one of the outcomes of this for the design of the cover of my reader for special issue 5.