Latest revision as of 18:32, 27 March 2018

Raw audio to data

In a session with Andre we worked on ways to turn raw images into audio, and turn raw audio into images. Building upon that, I got interested in finding what characteristics of the sound influence certain aspects of the image.

During the session we used music and recorded sounds. For my further tests I however decided to use generated tones in audacity, to get full control over parameters like the type of soundwave and the frequency.

How to

Audacity

Generate a tone
Export uncompressed -> format raw -> unsigned 8 bit

Imagemagick

convert -depth 8 -size 2102x2799 rgb:image.dat image2.png

Based on the recipe in Sonification.

The software draws the image based on the raw data starting from the upper left pixel, then it will go to the right. After drawing the most right pixel on the first line, it will draw the next line of pixels.

Frequency

The frequency influences the amount of times that the pattern repeats. The higher the frequency, the smaller and more detailed the pattern.

1Hz Sawtooth
5 Hz Sawtooth

types of tones

The shape of the soundwaves influences the appearance of the pattern.

5hz block tone
5 Hz Sine

Multiple tones

If you have multiple tones of different frequencies, you will hear interference. This interference is also visible in the pattern when you convert the audio to an image. Based on the frequencies the appearance of the pattern will change.

5hz and 1hz Sawtooth

Length

The lengt of the audio file influences the amount of raw data that is generated. To fill a canvas of 1000x1000 pixels, you need about 50 seconds of audio. If you have to little audio, part of the image will turn black. If the sound file is longer, the current pipeline will result in multiple images.

Volume

The volume of the sound (or the amplitude of the soundwaves) controls the contrast of the pattern. By fading out a tone, it's pattern will have a gradient going from opaque to transparent. Fading two different tones into each other, will also blend their respective patterns in the image.

Fading out a tone

Bitrate

The bitrate controls the amount of raw data generated per second of sound. With a higher bitrate, a sound can fill a bigger image than a sound with the same length but with a lower bitrate.

In the current pipeline I have to generate the sounds in Audacity. The downside of this approach is that Audacity doesn't offer a command line interface, which makes generating a big number of images time consuming. A next step might be to use sox to generate the sound, so I can generate a big number of different sounds and images using a bash script.

Another thing I need to find out is how to add color to the images. It is possible using noise as an input, but I haven't found out yet how to do it with tones generated in audacity. Adding a header with a color encoding indicator in imagemagick didn't work.

I used one of the outcomes of this for the design of the cover of my reader for special issue 5.

@@ Line 1: / Line 1: @@
 [[Sonification#converting_color_.28bit_depth.29|Sonification]]
-Audacity
+=Raw audio to data=
-Export uncompressed -> format raw -> unsigned 8 bit
+In a session with Andre we worked on ways to turn raw images into audio, and turn raw audio into images. Building upon that, I got interested in finding what characteristics of the sound influence certain aspects of the image.
+During the session we used music and recorded sounds. For my further tests I however decided to use generated tones in audacity, to get full control over parameters like the type of soundwave and the frequency.
-Length of the audio defines the amount of data to fill an image. The higher the resolution, the more data you need. Too little data: image becomes black after a certain point. Too much data, a new image file is written.
+==How to==
+'''Audacity'''
+* Generate a tone
+* Export uncompressed -> format raw -> unsigned 8 bit
-The shape of the wave influences the generated pattern.
+'''Imagemagick'''
+<pre>convert -depth 8 -size 2102x2799 rgb:image.dat image2.png</pre>
+Based on the recipe in [[Sonification#Raw_images|Sonification]].
+The software draws the image based on the raw data starting from the upper left pixel, then it will go to the right. After drawing the most right pixel on the first line, it will draw the next line of pixels.
+====Frequency====
+The frequency influences the amount of times that the pattern repeats. The higher the frequency, the smaller and more detailed the pattern.
+<gallery>
+File:1hz sawtooth 1min5s.jpg|1Hz Sawtooth
+File:5hz sawtooth 1min5s.jpg|5 Hz Sawtooth
+</gallery>
+====types of tones====
+The shape of the soundwaves influences the appearance of the pattern.
+<gallery>
+File:5hz block 2min-0.jpg| 5hz block tone
+File:5hz sine-0.png|5 Hz Sine
+</gallery>
+====Multiple tones====
+If you have multiple tones of different frequencies, you will hear interference. This interference is also visible in the pattern when you convert the audio to an image. Based on the frequencies the appearance of the pattern will change.
+<gallery>
+File:Image232-0.png| 5hz and 1hz Sawtooth
+</gallery>
+====Length====
+The lengt of the audio file influences the amount of raw data that is generated. To fill a canvas of 1000x1000 pixels, you need about 50 seconds of audio. If you have to little audio, part of the image will turn black. If the sound file is longer, the current pipeline will result in multiple images.
+====Volume====
+The volume of the sound (or the amplitude of the soundwaves) controls the contrast of the pattern. By fading out a tone, it's pattern will have a gradient going from opaque to transparent. Fading two different tones into each other, will also blend their respective patterns in the image.
+<gallery>
+File:Image230-0.png|Fading out a tone
+</gallery>
+====Bitrate====
+The bitrate controls the amount of raw data generated per second of sound. With a higher bitrate, a sound can fill a bigger image than a sound with the same length but with a lower bitrate.
+==Next==
+[[Image:Image237.png|thumbnail|right|250px|Cover for the reader. Color added with GIMP]]
+Using generated tones makes it easier to identify which parameters of the sound influences specific paramets of the resulting image. This can help in getting an idea how 'real world sounds'  would look as an image.
+In the current pipeline I have to generate the sounds in Audacity. The downside of this approach is that Audacity doesn't offer a command line interface, which makes generating a big number of images time consuming. A next step might be to use sox to generate the sound, so I can generate a big number of different sounds and images using a bash script.
+Another thing I need to find out is how to add color to the images. It is possible using noise as an input, but I haven't found out yet how to do it with tones generated in audacity. Adding a header with a color encoding indicator in imagemagick didn't work.
+I used one of the outcomes of this for the design of the [[User:Joca/Reader Joca|cover of my reader]] for special issue 5.

User:Joca/raw-audio-bitmap: Difference between revisions