Revision as of 17:58, 27 March 2018

Raw audio to data

In a session with Andre we worked on ways to turn raw images into audio, and turn raw audio into images. Building upon that, I got interested in finding what characteristics of the sound influence certain aspects of the image.

During the session we used music and recorded sounds. For my further tests I however decided to use generated tones in audacity, to get full control over parameters like the type of soundwave and the frequency.

How to

Audacity

Generate a tone
Export uncompressed -> format raw -> unsigned 8 bit

Imagemagick recipe used -> link to wiki

The software draws the image based on the raw data starting from the upper left pixel, then it will go to the right. After drawing the most right pixel on the first line, it will draw the next line of pixels.

Frequency

The frequency influences the amount of times that the pattern repeats. The higher the frequency, the smaller and more detailed the pattern.

types of tones

The shape of the soundwaves influences the appearance of the pattern.

Multiple tones

If you have multiple tones of different frequencies, you will hear interference. This interference is also visible in the pattern when you convert the audio to an image. Based on the frequencies the appearance of the pattern will change.

Length

The lengt of the audio file influences the amount of raw data that is generated. To fill a canvas of 1000x1000 pixels, you need about 50 seconds of audio. If you have to little audio, part of the image will turn black. If the sound file is longer, the current pipeline will result in multiple images.

Volume

The volume of the sound (or the amplitude of the soundwaves) controls the contrast of the pattern. By fading out a tone, it's pattern will have a gradient going from opaque to transparent. Fading two different tones into each other, will also blend their respective patterns in the image.

Bitrate

The bitrate controls the amount of raw data generated per second of sound. With a higher bitrate, a sound can fill a bigger image than a sound with the same length but with a lower bitrate.

In the current pipeline I have to generate the sounds in Audacity. The downside of this approach is that Audacity doesn't offer a command line interface, which makes generating a big number of images time consuming. A next step might be to use sox to generate the sound, so I can generate a big number of different sounds and images using a bash script.

Another thing I need to find out is how to add color to the images. It is possible using noise as an input, but I haven't found out yet how to do it with tones generated in audacity. Adding a header with a color encoding indicator in imagemagick didn't work.

I used one of the outcomes of this for the design of the cover of my reader for special issue 5.

@@ Line 1: / Line 1: @@
 [[Sonification#converting_color_.28bit_depth.29|Sonification]]
-Audacity
+=Raw audio to data=
-Export uncompressed -> format raw -> unsigned 8 bit
+In a session with Andre we worked on ways to turn raw images into audio, and turn raw audio into images. Building upon that, I got interested in finding what characteristics of the sound influence certain aspects of the image.
+During the session we used music and recorded sounds. For my further tests I however decided to use generated tones in audacity, to get full control over parameters like the type of soundwave and the frequency.
-The converter reads the file from top to bottom, and generates the bitmap from top to bottom.
+==How to==
+'''Audacity'''
+* Generate a tone
+* Export uncompressed -> format raw -> unsigned 8 bit
-Length of the audio defines the amount of data to fill an image. The higher the resolution, the more data you need. Too little data: image becomes black after a certain point. Too much data, a new image file is written. Depth controls the amount of raw audio data needed per pixel.
+'''Imagemagick'''
+recipe used -> link to wiki
-The shape of the wave influences the generated pattern. The frequency controls the amount of times that the pattern repeats.
+The software draws the image based on the raw data starting from the upper left pixel, then it will go to the right. After drawing the most right pixel on the first line, it will draw the next line of pixels.
+====Frequency====
+The frequency influences the amount of times that the pattern repeats. The higher the frequency, the smaller and more detailed the pattern.
+====types of tones====
+The shape of the soundwaves influences the appearance of the pattern.
+====Multiple tones====
+If you have multiple tones of different frequencies, you will hear interference. This interference is also visible in the pattern when you convert the audio to an image. Based on the frequencies the appearance of the pattern will change.
+====Length====
+The lengt of the audio file influences the amount of raw data that is generated. To fill a canvas of 1000x1000 pixels, you need about 50 seconds of audio. If you have to little audio, part of the image will turn black. If the sound file is longer, the current pipeline will result in multiple images.
+====Volume====
+The volume of the sound (or the amplitude of the soundwaves) controls the contrast of the pattern. By fading out a tone, it's pattern will have a gradient going from opaque to transparent. Fading two different tones into each other, will also blend their respective patterns in the image.
+====Bitrate====
+The bitrate controls the amount of raw data generated per second of sound. With a higher bitrate, a sound can fill a bigger image than a sound with the same length but with a lower bitrate.
+==Next==
+Using generated tones makes it easier to identify which parameters of the sound influences specific paramets of the resulting image. This can help in getting an idea how 'real world sounds'  would look as an image.
+In the current pipeline I have to generate the sounds in Audacity. The downside of this approach is that Audacity doesn't offer a command line interface, which makes generating a big number of images time consuming. A next step might be to use sox to generate the sound, so I can generate a big number of different sounds and images using a bash script.
+Another thing I need to find out is how to add color to the images. It is possible using noise as an input, but I haven't found out yet how to do it with tones generated in audacity. Adding a header with a color encoding indicator in imagemagick didn't work.
+I used one of the outcomes of this for the design of the cover of my reader for special issue 5.

User:Joca/raw-audio-bitmap: Difference between revisions