Audio Visual Art and Extensions

1. Audio Visual Art.

"Certainly I have an aversion to everything that is demonstratively programmatic and illustrative. But this does not mean that I am against music that calls forth associations; on the contrary, sounds and musical coherence always arouse in me ideas of consistency and colour, of visible and recognizable form. And vice versa: I constantly combine colour, form, texture and abstract concepts with musical ideas. This explains the presence of so many non-musical elements in my compositions." György Ligeti, 1968

We’re receiving a variety of information from the world outside of our brains day and night, even when we’re sleeping. We listen to sounds with ears, see images with eyes. However, can we listen to images and see sounds? In my further research, I would like to dig more about audio visual art and technology.

Our eyes and ears are in charge of receiving visual and audio information. However, it is interesting to note that the human brain is structured so that there is one area can be called “audiovisual”, namely, the “superior colliculus”. This area receives input from both the ear and the eye and, because of this, its cells are called multisensory cells:” For example, if some multisensory cell responds to a light flash in the upper right portion of the visual field, that cell will respond to a sound only if it too comes from the same vicinity. Additionally, when visual and auditory inputs occur simultaneously, a multisensory cell responds more strongly than when either input occurs alone.”[R. Sekuler and R. Blake: Perception, Alfred A. Kopf, New York 1985, pag. 104]. Therefore, the structure of our brain provides the possibilities to translate audio to visual or visual to audio.

The most remarkable development in both modern music and the visual arts is the enormous expansion of the artist's territory. The composer need no longer confine himself to the traditional instrument played in the traditional manner, for since the development of concrete music he can use any sound that appeals to him and electronic and computer music have placed a whole arsenal of new possibilities at his disposal. The composer has also liberated himself from the formal restrictions of classical music: it is no longer necessary to have a well-defined beginning or a well-defined end, or indeed a clear structure. Chance and the imagination of the performer can playa part; the straight jacket of time (a compelling form and a fixed metre) has been unlaced and discarded.

The Visual arts, too, have refused to be restricted by the traditional instruments: oil paints and canvas stretched over a rectangular frame. Any material in any form is used. Tinguely and others introduced motion in art. Technological achievements, too, (the Laser beam, the computer, etc.) are now being employed. The diversity of the objectives - as well as that of the means - has also increased. It is no longer only the esthetics of the unique (hand) work of art that counts, but any and every way in which an idea can be expressed. The idea itself often becomes more important than its visualization.

As a simple result, there are now thousands of ways that are able to create connections between audio and visual.

2.”Image and Sound programme” and “Elecric Stimulus to Face”.

In Peter Struycken’s project – “ Image and Sound programme 1” 1970, he is concerned with the problem of how to obtain variations in large numbers of structures built up of similar component parts. Under certain conditions a relatively slight variation in a structural rule can bring about a high degree of variation in the visual and auditory effect. His Image and Sound programm I, is a model in which he investigates those conditions. The methods are similarly applied to image - changing in time - and sound.

he drew a basic pattern which consists of 8 x 8 units, each of which, is subdivided into 4 squares so that the entire field consists of 16 x 16 = 256 squares. He filled selected squares with black color so that different random patterns could be created, simply like the internet word : 010101, which means off and on.

In his point of view, time-factor plays an obvious important role in sound. However, this is not the case with images.Furthermore, he found that both image and sound can be seen to have something to do with alteration in time, thus, the characteristics can be the connections between image and sound. He used two characteristics to create the connections : the quantity (increasing or decreasing in the number of visual elements and sound elements) and the type(types of image and sound elements).

However, on one hand, sound and image he used could not start synchronously, which causes the unequal connections between sound and image. On the other hand, he demonstrated that his main interest was in making the degree in which sound and image change perceptible. The degree of change can be extremely large or small. He wanted to give an overall impression of the changes at every moment.

In another audio visual project – “Electric Stimulus to Face” 2009, artist Daito Manabe translated audio information to facial emotions. He connected sounds and facial muscles with electric pulses. Every beat of sounds will activate those electric pulses to stimulate facial muscles, thus, people’s will change their emotions by following the rhythm of sounds. Behind the project, Daito Manabe was initially thinking about how weird it would be if people could artificially copy human facial expressions. However, he realized that it was not possible to build a technical device that can synthetically copy human facial expressions. He did a lot experiments by pursuing his goal , and “ electric stimulus to face” was one of his experiments. Besides the video he made, Daito Manabe also had several performances of this project, which to stimulate facial expressions by sounds through electric pulses.

Apparently, there are broader diversities of audio visual art forms nowadays by comparing those two projects. Audio visual artists now are not only looking for the connections between audio and paintings or moving images, but also exploring in a wider visual way, for example: relationship between audio and performance, emotions, mechanical installations, etc. In addition, the changes are the simple result of improvement of technology. The relationship between audio visual art and technology is extremely close, and the huge improvement of science and technology provides a variety of possibilities and opportunities for audio visual art.

3. Audio Visual Art’s Extensions.

There are variety of methods to create connections and correspondences between audio and visual arts as mentioned. For example : sound sculptures, visual music, performance, abstract film, etc. As for my own research direction, I would mainly focus on the connections between audio and abstract animation / motion graphics.

A sound can be analyzed as different attributes: timbre, high key or low key, sharp or smooth, etc. Same to animation for different attributes: shape, color, scale, transparency, ways of movement, etc. Therefore, analyze and separate those different attributes and then create correspondences between them become possible. Moreover, several possibilities could be found during the procedure of exploration, for instance: Does animation can only be shown on screen, or it can be applied as a physical form as well? Is it necessary that audio needs to be organized in rhythm or it can be random clips? How abstract can a sound be and how abstract can a visual part be?

In my flipbook project, I collected some sounds from daily life like: brooming , cling, eating ramen, crumbling papers, etc. Also, I created animation loops based on those sound elements. By creating the possible correspondences, I mainly animated the shape and movements of abstract graphics both with 3D and 2D softwares. In addition, I mixed those sounds while mixing different animation loops in the same way, and it turned to a whole piece that contains different sound and visual elements. A flipbook is a physical printed object, in that case, I tried to find the possibility that to add sound effect to a physical book. Therefore, the form of audio visual in this project becomes visualizing a sound by printed graphics from visualizing a sound by digital animation. During the exploration, many possibilities keep popping out.

Moreover, I started to realized if any human perceptions can be translated from one form to another since audio visual art is basically translating audio information to visual information. Can tactile sensation be heard? Can smell be seen? Can light be heard? Can happiness be weighed?

4. Conclusion. Audio visual art replies a lot on the development of technology. Additionally, the combination of technology and imaginary provides unlimited space for audio visual art.