User:Alexander Roidl/new-new–projectproposal

What do you want to make?

New advanced algorithms allow for new methods of visual form production. We see ourselves confronted with a weird set of new phenomena: algorithms that generate endless new photo-realistic images from a certain data-set, deeply weird shapes emerging from deep dream and computer vision or images that manipulate themselves from text-input.

Even before computers were invented artists where working on algorithmic form creation. Later, with the use of program code, images where generated in ever more diverse forms. Programmers and technicians have only recently developed images that are generated by machine learning that imitate photo-realistic material. Images, that follow more than only simple rules, but are far from pure randomness. Images, that are created based on a model that is not readable by humans, a model, which is feed by a database of existing images. How can these images be categorised? Are they photos, drawing, renderings or collages? I want to investigate on the implications of this new kind of images.

I’m looking to answer the question: What does algorithmic image generation mean for cultural production in times of machine learning?

In order to answer this question I need to ask: How does machine learning change the generation of image. Why and how is it different from other forms of image generation? What are the implications on culture and art production using those generated forms. I want to understand how these images come to be and how I as an artist / designer can make use of them. These kind of algorithms have been used to generate image alike Van Gogh or other famous artists, but I want to challenge these algorithms to generate new, more »native digital« images. Why would you try to replicate traditional art forms while the nature of these techniques is so different?

On the visible and invisible

During the research I encountered a weird set of images that deal with visibility and invisibility. On the one hand we find machine learning algorithms that auto-remove objects from images (http://deepangel.media.mit.edu/) on the other hand there are new algorithms that detect if an image is manipulated (https://newatlas.com/adobe-ai-detect-image-manipulation/55179/). For the second one this means computers see something in images that is invisible for us (small repetitions of pixels that arrive from using the stamp tool in photoshop).

Another example that I found on google earth are »satellite calibration targets«, that were put up in the dessert to calibrate satellite vision. (see screenshot from google earth below). So this huge targets are invisible for most of us, but are physical places, that you can visit. And through the very same technology these images become visible for anyone again through google earth. So we can get a glimpse into satellite technique by the way it produces images, manifested in a physical space.

(Note: Programming Languages use NONE / NULL to set if something is not there -> could relate to invisibility)

How do you plan to make it?

I want start researching a short history of algorithmic image & form generation and want to connect it to recent technological developments. I want to draw connections from the technical to the cultural, which means outlining machine learning in relation to visual culture. In a comparison between existing image generation techniques and new A.I.-enhanced forms, I hope to find parallels and differences in aesthetic, use and function. I plan to make experiments going along with the research that will eventually accumulate to one final work. In a playful approach I want to bend models of machine learning generators, try to make them fail, produce new material, irritate algorithms. I want to examine the development of the image, from a 2D dimensional communication tool to a zero-dimensional array of points (Flusser, 2011).

In my practical work I want to work on how we can understand these machine learning models. How can we learn from errors or mis-functions of software producing images that help making the inner functions graspable. I am want to test these new algorithms in a playful way, I want to produce new visual material, that helps to understand – or at least helps to question these systems in a critical way.

What is your timetable?

Along the writing of the thesis which will be based on careful reading and research, I want to make experiments with image generation algorithms.

Basic outline of planning (Very sketchy)

1. October / November: Frame work + outline, research on history of algorithmic image generation & machine learning, small sketches in image generation
2. December: Further research on machine learning related to images as well as image culture. Starting with more elaborate prototyping on images using machine learning
3. January: Connecting machine learning with image culture. Deeper Research and writing on image culture and the implications of new machine learning images.
4. February: Further research and writing + prototyping
5. March: Finish and finetune writing. Translate prototypes into final project.
6. April / Mai / June: Finish & fine-tune final project.

Why do you want to make it?

Online you constantly see news about new machine learning algorithms. I encountered a lot of technical papers in the past, where technicians are talking about images (quite strangely). But instead of talking about the technical improvements that can be done on these images, I want to think about what these images mean for fields traditionally concerned with image-making, especially arts and design.

Furthermore machine learning is embedded in many contemporary digital systems that drive our world, and it is hard for us to understand them fully. So I think images are becoming a more important tool to make visible what these algorithms are doing and also to make visible where they fail. It is important to understand these systems and their implications in order to be able to influence them.

Images dominate the world and human perception (80%? of your perception is based on vision). We saw different phenomena arising from image culture: memes, new aesthetics, photo manipulation, the aesthetics of censorship… While Walter Benjamin saw himself confronted with a mechanical reproducibility of art with the rise of photography, we are now facing endless digital self-production and are even challenged in the way we see by machines. (Walter, 2008)

Who can help you and how?

• PZI Tutors > Research / Prototyping
• AI Now Institute https://ainowinstitute.org/ I reached out to them and hope to be able to get some insights in their research about the social impact of artificial intelligence
• XPUB Gang
• Interaction Station: Javier (They are researching about machine learning in creative practices right now)

Relation to previous practice

I'm trained as a graphic designer and have been fascinated by different kinds of visual material. In addition to that I gained interest in new media and technology that would enhance humans. I am interested in understanding these new phenomena and their effects. Furthermore I have been researching about database and database-art. The models of machine learning algorithms are taking this idea of the database onto another level. A level where only machines can read this databases anymore. »Machine is talking to machine – the keyboard or user can be plugged in if needed« (Kittler, 2001) Furthermore I already did a few experiments that were related to experimental creative use of machine learning. Without prior knowledge of machine learning I generated new images of coastlines out of an accumulated dataset.

Relation to a larger context

Nowadays machine learning is embedded in many contemporary digital systems that drive our world. Images dominate the human perception of the world and have been matter to digital generation and manipulation since a long time.

These systems and models of machine learning are really hard to understand, to the level where even the creator of such algorithms is not even able to have an insight, why machines make certain decisions.

References

Literature (most important selection)

• Benjamin, W., 2008. The Work of Art in the Age of Mechanical Reproduction, 01 edition. ed. Penguin, London.
• Berger, J., 2008. Ways of Seeing, 01 edition. ed. Penguin Classics, London.
• Bridle, J., 2018. New Dark Age: Technology and the End of the Future. Verso, London ; Brooklyn, NY.
• Burrell, J., 2016. How the machine “thinks”: Understanding opacity in machine learning algorithms. Big Data & Society 3, 2053951715622512. https://doi.org/10.1177/2053951715622512
• Designing Programmes, 2016.
• Flusser, V., 2011. Into the Universe of Technical Images, 1 edition. ed. Univ Of Minnesota Press, Minneapolis.
• Gere, C., 2008. Digital Culture, 2nd Revised edition edition. ed. Reaktion Books, London.
• Hoelzl, I., Marie, R., 2015. Softimage: Towards a New Theory of the Digital Image. Intellect Ltd, Bristol.
• Hu, T.-H., 2016. A Prehistory of the Cloud, Reprint edition. ed. The MIT Press, Cambridge, Massachusetts.
• Kittler 2001, TV Doku Spiegel Unberechenbarkeit, video recording, YouTube, viewed 01 October 2018, <https://www.youtube.com/watch?v=AavTap5FgSQ&feature=youtu.be&t=275>
• Mitchell, W.J.T., 1980. The Language of Images, Reprint edition. ed. University of Chicago Press Journals, Chicago.
• Müller, A.C., Guido, S., 2016. Introduction to Machine Learning with Python: A Guide for Data Scientists, 1 edition. ed. O’Reilly Media, Sebastopol, CA.
• Postdigital Aesthetics - Art, Computation And Design | D. Berry | Palgrave Macmillan, n.d.
• Press, T.M., n.d. The Allure of Machinic Life [WWW Document]. The MIT Press. URL https://mitpress.mit.edu/books/allure-machinic-life (accessed 10.15.18).

Relations

• In 2012 James Bridle established the term »New Aesthetic« in 2012, an ongoing collection of images on a tumblr blog.

Visual

Code

• https://github.com/paarthneekhara/text-to-image