User:Alexander Roidl/new-new-new-new-projectproposal

      WHAT                       HOW                              CONTEXT




+------------------+
| machine learning |
+------------------+
|                  |          +--------------+                +---------------+
|                  +----------+ software art +----------------+ art & culture |
|                  |          +--------------+                +---------------+
+------------------+
| neural networks  +-----------------------+
+------------------+                       |  architecture
|   resources      |         +-------------+---------------+
|   learning       |         |input   hidden layer   output|
|   loop           |         |                             |
+--------+---------+         |       +     +-+    +        |
         |                   |  +-+  |     +-+    |        |
         v                   |  +-+  |            |        |
   installation <------------+       |     +-+    |   +-+  |
   fun/software              |       |     +-+    |   +-+  |
                             |  +-+  |            |        |
                             |  +-+  |     +-+    |        |
                             |       +     +-+    +        |
                             |                             |
                             +-----------------------------+

What do you want to make?

»emphasizing the neglected aspect of computation, which involves the possibilities of virtuality, simulation, abstraction, feedback, and autonomous processes. «

Currently, there is a lot of attention towards Artificial Intelligence and its implications, but there is little research about the algorithms and software itself, other than from an engineering point of view. What is visible from those algorithms is mostly the output, e.g. a self-driving car, faces being recognized in images or google assistant providing an answer. The actual process of learning and what that means is hidden under layers of programming frameworks and conceptual statistics. But how can this idea of a learning algorithm be approached at a level that allows for productive discussion and insight? Software Art provides an interesting framework »describing not merely software used to produce art, but rather software itself as the artwork« (Cox, 2007, p. 147). So in the manner of Software Art or Software Studies, this work doesn’t want to make an easy critique of A.I. or create magical applications but instead take the neural network as its primary object of study. So by concentrating on fundamental concepts and its architecture I want to find a different perspective on machine learning. With my work, I wish to create a comment that is very much focused on the algorithm itself rather than adding more to the recent questions about A.I. intervening in our daily life. Of particular interest within this framework of machine learning are neural networks that seem to mimic certain human features like learning and neurons. This architecture of connected neurons brings certain consequences concerning resources (regarding computing power and data), programming concepts (see fig 1) and human imagination about computation. Like in my written thesis this work follows up on the concept of Software Art and provides a practice that is instead focusing on the process than on outcome. It might take the form of an installation, funware, software or even a platform for discussion, including documentation of the process. Through learning the fundamentals of neural networks I want to intervene in their inner working and therefore create a work expressing the deeper semiotics of those algorithms – unlike other artists and designers, that are using these algorithms mostly as a tool to create surprising, generative outputs.

This is always accompanied by the question: Do these algorithms (neural networks) have the potential for deeper artistic engagement that is beyond »A.I. magic« (using A.I. as a tool to generate surprising but rather meaningless output)?

Neural Network in C

Neural Network running on an iPod

fig 1: Concept of machine learning algorithms (Chollet, 2017)

How do you plan to make it?

I will learn about the inner workings of machine learning especially with a focus on neural networks. I plan to make experiments going along with the research that will eventually accumulate to one final work.

To approach certain machine learning concepts I started creating small experiments trying to understand the techniques used. In a workshop I asked people to create their own dataset for an image classifier and documented their insight; specifically their imagination about the inner workings of those algorithms.

To go further into the core functions of these algorithms I started focusing on neural networks and began writing a basic neural network in C and installed it on an iPod as a first prototype reflecting on resources and the complex structures that are usually hidden in high-level programming approaches. Furthermore, I want to use this simple program code and experiment with it on different media. I want to create different digital objects that deal with certain specific features (as stated above: resources, architecture and imagination) of a neural network. This might literally take the form of a network, that can be rearranged and made tangible like it is the case with an analog synthesizer (fig 2). I'm planning to add a story to the different objects and create a narrative, that can be quite poetic and therefore connect their meaning.

fig 2: tangibility of circuits on an analog synthesizer through patching

What is your timetable?

Alongside the writing of the thesis which will be based on careful reading and research, I want to do experiments with basic structures of neural networks.

A basic outline of the planning

1. October / November: Framework + outline, research on machine learning and Software Art/Studies.
2. December: Further research on machine learning algorithms and their inner workings. Starting with more elaborate prototyping, intervening in the structure.
3. January: Deeper Research and writing on Software Art and the implications of new machine learning algorithms.
4. February: Further research and writing + prototyping. Test the prototypes with an audience. (Can also be in the form of a workshop or performance)
5. March: Finish and fine tune writing. Translate prototypes into a final project.
6. April / Mai / June: Finish and after fine-tune final project. Presenting the final work.

Why do you want to make it?

Firstly I see the possibility of an interesting intervention into a very recent topic, that is much debated and feeds into the broader context of artificial intelligence. Machine learning shouldn’t only stay in the realm of software development, where it is used as a pure tool, or for marketing purposes. This is pretty much following up in the sense of Software Art and Software Studies. Secondly, I encountered a lack of artistic engagement with the deeper structures of machine learning algorithms, while I think it is essential, to investigate further in these very concepts. The high level of frameworks that allow simple implementation of machine learning in applications blur the core ideas of the algorithm and abstracts it to a magical black box. I want to unravel that black box by researching the basic structures of neural networks.

»High-level programming approaches can be very successful in achieving certain ends, but the very imposition of higher-level constructs and metaphors also limits awareness of how code operates in and for itself and what may be achieved through that. Arguably it is the changes in low-level systems that have provoked the biggest paradigm shifts, such as the development of binary computation and Turing machines, and such as Wolfram is suggesting will be the case in fully understanding simple programs.« (Yuill 2004)

Who can help you and how?

• PZI Tutors > Research / Prototyping
• AI Now Institute https://ainowinstitute.org/ I reached out to them and hope to be able to get some insights in their research about the social impact of artificial intelligence
• XPUB Gang, getting feedback and discussing with people in the same field
• Interaction Station: Javier (They are researching about machine learning in creative practices right now)
• _V2 AI-Lab / AI Hands on evening > talking to other people researching the same issues
• Manetta, as she did some projects and workshops around machine learning

Relation to previous practice

As a designer and artist, I worked quite intensively on computational structures, including algorithms, networks and databases. In addition to that, I gained an interest in new media and technology that enhances humans. I am interested in understanding these new phenomena and their effects. Furthermore, I became interested in the hype around machine learning and I already did a few experiments that were related to the experimental creative use of machine learning. Without prior knowledge of machine learning, I generated new images of coastlines out of an accumulated dataset.

previous experiment on machine learning

After further engagement I realized that these kinds of experiments didn’t really have a further relevance and can be found all over the net, following the hype around artificial intelligence. So with my work I want to find a way to further investigate these new kinds of algorithms and integrate them in my artistic practice in a meaningful and lasting way.

Relation to a larger context

Nowadays machine learning is embedded in many contemporary digital systems that drive our world. These systems and models of machine learning are tough to understand, to the level where even the creator of such algorithms is not even able to have an insight, why machines make certain decisions. New methods of machine learning have also found their way into the arts, where artists are trying to make sense out of these new algorithms. They are focusing on the outcomes and consequences of machine learning, but there is no interaction with the inner workings of these algorithms, as it is done in the realm of Software Art for instance. These algorithms also bury a potential for artistic study and even poetics. It is important to understand these systems and their implications to be able to influence them, which means to be able to decide if, how, when and why we want to make use of them. In the same way, this project is inspired by the tactics of Software Studies it is meant to feed the generated knowledge back into this field, contributing to the extensive research around Software Culture.

References

• Benjamin, W. (2008). The Work of Art in the Age of Mechanical Reproduction. London: Penguin.
• Berger, J. (2008). Ways of Seeing (01 edition). London: Penguin Classics.
• Bridle, J. (2018). New Dark Age: Technology and the End of the Future. London ; Brooklyn, NY: Verso.
• Broeckmann, A. (2006). Software Art Aesthetics. Retrieved 11 November 2018, from http://www.mikro.in-berlin.de/wiki/tiki-index.php?page=Software+Art
• Burrell, J. (2016). How the machine ‘thinks’: Understanding opacity in machine learning algorithms. Big Data & Society, 3(1), 2053951715622512. https://doi.org/10.1177/2053951715622512
• Charly Onrop. (n.d.). TV Doku Spiegel Unberechenbarkeit 1/3. Retrieved from https://www.youtube.com/watch?v=AavTap5FgSQ&feature=youtu.be&t=275
• Chollet, F. (2017). Deep Learning with Python (1 edition). Shelter Island, New York: Manning Publications.
• Cox, G. (2007). ‘Generator: The Value of Software Art’, in Rugg, J., & Sedgwick, M. (ed.) Issues in Curating Contemporary Art and Performance. Intellect Books, pp. 147-162.
• Deep Angel, The Artificial Intelligence of Absence. (n.d.). Retrieved 1 November 2018, from http://deepangel.media.mit.edu/
• Finn, E. (2017). What Algorithms Want: Imagination in the Age of Computing. Cambridge, MA: The MIT Press.
• Flusser, V. (2011). Into the Universe of Technical Images. (N. A. Roth, Trans.) (1 edition). Minneapolis: Univ Of Minnesota Press.
• Gere, C. (2008). Digital Culture (2nd Revised edition). London: Reaktion Books.
• Gerstner, K. (2007). Karl Gerstner: Designing Programmes (3rd, and enlarged ed.). Baden: Lars Müller.
• Haridy, R. (2018). Adobe’s new AI can identify altered images. Retrieved 1 November 2018, from https://newatlas.com/adobe-ai-detect-image-manipulation/55179/
• Hoelzl, I., & Marie, R. (2015). Softimage: Towards a New Theory of the Digital Image. Bristol: Intellect Ltd.
• Hu, T.-H. (2016). A Prehistory of the Cloud (Reprint edition). Cambridge, Massachusetts: The MIT Press.
• Minsky, M., & Papert, S. A. (1987). Perceptrons: An Introduction to Computational Geometry, Expanded Edition (expanded edition). Cambridge, Mass: The MIT Press.
• Mitchell, W. J. T. (1980). The Language of Images (Reprint edition). Chicago: University of Chicago Press Journals.
• Müller, A. C., & Guido, S. (2016). Introduction to Machine Learning with Python: A Guide for Data Scientists (1 edition). Sebastopol, CA: O’Reilly Media.
• Postdigital Aesthetics - Art, Computation And Design | D. Berry | Palgrave Macmillan. (n.d.). Retrieved from //www.palgrave.com/gp/book/9781137437198
• Press, T. M. (n.d.). The Allure of Machinic Life. Retrieved 15 October 2018, from https://mitpress.mit.edu/books/allure-machinic-life
• Simard, P. Y., Amershi, S., Chickering, D. M., Pelton, A. E., Ghorashi, S., Meek, C., … Wernsing, J. (2017). Machine Teaching: A New Paradigm for Building Machine Learning Systems. arXiv:1707.06742 [Cs, Stat]. Retrieved from http://arxiv.org/abs/1707.06742
• Yuill S. (2004). Code Art Brutalism: Low-Level Systems and Simple Programs” in Goriunova O. and Shulgin A. (ed.) Read_me: Software Art and Cultures, Aarhus:Digital Aesthetics Research Centre.