User:Alexander Roidl/new-new-new-new-projectproposal

      WHAT                       HOW                              CONTEXT




+------------------+
| machine learning |
+------------------+
|                  |          +--------------+                +---------------+
|                  +----------+ software art +----------------+ art & culture |
|                  |          +--------------+                +---------------+
+------------------+
| neural networks  +-----------------------+
+------------------+                       |  architecture
|   resources      |         +-------------+---------------+
|   learning       |         |input   hidden layer   output|
|   loop           |         |                             |
+--------+---------+         |       +     +-+    +        |
         |                   |  +-+  |     +-+    |        |
         v                   |  +-+  |            |        |
   installation <------------+       |     +-+    |   +-+  |
   fun/software              |       |     +-+    |   +-+  |
                             |  +-+  |            |        |
                             |  +-+  |     +-+    |        |
                             |       +     +-+    +        |
                             |                             |
                             +-----------------------------+

UPDATE

The main question shifted from a focus on neural networks to a broader perspective on software, because I became increasingly interested by the need to engage with software and software art. Leading to the new research question: How can artistic methods be used to elicit critical reflection on software as a cultural object beyond the interface.

What do you want to make?

Currently, there is a lot of attention towards artificial intelligence and its implications, but there is little research about the algorithms and software itself, other than from an engineering point of view. What is visible from those algorithms is mostly the output, e.g. a self-driving car, faces being recognized in images or google assistant providing an answer. The actual process of learning and what it means is hidden under layers of programming frameworks and abstracted statistics. But how can this concept of a learning algorithm be approached on a level that allows for productive discussion and insight? Software Art provides an interesting framework »describing not merely software used to produce art, but rather software itself as the artwork« (Cox, 2007, p. 147). Software Art marked a shift from pure syntax to something semantic, something that is aesthetically, culturally and politically charged.(Cramer, 2003) Thus, in the manner of Software Art or Software Studies, this work doesn’t simply want to make a critique of A.I. or create fancy applications but instead take the neural network as its primary object of study. By concentrating on fundamental concepts and its architecture I want to find a different perspective on machine learning.

With my work, I wish to make a comment that is very much focused on the algorithm itself rather than adding more to the recent questions about A.I. intervening in our daily life. Of particular interest within this framework of machine learning are neural networks that seem to mimic certain human features like learning and neurons. This architecture of connected neurons brings certain consequences concerning resources (computing power and data), programming concepts (see fig 1) and human imagination about computation. As in my written thesis, this work follows up on the concept of Software Art and provides a practice that is focusing on the process rather than on the outcome. It might take the form of an installation, funware, software or even a platform for discussion, including a documentation of the process.

By learning the fundamentals of neural networks I want to intervene in their inner working and create a work expressing the deeper semiotics of those algorithms – unlike other artists and designers, that are mostly using those algorithms as a tool to create surprising and generated outputs.

This is accompanied by the question: Do these algorithms (neural networks) have the potential for deeper artistic engagement that is beyond »A.I. magic« (using A.I. as a tool to generate surprising but rather meaningless output)?

Neural Network in C

Neural Network running on an iPod

Claude Shannon demonstrates machine learning

fig 1: Concept of machine learning algorithms (Chollet, 2017)

How do you plan to make it?

I will learn about the inner workings of machine learning especially with a focus on neural networks, »emphasizing the neglected aspect of computation, which involves the possibilities of virtuality, simulation, abstraction, feedback, and autonomous processes. « (Fuller, 2008). I plan to make experiments along with the research that will eventually accumulate to one final work.

I started creating small experiments in order to understand the underlying technology of certain machine learning concepts. In a workshop I asked people to create their own dataset for an image classifier and documented their insight; specifically their imagination about the inner workings of those algorithms.

To go further into the core functions of these algorithms I began writing a basic neural network in C and installed it on an iPod as a first prototype. This reflects on the resources and the complex structures that are usually hidden in high-level programming approaches. Furthermore, I want to use this simple program code and experiment with it on different devices. I want to create different digital objects that deal with specific features (as stated above: resources, architecture and imagination) of a neural network. This might literally take the form of a network, that can be rearranged and made tangible like it is the case with an analog synthesizer (fig 2). I'm planning to add a story to the different objects and create a narrative, that can be quite poetic.

fig 2: tangibility of circuits on an analog synthesizer through patching

What is your timetable?

Alongside the writing of the thesis which will be based on careful reading and research, I want to do experiments with basic structures of neural networks.

A basic outline of the planning

1. October / November: Framework + outline, research on machine learning and Software Art/Studies.
2. December: Further research on machine learning algorithms and their inner workings. Starting with more elaborate prototyping, intervening in the structure.
3. January: Deeper Research and writing on Software Art and the implications of new machine learning algorithms.
4. February: Further research and writing + prototyping. Test the prototypes with an audience. (Can also be in the form of a workshop or performance)
5. March: Finish and fine tune writing. Translate prototypes into a final project.
6. April / Mai / June: Finish and after fine-tune final project. Presenting the final work.

Why do you want to make it?

Firstly I see the possibility of an interesting intervention into a very recent topic, that is much debated and feeds into the broader context of artificial intelligence. Machine learning shouldn’t only stay in the realm of software development, where it is used as a pure tool, or for marketing purposes. This is following up the idea of Software Art and Software Studies. Secondly, I encountered a lack of artistic engagement with the deeper structures of machine learning algorithms, while I think it is essential, to investigate further into these very concepts. The high-level frameworks that allow simple implementation of machine learning in applications blur the core structures of the algorithm and abstract it to a magical black box. I want to unravel that black box with my approach.

»High-level programming approaches can be very successful in achieving certain ends, but the very imposition of higher-level constructs and metaphors also limits awareness of how code operates in and for itself and what may be achieved through that. Arguably it is the changes in low-level systems that have provoked the biggest paradigm shifts, such as the development of binary computation and Turing machines, and such as Wolfram is suggesting will be the case in fully understanding simple programs.« (Yuill 2004)

Who can help you and how?

• PZI Tutors > Research / Prototyping
• AI Now Institute https://ainowinstitute.org/ I reached out to them and hope to be able to get some insights in their research about the social impact of artificial intelligence
• XPUB Gang, getting feedback and discussing with people in the same field
• Interaction Station: Javier (They are researching about machine learning in creative practices right now)
• _V2 AI-Lab / AI Hands on evening > talking to other people researching the same issues
• Manetta, as she did some projects and workshops around machine learning

Relation to previous practice

As a designer and artist, I worked quite intensively on computational structures, including algorithms, networks and databases. In addition to that, I gained an insight into new media and technology that enhances humans. I am interested in understanding these new phenomena and their effects. Furthermore, I became fascinated by the hype around machine learning and I already did a few experiments that were related to its creative usage. Without prior knowledge of machine learning, I generated new images of coastlines out of an accumulated dataset.

previous experiment on machine learning

After further engagement I realized that these kinds of experiments didn’t really have a deeper relevance and can be found all over the internet, simply following the hype around artificial intelligence. So with my work I want to find a way to further investigate these new types of algorithms and integrate them in my artistic practice in a meaningful and lasting way.

Relation to a larger context

Nowadays machine learning is embedded in many contemporary digital systems that drive our world. These systems and models of machine learning are tough to understand, to the point where even the creator of such algorithms is not able to have an insight, why the machine makes a certain decisions. New methods of machine learning have also found their way into the arts, where artists are trying to make sense out of these new programming methods. They are focusing on the outcomes and consequences of machine learning, but there is no interaction with the inner workings, as it is done in the realm of Software Art for instance. These algorithms also carry a potential for artistic study and even poetics. It is important to understand these systems and their implications to be able to influence them, which means being able to decide if, how, when and why we want to make use of them. In the same way, this project is inspired by the tactics of Software Studies it is also meant to feed the generated knowledge back into this field, contributing to the extensive research around Software Culture.

References

• Benjamin, W. and Underwood, J. (2008). The work of art in the age of mechanical reproduction. London: Penguin Books.
• Bridle, J. (2018). New Dark Age: Technology and the End of the Future. London ; Brooklyn, NY: Verso.
• Broeckmann, A. (2006). : Software Art. [online] Mikro.in-berlin.de. Available at: http://www.mikro.in-berlin.de/wiki/tiki-index.php?page=Software+Art [Accessed 23 Nov. 2018].
• Burrell, J. (2016). How the machine ‘thinks’: Understanding opacity in machine learning algorithms. Big Data & Society, 3(1), p.205395171562251.
• Chollet, F. (2018). Deep learning with Python. Shelter Island, New York: Manning Publications Co.
• Cox, G. (2007). ‘Generator: The Value of Software Art’, in Rugg, J., & Sedgwick, M. (ed.) Issues in Curating Contemporary Art and Performance. Intellect Books, pp. 147-162.
• Cramer, F. (2003). Exe.cut[up]able statements: the Insistence of Code. in Stocker G. & Schöpf C. (eds.), Code – The Language of our time, Linz: Hatje Cantz, pp. 98-103
• Finn, E. (2017). What Algorithms Want: Imagination in the Age of Computing. Cambridge, MA: The MIT Press.
• Flusser, V. (2011). Into the Universe of Technical Images. (N. A. Roth, Trans.) (1 edition). Minneapolis: Univ Of Minnesota Press.
• Fuller, M. (Ed.). (2008). Software studies: a lexicon. Cambridge, Mass: MIT Press.
• Gere, C. (2008). Digital Culture (2nd Revised edition). London: Reaktion Books.
• Hoelzl, I., & Marie, R. (2015). Softimage: Towards a New Theory of the Digital Image. Bristol: Intellect Ltd.
• Hu, T.-H. (2016). A Prehistory of the Cloud (Reprint edition). Cambridge, Massachusetts: The MIT Press.
• Minsky, M., & Papert, S. A. (1987). Perceptrons: An Introduction to Computational Geometry, Expanded Edition (expanded edition). Cambridge, Mass: The MIT Press.
• Mitchell, W. J. T. (1980). The Language of Images (Reprint edition). Chicago: University of Chicago Press Journals.
• Müller, A. C., & Guido, S. (2016). Introduction to Machine Learning with Python: A Guide for Data Scientists (1 edition). Sebastopol, CA: O’Reilly Media.
• Johnston, J. (2010). The allure of machinic life. Cambridge, Mass: MIT Press.
• Simard, P. Y., Amershi, S., Chickering, D. M., Pelton, A. E., Ghorashi, S., Meek, C., … Wernsing, J. (2017). Machine Teaching: A New Paradigm for Building Machine Learning Systems. arXiv:1707.06742 [Cs, Stat]. Retrieved from http://arxiv.org/abs/1707.06742
• Yuill S. (2004). Code Art Brutalism: Low-Level Systems and Simple Programs in Goriunova O. and Shulgin A. (ed.) Read_me: Software Art and Cultures, Aarhus:Digital Aesthetics Research Centre.