Re: Software Art: The Learning Algorithm

/////////this is a comment, help for the structure

/////////still sketchy – this is not so polished yet, especially when looking at the language, sorry!

Placing learning algorithms in the realm of Software Art enables a productive discussion about their cultural implications.

After recognising common patterns in the discussion around Software Art / Generative Art and in the current development of machine learning algorithms I want to analyse Software Art. The insights will then be helpful in order to discuss potential approaches towards learning algorithms.

»Software Art«

In this chapter I want to describe the approach of Software Art to then point out the important realizations of it in relation to artistic engagement with software and algorithms. Further, I will relate that insight to machine learning and current approaches towards artificial intelligence in the field of art. In a broader sense I hope this will also be applicable to other kinds of "new" algorithms.

/////////general definition of software art, tracing different opinions / movements

What is Software Art?

»The term ‘software art’ has become popular to describe the contemporary artistic preoccupation with software production« (Cox, 2007, p.147). According to Cox Software is being used as the main subject of the artwork instead of a pure tool (2007, p.147).

There are several approaches to outline Software Art. As Cramer realizes, there have been two major influences: Firstly, the group around Matthew Fuller / Graham Harwood and secondly, the movement around "Adrian Ward (whose Auto-Illustrator won the transmediale.01 software art prize) and Alex McLean (whose forkbomb.pl won the transmediale.02 software art prize) [and] the theoretician Geoff Cox", both part of the so called eu-gene group (Cramer, 2002b). "While Fuller/Harwood regard software as first of all a cultural, politically coded construct, the eu-gene group rather focuses on the formal poetics and aesthetics of software code and individual subjectivity expressed in algorithms" (Cramer, 2002b). (Later Fuller took his ideas about cultural software further, initiating Software Studies, reflecting on Software as a cultural / political phenomenon.) So this two different approaches can be used to further refine the understanding:

"[S]oftware art could be generally defined as an art of which the material is formal instruction code and/or which addresses cultural concepts of software." (Cramer, 2002b) This takes the different possible engagements into account and reflects on the main aspects of Software Art.

/////////examples on runme as support for definition of software art

runme.org is a repository that collects projects around Software Art. An attempt of definition and positioning can be found on http://runme.org/faq.tt2. It is a very practical attempt to make the engagement visible and I think it makes the definition of Software Art even clearer. What also becomes visible, unfortunately, is that the site hasn't been updated since 2012. This decline of activity can also be determined with Software Arts. But through this essay I also want to argue that this fade of attention is not due to lost relevance, but rather because of other reasons, like the use of other terms. The opposite is the case indeed, the concept of Software Art is becoming more relevant with more complex software entering our lives. In this context cultural and artistic study of these phenomena is inevitable, just like Software Art and Software Studies propose.

Thus, Software Art provides an interesting framework »describing not merely software used to produce art, but rather software itself as the artwork« (Cox, 2007, p. 147). In contrast to that, other movements that happened in the beginning of the 2000s where much broader in their focus, e.g. New Media Art. But interesting about Software Art is the specific focus on software, taking it as an - to a certain degree isolated - object of study. What is of most importance for this thesis is the framework and the approach that Software Art chooses.

The history of Software Art

/////////relation to there is no software by Kittler (find reference)

The argument of the essay might not go without referring to Kittlers "There is no Software" essay. Even though Kittler was arguing that there is no Software and it is intrinsically connected to its hardware, Cramer points out that »if any algorithm can be executed mentally, as it was common before computers were invented, then of course software can exist and run without hardware« (2002) Following this argument of Cramer this also points to the idea of software in a very conceptual way, not only defining software as a program that is running on a certain hardware.

/////////relation to conceptual art

This is also what enables Software Art to find its own pre-computational history partly in Conceptual Art. What connects them is, that they both use language as their main material, and in both "the collapsing of concept notation and execution into one piece" can be recognized (Cramer, 2002b). Connecting it to artists like Sol LeWitt, who used instructions for the execution of his artworks, not only creates a history but makes the ideas more profound and for the purpose of this essay it makes the connection between conceptual art and software art even clearer.

/////////relation to pre-software-availability artists

One reason that also contributed to the need for further engagement with software in itself, is the increase of existing software. Early computer artists had to write their own software in order to create art, while nowadays there is basically no need to write your own software to generate artistic output. (Cramer, 2002a) Even when writing your own program, there is a bunch of frameworks that abstract the actual programming. I will discuss this and its influences later on in the chapter [framework culture]. At the same time it is worth mentioning that there is rarely any work that can be done without the use of computers or software (Cramer, 2002). Every mail, every online search or even public means of transport are heavily based upon software. It’s only that we became so used to it and/or that it is hidden in invisible layers of our daily life.

Not used

»One thus could say that contemporary software art operates in a postmodern condition in which it takes pre-existing software as material — reflecting, manipulating and recontextualizing it.« (Cramer, 2002a)

"Software culture is the living culture of programmers and users, as active participants in a world of or mediated by software. In its heart it circumscribes the field of intensive immaterial production, if on the level of coding, use, speculation or critical reflection and at the periphery every aspect of human life which is somehow driven or controlled by software. Software art is reflecting the realities and potentials of this culture." Pit Schultz, from a closed mailing list discussion. -> Software that drives our life, can be also seen more and more by machine learning applications. -> Cultural discussion.

The approach of Software Art

focus on the actual thing & not take it for granted:

According to Cramer it is important to realize that artists often »take [Software] for granted« (2002). But within Software Art this shouldn’t be the case, it is rather supossed to »pay attention to how and by whom programs were written« (Cramer, 2002).

What we can see from this are important key questions and should be important for further research:

»not take software for granted«:
»by whom programs were written«: Google and Amazon provide infrastructures and frameworks, that are layered libraries, that make the core of the application hard to see.
»the software itself as the artwork« (Cox, 2007, p.147)

processbased, runtime

To put focus on the process instead of the end product is not new in the art world, but Software Art exemplifies this approach »appropriate to contemporary conditions« (Cox, 2007, p.147).

This means not only the relation to the processbased approach of Software Art, that I already mentioned above, it also includes new influences, pointing for instance on performance. While the result is not nescessary a fixed product, that is visible, it can be a runtime application, that never reaches the state of finishing. An approach like this opens up new discussions and new ideas.

material vs. tool

By taking software as a primary object of study it acknowledges the role of Software in a cultural manner, and realizes that software »is not merely a functional tool, but is itself an artistic creation. (amsterdam.nettime.org/Lists-Archives/rohrpost-0101/msg00039.html) This implies also that the code, which software is made of, becomes the material [Think this is from Cox, but need to check].

other values:

software determinism: What can be seen from examples of process based art: » They demonstrate how the producer can concede control to some extent – and this is an important qualification – over the production of the work but that human intervention is paramount to (software) production« (Cox, 2007, p.150)

not used:

»Although one can of course use computers without programming them, it is impossible not to use programs at all. The question only is who programs. There is, after all, no such thing as data without programs, and hence no digital arts without the software layers they either take for granted, or design or manipulate themselves. To discuss “software art” simply means to not take software for granted, but pay attention to how and by whom programs were written« (Cramer, 2002)

Software needs Hardware to run on. »This link to performance also clarifies something about the use of the term ‘software art’, in describing not merely software used to produce art, but the software itself as the artwork.« (Cox, 2007, p.147)

Relations

Software Art & Free Software

»The first subculture which understood software as something intertextual was the Free Software and Open Source movement, i.e. the movement which produced software like GNU and Linux, from which Steven Levy’s notion of the “hacker” was largely derived.[…] GNU/Linux provides many examples of software which would be wildly successful as software art if it only were properly advertised or, if you prefer, contextualize« (Cramer, 2002a)

Why Software Art and not Algorithmic Art?

When looking at learning algorithms like deep learning and neural networks it might seem confusing to approach them within Software Art, while there is also Algorithmic Art. In its use Algorithmic Art is a sub category of Generative Art (which will be discussed later) and mainly preoccupied with visual output.

On the other hand "Software Art can base itself on a number of different levels of software: source code level, abstract algorithm level, or on the level of the product created by a given piece of code" (Arns, 2005)

This means that Software Art includes indeed algorithmic approaches, that may or may not become Software in the end. The different terms reflect more on different ways of thinking than on the actual refinement of the field specified. Software Art and Algorithmic Art of course find overlaps and intersection, but I want to emphasis that the term Software and Algorithm are not equal [a reference would be good here]. Also there is no need for binary classification, it is only that Software Art provides the more interesting approach for the use of this reseach.

not used yet

aesthetics of software art

"But as a contemporary art, the aesthetics of software art includes ugliness and monstrosity just as much as beauty, not to mention plain dysfunctionality, pretension and political incorrectness." (Cramer, 2002a)

Generative Art?

[THIS IS STILL CHAOS]

In comparision to Software Art the term Generative Art has been around for way longer, following up on Computer Art. They both seem to have their own use and field, while the term Generative Art seems to be more popular. In this part I quickly want to elaborate on the differences between the two movements.

»[G]enerative art stress the formal rule-based and syntactical properties of software, and thus do not place sufficient emphasis on semantic concerns and social context« (Cox, 2007, p.148)

Definition by Philip Galanter:

»Generative Art refers to any art practice where the artist uses a system, such as a set of natural language rules, a computer program, a machine, or other procedural invention, which is set into motion with some degree of autonomy contributing to or resulting in a completed work of art.« (Galanter 2003)

Galanter and McCormack emphasise focus on end product

Generative Art as a term that is preoccupied by this -> a nesseciary shift towards the process, as Cox proposes »generator […] that which generates« (2007, p.151)

Inke Arns describes some key differences of Software Art and Generative Art in her essay Readme, runme, execute_me. As she points out Generative Art (following the definition of Philip Galanter, 2003):

runs autonomously or in a self organized way
generates unforeseeable results

"It does not regard software as a pragmatic aid that disappears behind the product it creates, but focuses on the code it contains" (Arns, 2005)

»There is broad agreement that generative art is a term applied to artwork that is automated by the use of instructions or rules by which the artwork is executed« (Cox, 2007, p.148)

There is a tension between the understanding of Generative Art and Software Art that can be productive and helpful to understand new technologies.

Software Studies

Next to Software Art computational phenomena are also investigated by Software Studies. While leaving out the Art it engages more with Software (in a broad sense digital objects and phenomena) on a philosophical and theoretical (?) level.

"Software Studies uses and develops cultural, theoretical, and practice-oriented approaches to make critical, historical, and experimental accounts of (and interventions via) the objects and processes of software." (Cox, 2013)

Framework culture

The stack of libraries and frameworks make creating a neural network super easy, but at the same time very abstract.

»The now wide availability of authoring software has changed the conditions for the production of software art by the artist-programmer. It is with some of these issues in mind that Richard Wright traces the ‘divergence between programmers and program users’, based around the issue of whether a computer is considered a medium or a tool (Wright 2004). In a hierarchy of programming languages, Wright points out that not all programming practices are equal. He is thinking of the predominance of scripting languages such as Flash Actionscript (but also Lingo, Perl, MAX, JavaScript, Java, C++, as well as other programming and scripting languages) that use libraries of functions and a certain shared, if not prescribed, vocabulary of style.« (Cox, 2007, p. 153)

The question is also: what kind of programming is being used (Cox, 2007, p.153)?

As mentioned earlier former artists had to write Software to generate, nowadays Software is widely available, so it is not necessary to engage with it. This of course means a decline in engagment with software and comes with the risk to take software for granted, without qustioning it. But the positive consequence is that this frees programming from certain aspects and gives room for a new engagement, "just as previously the invention of photography perhaps freed painting from figurative representation" (Cox, 2007, p. 155)

Code Brutalism

Relation to the rawness of code. What Software Art means for the interaction with code and how code brutalism offers an interesting concept for programming.

»High-level programming approaches can be very successful in achieving certain ends, but the very imposition of higher-level constructs and metaphors also limits awareness of how code operates in and for itself and what may be achieved through that. Arguably it is the changes in low-level systems that have provoked the biggest paradigm shifts, such as the development of binary computation and Turing machines, and such as Wolfram is suggesting will be the case in fully understanding simple programs.« (Yuill 2004)

Software Art and Cultures (readme book) “software art is the political and critical art form of contemporary culture”

Software Art poetics of software iPod / low, obsolete tech neural networks narrative of computation

neural network investigating itself having sensors recognize signals from the networks

if you take the nn out of its function it can reveal something about itself suddenly it is not threatening anymore

working with familiar objects might make neural nets more tangible

the limit-ness of the data-set (the dataset is limited and the net won’t go beyond it) the resources of machine learning (that is needs a lot of energy / processing power) the imagination of neural networks as a living character inside the system / intelligent classical narratives around AI: self-driving car to hit 2 groups of people, decide which on, dilemma

low tech objects and high level software the narrative around software

digital objects

the promise of the neural network (what do we expect from software / neural networks?) We want it to be clever but not to intervene into our life too much. So we expect Software to be better than ourselfs, but at the same time not, cause we are afraid of everything that might be better than us. So what is the promise of Software? Automation of boring tasks? It should assist you as good as possible, but no more. “For instance, it has become a truism to emphasize that code is not merely functional but can have poetic qualities, and political significance.” (read_me, p. 161)

“leading to the idea of software as potential literature” (read_me, p. 162)

“any essay or piece of code is only ever a work in progress and should be subject to active criticism and upgrade” (read_me, p. 162)

Is there a poetic to neural networks? What is the poetics of neural networks? And how does it change how we perceive neural networks / machine learning … ?

Software Poetics

my main interest: 1. how software influences people, with or without knowing. 2. how versatile software is in its creation and execution. 3. how software works and how it creates infrastructures 4. what are the poetics of software? How can it help to understand the software itself. 4.1. what comes with software besides the intended function 4.2. The in-between space of GUI and Hardware

How much are we dependent on / influenced by software. In times, when Software becomes more and more abstract and black boxed.

Nonfunctional Software (The function of software) Software always comes with a dedicated function or purpose. Lately, especially for proprietary software, user and creator seem to have diverging interests. After the big data and currently the AI hype companies became increasingly interested in data collections and using them for economical benefits. While this enables users to get software cheaper they not only lose their control, but also quality in software. But nevertheless

What, if Software has no function? Software plays an important role in our lives. Yet it is such a versatile and subtle medium. We can get a spark of what this means and how Software really works, when it fails or when it is taken out of its context. So in the following I want to argue that for a serious engagement with software it is also necessary to look at the nonfunctional and the stuff that is in-between the pixels and conducting paths. Software is primary made to function, but what if software fails or malfunctions on purpose?

malfunctions of software

While The Alliance for Code Excellence imagines ‘[a] world where software runs cleanly and correctly as it simplifies, enhances and enriches our day everyday life is achievable’ I argue that the malfunctioning of code can also be something positive that is revealing and holds a value. The great thing about an unstable setup can be that the user knows that there is a potential for crashes and therefore engagement is necessary. At the point when it crashes you will be able to get a glimpse of the inner workings of software.

unintended use

The unintended use of software can arise from an uninformed user or a user trying to stretch the potential of the functions. People are collecting misuses of software and operating systems online ( https://thoughtcatalog.com/michael-koh/2014/01/21-people-share-the-greatest-software-misuses-theyve-witnessed/). The open source license

Self-destructive Software

The influence of Software on our life Example of Lieferando / GPS navigation The drivers follows the route of the software, which may not necessary make sense, but still the human relies on the software to calculate the right system. Which is also a trust in the data that has been given to the system. (This is separate from hardware, in the sense, that when hardware fails, you would immediately know and figure out other ways to reach the aim. But if software fails, it doesn’t necessarily mean that one can notice it. You would for instance reach the wrong goal or go the wrong way)

Whiteboxing (see http://thepiratebook.net/category/articles/)

Programming Code Considering Software as a product, Code would consequently represent its material. As David M. Points out Code does not simply consist of one single representation. Besides the layer of human readable code, there is also the form of machine code and when the code is actually being executed on the machine. The first of the mentioned states, the textual representation of code, that is in a human readable format, written in a specific programming language, can also exist as literature. Following the history of computation it is mostly agreed on that code is more than just machine instructions. (The Philosophy of Software Code and Mediation in the Digital Age David M. Berry, p. 28ff.) Knuth for examples also argues for an aesthetic in code distinguishing between ‘good’ and ‘bad’ code. (look in The Art of Programming, Knuth)

A quine: A computer program that displays its own code. Techno-Galactic Guide to Software Observation

Shorter and cleaner code / one line C program a one line neural network

Software as a Service (SaaS) From free software to proprietary software to software on servers, which doesn’t even hand out the executable. The risk of this server software is of course that it gathers data and leaves backdoors for the software owner to change the software. This means it is equal to spywhere as pointed out by Richard Stallman. http://bostonreview.net/richard-stallman-free-software-DRM

Sketching on an iPod

What is poetics?

On the one hand poetics describes a textual genre, on the other hand poetics, in a broader sense, # can also describe an aesthetic quality.

Traditionally poetics describes the structured analysis of poems or literature, but recently the term poetics has also been used in relation to fields other than the textual genre, like for example computation.

“What I cannot create, I do not understand”, said the famous physicist Richard Feynman

For computation this applies also to code and software. Firstly code can be seen as a form of poetry. This means that the syntax of code itself can be considered as a form of poetry – including for instance the structure, the functioning and the execution. Secondly the “effect of code”, meaning the execution of program code, can hold poetic value. This doesn’t imply that the code generates a poem.(http://poeticcomputation.info/chapters/ch.1/) When looking at examples of software art, this poetic expression can take various forms.

The origin of the word poetics is Poïesis, which means to create and give form.

having an imaginative or sensitively emotional style of expression

What software tells Creating stories about software

deliveroo narrative: siri (gags)

The software narrative The digital medium offers new ways of telling stories. This becomes obvious not only due to different structures, like the form of the database as Lev Manovich points out, but also because of the different modes of intervention software takes in our life. (Database as a Symbolic Form - Manovich. (1999). Retrieved 8 February 2018, from http://www.mfj-online.org/journalPages/MFJ34/Manovich_Database_FrameSet.html) Furthermore the medium keeps evolving at inexorable speed and so does software, leaving space for new ways of how to tell and what to tell about computation.

That humans tend to anthropomorphize not only their surroundings but also computers and technology in general has been a well research topic among computer sciences. In addition to that humans have a vivid and diverse imagination about processes that are invisible. This includes computation. Often digital media black boxes certain processes and therefore provides a lot of space for imagination and narratives that can be constructed around it.

Narratives have been used for the purpose of marketing and there have been attempts to create relatable stories within applications. The need for a human approach to software becomes also visible from the emerging need for Graphical User Interfaces. The so called GUI, is not part of the traditional imaginary of computation, where commands are being filled in via a command line. But todays average user is only surrounded by software displayed via a “window”, encountering the terminal only by chance. Not only does the GUI simplify commands into buttons and mouse-actions, but also does it make software more human. A button that has a 3D effect (Software Studies → Button), the on/off function is displayed via a switch, the mouse transforms into a hand or the form that looks like a letter, which off course you fill in by pressing a pen symbol. So what we can see from this and many other real life examples is, that software itself anthropomorphizes, by it’s concrete functions (Maybe add more examples? Like naming of functions, software visuals/logos, etc.). On the other side, among others, “The media equation” had shown, that we as humans also consciously and unconsciously anthropomorphize computers. This also goes hand in hand with the imaginations that humans create around computation and specifically also software. A well known example is Joseph Weizenbaum’s Eliza, a digital application, that acted as a therapist, chatting with the user. This piece of software gave impressive proof of how humans anthropomorphize even simple digital applications. (Expressive Processing, p.27)

New poetics that software enables

The way how computation works enables new ways of expression. Therefore new poetic expressions can be revealed. Software creates its own infrastructures. The way software works, different pathways often become hidden or blurry, but when revealing / making visible these tracks, this can provide a quite strong insight.

Lem seems to hint that the ultimate drive of instrumental reason, of continually interrogating our world in the quest for answers, may only be madness: fantasies that we project onto the world in order to construct a story about truth, just as the characters in the novel must grapple with figures projected from their pasts.

The narrative of the tech device / sketching on an iPod

The realization, that humans project emotions onto digital devices & software, enables us to intentionally tell story using the digital medium. This enables the creator also to comment or question the used technique. As an experiment I wanted to think of a random digital object (can be soft- or hardware). I happened to have an old iPod classic at home, so this is gonna be my example for the following thought-experiment. So how can we construct narratives around this device and including the software in it? The iPod itself suggests a story already through its history. It is a rather deprecated device, which had once been substituted by the multi-functional smartphone. It’s future is uncertain, it’s basically obsolete. But still I have a certain appreciation for it’s interface, the scroll-wheel and its focus on some basic functions: playing music. Certainly one can relate to the story of the poor old device, that is lost and looking for a new use.

References

Arns, I. (2005). README , RUNME, EXECUTE_ME. In: O. Goriunova and A. Shulgin, ed., read_me: Software Art & Cultures, 1st ed. Aarhus University Press.

Cramer, F. (2002a). Contextualising Software Art. [pdf] Cramer.pleintekst.nl. Available at: http://cramer.pleintekst.nl/all/conceptnotationssoftwareart/softwaredecontextualizaton.pdf [Accessed 6 Dec. 2018].

Cramer, F. (2002). Concepts, Notations, Software, Art. [online] Cramer.pleintekst.nl. Available at: http://cramer.pleintekst.nl/all/conceptnotationssoftwareart/conceptsnotationssoftwareart.html [Accessed 6 Dec. 2018].

Cramer, F. (2003). Exe.cut[up]able statements: the Insistence of Code. in Stocker G. & Schöpf C. (eds.), Code – The Language of our time, Linz: Hatje Cantz, pp. 98-103

Cox, G. and McLean, A. (2013). Speaking code. Cambridge, Mass.: The MIT Press.

Cox, G. (2007). Generator: The Value of Software Art. In Rugg, J., & Sedgwick, M. (ed.) Issues in Curating Contemporary Art and Performance. Intellect Books, pp. 147-162.

Galanter, P. (2003). What is Generative Art?. [pdf] Available at: https://www.philipgalanter.com/downloads/ga2003_paper.pdf [Accessed 6 Dec. 2018].

Yuill S. (2004). Code Art Brutalism: Low-Level Systems and Simple Programs in Goriunova O. and Shulgin A. (ed.) Read_me: Software Art and Cultures, Aarhus:Digital Aesthetics Research Centre.

User:Alexander Roidl/more chapter

Contents