User:Riviera/18th international context meeting presentation

This text was prepared for a presentation I gave at the 18th International ConTeXt Meeting which took place in Lutten, Overijssel in late August 2024. Accompanying slides are available at: https://meeting.contextgarden.net/2024/talks/riviera/slides.pdf

Introduction

I will begin by speaking a bit about the live coding software known as Tidal Cycles. Then, I will improvise a piece of music with this software for about ten minutes. Afterwards, I will expand upon Open Sound Control (OSC), a protocol which Tidal Cycles implements in a specific way. Tidal Cycles is, at least in part, an extensible OSC client. By creating a new boot script and a new OSC server, Tidal Cycles can be configured to live code ConTeXt documents. I will conclude with a live performance that illustrates this.

What is Tidal Cycles?

Tidal Cycles is live coding software. It’s an actively developed, domain specific language which was written in Haskell, by Alex McLean initially, over a decade ago. Patterns, which McLean has described as “any function of time” are found throughout the software (McLean in mylarmelodies, 2024). In Tidal Cycles, time progresses in cycles, as the name suggests. By default the software sends a stream of data to Superdirt, which is a Supercollider extension, via the OSC protocol. To do so, it utilises an OSC library written in Haskell. For example, when the following piece of code is evaluated, Tidal Cycles instructs Superdirt to play a sequence of eight bass drum samples each cycle. There is an OSC message sent for each eighth of a cycle.

d1 $ n (run 8)
  # sound "bd"

SuperDirt ships with over 200 sample packs and several synthesisers that Tidal Cycles can sequence or play. The patterns may sound like four-to-the-floor beats or mix randomness and predictability to form a satisfying series of sonic permutations. Overall, Tidal Cycles has a broad, extensible and developing user interface. There are many comments throughout the source code which support people like me to better understand the software.

Live Coding Audio with Tidal Cycles

I’ve spoken briefly about Tidal Cycles, software which I have been using for approximately two years. For the next ten minutes, I will perform live coding with the software and produce some semi-improvised algorithmic music.

Tidal In The Browser

If you’re curious to try Tidal Cycles I encourage you to have a look at Strudel, Flok or Estuary. These websites are similar insofar that they permit visitors to live code within the browser. There’s no need to create an account on the websites. On the one hand, Flok and Estuary make setting up a real-time, collaborative, networked performance very simple. Strudel, on the other hand, implements Tidal Cycles in JavaScript.

Open Sound Control (OSC)

OSC is a protocol which Tidal Cycles relies upon and implements in a particular way. The specification was designed by Matt Wright and Adrian Freed around the late 90s / early 2000s. The intention was to aide communication between pieces of technology in musical performances. In their words, “Open SoundControl is an open, efficient, transport-independent, message-based protocol developed for communication among computers, sound synthesizers, and other multimedia devices” (Freed and Wright, 1997). OSC libraries are available in programming languages such as Lua, Haskell, Python, Common Lisp and Rust to name a few. The essential data types supported include ASCII strings, 32-bit floats and integers and pieces of binary data. The specification also details how additional data types (including True, False and 64-bit numbers) may be encoded by clients and decoded by servers.

The composition of OSC messages

OSC messages can be divided into a few different parts. Here’s an example of an OSC message which was sent from Tidal Cycles to SuperDirt and captured with Wireshark:

Message: /dirt/play ,sssfsfsfsfsiss
    Header
    String: _id_
    String: 1
    String: cps
    Float: 0.666667
    String: cycle
    Float: 581.25
    String: delta
    Float: 0.1875
    String: n
    Float: 2
    String: orbit
    Int32: 0
    String: s
    String: bd

The message begins with an address which looks like a Uniform Resource Indicator. OSC’s address space is open and so messages can be addressed to anywhere. After the address there is a string which details the types of the arguments which follow. In this case there are 14 arguments which form the third part of the message.

Repurposing OSC

As I mentioned, Tidal Cycles sends OSC messages to SuperDirt by default. With live, musical performance as its objective, this client-server relationship is a great example of a software-based, OSC application. By contrast, writing ConTeXt documents with Tidal Cycles deviates from the basic intention of OSC. As such, two things are foregrounded:
- Tidal’s OSC client capabilities
- OSC’s ability to send typed data across a network in real-time.

In what follows I will outline some of the issues, conflicts, quirks and limitations that crop up when attempting to write ConTeXt documents with live coding software.

Let’s begin by replacing SuperDirt with a Python script. Importing the osc4py3 library allows for creating an OSC server that listens on a UDP port. Likewise, Tidal Cycles is instructed to send OSC messages to this target. The Python script is responsible for writing the ConTeXt document. The server is configured to closely reflect the client. The addresses must match up on both sides of the application. Also, all the arguments require unpacking on the server side in the same order in which they were supplied by the client. To illustrate what this means in practice, let’s examine how various commands might be implemented and the challenges which arise as such.

Implementing \blackrules (Step 1)

let inheritsSetupblackrules = [("width", Just $ VF 1),
                               ("height", Just $ VF 1),
                               ("depth", Just $ VF 0),
                               ("distance", Just $ VF 0),
                               ("n", Just $ VI 3),
                               ("alternative", Just $ VS "a"),
                               ("style", Just $ VS "STYLE COMMAND"),
                               ("color", Just $ VS "black"),
                               ("variety", Just $ VS "yes"),
                               -- alias for ("type", Just $ VS "yes")
                               ("mp", Just $ VS "NAME")
                               ]
                               
let blackrulesMessageFormat = [OSC "/blackrules" $ ArgList inheritsSetupblackrules]

\blackrules is a command which inherits ten options from \setupblackrules. This piece of Haskell code, which assigns a list of pairs to a variable, amounts to a list of arguments. Each pair features an option in the form of a string and a value. The value acts as the default value for the given option. Tidal Cycles only sends an OSC message when all options are supplied a value. Therefore, providing defaults for every option automatically ensures the OSC message will be sent. In keeping with the OSC specification, the list of options and default values is associated with an OSC address. Notably, each value has a type corresponding to the data types supported by OSC. Tidal Cycles implements five OSC data types via five constructors, as illustrated in the table.

(OSC) Type	Value constructor
String	VS
Integer	VI
Float	VF
Boolean	VB
Binary “blob”	VX

Implementing \blackrules (Step 2)

:{
    let width = pF "width" 
        height = pF "height"
        depth = pF "depth"
        distance = pF "distance"
        n = pI "n"
        alternative = pS "alternative"
        style = pS "style"
        color = pS "color"
        variety = pS "variety"
        mp = pS "mp"
:}

Next, Tidal Cycles must be informed about the type of value that each parameter expects. For each parameter, a variable is declared which makes this explicit. Consequently, if an integer is passed to an option which expects a string during a live coding session, the interpreter will complain.

Implementing \blackrules (Step 3)

let setupblackrulesMap = [(contextTarget, setupblackrulesMessageFormat)]

setupblackrulesStream <- startStream (defaultConfig) setupblackrulesMap
blackrules = streamReplace setupblackrulesStream 1

Finally, the message format is paired with the target and stored in a variable. This information is used to create a stream of data which is ultimately accessed via the blackrules function when live coding. There were one or two issues which I encountered when implementing this ConTeXt command in the OSC application. The first issue was that \blackrules has an option called type which is a built-in function in Haskell. I was hesitant to redefine this built-in during step two. Haskell might allow it, although ultimately I did not attempt this. Instead, I used a variable named variety. Secondly, I could not figure out how to pass a value containing a backslash to an option when live coding. The implication being that dimensions for options such as “width” are hard-coded in the python application.

Implementing \framed

Let’s now consider \framed, a complex command which inherits 50 options from \setupframed. Some options to framed, such as corner, framecorner and backgroundcorner can accept values such as “round”, “rectangular”, or any number between zero and 33. Also, backgroundoffset can accept “frame” or a dimension. Likewise both the width and height option can accept a dimension or “fit”. Recall that in step two Tidal Cycles was told about the type of data that each parameter could work with. Given that each parameter can only work with one type of data, these options pose a clear problem.

Conclusion

I’ve offered a three-step overview of how Tidal Cycles works with OSC. I illustrated this process by discussing issues which arose when implementing the \blackrules and \framed commands. Ultimately, parameters in Tidal Cycles capable of accepting multifarious data are not available by default. Perhaps this is a good first issue that I could put forward to other Tidal Cycles users. If so it would be beneficial for me to find an example use-case which would justify implementing such functionality in keeping with the primary purpose of Tidal Cycles. After all, Tidal Cycles was designed for making music, not for typesetting documents. On the other hand, perhaps what is needed is a different protocol, one designed with TeX data types in mind. Such a protocol could then be implemented in purpose-built, live coding software written in Lua. In the meanwhile, I’d like to illustrate what live coding a context document with Tidal Cycles looks like through the medium of performance.

Bibliography

mylarmelodies (2024) ‘How to make music for free, with code: Why We Bleep podcast with ALGORAVE’, [Online]. Available at https://www.youtube.com/watch?v=NUgJgCvX4Y4 (Accessed 9 August 2024).

Wright, M. and Freed, A. (1997) ‘Open SoundControl: A New Protocol for Communicating with Sound Synthesizers’.