23 Apr '21
A short list of Extempore livecoding tricks
Yesterday’s LENS class discussion turned into an AMA of how I do certain things when I’m livecoding in Extempore. As promised, here’s a blog post where I’ve put together all of the things we’ve discussed (with a bit more explanation). If you’re in the LENS ‘21 class this’ll hopefully be a helpful complement to yesterday’s class discussion. If you’re not in the class, then maybe you’ve always been curious about certain things I (over)use in my livecoding sets? This is maybe a bit too niche to go in the general Extempore documentation, but if you’ve got any questions then you can hit me up on Teams (for LENS students) or the Extempore mailing list (everyone else).
table of contents
- adding instruments to the sharedsystem setup
- markov chains
- weighted random selections from a list
relfor relative pitches
nofand the macros vs functions distinction
- quasiquote (
`) vs regular quote (
- modulating filter (or other) params over time
- playing a loop which then stops
- multi-laptop Extempore clock sync
- Selecting the correct audio device
- Audio underflow: are you pushing extempore too hard?
- Creating a “startup” file
cosr is just a convenient way of sinusoidally modulating something with a
period synced to the tempo.
When it came to writing up this part of yesterday’s discussion I had déjà vu—I was sure I’d written this stuff up elsewhere. Turns out I have—it’s in the main Extempore docs as part of the pattern guide.
One other tip which I mentioned yesterday: if you use fractions for the period
(final argument) which have a co-prime numerator & denominator (e.g. 7/3 or
15/8) then you’ll get interesting accenting patterns, but which will still
repeat (so it gives a bit more structure than just using
One other related trick that I use is doing “mod” calculations on the
variable (which is an explicit argument in most temporal recursions), but is
also implicitly bound in the pattern expression if you’re using the pattern language:
(if (= (modulo beat 2) 0) (println 'downbeat)) (if (= (modulo beat 2) 1) (println 'upbeat))
adding instruments to the sharedsystem setup
The default sharedsystem instruments are defined near the top of the
(make-instrument syn1 analogue) (make-instrument syn2 analogue) (make-instrument syn3 analogue) (make-instrument kit dlogue) (make-instrument samp1 sampler)
If you want to use different instruments, you can add them to the various DSP
dsp5 (this multi-DSP-function setup is to allow the audio
engine’s work to be distributed across multiple cores).
In each of the DSP functions (e.g.
dsp1), the actual work of getting the
“signal” out of the relevant instrument happens in a line like this:
(set! out (syn1 in time chan dat))
To add e.g. an
fmsynth to the sharedsystem setup, there are two steps:
fmsynthinstrument somewhere with
(make-instrument fmsynth fmsynth)
fmsynthfunction in one of your DSP functions (doesn’t matter which one—they all get summed in the end) callbacks and make sure the return value is added to one of the
Here’s an example (in
dsp1 as per the previous example):
(set! out (+ (syn1 in time chan dat) (fmsynth in time chan dat)))
Then, any notes you play (using
play) on the
fmsynth instrument will make
their way into the audio output.
As discussed yesterday, Scheme (most lisps, really) makes it pretty easy to represent markov chains.
Consider the following temporal recursion, which (in addition to the usual
dur arguments) also takes a
pitch argument. This is the exact
code I wrote yesterday:
(define (obi-lead beat dur) pitch (play samp1 pitch (cosr 75 20 5/3) dur 2) (callback (*metro* (+ beat (* .5 dur))) 'obi-lead (+ beat dur) dur (random (cdr (assoc pitch '((60 60 63 67) (63 67 60) (67 58) (58 60))))))) (obi-lead (*metro* 'get-beat 4) 1/4 60)
The markov chain happens in that
(random (cdr (assoc ...))) line at the end of
the definition of the
obi-lead function. A couple of things to note:
cdris only necessary to make sure that a “self-transition” (i.e. the pitch staying the same) only happens if you explicitly add the same pitch as an element other than the first element of the list. Otherwise, you wouldn’t be able to make a markof chain that always went to a different pitch from the current one.
randomcall isn’t a special “markov” random, it’s just the normal
randompicking from a list of pitches (so you can use the same “use the same element multiple times for weighted sampling” trick mentioned below)
you can do whatever you want with the
pitchargument inside the body of the
obi-leadfunction—the fact that we
playit here is a common pattern, but you could e.g. fire out an OSC message, etc.
this is just a specific example of the more general class of algorithms which use a temporal recursion to pass variables to subsequent callbacks, and potentially do tests/operations on said variables to change them for future callbacks
the onus is on the livecoder to make sure that each possible state (i.e. each value that
pitchcan take) is represented as the head (
cdr) of one of the lists. But as long as you satisfy that invariant then the
obi-leadmarkov process will just keep on playing the
pitchas it travels through time and space1.
weighted random selections from a list
As promised (Caleb!) here’s the syntax for doing a weighted random sample: the
key is that the argument to
random isn’t a list of
cons pairs, it’s multiple
cons pair arguments, each one of the form
(cons WEIGHTING VALUE).
;; here's an example: note that it's in a "do 10 times" loop to show that the ;; values are indeed sampled using the appropriate weighting (dotimes (i 10) (println 'i: i (random (cons 0.1 1) (cons 0.9 -1)))) ;; printed output: ;; ;; i: 0 -1 ;; i: 1 -1 ;; i: 2 -1 ;; i: 3 -1 ;; i: 4 1 ;; i: 5 1 ;; i: 6 -1 ;; i: 7 -1 ;; i: 8 -1 ;; i: 9 -1
Note that if you’re happy with just rough “this value is twice/three times as likely as the others” then it’s usually simpler to just randomly sample (equally-weighted) from a list, including duplicate values for the things you want to turn up more often in the output:
;; 0 will be twice as likely as 3 or 7 (random (list 0 0 3 7))
rel for relative pitches
From here on, all the following code snippets assume you’ve loaded the pattern language with
Note that the pattern language library is loaded when you load the sharedsystem as well.
Based on the (global)
*scale* variable, you can use
rel to calculate pitches
“relative to” a starting pitch.
So, if you’re starting with a middle C (midi note
60) and you want to go
notes up the scale (and you haven’t changed value of the
*scale* variable from
the default “C natural minor” scale) then you can use:
(rel 60 2)
This can be handy when paired with the
range function (which just generates
lists of integers) for running your scales:
(:> scale-runner 4 0 (play samp1 (rel 60 @1) 80 dur) (range 8))
Note that this is just a slightly terser version of
libs/core/pc_ivl.xtm) function, which doesn’t use the
*scale* variable by
default. Note further that
rel takes an optional third argument for
providing a different scale, if e.g. you want to use a different scale for your
“relative pitch” calculation than you’re currently using elsewhere in the piece.
Here’s an example:
;; set scale to F natural minor (set! *scale* (pc:scale 5 'aeolian)) ;; use a Fm7 chord (a subset of F natural minor) for the relative pitch calculation (rel 65 (random 4) (pc:chord 5 '-7))
println the output of that
(pc:chord 5 '-7) (Fm7) function, you’ll
get the result
(5 8 0 3), which corresponds to the following pitch classes:
which are the pitches from an Fm7 chord, so it all checks out.
As one final tip, you can just skip the call to
pc:chord altogether and do
(rel 65 (random 4) '(5 8 0 3))
This makes it super-easy to make quick edits, e.g. if you want to flatten the
fifth (C). But it does make it a little less readable for the audience (and
let’s face it, reading
(pc:chord 5 '-7) was already pretty tough going for
most folks outside the music theory ∩ computer programmer intersection).
nof and the macros vs functions distinction
nof (think “give me n of these”) is a Scheme macro for creating a list by
repeatedly evaluating a form.
So, one way to get a list of 10
(nof 10 0)
To get a list of 4 random integers between
9 (inclusive) you could use:
(nof 4 (random 10))
I just evaluated the above form on my machine; the result was
(0 9 9 8). Note
that the numbers are different; so the
nof macro is obviously not just taking
the result of a single call to
(random 10) and repeating it
4 times to
create a list.
This is where the fact that it’s a macro—not a function—comes into play.
One trick for looking at what a macro form “macroexpands” out to is calling
macro-expand (note that the
nof form has been quoted using
(println (macro-expand '(nof 4 (random 10)))) ;; prints: ;; (make-list-with-proc 4 (lambda (idx) (random 10)))
So, the list is actually formed by four repeated calls to a
function, and that’s why the four random numbers in the above list are different.
If this isn’t actually the behaviour you want—if you want the same random
number repeated four times, there’s a
repeat function which you probably want
to use instead. Notice the difference:
(println (nof 4 (random 10))) (println (repeat 4 (random 10))) ;; prints: ;; nof: (1 3 3 6) ;; repeat: (5 5 5 5)
So keep that in mind when you’re using
nof in your pattern language. It’ll
probably just work, but this subtle macro vs functino thing may be the cause of
errors or weird behaviour that you see.
`) vs regular quote (
The quasiquote (
`) symbol (which is also called the tilde) is like the
normal quote operator (
'), except that you can “undo” the quoting (i.e. eval)
inner forms as necessary using the unquote operator (
That’s not easy to get your head around when explained in words, but here’s an example:
;; this is kindof tedious to write '(c3 | | | | | d3 e3) ;; so instead we write `(c3 ,@(nof 5 '|) d3 e3) ;; don't forget the @ (splicing) part; this is probably not what you want... `(c3 ,(nof 5 '|) d3 e3) ;; result: (c3 (| | | | |) d3 e3)
modulating filter (or other) params over time
One way to do it is show in the example file
examples/sharedsystem/analogue_synth_basics.xtm. Have a look around line 39,
where it says:
;; and now add a second pattern to 'sweep' the filter (:> B 4 0 (set_filter_env syn1 40.0 100.0 (trir 0.0 1.0 1/32) 100.0) (nof 16 0))
playing a loop which then stops
The pattern language isn’t really designed for playing one (or two, or three, or n for n < ∞) shot loops. It’s really designed for things which will keep on looping until you stop the pattern.
If you want to play a sequence of events which runs for a while and then stops, then using a standard temporal recursion is probably best.
Here are a couple of examples. First, this recursion will keep playing the notes
pitch argument gets to
(define (ascending-chromatic beat dur pitch) (play syn1 pitch 80 dur) (if (< pitch 72) ;; note the `callback` is inside the `if` (callback (*metro* (+ beat (* .5 dur))) 'ascending-chromatic (+ beat dur) dur ;; pitch gets incremented by 1 in each subsequent callback (+ pitch 1)))) ;; kick it off - note that we're passing the initial pitch argument (ascending-chromatic (*metro* 'get-beat 4) 1/4 60)
Now, both the “increment” part
(+ pitch 1) and the “stopping criteria” part
(if (< pitch 72) ...) are both just Scheme code, so it’s very
flexible. You can handle those things however you like.
One other approach to doing this is a recursion approach straight out of the
functional programming handbook2, just with a temporal twist.
The key idea: start with a list, recurring on the
cdr (i.e. the tail of the
list) each time until it’s empty, then stopping.
So, here’s a way of playing a scale—ascending, then descending—then stopping.
(define (ascending-descending-scale beat dur plist) ;; note the (car plist) since plist is a list, not a number (play syn1 (car plist) 80 dur) (if (not (null? plist)) (callback (*metro* (+ beat (* .5 dur))) 'ascending-descending-scale (+ beat dur) dur (cdr plist)))) (ascending-descending-scale (*metro* 'get-beat 4) 1/4 '(60 62 64 65 67 69 71 72 71 69 67 65 64 62 60))
There are a couple of nice variations on this one:
(rotate plist -1)to make it cycle through the pitches, but then go back to the start (have a think about what
rotatedoes to convince yourself that this works)
you can re-trigger this temporal recursion several times, either with the same
plist, or even with different
plists—since all the temporal callback chains will be independent they’ll all just run nicely over the top of one another, which can lead to some interesting musical “layerings”
multi-laptop Extempore clock sync
If you’re jamming with another Extempore laptop musician, you often want to make sure your tempos & metronomes sync up. One way to do this is to use the topclock protocol (the name comes from TOPLAP). You’ll need a network connection between all the machines—wired is best if you can manage it, or at least on a private-ish wifi LAN (you’ll probably have a bad time trying to do it over ANU Secure).
There’s an example in
examples/core/topclock_metro.xtm which will show you the details.
The alternative (manual) way to sync up two laptops is to:
make sure you’re both using the same tempo:
(*metro* 'set-tempo 140) ;; or whatever tempo you like
both starting “playing time”, i.e. something simple which is easy to listen to and feel the tempo
listening carefully, try and figure out if you’re rushing or dragging, and then executing the appropriate
;; if you're rushing (*metro* 'pull) ;; if you're dragggin (*metro* 'push)
Pros with this approach: no network connection required, and it’s usually not too tricky to get the timing close enough to jam together. Cons: it’s pretty manual (and requires some careful listening, which is a skill that takes time to master), and it also doesn’t help with getting the exact beat clock synced up (so that you’ll still have to be careful that e.g. your 4-beat bars line up, and you may have to keep fiddling with some offsets to make it all work).
Selecting the correct audio device
We discussed this during the demo day class, but you should visit the doco
to see how to do it. The tip about using
--device-name instead of
is particularly good advice when you’re trying to bump in quickly in a gig
Audio underflow: are you pushing extempore too hard?
The sharedsystem is actually kindof heavyweight3 (or at least medium-weight) from a CPU use perspective. It loads up 4 analogue synths and a sampler, plus some FX (e.g. global convolution reverb) and tries to distribute them across multiple cores on your machine.
If you’re getting lots of “audio underflow” messages, you’ve got a few options, in order of easiest fixes to most difficult:
don’t run any other software on your machine that you don’t absolutely need for the gig
try starting Extempore with a larger frame size (e.g.
./extempore --frames 8192)
if you don’t need MIDI I/O, just load
(sys:load "examples/sharedsystem/audiosetup.xtm")instead of
remove some of the
N= 1..5) functions from the signal chain in
examples/sharedsystem/audiosetup.xtm(e.g. if you’re only using
syn2you could remove
dsp3from the signal chain, which uses
dspmtas the final output sink)
get your hands on a beefier laptop (not an option for many people, obviously)
If you go with option #4, remember that you can create a new copy of
audiosetup.xtm and modify it to your heart’s content. Even if you’ve just been
messing with the original
audiosetup.xtm file directly, remember that you can
get a “pristine” version at any time from
Creating a “startup” file
sys:load can load any file on your computer (even files outside your
extempore directory if you provide a full path). So it’s a good idea to put
everything required to set up your Extempore session into one file, then you
can just load that file with e.g.
and then once that’s loaded you’re good to go.
Extra tip: also memorise a line of code that you can quickly type & execute to check if there’s sound coming out.
I mean, it’s still supposed to work on a half-decent laptop—it shouldn’t require a real beast—but if you’re on a particularly old/wheezy machine then even with the below tricks the sharedsystem might not be a good choice. The other examples (e.g.
examples/core/fmsynth.xtm) show how to create a lighter-weight DSP chain, and from there you could add only the instruments & effects that you need. ↩