This isn’t really too much of a stretch. Audio unit generators won’t be built into the core language, but instead will be stored as separate modules and imported as needed; The same would be true for visual modules. Musical GUI elements such as sliders are already visual in nature. There are existing audio/visual systems out in the wild, such as Max/MSP/Jitter and Impromptu. Various functions and objects designed for music could also be applied to visuals; The same envelope that controls frequency can be used to control the position of a circle:

t = 10                          # time = 10 beats
my_circle = Circle(0, 200, 15)  # x position, y position, radius
env = line(0, t, 1)             # start value, duration, end value
sine(1.0, 440 * env)            # amp, frequency
my_circle.xpos(env * 400)       # x position over time

The example is a bit crude and omits many practical things, but the idea of syncing visuals with audio is there.

Beyond music and visuals, anything that is time-based would theoretically work in slipmat, providing a module is written.

This is a demonstration of the software i wrote for my MSc In Music Technology at DKIT under the supervision of Rory Walsh. The software is used to create custom multitouch user interfaces for controlling Csound Instruments. It allows users to define gui elements, such as sliders and buttons, in their csound file. The .csd file is parsed by CsMultitouch to retrieve this information. Said information is then used to create the multitouch interface using the PyMT framework.

I want this.

I just released a new Slipmat package at sourceforge. This latest version comes with three new examples, including one that uses a basic Java GUI. Four out of the five examples are now pre-rendered as CSDs for convenience. There are also a handful of new synth Modules to play with.

The documentation has been improved, including better Javadoc support. The Javadocs are not pre-rendered as to keep the size of the release to a minimum, so you’ll have to generate them yourself. Many IDEs, including NetBeans and Eclipse, will generate them for you.

There is also the PseudoTutorial example that gives a broad overview of the design of Slipmat and how to use it.

And in case you’re wondering, Slipmat is “A Java-based modular computer music library built on top of the Csound API.”