[February 12, 2002: updates pending...]

 Hardware

for connecting the physical world and the computer:

• Purpose-built sensors-to-MIDI units
• Microcontrollers
• Sensors

Software

for real-time DSP, and interactive composition and performance:

• Macintosh

Institutions researching live, interactive and physical sensing applications.

My involvement in this: the "funny-fiddle" project.

Performing the studio: "liveness" and aesthetics in the digital studio.

Disclaimer

Purpose-built units for turning physical-world sensor data into MIDI:

MIDI Brain Low-cost self-assembly kit for sensor-to-MIDI conversion.

AtoMIC (via "Products and Services"... "Software"...) Flexible and configurable converter box from IRCAM. Has 32 x 8-bit analog-to-digital, 8 digital in and out lines, 4 MIDI-out ports. ~$800

I-Cube from Infusion Systems. Popular and powerful, the only unit I've found to offer multiple 12-bit ADC channels, along with digital I/O lines, flexible MIDI, and control objects for MAX. I'm told it's not so rugged, but at $615 nor is it a throwaway item. Infusion also sell a wide range of (pricey) sensors ready to plug and play.

Sensorlab from STEIM. Multiple a-to-d channels (8-bit), digital in/out lines, drivers for ultrasonic distance measurement. Rugged, designed to be worn on stage, with a single light tether cord to its power supply and your MIDI gear. Its own scripting language on the Mac.

Commercial "control surfaces", fader boxes, drum triggers: a popular starting point is to modify the Peavey PC1600x as it has multiple analogue faders, switches and flexible MIDI options.

JL Cooper make various control devices. The FM3 "Faderbaby" (discontinued?) is a $99 miniature fader box that can be hand-held. I've modified one to add two joysticks.

The ADB I/O from Beehive Technologies Inc. has 4 x 8-bit ADC and 4 relay outputs, and connects to the Apple Desktop Bus (ie not directly MIDI). Obsolete, but may still be obtainable (Beehive is no more). Drivers for MAX, Applescript, Director 7 and others. Low sample rates only. ~$200

Purpose-built units abused, requiring more hardware/electronics intervention:

Redundant synthesizers / cheap new computer-music keyboards ($0 - $100): for applications that require multiple switches rather than analogue conversion, remove the "brain" - often just a small circuit board - from a MIDI keyboard. It may also offer a few analogue channels. Keyboards with no velocity sensing are easier to wire, just needing a push-to-make switch (and maybe a diode per switch, to replicate the logic of the keyboard matrix). Velocity-sensing keyboards are trickier: need break-before-make changeover switches, with a suitable transition time, but potentially rich results. With a small amount of external electronics, can do pulse-duration measurement.

Redundant computer keyboard (<$10): as above - unsolder the switches, decipher the key matrix, saw off the rest of the PCB from the "brain", re-use the switches in the target application. Since this will probably be daisy-chained with the regular keyboard (eg on Macintosh ADB) it's best to wire switch positions that will not interfere with your normal keyboard activity during the development-cycle. A nice side-effect is that you can test the "responding" software by pressing keys on the regular keyboard.

back to top

Microcontrollers

If your sensor data will be processed in the computer before being output to synthesizers (or video control or whatever), the data doesn't have to arrive as MIDI. Microcontrollers, while not specifically MIDI-oriented, are tiny self-contained computers that typically offer digital I/O lines, often ADC or DAC, serial interface to communicate with host (or used free-standing) and non-volatile program storage. Ubiquitous in industry, washing mashines, etc. they're cheap ($30 and up) and small, so you build them into projects without needing to re-use. Often good price breaks on quantity purchase.

Programming requirements vary, as do the learning curve and investment in development tools. Some need significant hardware and software ability; others don't. If you're in an academic institution, talk to your engineering/physics/computing colleagues about their experience with "imbedded systems".

A selection of the many types available:

PicStic, DOMINO and AnswerMAN from Micromint.

AnswerMAN Junior: serial to 57600 bps, 4x8 bit ADC, 8 bidirectional digital lines. Senior version adds 2x12 bit ADC/DAC and better serial line drivers. Looks promising for the non-programmer - communication is by "query", "set" and "reply" strings of ASCII characters. Should be easy via MAX serial object. From $49.

The PicStic comes in various flavours; one is pin- and code- compatible with the BASIC Stamp 1, but gives access to interrupts and timer. Micromint claim their BASIC is 15 times faster (because compiled rather than interpreted); you can also use assembler, C, or a mixture. The compiler provides serial routines up to 9600 bps - my guess is you could code MIDI-speed routines. Variants include 2x12 bit ADC and realtime clock/calender. Flexible, but investment in developer environment. Physically tiny. Cost from $29.

The DOMINO is a general microcontroller running a floating-point BASIC interpreter (which also allows assembly language routines). Serial interface to 19200 bps, 2x12 bit ADC, I2C bus. From $99.

back to top

The following are outline notes only. More critical assessment in due course...

Basic Stamp from Parallax Inc.

Popular with non-specialist-engineers; runs interpreted tokenized BASIC but no assembler; B-Stamp 2 runs ~4000 instructions/sec; PC dev environment, or Mac running Soft PC or Soft Windows 2.0; PIC16C56 (BS1); PIC16C57 (BS2); timer is used internally; no external interrupts; BS2 can hold 5-600 lines of PBASIC2 code; efficient IO commands; 16-bit integer arithmetic; BS2 "can handle MIDI serial speed"; comparing BS2 schematic & PIC16C57 data sheet indicates serial port (eg for MIDI) is handled by software on general-purpose IO pins (so nothing else happens while serial bytes are being sent). Can be a tidy solution if you don't have a huge volume of data throughput.

BASIC Tiger / Tiny Tiger from Wilke Technology GmbH

Modules, look to have a lot of power for the price; Multitasking - useful or a liability? 12-bit ADC "with oversampling"; Windows dev environment; TT from $59 - dev kit $149; BT from $89 - dev kit $299

BasicX by NetMedia Inc.

Single chip, like core of BASIC Tiger; multitasking; cheap - dev kit $50 - includes chip, etc; single chip $40, $20 for 10-off; PC dev environment, needs parallel port; has "two high speed bidirectional buffered serial ports"; 65,000 lines of code per sec.; small progs don't need external EEPROM or RAM

RP3

Larger circuit board; Nice range of features but $400 for RPC-330; BASIC, C, assembler; library of drivers; 8x12bit adc, serials, 2 fast counters, etc.

FIRECARD from Fire, Wind & Rain

Pcb, but small; good features, 8x12bit adc, serials, floating-point BASIC etc; flash memory cards; price? serial speed?

AVR series from ATMEL

Powerful RISC-based chips, with wide range of on-chip peripherals. Heavy-duty data sheets and development environmnet. Not for the faint hearted?

 

Some Macintosh-based development tools can be found at MacRobotics and at Francis J. Deck's site (for PIC programming); DesignWorks from Capilano Computing (circuit schematic capture); and Douglas Electronics for PCB design. PCB prototype manufacturers include Douglas, AGCO Printed Circuitry Inc., ECD PCB Express and Alberta Printed Circuits Ltd.. Newark Electronics, Jameco Electronics and Digi-Key are suppliers of electronics parts in the USA.

I found the magazine Circuit Cellar INK a useful source here (in its printed form). Thanks also to colleagues on the MAX-list.

back to top

Sensors

A directory of sources for input technologies prepared by Bill Buxton.

I-Cube ready-rolled.

IRCAM - see below under Institutions

Robotics sites are a good source of sensor ideas - try Acroname.

Sensors designed for home security, such as pressure mats and motion detectors, are available from many mail-order electronics suppliers, Radio Shack, etc.

Max/MSP is "an interactive real-time graphic programming environment for multimedia," a toolkit for building anything from quick-and-dirty MIDI patchers to complex, polished DSP systems. Max/MSP is widely used by performers, composers, sound designers, researchers and educators. Smooth handling of MIDI data is central to Max (a non-audio version is available), but it can do much more. You can also build license-free standalone applications.

Max/MSP users maintain (a) strong community spirit(s). The software's makers Cycling '74 actively support this with a community page, including links to discussion lists and an extensive Max/MSP Annotated Resource Guide. (There is some overlap here with the material of this page.)

Highlight include a set of tutorials by Peter Elsea.

UnMAX maintains a download resource of Max/MSP/nato objects, patches and add-ons.

Extra MAX patches Useful add-ons. IRCAM's site includes excellent free/shareware MAX objects to add AIFF file playback, speech control etc. to the basic MAX environment. (A good half-way step to the non-free MSP...)

For video manipulation, nato.0+55 is a powerful and evolving extension to the Max environment. The author has a unique net identity/image.

SuperCollider "A real time audio synthesis programming language." Not for the beginner: a C++/SmallTalk -like text based language. But very slick, very fast; clean audio (eg anti-aliased oscillators, sorely lacking from many r-t synthesis applications); easy graphic interface building and MIDI mapping; and the new version multitasks with other application (including MIDI and audio). Program development is fast when you get used to it.

Also check STEIM's real-time performance software.

back to top

The MIT Media Lab is a high-profile, high-budget academic research institute. Cutting-edge R&D in cognitive, perceptual, algorithmic etc. engineering seems to take precedence over aesthetic production.

The University of Surrey, UK, had recently established the Institute for New Media Performance Research within its School of Performing Arts. The focus is on making collaborative and interactive music/dance/theatre, supported by a framework of academic critique.

IRCAM in Paris has an excellent site on Gesture Research in Music, run by Marcelo M. Wanderley. This draws together research and resources across the aesthetic-technological-critical spectrum.

STEIM The STudio for Electro Instrumental Music in Amsterdam, the Netherlands. New (art-)music performance technologies, R&D, artist residencies, performances. They sell live-performance oriented software for sampling, video, MIDI.

back to top

This material was drawn together as part of an ongoing personal project. I have developed a violin-like control instrument, putting a live solo performer at the focus of the electronic sound-world. I sought to draw on my training as a violinist, without the sounds produced having any necessary "string" component. (This is a problem for me with the instrument-plus-sensors approach of, for example, the Zeta MIDI violin.) Violin-like gestural activity is translated not only into what sounds happen when, but also where in the performance space they sound. Present development of the instrument is focused on deepening its sensitivity to the control of micro-gesture that a classical training yields. (It'll get four strings, too.)

The compositional challenge is to make a rich musical work that integrates vitally with the performance showmanship. A first work for the instrument, Gipsy fugue (1996-7), was very well received in its two concert performances at UEA and was featured in a network television documentary for the Open University.

back to top

This discussion of tools has implicitly focused on situations where live performance is "expected" or is the foreground activity. Bringing these tools into the computer studio opens a space of speculative activity informed by physical gesture.

Here is a small practical example: many digital mixing desks send out MIDI for movements of their faders, knobs and push-buttons. If you can temporarily spare the mix channels, use the desk as a generalized control surface. Map the faders etc to parameters of a software or hardware sound-processor or synthesizer (not just level, pan, EQ). Improvise sonic trajectories. Listen in real time. What happens when you think about the mapping of controllers to process parameters? What happens if you think about the mapping of physical gesture to sonic result (and ignore the process parameters)? Rehearse, vary, riff, counterpoint?

An exploration of the aesthetic inflections, transplantations and transgressions implied here is the rich subject of research, inside and outside the academy, by practitioners and by theorists. Here, in keeping with the spirit of this page, I offer the practical challenge.

I have no financial interest in any of the companies mentioned (other than my own search for cost-effective tools). Some of these tools I've used, some I scanned manufacturer's data sheets, some may be hear-say. Prices and specs may change, etc., etc. Suggestions that involve "engineering" assume you have enough knowledge to look after your own and your equipment's physical safety. Any use of this information or of the products described is entirely at your own risk.

Please help me refine and expand this information. Email your responses.

back to top