SamplerBox and Pure Data: Theory and Practice

Hello everyone,

for the past few months I have been (successfully) working on integrating SamplerBox with Pure Data, and after many conversations with Hans we decided to open up forum dedicated to the subject. Hopefully others find this intriguing, and are interested in trying it themselves, but first a quick introduction to Pure Data.

Pure Data ( https://puredata.info/ ) is an open source visual programming language developed by Miller Puckette, similar to Max/MSP (..which was also originally developed by Miller Puckette). It was developed with the idea of making programming accessible to artists and musicians, it is able to to process audio, control art installations, or build entire virtual synthesizers.

Once you understand the basics it's incredibly easy and fast to use (some musicians and performance artists use it for live coding, creating unique compositions and sounds on stage). Previously it was relegated to use on desktop computers or laptops, and was a fairly obscure tool, but as the raspberry pi has gotten more powerful it has experienced a bit of a resurgence and is showing up as the basis for many dedicated hardware instruments, such as standalone Synthesizers, Eurorack modules, and effects processors.

coding in Pure Data is very much like operating a modular synthesizer, or putting together a pedal board. You have individual objects (such as oscillators, math functions, logic gates, audio effects, etc etc) that you connect together using virtual patch cords. These patch cords can be changed and manipulated in real time, completely changing the program. For our purposes we wont be doing any sort of live code manipulation, but this makes it incredibly easy to develop and test our program.

a basic Pure Data file (called a Patch) looks like this:

   


there are lots of resources online for learning PD, as well as many examples (like the one above) included in the pure data help menu.

I found the examples in this tutorial to be particularly helpfull http://www.pd-tutorial.com/
there are also lots of incredible examples over at pure data patch repo but most are too complicated for the Pi to run
and patch storage has dedicated pages for pure data as well as several Pi and PD based instruments (those can get pretty hardware and OS specific and take some work to adapt)

Patch Storage: pd-vanilla
Patch Storage: ORAC
Patch Storage: Organelle
Patch Storage: Bella

Integration with SamplerBox

My initial idea with this was to add a non-sampled digital synthesizer voice to accompany the sampled instruments, making SamplerBox function something like the Prophet X by Sequential Instruments, but really the sky is the limit (or rather... CPU power is the limit) in what we could do, from basic audio processing to granular synthesis (you could directly record a few bars from SamplerBox and then using that as the source for granular synthesis, and have that play in tandem with the live audio from SamplerBox) or custom midi sequencers.

a side benefit of offloading audio effects from samplerbox to PD is that they are separate processes, and will run on separate cores, freeing up CPU resources for each, and as multiple instances of Pure Data can be opened at the same time (with as many audio inputs/outputs as you want, feeding into each other) we can further break down our individual audio processes for more efficient hardware usage. Using CPU heavy effects like Reverb directly in SamplerBox can cause issues with headroom and limit what it's capable of, but by offloading the effect to pure data we no longer have that problem, as each will operate on separate cores.

to create and test my Pure Data patches directly on the Raspberry Pi I've installed a desktop environment and am using a VNC to access it, but everything can be setup to boot and run "headless", so we wont need the desktop to run our pure data patches (I have yet to actually set this up though).

here are some videos of the setup working. These are a bit dated as I've been completely rebuilding the synth engine.

sampled DX7 and pure data based drum machine

piano from samplerbox and custom FM synth from pure data

a drum machine, sequencer, and BPM based dual delay made entirely in pure data

a closer look at the FM synth engine

to feed the audio from SamplerBox we are going to use a two audio cards, I'm using a HiFiberry clone as the default card for samplerbox, and a USB audio card with stereo audio in and out to be the default card for Pure Data (I'm using a behringer UCA202), we simply use RCA cables to feed the output of samplerbox into the USB audiocard. If you didn't care to process the audio from samplerbox itself you could use a setup with separate stereo outs for each program, as well as stereo in for pure data (making a device more akin to the organelle)

(for those following along this is an update to my previous method of using Jack Audio to internally route the audio from samplerbox to pure data, which had it's share of problems)

Future Development/Plans/wishes

Intercommunication and Menu System: I would like to get pure data and python directly talking to each other, namely I want samplerbox to send menu data as message to pure data, or the reverse, and relay those combined messages to a microcontroller and display. Theres a few ways to go about this but I've only just begun investigating. the existing pure data based hardware synths out there use OSC to talk to python and create a menu system, but instead of going to the trouble of implementing that I would rather use serial communication (the "comport" extension object in pure data) to the microcontroller,  and TCP/UDP (Netsend/netrecieve in PD and Sockets in Python), or somehow using this python extension in PD pyext, or have each send info to the microcontroller separately. Final and tedious possible method would be to assign each message a MIDI CC number and value, as I'm already able to connect everything to midi via Jack, but this would mean that any changes I make to the PD patch will have to be redefined 

oscilloscope: you may have noticed the oscilloscope in pure data, I find it very helpful in understanding how a sound is being changed/effected. theoretically I should be able to regularly dump the oscilloscope data as coordinates and plot them on a display (using serial plot on the arduino), I then would simply need to create an argument to switch to displaying the menu data whenever a midi CC control is changed


I would like to completely offboard the reverb effect, as it eats up too much CPU and could limit the possibility of polyphony in the synth engine. I've started working on adapting a circuit that uses 4 PT2399s (a very cheap digital delay IC that was made for adding delay/reverb effects to karaoke machines, I bought a tube of 12 of them for $24) to create a simulated spring/plate reverb effect. They run on 5v so it should be relatively simple to integrate, without any special power supply needs.

I'm also interested in adding an analog voltage controlled filter to the signal path (digital filters sound fine but there's nothing like the real thing), using these reproduction VCF ICs AS3320, and using a digital to analog converter to output a control voltage from the raspberry pi for an LFO and midi controls, but these require +-15V, so I would also need a switching power supply for that, and a step down converter and regulator for powering the pi such as this, which is getting a bit complicated and may be an idea reserved for a future project (the overly ambitious part of my brain is imagining a digital/analog hybrid synth, using PD as a multi-engine and CV source. you could apply digital FM to analog oscillators! or use wavetables for an LFO! or even have a touch screen for drawing waveforms! and that's just the CV output. anyways...)

--Bryan

Step by Step guide to installing Pure Data to the SamplerBox OS (Updated for the January 2022 image!)

I finally had the energy and motivation to start with my fresh build of this project!

first things first, you'll want a larger SD card for installing your OS, as we will need to install a lot more than Pure Data to get everything working, I'm using a 32gb card.



Start by installing a fresh copy of SamplerBox to your SD card, we will next need terminal access, either by connecting a keyboard and monitor to your Pi or using SSH (you'll want to use remote access later so you might as well start now, I'm using Terminus on Windows)


We need to modify our file system (we need to enlarge both the Samples partition as well as the root partition so that we can), this is easily done if you have a linux system or a second SD card with the standard Raspbian OS and a card reader handy, here I'm using gparted to resize the drives, notice that I've increased the root folder to 8+gb (should be more than enough), and gave the remaining 19gb to the samples directory, this can all be done from the terminal but I find the GUI helps me make sure I'm not screwing up



   


boot up your new samplerbox image and enter 'devel' into the terminal, this is development mode, which remounts the drives as read/write and stops samplerbox.py

enter:
sudo raspi-config

and go to localization, update the timezone and press finish

update the local time as directed by the prompt when you entered development mode, run

sudo apt-get update

return to raspi-config and under 'interface' enable VNC, this will take awhile

we still need to install a desktop environment before we can boot into vnc,
enter the lines below and while you're at it go ahead and install synaptic (it'll be useful soon)

Code:

sudo apt install xserver-xorg
sudo apt install xfce4 xfce4-terminal
sudo apt install synaptic

We should now be able to boot up and go to a desktop environment with VNC

in the next post we will compile the latest version of Pure Data from scratch (all the raspberry pi repositories are using a version from several years ago, and there have been some substantial improvements since then)

We can now access the desktop environment that we just installed via VNC.

on reboot access your Pi with your SSH client, we need to enter development mode again if we're to boot into the VNC desktop environment

now you can start the VNC client on your computer and you should be see something like this:


   

(note that you can see my SSH client in the background)

if you get an error saying the desktop cannot be displayed, try changing the default resolution using raspi-config. This is a common/known error with the Pi and VNC.

we basically have a full desktop, but a lot of your standard software isn't installed, but we did previously install synaptic, so adding whatever you want is now a breeze

you'll want to install a text/code editor (I've gone with emacs) and you might want a web browser (Firefox works well, chromium doesn't like that we're logged in as root) for times you're using a monitor instead of VNC

DO NOT install puredata with synaptic, none of the repositories carries the latest version (which has some major feature upgrades, like the 'clone' object that is handy for polyphonic synth voicing). Instead we are going to compile it manually;

(tutorial taken from here, https://forum.pdpatchrepo.info/topic/116...to-pd-51/2, but I'll include everything in this post)

either open a terminal window in the VNC viewer or use the SSH client, and enter;

Code:

mkdir src
cd src
sudo apt install build-essential autoconf automake libtool gettext git libasound2-dev libjack-jackd2-dev libfftw3-3 libfftw3-dev tcl tk
sudo wget http://msp.ucsd.edu/Software/pd-0.51-4.src.tar.gz
sudo tar -xzf pd-0.51-4.src.tar.gz
cd pd-0.51-4
sudo ./autogen.sh
sudo ./configure --enable-jack --enable-fftw
sudo make
sudo make install

(note: make takes awhile, maybe go grab a coffee)


you can now start pd!
you can just enter 'pd' into the terminal (on the vnc desktop, if you want to open it in the ssh terminal use the '-nogui' flag or try 'pd -h' to see the available boot flags) to see if everything worked, but for our purposes we want to have extra audio in and out ports so we want to boot pd using;

Code:

pd &

you can create a desktop launcher using the same command line.



in pd go to the help menu and click on Browser and go to Pure Data/3.audio.examples/A01.sinewave.pd

this will open a simple example patch, click on the dsp on button in the patch and you should hear a sine tone, congratulations you have successfully installed puredata!

a little update:

I've been working on getting this setup working with Hans' latest beta, and although I've been able to replicate my setup I now hear a large amount of clicks, pauses, and interrupts. I determined that this was coming from Alsa_in, and not pure data. very disappointing but the Alsa_in/loopback device solution was always less than ideal, and without a complete rewrite of samplerbox's audio engine I have determined that a purely software based solution is not viable going forward. I have since switched to a fully hardware solution and it works wonderfully. I've added a USB audio card with stereo in, and have the hifiberry that samplerbox uses feed into that (I believe that when I first came up with the idea of blending samplerbox with pure data Hans suggested I use two cards and I should have listened but I was being cheap   ). There's zero latency and the sound is crystal clear. 

I've bought a cheap behringer ucontrol uca202, removed it from it's plastic case, and mounted it along with the pi to a piece of laser cut acrylic, very tidy, I just need to make some custom length RCA cables for when I build a case. The ucontrol is class compliant so there was no need for drivers but I'm still using Jack audio to route pure data directly to that card. The card is marketed for transferring tapes and vinyl to digital, and an interesting feature is that it has a headphone monitor out, that only monitors the pre-processed input signal (as in, a separate output of samplerbox before being processed by puredata), what this means is that I can take that headphone output and wire it to left and right mono jacks, giving me 2 sets of stereo outputs, one set with just the signal from samplerbox, and another set with the samplerbox/puredata mix.

Hi bbishop

I've just read your thread and it looks fantastic to me !
I'm gonna try to follow your instructions and make it work here !
thx for sharing
Pierre

Hi bbishop

I've just read your thread and it looks fantastic to me !
I'm gonna try to follow your instructions and make it work here !
thx for sharing
Pierre

Glad to hear some interest! I know there's something I forgot on the alsa_in setup instructions so please review the tutorial on the link I included (I tried to simplify it, the page is very dense).

I enjoy reading your posts, you really seem to be delving deep into the functions of samplerbox, I look forward to hearing what you come up with the PD implementation, and if you have any ideas or make any progress past what I've done I'd love to hear it! 

I still have to put together a startup script for running headless and need to figure out a UI and midi implementation to build a patch around (I'm thinking 4 encoders and 4 slide resistors, with banks, handled by an arduino)

..when I first came up with the idea of blending samplerbox with pure data Hans suggested I use two cards and I should have listened..

Hi Bryan,
This option was really worth a try (that was my following remark back then as well) and I appreciate your efforts in this. I encountered some less successful experiments as well 
Hans

..when I first came up with the idea of blending samplerbox with pure data Hans suggested I use two cards and I should have listened..

Hi Bryan,
This option was really worth a try (that was my following remark back then as well) and I appreciate your efforts in this. I encountered some less successful experiments as well 
Hans

it's a huge improvement on performance and I would recommend anyone interested in this experiment to go that route. It's easy enough to try out with any linux compatible IO box (I first tested it with a focusrite itrack solo, a box meant for recording on ipads), any class compliant audio interface should work!

A little video of my current setup, with the HiFi Berry feeding into the Behringer USB card. The other board you see is a simple gpio extender, just giving me access to the pins not being used by the soundcard. 

https://photos.app.goo.gl/4uCsVqyQNfbJhyBC6

I'm running PD from the terminal without the GUI. Reverb and sine wave synth voice (you can hear it under the piano) are both via pure data. This is the closest to headless I've gone, my only problem is in creating a permanent or auto midi connection to pure data, I'm having to connect my keyboard each time with aconnectgui. Any help or insight into figuring this last part out would be greatly appreciated (some sort of script for the alsa sequencer? Or some way to create a permanent connection from the samplerbox midi thru to pure data?? This is not the part I'm good at, unfortunately)

I went back to a simple patch because my previous FM synth patch was way too complex (not so much resource wise, but I would need to build a midi controller with dozens of controls), once I have this permanent midi connection/headless system figured out I will focus on building the of patches from the ground up

Hi Bryan,
Well, I think you could use the internal midi through of the PI and route the midi through of samplerbox to that.

Code:

keyboard -> samplerbox with through enabled for port14 -> alsa through (port14) -> PD reading 14

We'll have to sacrifice the internal midiplayer which is not a big loss in this case imho (just a choice depending interests / usage approach of SB which differ a lot, again imho). Background: the smfplayer is a separate thread writing to alsa midi through (port14), which in turn is opened by samplerbox as input. I'm suggesting to do the opposite here..
Perhaps I can help you (& even should in order to keep your extension compatible with future development) with the samplerbox part. For the PD it means you'll have to open port 14 similar as what samplerbox does for the smfplayer. You may have figure out the specific way for opening the alsa through port in PD.
This way you leave detection of the midi input to samplerbox and get  a CC of that in PD.
Please feedback whether we're on same page here, so I can start thinking of making the SB part.
Hans