Dataflow tutorials

While a PD user (which is, lovingly enough, a PD programmer at the same time) can learn how to use it just by playing around and trying new things, there are important functionalities that are not immediately apparent through play, trial and error. It may take some time to understand underlying functionalities of objects and messages. Following tutorial(s) try to explain and practically demonstrate in quick simple way some of the more important 'grammatical' aspects of this patcher language graphical programming environment.

All examples in this tutorial(s) are available as PD patches too. As some of them demonstrate certain functionality much more vividly when opened inside the Pure Data it is recommended to download them and try them out while reading the tutorial. Get the zip here: http://en.flossmanuals.net/floss/pub/PureData/DataFlow/DataFlowTut_patches.zip

These tutorials can be used in two ways: they can be followed from start to finish (there is slight gradation of difficulty of material and examples), but they can be accessed also as some kind of simple reference. So if something is too obvious to the reader, she can skip a section or two (or just check the screenshots).

Messages

In the same way as basic atoms of a spoken language are words, PD's objects intercommunicate using messages. When data is "sent" from outlets, "travels" along the connections and is "received" at the inlets of objects in the patch it is "understood" or "decoded" by objects in a specific way. Apart from audio signals most (all?) other data is known to be "messages". As with nouns, verbs and pronouns, PD messages can be of different type: numbers (also known as floats - floating point numbers), words (known as symbols) and lists (of numbers and/or symbols), to name the most frequent ones. To help an object to understand what kind of data it is receiving a word called "selector" is frequently present before each data.

A [print] object is PD user's best friend when it comes to determining data types, or rather, their selectors. In above simple demonstration [print] prints a selector "list" before four symbols that are packed by [pack] object which receives special message bang, which is an instruction that is understood by all objects as a "do it!" command. Apart from bang message, if a message has no selector before it, it is assumed that message is a numerical value (PD doesn't distinguish between floating point and integer numbers - all are floats). In the above example, therefore, [pack] expects four symbols in its inlets and will report error if incoming data will not have correct selectors. It is possible to convert data types with objects like [trigger] and [symbol].

Math

Numerical values (or streams of them) can be mathematical manipulated with numerous arithmetic objects in PD:

While it is possible and quite normal to build equations using multiple basic math objects in PD, there is very useful object called [expr] which offers many possibilities with more complex computations, where connecting many boxes is less convenient then writing a formula into [expr].

 

<examples of [expr]???> 

 

But first some truly important dataflow basics.

Three little bits - temperature, order and depth 

Once upon a time there was a PD object. Despite being a somewhat virtual construct by a certain Puckette, it had, curiously enough, a hot and cold side. Not only that, it demanded to be triggered from the right side and was extremely pedant in finishing what was started. Or was it? Not sure what this all means? Read on!

 

Inlets: hot and cold

Because PD is not a traditional text-based programming language, scheduling actions at the inlets and outlets of an object is done with some special rules which need to be kept in mind before they become 'natural'. How and when processing and output of the object is triggered by input is defined by a 'hot and cold inlets' concept: the first leftmost inlet of any object is always a hot inlet. Whatever an object receives on hot inlet it triggers processing and output from the object. All other (right) inlets are cold inlets. Whatever the object receives in them, it stores that value, but does not output. This can be seen at work with a simple counter example:

A message "bang" to a hot inlet is a special message that causes an object to process and produce output using previously initialized, set or default values ([float] for example outputs 0 if nothing has set its value before). At its cold (right) inlet [float] object stores the result from addition object and does not output before it receives anything at hot inlet. In other words, when sent a "bang" message, float] sends a value (0 or anything stored from before), to this a 1 is added in [+ 1] object and result is sent to the cold inlet of [float], where - because it's a cold inlet - this value is stored until next bang or other action at its hot inlet. This is why above construction does not produce an endless loop (and crashes your PD session or report stack overflow) unlike an example below:

A simple example in which it is easy to see hot and cold inlets at work is also any basic math object [+ ], [* ], [/ ] or [- ].



In the above example it is necessary to send a value or a bang to hot inlet last in order to receive correct result.

Similarly the [random] will not output anything unless banged on its hot inlet:



Another example of hot/cold inlets is [pack] which packs individual messages into a list. In the following example a [line] object produces ramps of numbers in time. The list of two object received by [line] determine the target value and time interval within which target value has to be reached.

Order of connecting and [trigger]

While multiple incoming connections to the same inlet are rarely problematic, care has to be taken when order of operations is important and when making multiple outgoing connections from a single outlet to various inlets in a patch. The order what goes out sooner or later is determined which connection was made before or ofter some other connection. This is later invisible in the patch itself (unless one inspects saved patch as a text) and is therefore discouraged in order to produce readable code. When copies of data are needed [trigger] is programmer's friend. Trigger takes a single incoming data, copies and converts it according to its arguments and outputs the copies through its outlets in order from right to left. Here's a brief demonstration (from which one can even figure out the (non visible) order of connecting with which message1 was connected to print objects:

A simple practical application of hot+cold and trigger would be the following simple control of frequency of oscillator:

<explain in-line, rather than only with comments?>

Consider this simple example of scheduling two bangs at inlets of [timer] object to count intervals between single bangs:

<explain in-line, rather than only with comments?>

Another practical application is slightly more complex generation of ramps from 0 to 1 and back whose speed varies in time, it never stops and it always reaches 0 or 1:

<explain in-line, rather than only with comments?>

 
 

Depth first message passing

There is one more rule of Pure Data programming that is important when scheduling of events and order of execution becomes crucial to your code. Also called 'depth first message passing' the rule states that at a forking point (as in [trigger] or multiple connections from a single outlet) a single scheduled action is not "finished" until its whole underlying tree is "done". In other words, an action will send down all messages it needs till it arrives at either cold inlet, end of chain of active messages or hot inlets or similar. To put it in even simpler terms, the message will first go all the way down the well until the next scheduled message down an other well. To imagine better what this means consider this simple demonstration example, which at first sight might look intimidating, but it's nothing more difficult than following with the finger the path through the labyrinth. Try it and remember the trigger's right to left order and depth first rule:

The resulting number will be always the same as the input number as the scheduling logic is taken care of according to rules we defined so far.

Consider again the wrongly connected counter example that can crash your PD session (or report stack overflow) because of infinite loop:

From the point of view of depth, the above example represents infinite depth - the message passing is never finished.

In conclusion of these three important dataflow rules it must be said that

1. hot and cold inlets
2. order of connecting + use of trigger object, and
3. depth first message passing

are in most cases intertwined and dependent on each other. In other words, these concepts and its practical applications appear constantly in the PD code. In this sense, they form some of the most basic rules of this language.

 

Invisible connections, crossing borders, reusing code

While the main strength of PD seems to be its graphical/diagramatical nature where connections between objects represent actual path of data, it is only when data can be be exchanged between different windows, patches and applications and also when code can be abstracted and reused in multiple instances it is possible to create truly complex programs suited to one's needs.

Send, receive, throw and catch

Soon after some introductory patching, a PD user will soon discover the slight inconvenience of connection lines running over objects to reach other objects. For a while, with some imagination this tendency towards mess can be avoided, however when serious and more complex patching starts to take shape inconvenience turns into a drawback. Luckily, there's a solution.

Using [send] and [receive] data can be sent from one part of the patch to another (or even into another window) without connecting lines. Both objects need an argument that tells from which [send] does a [receive] receive data. In this way we can use more [send]s and [receive]s. In the following patch

[metro 1500] generates bangs at the interval of 1.5 second (1500ms) and is sending them to [send beat] object. These bangs are picked up by [receive beat] objects because [send] and all [receive]s have the same argument - "beat". An argument required by [send] and [receive] can be arbitrary (but not numeric only). The rest of the code (osc, delay, dac) produce short beeps. Try it to hear it. There is no limit in number of [send]s and [receive]s with same argument. It is possible to have many sends. Just add to the example above more [metro] objects:

[send] and [receive] are for control data - messages, symbols, lists. For audio signals a 'tilde' version of these objects are needed. [send~] and [receive~] can be used to receive a single audio signal at many places. The slightly more complex example would be to create a multitap delay:
 

To consider above example: taking a [send beat] from previous examples and a single generator of a beep an audio signal is sent with a [send~] to 'mtapdelay' and 'out'. The latter is received and send to dac~ for immediate output (dry signal). Beeps are received also by [r~ mtapdelay] ([r~] being a working abreviation for [receive~] - there is no difference in functionality) and fed into four separate delay lines (notice dline1, dline2, dline3... and differing delay times in delread~ objects) and sound is also attenuated with [*~ 0.4] and alike before send to one of the two channels representing a stereo output.

Notice however that outputs from delay lines are not sent with [send~] back to [r~ out] for example. A reason behind that is that with audio signals there can only be one [send~] for many [receive~]s. While there are technical reasons for this difference, a handy pair of audio objects that help to achieve many-to-one sending are [throw~] and [catch~] which work in the sense of a summing audio bus: many [throw~]s can send audio signals to one [catch~] that simply sums all the signals:

 

Using [catch~] it is then possible to further control and process audio (i.e.: volume control, VU metering, limiting, reverbs, etc...).

Coincidentaly, all objects we described above ([send], [receive], [send~], [receive~], [throw~], [catch~], as well as [delwrite~] and [delread~]) all work across different patches, subpatches and abstractions (the latter two are explained in next sub-chapter). To build on the example above, the simple matrix of delay-lines could be in a separate patch. Furthermore, it is actually not necessary to have four delay-lines to achieve multitap delay. There can be only one [delwrite~] and more [delread~]s that read from that delay-line at different delays:

 

In conclusion, the objects described above are powerful tools  to not only send and copy data and audio around a single patch without messy connections, but to create connections between individual patches, subpatches (patches within patches) and abstractions (reusable "classes" of which multiple instances can be created) as well. A word of warning though: the arguments passed to these objects are always global - they are accessible from all patches and abstractions opened in a single PD session. This simply means that a situation can arise with unwanted 'crosstalk' of data or multiplies defined. Care has to be taken on names of arguments, while at the same time a technique exists to localize arguments using dollarsigns (see chapters on abstractions and dollarsigns).

Subpatches

After a PD user discovers and learns how to use [send] and [receive] she is spared of messy criss-crossed patches, but not for long. With even more complex coding, user would either store code in different individualy saved patches - but would then have to open them separately, or expand its canvas to unscrollable size. However, there is a solution to 'hide' or store parts of a patch in a so-called subpatch.

It is useful to think of subpatches as container or drawers, where code is organized and put away. A subpatch is created by typing [pd any-name] into an object box, where "any-name" can be arbitrary word. When creating subpatch like this, a new empty subpatch window will appear. Consider the following practical example:

Subpatches can also have inlets and outlets (for data and audio respectively) which are created by using an object [inlet] or [outlet] (and [inlet~] or [outlet~] for audio signals) inside a subpatch. Example above uses all these. In a single patch saved as subpatches.pd a subpatch called "my-beeper" is created that contains a bit of code from previous examples - a simple switchable beeper, whose frequency of a beep and time interval can be set - from without the subpatch itself using [inlet]s. The horizontal order with which they appear in a subpatch (a little window on the right side) determines the corresponding order of inlet at the object box. "my-beeper" subpatch produces audio signal which is fed into corresponding [outlet~]. A copy of this signal is sent to one of the output channels (presumably your left speaker) and into "my-delay1" subpatch which contains simple code of and audio [inlet~], message/control [inlet] and a delay-line. This delayed audio signal is sent to the other output channel - presumably your right speaker.

By closing the windows of subpatches the code is not lost but still exists inside the subpatch objects ([pd my-beeper] and [pd my-delay1]). They are saved within the patch subpatches.pd. Subpatch windows can be reopened by left-clicking on subpatch objects or by rightclicking and choosing "Open" from menu. Subpatch objects can be freely copied, by which unique copies are created, that can be individually edited - changes are not reflected in any other subpatches, even if they have the same name.

 

Care has to be taken, however, as in above example, subpatch my-delay1 uses delay-line called "dline1" and copies of that subpatch should not all use the same delay-line, but rather each should use it's own one (i.e.: "dline2" and "dline3"). If there is two or more [delwrite~]s that write to the same delay-line PD reports "multiply defined" and delays do not work. It also important to remember that arguments in an object name cannot be passed to subpatches - unlike in abstractions.

Abstractions

Subpatches are useful to put away unique piece of code from the main canvas, to store functionality specific to the opened patch or to have similar but slightly different pieces of code that can be edited separately. However, sometimes precisely the same code is used again and again at different times in different patches, or same exact construct needs to be used multiple times, in which case it would be less convenient to create copies of subpatches, especially when this construct is improved and then improvement should be reflected in all subpatches. In all these cases it is much more useful to abstract this code, to make it available from the outside of the patch, so it is reusable by calling it from within as an instance. A possible analogy to this would be a radio broadcast - it is produced live in a radio studio, but many instances of it are heard across the country. Changes to it at the original location are reflected at all instances.

Consider a situation where a random note on minor C scale converted to frequency is needed multiple times in one patch. A basic construct for this would be:

Every time [random] is banged, one of the displayed numbers will be transposed + 50 and through [mtof] converted to frequency. It's a construct that's inconvenient to reproduce many times in a patch. To abstract the code, it should be handled like it's a subpatch and inlet's and outlets are added but should be saved as a separate patch:  

 

This patch needs to be saved on a path (folder) that PD looks into each time an object is created. That path (folder) can also be defined in PD preferences however the simple usage is to have this patch in the same folder where the calling patch is saved - a patch from within which this abstraction is called. Consider a main patch "cminor-oscilations.pd" saved in /home/user/puredata/ (or c:/pd-work/) and "cminor.pd" in the same folder. The abstraction (or an instance of it) is called simply by typing the name of the patch (without extension .pd) into an object box, like this:

By clicking on the [cminor] (or rightclicking and choosing "open") the abstraction is opened in new window, just like subpatch. Alike, its inlets and outlets are defined by [inlet] and [outlet]. However now a separate patch (cminor.pd) is being edited. This means when changes are saved all instances in the calling patch are updated. In an example with more instances:

an inlet is added to [cminor] abstraction by opening it, adding code for "automatic" change of frequency and saved. This is immediately reflected in all instances by two inlets appearing on them:

 

Dollarsigns

In the same way as objects like [metro], [random] or [osc~] can (and need) to accept arguments (as in [metro 1000]) an abstraction can accept arguments that can be used inside of it. Consider an abstraction that combines [metro] and [random] objects to produce random numbers that also denote time intervals at which the are produced. In its basic form it could look like this:

The abstraction above has two inlets, at left it would receive on/off (1/0 float) input and at right the range for the [random] which will then pass random numbers to [metro] and at abstraction's outlet. As it can be seen, the abstraction will initialize with 1000ms to [metro] object and range from 0 to 1000 to [random] object. To change the value of random object dynamically that value will have to be send at abstraction right inlet. However, this can be done differently by passing arguments to the abstraction at the creation time using dollarsigns inside the abstraction. Consider this change including demonstration of usage:

 

At the creation time two arguments are passed to an abstraction [randometro1]. Inside the abstraction, $1 is substituted with the first argument, and $2 with the second. The effect (which was goal in the first place) is to be able to define the min-max range (as opposed to only 0-max) at which abstraction works. Because [random] inside the object needs a 0-max range, first argument (presumably smaller) is subtracted from the second. The result is passed to random to produce random numbers which are then added to the first argument. In demonstration of usage in the window behind the abstraction this construct produces random numbers between 1000 and 1100 in the first case, and 500 and 600 in the second.

While $1, $2, ... and so on represent first, second, etc .. argument to the abstraction, there is one special dollarsign that is in Pure Data extremely useful. $0 is a variable that is internally substituted by unique four-digit number per patch or instance of abstraction. In other words, PD takes care that each instance of an abstraction or patch will be assigned this unique number and stored in $0 variable. The usefulness of this is immediately apparent in the following example of simple delay abstraction where delay-lines with the same name in multiple instances of same abstraction must be avoided:

 

It is important to understand that, despite $0 isn't actually substituted with the unique number inside the delwrite~ object, the latter actually writes audio signal to delay-line named "1026-dline". $0 variable is assigned in every opened or called patch, which also solves the problem of two or more instances of same patch (i.e.: simple synth). $0 also saves from situations from unwanted crosstalk of frequently used variables in different patches. An attentive reader/user could also point out a possibility to use $1, to use an argument passed to an abstraction (like "one" and "two" in above example), in which case care must be still taken to assign unique arguments to abstractions used in the same PD session.

$0 is at times called localized variable, however, in my view, that is not entirely true. A variable constructed with $0-something can still be accessed from the global namespace by simply finding that unique number and than calling that appropriate variable (like for example to read the delay-line named 1026-dline from above example from within another independent patch). In fact this can sometimes be even useful. It is however true that using dollar variables is a localization technique.

A frequent confusion arrises from the use of dollarsigns in message boxes. It is important to understand that dollar variables in message boxes are actually totally local to that message box itself regardless where they appear. They will be substituted only by what a message box receives on its inlet. In an example of abstraction within which both types of dollar variables are used:

 

[shotline] abstraction, which has a goal of producing a ramp of values in specified time from some starting value to ending value, takes three arguments - speed, from-value and end-value. These variables are accessed inside the abstraction with $1, $2 and $3 in the [pack object]. The latter sends a list of those three arguments to message box, in which $1, $2 and $3 represent only elements of an incoming list and not directly arguments of the abstraction. Message box first send the second element, followed by a comma - so it resets line to that value, and then a pair of third and first element which correspond to target value and time-frame of a ramp.

Pretty interfaces and two-dimensional data

Graph on parent

In pure data it is extremely easy to create interfaces that include sliders, buttons, number boxes, toggles, coloured backgrounds... how to use them, look at the "GUI objects", or simply right-click>help on one of them. However they still need to be connected and to use them away from the data inlets that they control, they have to be repeatedly created in order to function the way we want. Consider an example of a delay abstraction (already used above) that takes at it's second inlet a value for time of delay which we want to control with a slider:

 

So, everytime when an abstraction like that is created, when it is desired to be controled by a slider, many steps are needed to recreate the same visual and programmatic construct. Luckily, there is a very powerful feature od PD: graph-on-parent. It enables a subpatch or an abstraction to have a custom appearance at the parent 'calling' patch. Instead of plain object box with the name of abstraction and arguments, it can have different size, colour, and all the gui object inside. Here's how it's done, continuing on delay: inside the abstraction or subpatch, rightclick on white underlying canvas and choose properties. Inside a dialog that appears, enable toggle for graph-on-parent:

 

Applying this will create a grey-bordered box within the abstraction. This box represents the shape and form of the abstraction on the parent canvas (the calling patch). Whatever the size and contents of that grey box will be visible excluding connections, object boxes and message boxes. In the properties of the abstraction below the graph-on-parent option two rows of four values represent X and Y settings. Size will set the size of the box while margins will only set the position of that grey box within the abstraction. Adjusting these setting accordingly:

 

Inside the grey box it is now possible to create a suitable interface, according to users needs and aesthetic preferences needed for functional and pleasurable control of parameters. See properties of individual GUI objects (like canvas, slider, etc) and experiment what can be done with them. Simple delay abstraction in this case receives an underlying colour canvas and two sliders, one for delay-time and the other for incoming level:

While editing the abstraction with graph-on-parent, abstraction is greyed-out on the parent canvas until the abstraction window is closed. Only then the final appearance can be seen:

The purpose of a pixel wide transparent gap between the gray border and canvas in the abstraction is to reveal inlets and outlets at the parent window - however with sizing of inlaid canvas, even black borders can be hidden. Calling this abstraction as usual - by creating an object box and typing the name of abstraction without the extension .pd - will always instantly create this GUI:

that needs nothing more than to connect to audio signals and adjusting controls:

Arrays + graphs = tables

In programming and building applications - and patches in PD are many times some kind of applications - we frequently need a way to conveniently store bigger amount of data and to be able to instanly access it. To store data in float boxes or dollarsign variables is inconvenient when more then four, five or ten numbers need to be stored. An "array" can be thought of as a container in a computer memory with neatly indexed drawers with data that can be looked up instantly. Arrays are visualy contained in graphs with X and Y axis. In PD arrays contain only numbers and together with graphs encapsulate the concept of tables. In other words, tables are graphical arrays that contain numbers.

To create a table choose "Array" from "Put" menu and a dialog appears:

Here the name and size of array can be defined. The name of the table should be unique and $0 can be used in a name (i.e.: $0-sample1) to avoid crosstalk. Size of the array defines how many elements it will hold. If table will be used to control a 16-step sequencer only 16 elements is needed. But if it will contain a two seconds long sound sample (at 44100hz sampling rate) the array should be long 88200 elements, however the table can be resized also later. Save contents will save the contents of an array within the patch file. This can be desired if table will be used for waveforms for oscilations or to control an envelope of the sound, or undesired if soundfiles will be loaded in it - in which case the patch file (something.pd) will become rather big and despite of being a text file will contain sound-wave data. Next three options 'draw as points', 'polygon' and 'bezier curve' define how data will be visualized: as discreet points (horizontal lines), as cornered zigzagging connected lines or smoothed bezier-curved line:

 

 

In play mode, it is possible to draw inside the table with the mouse, which can be useful with few elements in a table. Sometimes tables can be used to display the waveform of sound signals. Using tabwrite~ sound signals are recorded into table. Every time a [tabwrite~] receives a bang, it will start recording (sampling) audio signal into the array, graphing it when reaching the end of array:

 

In above example, [tabwrite~] is banged every half second to continuously display the waveform produced from two [phasor~]s, and a [clip~] object. Data can be put as values into tables too, simply by sending an index number (X-coordinate) and a value (Y-coordinate) to [tabwrite] (no tilde!) object:

In above example for each index number  (they are produced with a counter and start from beggining (0) with [mod 100] at 100) a random value between -1 and 1 is writen to a table.

Tables can be read (looked up) in two ways: to get discrete numbers, or to directly read them as audio waveforms. With [tabread] an index number is taken as an X-coordinate and value in the table (Y-coordinate) is output. In the following example an array is used in a repeating sequencer-like fashion as a simple rudimentary control for an sawtooth oscillator:

With [tabosc4~] table data is used as an oscillating waveform - like sinewave is used in sinewave oscillator [osc~] and sawtooth wave is used in [phasor~]:

In above example an oscillating waveform from table7 is used to modulate frequency of an oscillator that is using the same waveform to synthesize sound. Changing the table in realtime will influence the modulation and oscillation. Source for hours of fun!

Another way to read data from a table is to play it as a sound recording - which usually is, especially if array is filled with data from a sound file. For this [soundfiler] object comes handy, as is shown in the following examples. In first, array is played using simple and straightforward [tabplay~] object, which offers flexibility of playing from a specific point for a specific length. Remember, digital sound recording is, simply put, high frequency measurements (sample rate, i.e.: 44.1kHz) of sound vibrations. In PD, when soundfile is loaded into a table, every single measurement (sample) can be accessed. That is why, 44100 samples equals 1 second (in most cases).

 

Following to the aforementioned possibility of accessing individual samples within a sound recording that's been loaded into an array, a [tabread4~] object allows more computational flexibility. Below, [phasor~] object produces ramps (sawtooth wave) from 0 to 1 at the audio rate (commonly 44100 times in a second). If frequency of the [phasor~] oscilator is 1Hz, it will output a ramp from 0 to 1 in exactly one second. If multiplied by 44100 and sent to [tabread4~], it will read first 44100 indices (indexes) in a second and output the values as an audio signal - example below tries to demonstrates that with a twist or two:

First twist comes from an idea of changing the frequency of phasor, and this way slowing down the ramps. This would however shift the pitch of the sound - like changing speed of a vinyl record. This is prevented by multiplication with higher number of samples, which effectively turn the parameter into the length of a sample that is being looped instead of slowing it down. Looping is here because [phasor~] starts again at 0 after it has reached 1. The other twist is the starting point, which simply shifts the whole loop by adding number of samples (seconds multiplied by 44100).