Psychtoolbox - Tutorial

The Psychophysics Toolbox is a free software package for vision research. (It's available for download.) Its routines provide an efficient interface between a high-level interpreted language (Matlab) and the display hardware on Macintosh and Windows computers. It also provides interfaces for timing, sound, keyboard, and the serial port. Example programs are included.

Here are the key ideas (from several domains) that will orient you in writing code for your experiments in this environment:

1. Strengths of Matlab & Psychtoolbox

Interpretive general language & a good interface to hardware

Unlike most software packages for experimental psychology and visual psychophysics, the Psychtoolbox is very general. It doesn't contain specific support for generating gratings, running trials, etc. Rather, it provides the user of a high-level interpreted language (Matlab) with a well-defined interface to the graphics hardware (frame buffer and lookup table) that drives the display. In this sense, it is a very generic tool. The power comes from the fact that once you can write arbitrary matrices into the frame buffer and lookup table as fast as the machine can go, everything else is easy to program in Matlab.

Matlab is a high level interpreted language with extensive support for numerical calculation (The MathWorks, 1993). The Psychophysics Toolbox provides a Matlab interface to the computer’s hardware. The core Psychtoolbox routines provide access to the display frame buffer and color lookup table, allow synchronization with the vertical blanking, support millisecond timing, and facilitate the collection of observer responses.

The Psychtoolbox doesn't limit the user—if the experiment can be run on the hardware, it can be run with the Psychtoolbox. In comparison, other environments for creating perception experiments provide very evolved support for specific experiments. Our experience with software packages not based on a general-purpose programming language has been that the very first thing we wanted to do turned out to be impossible.

We think the Matlab-Psychtoolbox combination has four winning features that we’d recommend for any experiment-design environment:

The Psychtoolbox also provides interfaces for timing, sound, keyboard, and the serial port. And it includes many useful Matlab routines, such as color space transformations (Brainard, 1995; Brainard, Pelli, and Robson, 2002) and the QUEST (Watson and Pelli, 1983; Pelli and Farell, 1995) threshold seeking algorithm.

2. Transforming numbers into movies

Using a high-level platform-independent language like Matlab, it's easy to produce a matrix of numbers specifying the desired luminances of all the pixels in the displayed image. Today's off-the-shelf personal computers can copy those numbers from memory to video memory quickly enough to show a new image on every frame of a CRT monitor. However, high-level languages generally provide only rudimentary control of the vital transformations from number to color, and of the rate at which successive images are displayed.

That is where the Psychtoolbox comes in, providing simple but powerful functions to control the pixel transformation and timing synchronization of the computer-display interface.

Here’s a quick sketch of how computers display images. (See Brainard, Pelli, and Robson, 2002, for a fuller treatment.) Once the matrix of numbers has been loaded into frame buffer memory, the subsequent transformation from number to luminance (or color) is complicated, but usefully simplified to three steps. First, at video rates (e.g. 100 million pixels per second), each number passes through a lookup table, typically one 8-bit number in and three 8-to-10-bit numbers out, each driving an 8-to-10-bit digital-to-analog converter. Second, the three analog video signals drive the three guns of a color CRT. The luminance of light emitted by each monitor phosphor is proportional to the corresponding gun's beam current, which is an accelerating function of drive voltage; this is called the monitor's "gamma" function. Third, the luminous image is blurred by the point spread function of the beam.

Most graphics cards have adjustable pixel size, typically 8, 16, or 32 bits per pixel. Furthermore, while most have 8-bit Digital to Analog Converters (DACs), a few have 9- or 10-bit DACs. Many users write to ask what these numbers of bits mean. In the pixmap, each pixel is assigned a certain number of bits, 8, 16 or 32. The number of bits per pixel determines how many different colors you can have in one frame: 256, thousands, or millions. When you actually display an image, the pixel value is used as in index into a lookup table on your graphics card. The values in the lookup table are typically 8 bits per channel, but some cards have 9 or 10 bits per channel. Those values, output from the lookup table then drive digital to analog converters (DACs) with a corresponding precision, 8 to 10 bits. In 8-bit mode you can select any 256 colors. Within the lookup table, each color is specified by three 8-10 bit numbers. If instead you use the 32-bit mode (millions of colors) then the pixel is considered to be made up of three 8 bit values, one per channel (plus 8 bits of padding), each of which goes through a one-channel lookup table, again with 8-10 bit outputs. 16-bit mode is rarely useful. In that mode 5 bits are assigned to each channel (plus 1 bit of padding), allowing only 32 values per gun. Again, we have a longer treatment of this issue in our Display Characterization chapter (Brainard, Pelli, and Robson, 2002).

The basic idea is that you use Matlab to compute images or movies, and use new Matlab functions provided by the Psychtoolbox for accurate display. The Psychtoolbox routines treat the computer (Mac or Windows) as a display device: a frame buffer with a color lookup table. (To read about how to use frame buffers for visual psychophysics, see our psychophysics bibliography.)

The software has three layers. First, there is Matlab code that you write and some Matlab utilities that we supply, e.g. to compute color lookup tables and implement the QUEST staircase procedure. Second, there are a set of Matlab extensions (MEX or DLL files) that are written in C but callable from within Matlab. Third, the extension files, in turn, use VideoToolbox software (for Mac, or work-alikes for Windows) for low-level control of the hardware. (You won't need to have the VideoToolbox—it's already linked into the extension files—unless you want to build your own extension files.)

Screen

Screen.mex is the heart of the Psychophysics Toolbox, providing fifty subfunctions (selected by a text argument) that control the display screen(s). Experiments typically begin with a call to Screen('OpenWindow') and end with a call to Screen('Close'). (Which, in turn, call corresponding VideoToolbox routines, but you don't need to know that.) Anywhere in between, you may copy an image from a Matlab matrix onto the screen using Screen('PutImage') and change the lookup table using Screen('SetClut'). Typically you'll create a window on each screen that you're using in your experiment. Copying within or between windows is very fast (e.g. 48 MB/s on a PowerMac 7500, 180 MB on a G4 Cube). And you can create an unlimited number of offscreen windows (in memory, not visible) that can then be shown, one after another, as a movie, by copying to an onscreen window. Other Screen functions display text and dialogs and provide frame-accurate timing.

You can use the Screen function to write Matlab scripts that intermix graphics operations, calculations, and wait for observer responses. If you run the routines interactively from the command window, there will be a certain level of chaos as Matlab's windows overwrite parts of the experimental window. Still, this mode can be useful for debugging, especially if you restrict the window sizes to avoid overlap, or you have a second monitor.

Operations such as synching to vertical blanking and writing color lookup tables depend on the kind of video card(s) you have, and their video drivers. New versions of the computer operating system often include new video drivers. The Psychtoolbox provides a uniform interface, but you should check the timing on your computer, by running ScreenTest.m.

Note that Matlab has a number of built-in graphics commands, like BAR, that can draw into Matlab "figure" windows. Those commands won't draw into a Screen window. Use Matlab commands to draw into Matlab figures; use Screen to draw into Screen windows. For example, if you have an open Screen window, you can draw a black filled rectangle (10x25) in it by saying: Screen(window,'FillRect',BlackIndex(window),[0,0,25,10]). You can erase the whole window by overwriting with white: Screen(window,'FillRect').

Rush

A major challenge in doing psychophysics on modern personal computers is that operating systems are becoming more and more aggressive about stealing time away from your display code to do other things. Rush is a MEX function that executes a piece of your Matlab code with little or no interruption. This allows you to keep your computer running more or less normally, with lots of background processes, yet grab complete control for the second that it takes to present your stimulus.

Other Psychophysics routines

In addition to Screen and Rush, there are routines to satisfy all the needs of psychophysical experiments: unbuffered keyboard i/o, mouse i/o, serial i/o, timing, sounds, and threshold-estimation staircase. Other routines move data to and from the clipboard, interface to PhotoResearch PR-650 color meter, and save images as EPS files.

4. Displaying a grating with Screen.mex

Let's write a program to display a grating. We’ll open up a window on the screen, write a Matlab matrix into our window (i.e. into the frame buffer), and then close the window. These functions, Screen('OpenWindow'), Screen('PutImage'), and Screen('Close'), along with functions to load the lookup table and sync to the vertical blanking, are the heart of the Psychtoolbox.

Next we figure out what numbers will produce white, gray, and black, and fill the whole window with gray.

Now we use Matlab functions to compute a gabor patch (a grating vignetted by a gaussian envelope), and put that image into our window.

Now we pause, displaying the grating until the observer types any character, and finally close the window.

That’s a complete program that will run on any computer with Matlab and the Psychtoolbox. It’s a slightly abbreviated version of GratingDemo.m, which is included in the Psychtoolbox.

In the above, the call to Screen(‘PutImage’) slowly translates the Matlab double precision matrix into the pixmap format of the frame buffer. You won't want to do that while showing a movie. In that case you’d create an "offscreen" (i.e. invisible) window, which is allocated in your computer’s memory,

Now, whenever you like, you can blit (i.e. copy) very quickly (up to 200 MB/s depending on your processor clock rate and bus speed—run ScreenTest to time your hardware) from offscreen to onscreen graphics memory:

If you create multiple offscreen windows in advance, then you can show them, one after another, one per frame, to create a movie. You’ll need to synchronize your program with the display, by waiting for blanking before showing each new frame.

Incidentally, if you display stimuli on the main screen, as we often do, then the Screen window will hide the main menu bar and obscure Matlab’s command window. That can be a problem if your program stops (perhaps due to an error) before closing the window. The keyboard will seem to be dead because its output is directed to the front most window, which belongs to Screen not Matlab, so Matlab won’t be aware of your typing. It’s ok. Remain calm. Typing command-period (Mac) or Ctrl-C(Win) will stop your program if hasn't stopped already. Typing command-zero (on the Mac) or Alt-Tab (on Windows) will bring Matlab’s command window forward. That will restore keyboard input. The screen might still be hard to make out, if you’ve been playing with the lookup table. Typing

will cause Matlab to flush Screen.mex. Screen.mex, as part of its exit procedure, cleans up everything it did, closing all its windows and restoring the lookup table of all its displays. And everything will be hunky dory again. Remember the magic incantations: command-zero (Mac) or ALT-tab (Win) to bring the command window forward, and "clear screen" to restore the displays to normal.

5. Examples

You got yourself a computer, bought Matlab, and installed the Psychtoolbox. Now you want to get your experiment running. Where to begin?

George Sperling pointed out to us recently that writing software from scratch is hard. It's much easier to edit an already working program that does something similar. The PsychDemos folder includes a variety of short programs that show how to do various specific things, including synthesizing and displaying a movie. Type

The Psychtoolbox website includes a Library page with links to programs written by other users. We invite everyone to send software to the Psychtoolbox forum, which automatically archives your message and enclosure. (Please include the keyword DONATE in the subject, so we can all search the forum for software.) We add links on the Library page to programs in the forum that appear to have enduring value.

By the way, don’t shortchange yourself. Buy enough memory ($0.4/MB) and disk space ($3/GB) to work comfortably.

6. Online help

The Psychtoolbox has no manual. Matlab has manuals, but we hardly ever use them. Instead we use the HELP command. Typing

will list a variety of broad topics on which Matlab offers help. You can ask for help on any function, including Matlab’s functions and any function in the Psychtoolbox. For example,

you will get an overview of the hierarchical organization of the Psychtoolbox. For any of the subdirectories listed, you can get a synopsis of the functions in that subdirectory. So

will give you a synopsis of the core toolbox routines. The HELP facility is a fast way to explore Matlab and the Psychtoolbox, and we use it all the time.

Some of the Psychtoolbox functions, like Screen, have a large number of subfunctions, making it impractical to include all the information in the HELP display. Simply typing

will give you a synopsis of all the Screen subfunctions. For more detail on a specific subfunction, call Screen itself, adding a question mark to the subfunction name.

will type out helpful text for ‘CopyWindow’. You can omit the parentheses and quote marks, because Matlab considers this

In our help text for specific functions, we've mostly followed Mathworks's help-text conventions. But note that we designate optional arguments to function calls by embracing them with square brackets. You're not meant to include these brackets when you actually call the function. For example, "help Snd" will tell you this:err = Snd(command,[sig],[rate]). What this means is that the "command" argument is required and the "sig" and "rate" arguments are optional. Thus, a typical call to Snd looks like this, and has no brackets: Snd('Play','Quack'). If you would like to force an optional argument explicitly to its default, you can typically pass the empty matrix. This is useful for functions with more than one optional argument where you'd like to (e.g.) accept the default on the first but explicitly pass the second.

Matlab usually ignores case (at least on Mac and Win platforms), except in variable names and built-in functions. The Psychtoolbox, by default, ignores case, but this is a user-settable preference. Although lazy typists can type everything in lower case, keep in mind that this practice may lead to portability problems somewhere down the line.

7. Parallelize

Matlab is quick. Running on a 250 MHz PowerBook G3, the loop overhead is only 1 microsecond per iteration (after the first). Because it’s an interpreted language, it takes time (7 microseconds) to process each statement. However, one statement can perform a large number of elementary operations, e.g. adding two 100 element matrices requires 100 adds. Matlab does the elementary operations very efficiently. The large 76:1 ratio of the 7 microsecond statement overhead to the 0.09 microseconds per elementary operation (~ + - * / == & | sin sign) is a defining characteristic of the language. You can run the Psychtoolbox program SpeedTest to assess these parameters for your own computer.

The implication, worth remembering, is that the run time of statements that operate on fewer than 76 elements is mostly spent processing the statement, not the elements. An example may help.

This creates two 100 element arrays. (They’re 10x10 square matrices.) In languages, like C or BASIC, that lack matrix operations, one would add x and y by writing a loop.

That works in Matlab too, but it runs very slowly, taking 800 microseconds (i.e. 8 microseconds per iteration, for 100 iterations). The right way to do this in Matlab is to operate on the whole matrix at once,

Again, the thing to remember is that the run time of statements that operate on fewer than 76 elements is mostly spent processing the statement, not the elements. An important part of learning Matlab is learning how to operate on lots of elements at once, as in the above example.

All of the timing above is for compiled code. Matlab compiles functions and loops before executing them, so you’ll usually benefit from the compilation without having to think about it. The one case you should avoid is calling a script (i.e. not a function) repeatedly; you should convert that script into a function.

8. Use the debugger

Matlab has a great built-in debugger, allowing you to step through your program, examine and modify variables, and set breakpoints. However, in the Mac version, the way you start it up is confusing, at least the first time you do it, which discourages many people enough that they never discover how useful the debugger is.

Be warned that, on the Mac, the debugger has slight difficulties with files that are in the Matlab toolbox folder (which includes the Psychtoolbox) and that the debugger may give a spurious error beep if you choose to debug a file whose name has any uppercase characters. For best results, debug a file outside the toolbox folder with a filename that’s entirely lowercase. Later on, once you've got the hang of using the debugger, you can ignore this restriction, but, as a beginner, it'll be less confusing to respect it.

Suppose you just wrote a function called foo.m, and you’ve got the file open, in a window called "foo.m". Click on the debugger icon (a green bug) in the window’s title bar. This will open the debugger window, which (confusingly) is also called "foo.m". Note the debugger's flow control icons

at the left end of the title bar. Now set a breakpoint somewhere in your program by clicking one of the dashes "—" that appear in the left margin of the window, next to each statement. Clicking the dash turns it into a red dot

, a breakpoint. (You can set multiple breakpoints, if you like.)

Sometimes when you try to set a breakpoint, you'll get a beep and no red dot. This usually means that Matlab is having trouble finding your file. (Which is sad, considering that it's got the file open.) Setting a breakpoint seems to be implemented in effect by issuing a command like "dbstop foo 17". This will fail if foo is neither in Matlab's path (a list of folders of likely places to find stuff) nor the current directory. You fix this by using the Matlab CD command to set the current directory to be the folder that contains the file your debugging, foo.m. If that succeeded, you should be able to open foo by typing "edit foo" in the command window. Now you should be able to set breakpoints without difficulty.

Now you’re ready to run your program. You’d have thought that you could just click something to say "Go!". No such luck. You must now go back to the Matlab command window. Using the keyboard shortcut, type command-zero (on a Mac) or xxx (on Win) to bring the command window forward. Now run foo, by typing its name:

The program will begin execution and halt when it gets to your breakpoint. The command window will display a special prompt

indicating that you’re in the debugger. You can issue any Matlab command you like. Mostly you’ll simply type variable names to see what values they have. You can resume execution by typing

but, instead, you’ll probably find it more convenient to go back to the debugger window by typing the shortcut command-4 (Mac) or xxx (Win), and use the flow-control icons in the title bar. You can single step

, descend into a subroutine

, ascend to the calling program

, continue

, or stop

. You can also add or remove breakpoints

. When you’re done, you should remove all the breakpoints.

9. Measuring threshold

You can measure whatever you like, but it is often useful to measure the stimulus intensity that yields a criterion level of observer performance (Pelli and Farell, 1995). The Psychtoolbox includes Matlab code implementing the QUEST procedure for estimating threshold.

Experiments are usually organized as a run (e.g. 40 to 100) of trials. Each trial presents stimuli to the observer and waits for a response. Each trial takes several seconds. To measure threshold you’ll write a loop, with one iteration per trial.

Before starting the loop, you’ll initialize QUEST, giving it a rough guess for the value of threshold. You may also want to ask for the observer’s name and so on.

Within that loop are the guts of your experiment. Typically you might call QUEST to ask it to suggest a good contrast to test at, based on the initial guess and all the observer’s responses so far. Then you’d compute an appropriate stimulus and display it briefly in a window. If you’re using a two-interval forced choice paradigm you’ll have two intervals, announce by beeps, and display the signal in only one of them. Then you’ll wait for the observer’s response, typically a keypress or mouse click. Finally, tell QUEST what contrast you actually tested at and whether the observer’s response was right or wrong. The Psychtoolbox demo program ContrastThreshDemo illustrates how QUEST is used in the toolbox environment. We recommend discarding the observer’s first response, just in case he or she wasn’t quite ready.

Finally, after the last trial, you’ll report QUEST’s threshold estimate and confidence interval.

Judging from email queries we’ve received from users, the most common beginner’s mistake is to forget to leave things in the same state at the end of the trial as they were at the beginning. If you open a window at the beginning of the trial (on- or off-screen) then close it at the end. Otherwise you’ll eat up memory fast, adding yet another window on each trial. The symptom of this programming error is that the experiment works perfectly for a few trials but eventually fails, when it runs out of memory.

We suggest that you avoid opening and closing windows (whether on- or off-screen) within a trial because it’s slow. It’s better to open all the windows you’ll need ahead of time and then just use them on each trial. Finally, after the last trial, you should close them all.

10. Calibration

Everyone says that you should calibrate your monitor so that you’ll know what you’re displaying, but rarely is software and a photometric instrument provided to help you do it. The Psychtoolbox, being free software, doesn’t include the instrument, but it does include software, in PsychCal, which should help, though it still isn’t as well documented as we’d like. Our measure page has some suggestions on what to buy. You may wish to read our chapter on display calibration (Brainard, Pelli, and Robson, 2002).

11. Good luck!

References

Brainard, D. H. (1995) Colorimetry. In Handbook of Optics: Volume 1. Fundamentals, Techniques, and Design. M. Bass (ed.). McGraw-Hill, New York, 26.1-54.

Brainard, D. H. (1997) The Psychophysics Toolbox. Spatial Vision 10:433-436 (PDF)

Brainard, D. H., Pelli, D.G., and Robson, T. (2002). Display characterization. In the Encyclopedia of Imaging Science and Technology. J. Hornak (ed.), Wiley. 172-188. (PDF)

Pelli, D.G. (1997) The VideoToolbox software for visual psychophysics: Transforming numbers into movies. Spatial Vision 10:437-442. (HTML)

Pelli, D. G. and Farell, B. (1995) Psychophysical methods. In Handbook of Optics: Volume 1. Fundamentals, Techniques, and Design. M. Bass (ed.). McGraw-Hill, New York, 29.1-13.

Pelli, D. G. and Zhang, L. (1991) Accurate control of contrast on microcomputer displays. Vision Research 31, 1337-1350. [pdf]

Watson, A. B. and Pelli, D. G. (1983) QUEST: a Bayesian adaptive psychometric method. Perception and Psychophysics 33, 113-120

Thanks to George Sperling for suggesting that a tutorial and examples would be useful.