Physiology, video and gaze in one observation lab

Emotion research, usability testing, parent-infant interaction, social stress protocols and consumer research share a common problem. The behaviour of interest is visible in the room, the autonomic state accompanying it is not, and the participant's gaze moves faster than either.

Video records what happened. Physiology records autonomic state, through heart rate variability, impedance cardiography and electrodermal activity. Eye tracking records where attention was directed. Coded behaviour converts the recording into analysable data.

Individually each is a partial account. Placed on one timeline they support a claim that none supports alone: that a specific event produced a specific autonomic response while the participant was looking at a specific thing.

The engineering problem is the timeline. This guide sets out how it is built, and the distinction that governs the whole room: the path the data travels is not the path the timing travels.

Two flows, and why they are separate

Establish this before any cabling is specified.

Data flow is where each recording is written. Physiology is written on the BioLab PC. Video is written by MediaRecorder. Gaze is written on the Tobii recording unit, on the participant. These are separate stores and they stay separate.

Trigger flow is what establishes the common timeline. It is a small number of low-latency signals whose only job is to mark the same instant in more than one recording.

Conflating the two is the usual source of failure. A stream can arrive reliably and still be useless for analysis if its timing arrived by a different route than its data.

Data flow: physiology

The MindWare Mobile is worn by the participant and transmits over wifi. That wifi is provided by a dedicated router supplied with the Mobile, and the router connects to the BioLab PC by a LAN cable into the Ethernet port.

The participant is therefore free to move. The BioLab PC sits at the far end of an Ethernet cable and never joins the wireless network itself.

The system acquires from up to 8 participants simultaneously, all written to the same BioLab PC. A dyad or a small group task is one record.

Data flow: video

BioLab starts MediaRecorder. The physiology record and the video therefore share a start point, and it is not reconfigured per session.

This is the one place where a data path also delivers timing, because the shared start is the alignment.

Trigger flow: stimulus onsets and responses

The MindWare TTL cable has an LPT termination at both ends. At the stimulus PC it connects through a USB to LPT converter supplied by MindWare, and the other end runs to the BioLab PC.

Most current PCs have no LPT port, which is usually true of the BioLab machine as well, so in practice a second converter is required at that end. Count the ports before ordering.

E-Prime or Python sends the code, and the TTL events are registered in millisecond accuracy alongside the ECG and the electrodermal activity in the same record.

That is what places stimulus and physiology on one timeline. Not a post hoc alignment, and not an inference from a log file.

For responses, add the MindWare adaptor for the Chronos. The standard MindWare TTL cable then fits the Chronos, and the far end goes to the BioLab PC as above. Response marks and stimulus onsets travel the same path.

Trigger flow: the eye tracker

This is the part most often misunderstood, so it is worth stating precisely.

The Tobii recording unit outputs a signal on an AUX cable whose far end has a termination fitting the 5th channel on the MindWare Mobile. That cable carries a synchronisation signal, not gaze data. The gaze recording itself remains on the Tobii unit and is retrieved separately.

What the channel carries is the Tobii Glasses 3 clock: a rectangular signal every 10 seconds, which appears in the physiology record as a clean pulse train. The gaze data is aligned to the physiology afterwards using that pulse train.

A mobile eye tracker with its own clock, on a participant who is walking, is tied to the physiology by one cable and a repeating square wave. Confirm the pulse train is present and regular in the physiology record before the participant is released. A cable that was not seated fully is best discovered then.

Running several Tobii Glasses 3 simultaneously is a separate question, and the answer depends on the intended acquisition method. Ask us and we will go through the options.

The shortcut we would not take

BioLab accepts UDP commands to start acquisition, stop it and log events over the network. It requires no additional hardware.

MindWare state in their own documentation that UDP commands to BioLab are subject to network and processing delays and are not a substitute for external trigger and event lines when precise timing is required.

Their published measurements make the case better than the caveat does. Testing 1000 pulses across four sessions on a wired gigabit connection, MindWare report an average delay of 6.2 ms, a standard deviation of 22.2 ms, and a maximum delay of 245 ms. The average is tolerable. The standard deviation and the maximum are not, and they are the reason the marks you intend to analyse belong on a cable.

MindWare also note that where millisecond precision is required, the synchronous digital ports on the 8-slot BioNex are the correct event capture route.

Use UDP for start and stop. Use the cable for events.

The Observer XT sits outside the acquisition synchronisation

This surprises people, so it is worth stating plainly. The Observer XT is not part of the acquisition synchronisation, and does not need to be.

It is where coding and analysis happen, after the session. Observer XT accepts external data from BioLab, from E-Prime and from the glasses, so video, physiology, stimulus events and gaze occupy one project on one timeline, and behavioural codes are added on top.

The alignment itself was performed in hardware while the data was being recorded. That places the weight on the cabling. Correct wiring gives Observer XT a straightforward job. Incorrect wiring cannot be recovered in Observer XT afterwards.

What is tied to what

Where we would not spend the money

Where the protocol is a seated screen task and the participant does not leave the chair, the glasses are the wrong instrument. A screen-based tracker such as Tobii Pro Fusion provides higher sampling rates, a simpler cable run and a lower cost, and it removes the pulse-train alignment step entirely. Specify glasses when the participant has to move, and not before.

The same applies to camera count. MediaRecorder scales to eight channels. Eight angles of a room in which nothing occurs in six of them is eight streams to store and review. Specify the angles that will be coded.

Commissioning checklist

• Confirm the Mobile connects to its router and BioLab finds it, before anyone is wired up
• Check both ends of the TTL run for an LPT port, and order a converter for whichever end lacks one
• Send a known sequence of event codes and confirm every one appears in the BioLab record
• Confirm the glasses pulse train is present in the physiology record and arriving on schedule
• Run a full-length dummy session, then compare the first minute against the last
• Import one complete dummy session into Observer XT before running a real one
• Label the cables. In a room this dense, a swapped cable costs a full session

The parts

NBT supplies MindWare, Noldus, Tobii and Psychology Software Tools in Israel, and installs them.

Send us the protocol and the room, and we will specify what has to be wired and what does not. Is the participant remaining in the chair, or moving?