Four Signals, One Flight: Measuring Workload with EEG, Eye Tracking, ECG and GSR

Human factors work has been drifting out of the tightly controlled lab and into tasks that look like the real thing. The problem with that move is synchronisation: the moment you add a second and third measurement stream, you need them aligned tightly enough that an event in one is meaningful against the others.

A recent post on the g.tec blog walks through a demonstration that does exactly this — EEG, ECG, galvanic skin response and eye tracking recorded together during a flight simulation, in a setup prepared in about half an hour.

The experiment

The demonstration ran as a workshop at University College London with Professor Tom Carlson, with students from UCL's MSc in Rehabilitation Engineering and Assistive Technologies setting up and running it.

Participants flew two complete cycles around Cape Town in the freely available GeoFS simulator — take-off, free flight and landing — using a standard Cessna and a standard computer keyboard. The task is easy to learn but demands continuous attention, visual navigation and motor coordination, which gives you varying workload inside a reproducible scenario.

What was measured, and how

The setup combined four modalities, all synchronised and recorded in real time:

• g.Nautilus Multi-Purpose, 28-channel wireless EEG
• ECG, for heart rate variability
• Galvanic skin response
• Tobii Pro Glasses 3 for gaze position and pupil diameter

Alongside these, real-time signal validity monitoring flagged connection quality during the recording — which matters more in mobile setups than most people budget for.

The metrics themselves were derived online. Engagement was estimated continuously from the β/(α+θ) power band ratio, mental workload from gamma-band power, and heart rate variability from the ECG using RMSSD, with all values averaged over consecutive five-second epochs. g.tec is candid about the limitations here: gamma-band power was generally a less reliable indicator of mental workload, though it appeared particularly sensitive during stressful moments. That is a useful admission, and the kind of thing pilot studies exist to establish.

Acquisition and processing ran in g.HIsys, a MATLAB Simulink-based environment with processing blocks for filtering, band power, feature extraction and physiological metrics such as RMSSD and pNN50 — so custom multimodal workflows can be built with minimal programming.

Thirty minutes to prepared

The preparation detail is worth noting because it is usually where multimodal studies bleed time. The EEG cap mounts quickly; ECG needs a single electrode placed opposite the EEG reference at A2; GSR sensors go on the index and ring fingers; and the trigger output of the Tobii glasses connects directly to the amplifier to synchronise gaze events with the physiological recordings. Once prepared, participants stay fully mobile.

Where the eye tracking earns its place

After recording, gaze markers generated in Tobii Pro Lab are imported into g.BSanalyze through a batch tool. Those markers identify events such as take-off, approach and landing — but they can also define Areas of Interest, detect when a participant looked at a specific flight instrument, segment EEG epochs and support event-related potential analysis such as P300.

That is the real argument for doing this in one platform. Gaze stops being a parallel dataset you correlate afterwards and becomes the event structure the EEG analysis is built on.

Adapting it

The model is deliberately generic. Sensitivity of each parameter can be adjusted because the recorded data are not normalised during acquisition, cognitive metrics can be computed from a single channel, a channel pair or an average across channels, and epoch length can be matched to the paradigm. For pilot work, this is how you find out which parameters and channels actually capture the effect you are looking for before committing to a full protocol.

NBT represents g.tec in Israel across the g.Nautilus, g.HIamp and g.USBamp ranges, along with g.HIsys, g.BSanalyze and the Unicorn platform — and we also supply Tobii eye tracking. Contact us to talk through a multimodal setup for your lab.

Source: g.tec INSIGHTS blog.