Usability Kit
The session began with demo of the "usability kit" which was developed by Máirín Duffy and Ray Strode which records the keyboard, screen, and the user's face. The box contains a DVR-style device which records all four video inputs. It's using embedded Linux. Total cost of all hardware including VGA scan converter: $800.
A set of tasks are given by the test giver to be accomplish tester. The analysis is done after-the-fact where she fills in the user's behavior when given the task to complete. The video provides actual documentation between the lag of a user click and the responsiveness of the UI (especially helpful on a web site design).
She tests a wide range of users: developers and regular users.
Máirín recommends narrowing the scope down to a very specific set of tasks. Don't just say, "I'm going to test GNOME." Narrow it down to "File Management" and then come up with a list of tasks related to that and measure those.
Ray finished the session by showing how they use gstreamer to stitch the four separate video feeds in to a single video.
The floor was opened to discussion on how to do constructive user-army sourcing of input in a way that is useful for improving usability.