Whiteboards let us quickly sketch ideas in a visual way. And vision-based augmented reality lets us align virtual content with visual information. As a by-product of this process, vision-based augmented reality takes measurements of the world around us. In this project we combine these concepts in a new example application.

The on-board camera of the smart glasses is used to measure content drawn on a whiteboard in real-time. As the user draws lines and curves on the whiteboard, the smart glass display gives a graphical confirmation that the system has registered what was drawn. The shape of curves seen by the camera is then processed by the smart glass computer and output as sound waves.

This allows the user to draw sounds and hear them. We may think of this as a ‘whiteboard sound synthesizer’, hence the project name.

User draws waveform on whiteboard whilst wearing smart glasses.
Smart glass camera measures waveform and shows overlay for confirmation.
Smart glass computer processes measurements from camera and outputs as sound waves for user to hear.