Web Gestures

a tQuery way to interact with your Browser

Jerome Etienne

Who Am I

Learning Three.js


So what is this ?

What are Augmented Web Gestures


  1. User moves in front of the camera
  2. Browser analyse the images from the webcam
  3. They are converted into user events


Show, Dont tell!

Doom: a new workout for geeks!

"Show me the code" - linus torwald

Coding with augmented gesture

Initialize it

var aGesture	= new tQuery.AugmentedGesture();

Add Debug View

// get debug texture
var texture	= aJoystick.texture();	
// put the debug texture on top on the screen
var domElement	= texture.image;

Get a Gesture out of Pointers

Get Pointers Coords

aGesture.bind('update', function(pointerR, pointerL){
	console.log(pointerR, pointerL);

Recognize the Gestures

var gestureR	= new tQuery.GestureRecognition();
aGesture.bind('update', function(pointerR, pointerL){
	var eventR	= gestureR.update(pointerR.x, pointerR.y, canvasW, canvasH);
	// eventR === null if nothing recognize
	// eventR === 'punch' if a punch if recognize

Pack it with Doom Character

Get MD2 character

var character	= new tQuery.RatamahattaMD2Character();

Action from Gestures

if( eventR === 'punch' ){
    character.animation('crdeath');   // make character move
    // play a sound
    tQuery.createSound().load('eatghost.mp3', function(sound){;
    // change the weapon
    character.setWeapon(Math.floor(Math.random()*11 % 11));

Not bad, what is under the hood ?

Augmented Web Gestures : Fact sheets

All open technology

All inside your web browser. pure web

Very efficient image processing

Image Processing Requirements



Easy to setup

Comparison with alternatives

Previous work

Augmented Web Gesture

What are the capabilities ?

Localize Two Hands

two pointers


hmm what about a actual application ?

Doom: a new workout for geeks!

Seems promising... What's next ?

Future works

on Medical Apps

on Game

on other fields i dunno yet :)

That's all folks.

Question ?

Jerome Etienne