Augmented Reality in Three.js
What Will I Talk About ?
- Doing Augmented Reality
- Using Only Web Technology
- Using Only Open Source
To Do Augmented Reality
With Web Technology
And Open Source
First, Let's See What We Need.
Ingredients Needed For AR
- Get a camera feed
- Analyze it to localize AR Markers
- Generate 3d on top
- Finally display both, 3d and video, on the screen
How To Do It With The Web ?
With A Pinch of Web Standards
Camera Feed
- Simple color video like Webcam
- webrtc/getUserMedia - spec - example
- Supported on Desktop and Android mobile
- Not IOS unfortunatly - details
Display 3D
- WebGL - spec
- Standard to display 3d on the web
- Based on openGL 2.0
- Available on desktop and modern mobile phones
- Javascript to code the app
- CSS to layout elements on the page
The web contains all the Ingredients needed For Augmented Reality
Result
- We tried and it works!
- We have augmented reality in browsers.
- It EVEN runs in browser and mobile phones.
- All open source, so you can use it.
Sample Code
- It is provided as sample code for you to build on
- Thus you focus on your own app
- Available on github
Let's See What We Can Do With Them
Being Awesome - link
Dynamic Histogram in AR on Mobile
Display Info From a Database on per-marker basis
A Spoon of Open Source
- three.js to ease webgl display
- jsaruco to find AR markers within video streams
Display WebGL
- Three.js javascript library from mrdoob
- Leading library to display Webgl
- MIT license, so easy to integrate
- Runs on desktop and mobile.
AR Marker Recognition
Several possibilities, we focused on
- jsaruco - current implementation
- jsartoolkit - to be implemented
js-aruco
ArUco library is a minimal library for Augmented Reality
applications based on OpenCv.
Extensions For Augmented Reality
See README.md for Details
Examples walk through
- can be found in
/examples - link
- Good place to learn how to use the code
- Experiment with multiple type of applications
basic.html
basic.html
- source written to be simple and clear
- Good for getting started
- Display debug informations to ease understanding
data-visualization-histogram3d.html
data-visualization-histogram3d.html
- source
- Data visualisation with a 3d histogram
- Change the value the way you like
- Information source may be a REST API
- or from an industrial sensor
Up to you
- source
- Simple App to share contact in AR.
- Show informations on a per-marker basis.
- Informations are from database.
Looks Like That
Not too sexy but practical
Markers Details
- Each got a given ID
- From 0 to 1023
- Simple to recognize many markers
This One is 265
How to Generate Other Markers
They are the base
Let's see how to handle them
Display it on Mobile
Display it on Mobile
- Load the image on your phone or tablet
- Put the mobile in front of the camera
- CON: mobile screens are reflective
- PRO: highly available
Print It on Paper
Print It on Paper
- CON: Requires a printer
- PRO: No/little reflection - so ease recognition
- So paper is usually better
- Glue it on cardboard for robustness
How Does Web Compare with Native ?
PRO Web
- Lots less code to write
- no need for android version + IOS version
- Easier to learn
- More standard
- Less Varied
PRO Native
- Better controls on running platform
- e.g. can't read webcam in IOS browser
- but possible in IOS native
- More efficient
- native typically runs faster
- less latency
- more battery life
Choosing Between Native and Web
It depends on your goals
For serious, long term project, native may be better
For fun, short project, go for web technology
Whatever you choose, Native or Web
The importance is to experience AR
So now that we have all the ingredients
Time to cook it up!
To do AR is to work on the future
