Introduction
Eye and gaze tracking have been the domains of high end technology for a long time. But in the first week of June, computer scientists from Brown University have released a javascript library titled WebGazer developed by them that brings eye tracking within the reach of anyone with a website. Using the webcam of the computer or front cameras on mobile phones, this could be used by website owners to understand their users’ areas of interest better and provide them content aligned with those interest areas in real time.
The eye tracking model is based on self calibration by analyzing the pattern of where the user is looking when he clicks a particular location on the website. The entire development has two main parts – a pupil detector that can be used with any eye detection library and a gaze estimator that uses regression analysis based on user interactions. This allows the software’s self calibration to go on while the user is browsing the web. The images collected by the webcam during such browsing sessions can be analyzed to store what the user’s eyes look like. After this, similar interactions will also be stored and the two will be compared to understand the relationship between eye features with respect to on-screen gaze location. This will allow the gaze to be tracked even when the user is not interacting.
That being said, that is not the only way in which WebGazer tracks the gaze of the users. The second one treats the eye as a multidimensional feature vector and combines user interactions with regularized linear regression. This goes beyond the usage of clicks for tracking, it also takes into account cursor movement and eye-cursor coordination delay as vital inputs to estimate the gaze accurately.
The relative simplicity of WebGazer might be its strongest suit. It has no 3D capabilities that render it light and easy to implement into any browser and website. It makes up for the loss of accuracy there by constant calibration. Along with that, the constant calibration removes the need for any initial calibration session and lets the user be free to move during any tracking. Pretty soon, it might become internet norm to have websites tracking your gaze, quite like the crawlers of today
Getting WebGazer:
The entire WebGazer library is a simple javascript file available for download on their website.
To get it, simply go to https://WebGazer.cs.brown.edu/#download and download the file onto your machine. When using the library on a webpage, do remember to include it by including it as follows:
<script src=”WebGazer.js” type=”text/javascript” >
WebGazer Features:
Eye Tracking:
WebGazer uses the already available webcam on laptops, and front camera on mobiles to track the eye movement of the user by calculating the movement of the eye pixels at regular intervals. It includes a number of computer vision algorithms known as tracker modules. These are used to determine how the information of eye movement should be calculated. The three standard tracker modules that ship with WebGazer.js are .clmtrackr, js_objectdetect, tracking.js. However, other algorithms can also be configured allowing for higher degrees of customization.
Machine Learning:
The library continuously tracks user clicks and mouse movements and then uses the information to self-calibrate itself, improving the accuracy of prediction of eye movements. It uses a regression model to process the input.
Just like tracker modules, a number of regression modules are available for developers to choose from, like:
• ridge
• weightedRidge
• threadedRidge
• linear
Browser Support
WebGazer uses the Stream API that is supported in most major browsers. These include
• Chrome 47 and above.
• FireFox 44 and above
• Opera 36 and above
• Microsoft Edge 13 and above
Applications:
An efficient eye tracking tools has many applications and it is only limited by the capabilities of the hardware. Some of the significant ones are:
• e-Commerce – Online shopping information is one of the goldmines that almost every e-commerce sites track these days using crawlers and other means. But that highly relies on you actually interacting with the website, which might not always be the gaze. With this library, such ecommerce websites will now be able to track where you are looking, which product holds your gaze longer, what are the things you notice first on a product page and what gets ignored.
• Advertising – The principles of e-commerce also apply to online advertising in terms of tracking. An advertiser would definitely want to know if people are even looking at his ads, whether they are actually ignoring it or is it catching their eyes. This also helps website owners to place ads better so that they receive more attention. With possibilities of dynamic generation, analysing a gaze pattern over time might even lead to targetted advertising, with a website loading differently for users with different gaze patterns.
• Gaming – The ability to track the gaze opens multiple new doors for gaming. A player can analyze their gaze pattern over multiple matches to know where they are going wrong or figure out weak spots. It would be handy if you can figure out your blind spots in Dust II, wouldn’t it? Although there are other gaze tracking technologies, the lightness of this makes it ideal for gaming, where performance needs to be optimum. Beyond self analysis and improvement, gaze tracking in gaming can be a completely new method of interaction. Consider this, you are playing a highly detailed single player game and an object catches your attention. Gazing at it ingame brings up detailed information and a zoomed in view of the object. These are only a couple of possibilties and there are endless new ways to game with this.
• Virtual Reality – If there is one area that can benefit endlessly from gaze tracking it is Virtual Reality. Even though there aren’t really any mainstream websites that are completely VR enabled , mobile VR already tracks the zone you are facing long enough to detect that as a click. With advanced gaze tracking, it is highly likely that simple interactions like scrolling, zooming in, minimising and maximising can be done via gaze changes.
Also, since rendering VR can be quite tasking for most processors and GPU’s, especially when it is within a browser, it might make sense to restrict high quality renders to the area where the user is looking.
Disadvantages:
Since this tool essentially democratizes tracking, there are several possible concerns:
• Privacy – We do not need to tell you explicitly that giving access to your webcam might open doors to hackers, letting them see you and your surroundings, but we still will. Unless you are completely sure of the websites credentials, or a completely new and reliable certificate based on this is released, it would probably be wise to grant access quite rarely.
• Performance – Any device, irrespective of how light and resource friendly it is, does have a lifetime. WebGazer essentially takes control of your webcam continuously, constantly recording the frames and analysing them to get a better estimate of your gaze. Unless you have a really high quality webcam, this might lead to an early demise of that device.
• Accuracy – Technically, we do look at what we are interacting with online, but it is not always completely precise. WebGazer can at the most get an estimate of the general are we are looking at for quite a while as of now. Even if it could precisely pinpoint the location of the user’s gaze, it might not necessarily be the exact thing we are interacting with and that parameter would vary from website to website.
Creating The WebGazer.
Here we create a simple program that uses WebGazer to generate a heatmap of the user over a period of one day
STEP 1. Create a simple HTML Page with a button in it which onclick will call a js function named “stopTracking”.
<button onclick=”stopTracking()” > Stop Tracking </button>
STEP 2. Include the WebGazer.js (https://WebGazer.cs.brown. edu ) and simpleheat.js (https://github.com/mourner/simpleheat) in the html file.
<script src=”WebGazer.js”></script>
<script src=”simpleheat.js”></script>
STEP3. Now add the following snippets in a script:
3a. Set the regression and tracker modules:
window.onload = function() {
WebGazer.setRegression(‘weightedRidge’)
//different regression modules can be used.. Refer documentation
.setTracker(‘clmtrackr’) //three trackers are available by default, refer documentation
.setGazeListener(function(data, clock) {
/* data: is an object containing an x and y key which are the x and y prediction coordinates (no bounds limiting) , clock: elapsed time in milliseconds since WebGazer.begin() was called */
})
.begin()
.showPredictionPoints(true); /* shows a square every 100 milliseconds
where current prediction is */
3b. The Setup with the video in a frame can be done as follows:
var width = 320;
var height = 240;
var topDist = ‘0px’;
var leftDist = ‘0px’;
var setup = function() {
var video = document.getElem
entById(‘WebGazerVideoFeed’);
video.style.display = ‘block’;
video.style.position = ‘absolute’;
video.style.top = topDist;
video.style.left = leftDist;
video.width = width;
video.height = height;
video.style.margin = ‘0px’;
WebGazer.params.imgWidth = width;
WebGazer.params.imgHeight = height;
var overlay = document.
createElement(‘canvas’);
overlay.id = ‘overlay’;
overlay.style.position = ‘absolute’;
overlay.width = width;
overlay.height = height;
overlay.style.top = topDist;
overlay.style.left = leftDist;
overlay.style.margin = ‘0px’;
var myCanvas=get(‘canvas’);
myCanvas.width=screen.width;
myCanvas.height=screen.height;
document.body.appendChild(overlay);
var cl = WebGazer.
getTracker().clm;
function drawLoop() {
requestAnimFrame(drawLoop);
overlay.getContext(‘2d’).
clearRect(0,0,width,height);
if (cl.getCurrentPosition()) {
cl.draw(overlay);
}
}
drawLoop();
};
3c. Now let us check if Webgazer is ready and call the setup method as defined previously:
function checkIfReady() {
if (WebGazer.isReady()) {
setup();
} else {
setTimeout(checkIfReady, 100);
}
}
setTimeout(checkIfReady,100);
};//The window.onload function which we started in step 3a ends here
STEP 4. Optionally save the data into local web storage or clear it as you wish…
window.onbeforeunload = function() {
WebGazer.end(); //Saves the data even if you reload the page.
//window.localStorage.clear(); //
UnComment if you don’t want to save data across different sessions
}
Stopping the Webgazer and Creating a Heatmap
STEP 1.
Define the stopTracker function that is called on clicking the button
/*Helper Function*/
function get(id) {
return document.getElementById(id);
}
function stopTracking()
{
WebGazer.end(); //Save Webgaze Data var dataset=JSON.parse(window.
localStorage.WebGazerGlobalData); //
Read Data from Local Webstorage
//create data in the form of “simpleheat” input var data=[];
for(var i=0;i<dataset.data.length;i++)
{
data.push(dataset.data[i].screenPos);
}
//Hide the Video Overlay get(‘overlay’).style.display=’none’;
//Display The Heat Map get(‘canvas’).style.display=’block’;
generateHeatMap(data);
}
STEP 2.
Define the generateHeatMap function that renders the heat map
and hides the video overlay function generateHeatMap(data){
/*A heat map generator using “simpleheat.js” (https:// github.com/mourner/simpleheat) to show your most viewed areas in a heatmap representation*/
window.requestAnimationFrame=
window.requestAnimationFrame|| window.
mozRequestAnimationFrame||
window.webkitRequestAnimationFrame
|| window.msRequestAnimationFrame;
var heat = simpleheat(‘canvas’).data(data).max(18),
frame;
function draw() {
console.time(‘draw’);
heat.draw();
console.timeEnd(‘draw’);
frame = null;
}
draw();
}
Inferences:
• The tracking modules are a bit difficult to work with, clmtracker was the fastest of the lot, while trackingjs was relatively slower and less accurate.
• The different regression models tend to learn at nearly the same pace with the threaded Ridge model being the fastest (as data training was on a separate thread).
• Even with 2-3 hrs of training we were unable to achieve less than 500 pixel accuracy of gaze prediction.
Conclusion:
Computer vision is a rapidly expanding field in the science of computing with amazing potential.
WebGazer seems to be a unique way to leverage it in an attempt to better understand user interaction and interest ,that can in turn help us to further enhance user engagement as well as interaction.
The possibilities for WebGazer seem to be endless, from better understanding user choices and interest on an eCommerce website to gaze based interaction in gaming. Even in mobile applications, WebGazer represents an easy way for developers to harness a new way of user interaction. It can prove to be as disruptive in its innovation as VirtualReality and even more widespread because of its simplicity.
However the ethical concerns of allowing an application to access your webcam to ‘see’ you and your surroundings are significant. Such permissions would only be reasonable for completely trustworthy websites or apps and even then would be hard to acquire.
Although both Computer Vision and machine learning are preexisting technologies, this library is probably the first attempt to combine them in such a novel way. But above all it is the simplicity and ease of use of this javascript library that makes it a truly unique and democratized tool.
With the advent of dedicated vision processing units, this development might lead to interesting possibilities in the fields that we discussed earlier. A computer that can actually see, might be able to do a lot more with the ability to estimate your gaze. From extremely realistic human-machine interaction, to machines that can actually tell when you are having trouble focussing on your work, the machines of the future can and will truly look back at you.
