Getting touchy

Everything you (n)ever wanted to know about touch and pointer events

Getting touchy

Everything you (n)ever wanted to know about touch and pointer events

Patrick H. Lauke / Last major changes: 8 May 2017

about me...

github.com/patrickhlauke/getting-touchy-presentation

"evergreen" expanded version of this presentation
(and branches for specific conferences)

patrickhlauke.github.io/touch

Large number of mobile phone/tablet test devices

Touch/pointer events test results

CodePen: Basic slider - from mouse-only to mouse/touch/stylus enabled

my JavaScript sucks...
(but will hopefully convey the right concepts)

“how can I make my website
work on touch devices?”

you don't need touch events
browsers emulate regular mouse events

patrickhlauke.github.io/touch/tests/event-listener_mouse-only.html

patrickhlauke.github.io/touch/tests/event-listener_mouse-only.html

compatibility mouse events

(mouseenter) > mouseover > mousemove* > mousedown >
(focus) > mouseup > click

* only a single “sacrificial” mousemove event fired

on first tap

(mouseenter) > mouseover > mousemove >
    mousedown > (focus) > mouseup > click

subsequent taps

mousemove > mousedown > mouseup > click

tapping away

(mouseout) > (blur)

focus/blur only on focusable elements; subtle differences between browsers
Mobile/tablet touchscreen activation/tap event order

emulation works,
but is limiting/problematic

  1. delayed event dispatch
  2. mousemove doesn't track
  1. delayed event dispatch
  2. mousemove doesn't track

patrickhlauke.github.io/touch/tests/event-listener_show-delay.html

less of a problem in modern browsers, but for iOS < 9.3 or older Androids still relevant

patrickhlauke.github.io/touch/tests/event-listener_show-delay.html

  1. delayed event dispatch
  2. mousemove doesn't track

patrickhlauke.github.io/touch/particle/2

patrickhlauke.github.io/touch/particle/2

“we need to go deeper...”

touch events

introduced by Apple, adopted in Chrome/Firefox/Opera

(and belatedly IE/Edge – more on that later)

www.w3.org/TR/touch-events

caniuse.com: Can I use touch events?

touchstart
touchmove
touchend
touchcancel

patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html

patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html

Bug 128534 - 'mouseenter' mouse compat event not fired...

events fired on tap

touchstart > [touchmove]+ > touchend >
(mouseenter) > mouseover > mousemove > mousedown >
(focus) > mouseup > click

(mouse events only fired for single-finger tap)

on first tap

touchstart > [touchmove]+ > touchend >
(mouseenter) > mouseover > mousemove > mousedown >
(focus) > mouseup > click

subsequent taps

touchstart > [touchmove]+ > touchend >
mousemove > mousedown > mouseup > click

tapping away

mouseout > (mouseleave) > (blur)

some browsers outright weird...

Browser/Android 4.3

mouseover > mousemove > touchstart > touchend >
mousedown > mouseup > click

Browser/Blackberry PlayBook 2.0

touchstart > mouseover > mousemove > mousedown >
touchend > mouseup > click

UC Browser 10.8/Android 6

mouseover > mousemove > touchstart > (touchmove)+ > touchend >
mousedown > focus > mouseup > click

Touch/pointer events test results

shouldn't affect your code, unless you're expecting a very specific sequence...

<interlude >

simple feature detection for touch events

/* feature detection for touch events */

if ('ontouchstart' in window) {
    /* some clever stuff here */
}

/* older browsers have flaky support so more
   hacky tests needed...use Modernizr.touch or similar */

patrickhlauke.github.io/touch/tests/touch-feature-detect.html

/* conditional "touch OR mouse/keyboard" event binding */

if ('ontouchstart' in window) {
  // set up event listeners for touch
  ...
} else {
  // set up event listeners for mouse/keyboard
  ...
}

don't make it touch-exclusive

hybrid devices
touch + mouse + keyboard

Windows 2-in-1 touchscreen laptop/tablet devices
All-in-one windows desktop device with touchscreen
Tweet: a quick video showing the issue of naive 'if touch events then simply assume touch only' in flickr

@patrick_h_lauke showing a naive "touch or mouse" issue on Flickr
(which has since been fixed)

Bugzilla

Bug 888304 - touch-events on Firefox-desktop should be disabled until we can support them...

Issue 392584: TouchEvent API should be enabled consistently

Chrome now returns window.TouchEvent on non-touch devices

patrickhlauke.github.io/touch/tests/touch-feature-detect.html

even on "mobile" we can have multiple inputs...

HTC Hero touchscreen phone with built-in trackball

HTC Wildfire with optical trackball sends mouse events...

Android tablet-based laptops
Windows 10 Mobile phone using 'Continuum' - with a connected screen, mouse and keyboard

Windows 10 "Continuum" (mobile device acting as desktop)

Samsung Galaxy S8 plugged into a DeX Station - with a connected screen, mouse and keyboard

Samsung DeX (mobile device acting as desktop)

Samsung DeX brings a new Dimension to the Mobile Web

Android + mouse – like touch (mouse emulating touch)

touchstart > touchend > mouseover > mousemove > mousedown > (focus) > mouseup > click

Android + Chrome 58 + mouse – like desktop

mouse events (+ pointer events) + click

blink-dev: mouse on Android stops firing TouchEvents

Blackberry PlayBook 2.0 + mouse – like desktop

mouseover > mousedown > (mousemove)+ > mouseup > click

Blackberry Leap (BBOS 10.1) + mouse – like desktop

mouseover > mousedown > (mousemove)+ > mouseup > click

Windows 10 Mobile/Microsoft Edge + mouse – like desktop

mouse events (+ pointer events) + click

Windows 10 Mobile/Microsoft Edge + keyboard – like desktop

focus / click / blur

Android + keyboard – like desktop

focus / click / blur

iOS keyboard only works in same situations as on-screen keyboard
(e.g. text inputs, URL entry)

VoiceOver enables full keyboard access on iOS

iOS + keyboard – similar to touch (using VO+SPACE)

focus / touchstart > touchend > (mouseenter) > mouseover > mousemove > mousedown > blur > mouseup > click

iOS + keyboard – like desktop (using ENTER)

focus / click / blur

mobile Assistive Technologies
(e.g. screen readers on touchscreen devices)

iOS + VoiceOver (with/without keyboard) – similar to touch

focus / touchstart > touchend > (mouseenter) > mouseover > mousemove > mousedown > blur > mouseup > click

Android 4.3/Chrome + TalkBack – keyboard/mouse hybrid

focus / blur > mousedown > mouseup > click > focus

Android 6.1/Chrome + TalkBack – like touch

touchstart > touchend > mouseover > mouseenter > mousemove > mousedown > focus > mouseup > click

Android 6.1/Firefox + TalkBack – similar to touch

touchstart > mousedown > focus > touchend > mouseup > click

further scenarios?

note on trackpad gestures

no way to detect these cases...

Modernizr.touch detects touch events not touch devices

Modernizr: Detecting a mouse user

Stu Cox: You can't detect a touchscreen

hacks.mozilla.org - Detecting touch [...]

/* feature detection for touch events */

if ('ontouchstart' in window) {
    /* browser supports touch events but user is
       not necessarily using touch (exclusively) */

    /* it could be a mobile, tablet, desktop, fridge ... */
}

touch or mouse or keyboard

touch and mouse and keyboard

what about
CSS4 Media Queries?

W3C Media Queries Level 4 (Editor's Draft)

Interaction Media Features

pointer / hover relate to “primary” pointing device
any-pointer / any-hover relate to all pointing devices


/* the primary input is ... */

@media (pointer: fine)   { /* a mouse, stylus, ... */ }
@media (pointer: coarse) { /* a touchscreen, ... */ }
@media (pointer: none)   { /* not a pointer (e.g. d-pad) */ }

@media (hover: hover) { /* hover-capable */ }
@media (hover: none)  { /* not hover-capable */ }

hover: on-demand / any-hover: on-demand removed in recent drafts

Interaction Media Features

pointer / hover relate to “primary” pointing device
any-pointer / any-hover relate to all pointing devices


/* across all detected pointing devices at least one is ... */

@media (any-pointer: fine)   { /* a mouse, stylus, ... */ }
@media (any-pointer: coarse) { /* a touchscreen, ... */ }
@media (any-pointer: none)   { /* none of them are pointers */ }

@media (any-hover: hover) { /* hover-capable */ }
@media (any-hover: none)  { /* none of them are hover-capable */ }

hover: on-demand / any-hover: on-demand removed in recent drafts

caniuse.com: Can I use interaction media features?

-moz-touch-enabled in Firefox

Media Features and JavaScript


if (window.matchMedia("(any-pointer:coarse)").matches) { ... }

/* listen for dynamic changes */

window.matchMedia("(any-pointer:coarse)")↵
      .onchange = function(e) { ... }

window.matchMedia("(any-pointer:coarse)")↵
      .addEventListener("change", function(e) { ... } , false }

window.matchMedia("(any-pointer:coarse)")↵
      .removeEventListener("change", function(e) { ... } , false }

caniuse.com: Can I use matchMedia?

dev.opera - Interaction Media Features and their potential (for incorrect assumptions)

CSS4 Interaction Media Features tests on a touchscreen phone, with and without a paired bluetooth mouse - primary input remains touchscreen (so pointer:coarse and hover:on-demand), but additional capabilities of mouse are detected (any-pointer:fine and hover:hover also evaluated to true)

patrickhlauke.github.io/touch/pointer-hover-any-pointer-any-hover

primary input is unchanged (and Chrome/Android required full restart – Issue 442418)

/* Naive uses of Interaction Media Features */

@media (pointer: fine) {
    /* primary input has fine pointer precision...
       so make all buttons/controls small? */
}

@media (hover: hover) {
    /* primary input has hover...so we can rely on it? */
}

/* pointer and hover only check "primary" input, but
   what if there's a secondary input that lacks capabilities? */
/* Better uses of Interaction Media Features */

@media (any-pointer: coarse) {
    /* at least one input has coarse pointer precision...
       provide larger buttons/controls for touch  */
}

/* test for *any* "least capable" inputs (primary or not) */

@media (any-hover: none) {
    /* at least one input lacks hover capability...
       don't rely on it or avoid altogether */
}

Limitation: [mediaqueries-4] any-hover can't be used to detect mixed hover and non-hover capable pointers. Also, pointer/hover/any-pointer/any-hover don't cover non-pointer inputs (e.g. keyboards) – always assume non-hover-capable inputs

detect which input the users is using right now...

GitHub: ten1seven / what-input

Demo: What Input?

Office 2013 in Windows offers a 'Touch/Mouse mode' switch - shown by default on touch-capable devices, but can be accessed even on mouse-only devices; switching to touch mode increases spacing and touch target of all ribbon controls

if in doubt, offer a way to switch interfaces...
or just always use a touch-optimised interface

</ interlude >

touch events
vs
limitations/problems

  1. delayed event dispatch
  2. mousemove doesn't track
  1. delayed event dispatch
  2. mousemove doesn't track

patrickhlauke.github.io/touch/tests/event-listener_show-delay.html

less of a problem in modern browsers, but for iOS < 9.3 or older Androids still relevant

why the delay?
double-tap to zoom
(mostly anyway)

when does the delay happen?

patrickhlauke.github.io/touch/tests/event-listener.html

patrickhlauke.github.io/touch/tests/event-listener.html

touch / mouse events delay

touchstart > [touchmove]+ > touchend >

[300ms delay]

(mouseenter) > mouseover > mousemove > mousedown >
(focus) > mouseup > click

“how can we make it feel responsive like a native app?”

react to events fired before the 300ms delay...

touchstart for an “immediate” control
(e.g. fire/jump button on a game)

touchend for a control that fires after finger lifted

(but this can result in events firing after zoom/scroll)

don't make it touch-exclusive

/* DON'T DO THIS:
  conditional "touch OR mouse/keyboard" event binding */

if ('ontouchstart' in window) {
  // set up event listeners for touch
  foo.addEventListener('touchend', ...);
  ...
} else {
  // set up event listeners for mouse/keyboard
  foo.addEventListener('click', ...);
  ...
}

patrickhlauke.github.io/touch/tests/event-listener_naive-touch-or-mouse.html

/* DO THIS:
   doubled-up "touch AND mouse/keyboard" event binding */

// set up event listeners for touch
foo.addEventListener('touchend', ...);

// set up event listeners for mouse/keyboard
foo.addEventListener('click', ...);

/* but this would fire our function twice for touch? */

patrickhlauke.github.io/touch/tests/event-listener_naive-event-doubling.html

/* DO THIS:
   doubled-up "touch AND mouse/keyboard" event binding */

// set up event listeners for touch
foo.addEventListener('touchend', function(e) {
  // prevent compatibility mouse events and click
  e.preventDefault();
  ...
});

// set up event listeners for mouse/keyboard
foo.addEventListener('click', ...);

patrickhlauke.github.io/touch/tests/event-listener_naive-event-doubling-fixed.html

preventDefault() kills
scrolling, pinch/zoom, etc

apply preventDefault() carefully
(just on buttons/links, not entire page)

github.com/ftlabs/fastclick

Bootstrap slide-down navigation button has 300ms delay in iOS by default

patrickhlauke.github.io/touch/fastclick

YouTube: iOS/Safari 300ms click delay: vanilla Bootstrap and using fastclick.js

Bootstrap with simple FastClick.js addition - slide-down navigation button has no more delay

patrickhlauke.github.io/touch/fastclick/fastclick.html

YouTube: iOS/Safari 300ms click delay: vanilla Bootstrap and using fastclick.js

browsers working to remove double-tap to zoom delay

non-scalable/zoomable viewport and
"double-tap to zoom"

<meta name="viewport" content="user-scalable=no">

patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html

Firefox/Android support for user-scalable=no and click delay suppression.

<meta name="viewport" content="user-scalable=no">

patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html

... content="minimum-scale=1, maximum-scale=1"

patrickhlauke.github.io/touch/tests/event-listener_minimum-maximum-scale.html

Firefox/Android support for minimum-scale=1/maximum-scale=1 and click delay suppression.

... content="minimum-scale=1, maximum-scale=1"

patrickhlauke.github.io/touch/tests/event-listener_minimum-maximum-scale.html

Changeset 191072 - Web pages with unscalable viewports shouldn't have a single tap delay

(from iOS 9.3 onwards)

what about accessibility?

Chrome's 'Force enable zoom' setting

Chrome: Settings > Accessibility > Force enable zoom

Opera/Android's 'Force enable zoom' setting

Opera: Settings > Force enable zoom

Firefox/Android's 'Always enable zoom' setting

Firefox: Settings > Accessibility > Always enable zoom

Samsung Internet's 'Manual Zoom' setting

Samsung Internet: Internet settings > Manual zoom

When 'Force enable zoom' is turned on, users can still zoom, but the 300ms delay - even with user-scalable=no - returns. A fair compromise?

"Force enable zoom" reintroduces delay – a fair compromise?

patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html

Thomas Fuchs' tweet, including a screengrab from Safari/iOS10beta3 changelog: To improve accessibility on websites in Safari, users can now pinch-to-zoom

@thomasfuchs - Safari/iOS10beta3 ignores unscalable viewports

(no official Apple documentation about the change...but the 300ms delay is back)

New web platform features announcement: On Windows 10 Mobile devices, the user can now zoom into the page regardless of zoom settings defined by meta viewport

Windows 10 build 15007 on Mobile ignores unscalable viewports

"mobile optimised" viewport and
"double-tap to zoom"

Chrome 32+ / Android: content="width=device-width"
suppresses double-tap-to-zoom, still allows pinch zoom

Google Developers: 300ms tap delay, gone away

Bug 941995 - Remove 300ms [...] on "responsive" pages

[RESOLVED FIXED]

Bug 150604 - Implement viewport-width-based fast-click heuristic

(from iOS 9.3 onwards)

Bug 151077 - Fast-clicking should trigger when scale is equal to explicitly set initial scale

(from iOS 9.3 onwards)

WebKit blog: More Responsive Tapping on iOS

patrickhlauke.github.io/touch/tests/results/#suppressing-300ms-delay

suppressing 300ms delay

if your code does rely on handling click / mouse events:

  1. delayed event dispatch
  2. mousemove doesn't track

patrickhlauke.github.io/touch/particle/2

patrickhlauke.github.io/touch/particle/2

events fired on tap

touchstart > [touchmove]+ > touchend >
(mouseenter) > mouseover >
mousemove* > mousedown > (focus) >
mouseup > click

* mouse event emulation fires only a single mousemove

too many touchmove events prevent mouse compatibility events after touchend

doubling up handling of mousemove and touchmove

var posX, posY;

function positionHandler(e) {
    posX = e.clientX;
    posY = e.clientY;
}

canvas.addEventListener('mousemove', positionHandler, false);
var posX, posY;

function positionHandler(e) {
    posX = e.clientX;
    posY = e.clientY;
}

canvas.addEventListener('mousemove', positionHandler, false);
canvas.addEventListener('touchmove', positionHandler, false);

/* but this won't work for touch... */

the anatomy of Touch Events

interface MouseEvent : UIEvent {
    readonly attribute long screenX;
    readonly attribute long screenY;
    readonly attribute long clientX;
    readonly attribute long clientY;
    readonly attribute boolean ctrlKey;
    readonly attribute boolean shiftKey;
    readonly attribute boolean altKey;
    readonly attribute boolean metaKey;
    readonly attribute unsigned short button;
    readonly attribute EventTarget relatedTarget;
    // [DOM4] UI Events
    readonly attribute unsigned short buttons;
};

www.w3.org/TR/DOM-Level-2-Events/events.html#Events-MouseEvent

www.w3.org/TR/uievents/#events-mouseevents

partial interface MouseEvent {
    readonly attribute double screenX;
    readonly attribute double screenY;
    readonly attribute double pageX;
    readonly attribute double pageY;
    readonly attribute double clientX;
    readonly attribute double clientY;
    readonly attribute double x;
    readonly attribute double y;
    readonly attribute double offsetX;
    readonly attribute double offsetY;
};

www.w3.org/TR/cssom-view/#extensions-to-the-mouseevent-interface

interface TouchEvent : UIEvent {
    readonly attribute TouchList touches;
    readonly attribute TouchList targetTouches;
    readonly attribute TouchList changedTouches;
    readonly attribute boolean altKey;
    readonly attribute boolean metaKey;
    readonly attribute boolean ctrlKey;
    readonly attribute boolean shiftKey;
};

www.w3.org/TR/touch-events/#touchevent-interface

interface Touch {
    readonly attribute long identifier;
    readonly attribute EventTarget target;
    readonly attribute long screenX;
    readonly attribute long screenY;
    readonly attribute long clientX;
    readonly attribute long clientY;
    readonly attribute long pageX;
    readonly attribute long pageY;
};

www.w3.org/TR/touch-events/#touch-interface

(this has been amended/extended in later implementations – more on that later)

TouchList differences

touches
all touch points on screen
targetTouches
all touch points that started on the element
changedTouches
touch points that caused the event to fire

changedTouches depending on event:

patrickhlauke.github.io/touch/touchlist-objects

var posX, posY;

function positionHandler(e) {
    if ((e.clientX)&&(e.clientY)) {
        posX = e.clientX; posY = e.clientY;
    } else if (e.targetTouches) {
        posX = e.targetTouches[0].clientX;
        posY = e.targetTouches[0].clientY;
        e.preventDefault();
    }
}

canvas.addEventListener('mousemove', positionHandler, false );
canvas.addEventListener('touchmove', positionHandler, false );

TouchList collections order

Particle party with touch working correctly with moving finger on Android device

patrickhlauke.github.io/touch/particle/3

Particle field working on Android device

patrickhlauke.github.io/touch/particle/4a

patrickhlauke.github.io/touch/paranoid_0.5

www.splintered.co.uk/experiments/archives/paranoid_0.5

patrickhlauke.github.io/touch/picture-slider

implicit capture

(3) Naive mouse-driven fake slider (doesn't "capture")

Demo: without tricks, slider won't work when moving mouse outside

(4) Basic mouse-driven fake slider

Exercise/demo: expand mouse-driven code to also work with touch

(5) Basic mouse- and touch-driven fake slider

tracking finger movement over time ... swipe gestures

patrickhlauke.github.io/touch/swipe

patrickhlauke.github.io/touch/swipe

/* Swipe detection from basic principles */

Δt = end.time - start.time;
Δx = end.x - start.x;
Δy = end.y - start.y;

if ((Δt > timeThreshold) || ((Δx + Δy) < distanceThreshold)) {
  /* too slow or movement too small */
} else {
  /* it's a swipe!
     - use Δx and Δy to determine direction
     - pythagoras √( Δx² + Δy² ) for distance
     - distance/Δt to determine speed */
}

don't forget mouse/keyboard!

Brad Frost's responsive demo site, featuring a combination of swipe-able image carousel in addition to image thumbnails, allowing BOTH swipe and click/keyboard interaction

bradfrostweb.com/demo/mobile-first

touchmove fires...a lot!

do absolute minimum on each touchmove
(usually: store coordinates)

do heavy lifting (drawing etc.) separately, avoid queueing
(e.g. using setTimeout and/or requestAnimationFrame)

debounce / throttle events

GitHub: m-gagne / limit.js

Function.prototype.debounce = function (milliseconds, context) {
  var baseFunction = this, timer = null, wait = milliseconds;
  return function () {
    var self = context || this, args = arguments;
    function complete() {
      baseFunction.apply(self, args);
      timer = null;
    }
    if (timer) {
      clearTimeout(timer);
    }
    timer = setTimeout(complete, wait);
  };
};

window.addEventListener('touchmove', myFunction.debounce(250));
Function.prototype.throttle = function (milliseconds, context) {
  var baseFunction = this, lastEventTimestamp = null,
      limit = milliseconds;
  return function () {
    var self = context || this, args = arguments, now = Date.now();
    if (!lastEventTimestamp || now - lastEventTimestamp >= limit) {
      lastEventTimestamp = now;
      baseFunction.apply(self, args);
    }
  };
};

window.addEventListener('touchmove', myFunction.throttle(250));

patrickhlauke.github.io/touch/touch-limit

why stop at a single point?
multitouch support

interface TouchEvent : UIEvent {
    readonly attribute TouchList touches;
    readonly attribute TouchList targetTouches;
    readonly attribute TouchList changedTouches;
    readonly attribute boolean altKey;
    readonly attribute boolean metaKey;
    readonly attribute boolean ctrlKey;
    readonly attribute boolean shiftKey;
};

www.w3.org/TR/touch-events/#touchevent-interface

/* iterate over relevant TouchList */

for (i=0; i<e.targetTouches.length; i++) {
    ...
    posX = e.targetTouches[i].clientX;
    posY = e.targetTouches[i].clientY;
    ...
}

patrickhlauke.github.io/touch/tracker/multi-touch-tracker.html

iOS/iPad preventDefault() can't override 4-finger gestures

iOS7+ Safari/WebView preventDefault() can't override edge swipes

Multitouch tracker in iOS/Safari - starting a finger movement on the edge of the screen starts a back/forward gesture that can't be overridden
Multitouch tracker in iOS/Safari - starting a finger movement on the edge of the screen starts a back/forward gesture that can't be overridden

YouTube: iOS/Safari (iPhone) can't prevent edge swipe gestures

Multitouch tracker in iOS/Chrome - starting a finger movement on the edge of the screen starts a back/forward gesture that can't be overridden

Chrome/iOS (WebView) custom swipe, also can't be prevented

patrickhlauke.github.io/touch/tracker/multi-touch-tracker.html

multitouch gestures

/* iOS/Safari/WebView has gesture events for size/rotation,
not part of the W3C Touch Events spec */

/* gesturestart / gesturechange / gestureend */

foo.addEventListener('gesturechange', function(e) {
    /* e.scale
       e.rotation */

    /* values can be plugged directly into
       CSS transforms etc */
});

/* not supported in other browsers, but potentially
   useful for iOS exclusive content (hybrid apps/webviews) */

iOS Developer Library - Safari Web Content Guide - Handling gesture events

iOS7/Safari two-finger getures demo - showing raw scale/rotation numbers

patrickhlauke.github.io/touch/iosgestures

iOS7/Safari two-finger getures demo - showing scale/rotation applied to an image

patrickhlauke.github.io/touch/iosgestures/image.html

/* Pinch/rotation detection from basic principles */

Δx = x2 - x1;
Δy = y2 - y1;


/* pythagoras √( Δx² + Δy² ) to calculate distance */

Δd = Math.sqrt( (Δx * Δx) + (Δy * Δy) );


/* trigonometry to calculate angle */

α = Math.atan2( Δy, Δx );

Mozilla Developer Network: Math.atan2()

Android/Chrome demo of two-finger scale/rotation calculated from basic principles

patrickhlauke.github.io/touch/pinch-zoom-rotate

Picture Organiser demo on iOS7/iPad, with four images freely moved/scaled/rotated

patrickhlauke.github.io/touch/picture-organiser

not all old/cheap devices/OSs support multitouch!

HTC Hero – Android 2.1

HTC Hero - Android 2.1/Browser two fingers on the screen, but only the first registers as a touch point

LG Optimus 2X – Android 2.3.7

LG Optimus 2X - Android 2.3.7/Browser two fingers on the screen are averaged as a single touch point only

ZTE Open – Firefox OS 1.1

ZTE Open Firefox OS 1.1 only supports 2 fingers

Touch Events specification
grey areas...

do touch events fire during scroll/zoom?

different models and behaviours, particularly in old browsers

YouTube: Touch Events and scrolling/zooming (playlist)

e.g. older versions of Chrome fire touchcancel on scroll/zoom

YouTube: Google Developers - Mobile Web Thursdays: Performance on Mobile

not defined in spec (yet), but de facto
yes in most modern browsers

patrickhlauke.github.io/touch/gesture-touch

touchmove and scroll

patrickhlauke.github.io/touch/pull-to-refresh/...

non-standard iOS quirks...

making generic elements clickable...

Historically (until end of 2016?), Apple suggested adding dummy onclick="" handlers
iOS Developer Library - Safari Web Content Guide - Handling Events

dummy onclick="" handlers are unnecessary (at least since iOS6)
Apple quietly dropped this from their documentation...

patrickhlauke.github.io/touch/ios-clickable/

mouse + click event bubbling on touch
(important when using event delegation)

Quirksmode: Mouse event bubbling in iOS

if target is a button, the touch, mouse and click events all bubble like normal

patrickhlauke.github.io/touch/bubbling/button.html

if target is a link, the touch, mouse and click events all bubble like normal

patrickhlauke.github.io/touch/bubbling/link.html

if target is a div, only the touch events bubble

patrickhlauke.github.io/touch/bubbling/div.html

adding an empty onclick attribute on the target div makes touch, mouse and click events bubble as normal again

patrickhlauke.github.io/touch/bubbling/div-onclick-target.html

adding an empty onclick attribute on any of the target's ancestors (between the target and the actual event listener) makes touch, mouse and click events bubble as normal again

patrickhlauke.github.io/touch/bubbling/div-onclick-ancestor.html

adding an empty click event listner on the target div makes touch, mouse and click events bubble as normal again

patrickhlauke.github.io/touch/bubbling/div-addeventlistener-target.html

adding an empty click event listner on any of the target's ancestors (between the target and the actual event listener) makes touch, mouse and click events bubble as normal again

patrickhlauke.github.io/touch/bubbling/div-addeventlistener-ancestor.html

adding cursor:pointer CSS on the target div makes touch, mouse and click events bubble as normal again

patrickhlauke.github.io/touch/bubbling/div-cursorpointer-target.html

adding cursor:pointer CSS on any of the target's ancestors (between the target and the actual event listener) makes touch, mouse and click events bubble as normal again

patrickhlauke.github.io/touch/bubbling/div-cursorpointer-ancestor.html

mouse + click bubbling in iOS

Quirksmode: Mouse event bubbling in iOS

WebKit Bug 171105 - Normalizing touch events mouse compatibility event bubbling

real-world example: force bubbling by adding/removing noop mouseover event handlers

iOS quirks in a nutshell

if your code does rely on handling mouse events:

Touch Events extensions...

W3C Touch Events Extensions WG Note

/* Extension to touch objects */

partial interface Touch {
  readonly  attribute float radiusX;
  readonly  attribute float radiusY;
  readonly  attribute float rotationAngle;
  readonly  attribute float force;
};

Google Developers: Using rotationAngle and touchRadius

/* Touch Events – contact geometry */

partial interface Touch {
  readonly  attribute float radiusX;
  readonly  attribute float radiusY;
  readonly  attribute float rotationAngle;
  readonly  attribute float force;
};

patrickhlauke.github.io/touch/tracker/tracker-radius-rotationangle.html

YouTube: Touch Events: radiusX, radiusY and rotationAngle

Rick Byers - Paint (with rotationAngle and touchRadius)

/* Touch Events – force */

partial interface Touch {
  readonly  attribute float radiusX;
  readonly  attribute float radiusY;
  readonly  attribute float rotationAngle;
  readonly  attribute float force;
};

force: value in range 01. if no hardware support 0
(some devices, e.g. Nexus 10, "fake" force based on radiusX / radiusY)

patrickhlauke.github.io/touch/tracker/tracker-force-pressure.html

iPhone 6s with 3D Touch supports force...

(Safari and WKWebView, e.g. Chrome/iOS, but not UIWebView, e.g. Firefox/iOS)

patrickhlauke.github.io/touch/tracker/tracker-force-pressure.html

...while in non-3D Touch models (e.g. iPhone 5c) force == 0

3D Touch ≠ Force Touch

Safari Developer Library: Responding to Force Touch Events

Google Developers: Precision Touch for Precise Gestures

Bug 133180 - Touch events should allow for device-pixel accuracy

interface Touch {
    readonly attribute long identifier;
    readonly attribute EventTarget target;
    readonly attribute long float screenX;
    readonly attribute long float screenY;
    readonly attribute long float clientX;
    readonly attribute long float clientY;
    readonly attribute long float pageX;
    readonly attribute long float pageY;
};

W3C Touch Events Community Group

W3C Web Events WG - Touch Events errata

(early effort to clarify grey areas, simplify language used)

W3C Touch Events Level 2 (Editor's Draft)

(merges errata, touch events extensions, fractional touch coordinates)

Pointer Events

up to IE9 (Win7 / WinPhone7.5) only mouse events

in IE10 Microsoft introduced Pointer Events

Graphic showing 'Pen', 'Touch' and 'Mouse' in a funnel

unifies mouse, touch and pen input into a single event model

XBox One exposes Kinect interactions in IE (single hand gestures only?) as Pointer Events

...and some new inputs (though currently mapped to mouse)

Building Xbox One Apps using HTML and JavaScript

YouTube: Xbox One Edge Browser sends Pointer Events

not just some
not invented here
technology

submitted by Microsoft as W3C Candidate REC 09 May 2013

Pointer Events - W3C REC 24 February 2015

work now continues to enhance/expand Pointer Events...

W3C Pointer Events Level 2 (Editor's Draft)

caniuse.com: Can I use pointer events?

Bug 822898 - Implement pointer events

about:config / dom.w3c_pointer_events.enabled

...what about Apple?

Microsoft submitted a proof of concept WebKit patch

html5labs.interoperabilitybridges.com/prototypes/...

Bug 105463 - Implement pointer events RESOLVED WONTFIX

Maciej Stachowiak - [webkit-dev] pointer events specification - first editors draft

Tweet: 'this spec to easily support multi-input devices is rubbish' says hardware/software company with no multi-input devices

@patrick_h_lauke paraphrasing Apple's stance on Pointer Events...

Apple Pencil for iPad Pro advert

Apple Pencil currently indistinguishable from touch in browser / JavaScript
(no tilt information, fires touch events)

Apple Pencil currently indistinguishable from touch in browser / JavaScript
(no tilt information, fires touch events)

enum TouchType {
    "direct",
    "stylus"
};


interface Touch {
    ...
    readonly attribute float altitudeAngle;
    readonly attribute float azimuthAngle;
    readonly attribute TouchType touchType;
};

Safari/iOS 10.3 exposes tilt and distinguishes touch and stylus

github.com/w3c/touch-events/pull/81

w3c.github.io/touch-events/#touch-interface

the anatomy of Pointer Events
(sequence, event object, ...)

Microsoft Edge / Windows 10 pointer events / mouse events sequence

patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html

events fired on tap (Edge)

mousemove* >
pointerover > mouseover >
pointerenter > mouseenter >
pointerdown > mousedown >
focus
gotpointercapture >
pointermove > mousemove >
pointerup > mouseup >
lostpointercapture >
click >
pointerout > mouseout >
pointerleave > mouseleave

mouse events fired “inline” with pointer events
(for a primary pointer, e.g. first finger on screen)

IE10/IE11 quirks

Compatibility mapping note for devices that support both Pointer Events and Touch Events

Chrome, Edge (on mobile), IE11 (Windows Phone 8.1update1) support
both Touch Events and Pointer Events

w3c.github.io/pointerevents/#mapping-for-devices-that-do-not-support-hover

Windows 10 Mobile / Edge with both touch and pointer events enabled shows different event order...

patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html

about:flags in Microsoft Edge to turn on touch events on desktop
(e.g. touch-enabled laptops) – off by default for site compatibility

... when touch events also supported (Edge)

pointerover > pointerenter > pointerdown >
touchstart >
pointerup >
touchend >
mouseover > mouseenter > mousemove > mousedown >
focus >
mouseup >
click >
pointerout > pointerleave

Specific order of events is not defined in the spec in this case – will vary between browsers...
only problem if handling both pointer events and mouse events (which is nonsensical)

/* Pointer Events extend Mouse Events
   vs Touch Events and their new objects / TouchLists / Touches */

interface PointerEvent : MouseEvent {
    readonly attribute long pointerId;
    readonly attribute long width;
    readonly attribute long height;
    readonly attribute float pressure;
    readonly attribute float tangentialPressure; /* Level 2 */
    readonly attribute long tiltX;
    readonly attribute long tiltY;
    readonly attribute long twist; /* Level 2 */
    readonly attribute DOMString pointerType;
    readonly attribute boolean isPrimary;
}

/* plus all MouseEvent attributes: clientX, clientY, etc */

www.w3.org/TR/pointerevents/#pointerevent-interface

handling mouse input is exactly the same as traditional mouse events
(only change pointer* instead of mouse* event names)

handling touch or stylus is also exactly the same as traditional mouse events
(mouse code can be adapted to touch/stylus quickly)

simple feature detection for pointer events

/* detecting pointer events support */

if (window.PointerEvent) {
    /* some clever stuff here but this covers
       touch, stylus, mouse, etc */
}

/* still listen to click for keyboard! */
Keyboard-related note in W3C Pointer Events, referencing WCAG 2.0 Guideline 2.1

"don't forget about keyboard users" note in Pointer Events spec

/* detect maximum number of touch points */

if (navigator.maxTouchPoints > 0) {
    /* device with a touchscreen */
}

if (navigator.maxTouchPoints > 1) {
    /* multitouch-capable device */
}

"you can detect a touchscreen"
...but user may still use other input mechanisms

Chrome on Android and on a non-touch-enabled desktop, showing they both support Pointer Events, but only on Android it also registers navigator.maxTouchPoints greater than zero

patrickhlauke.github.io/touch/tests/pointer-multitouch-detect.html

Chromebook Pixel: navigator.maxTouchPoints == 16

do pointer events fire during scroll/zoom?

Non-normative note on situations when pointercancel is fired, including 'The user agent interprets the input as a pan or zoom gesture'

w3c.github.io/pointerevents/#the-pointercancel-event

once a browser handles scrolling, it sends pointercancel

patrickhlauke.github.io/touch/gesture-touch/pointerevents.html

pointer events
vs
limitations/problems of mouse event emulation

  1. delayed event dispatch
  2. mousemove doesn't track
  1. delayed event dispatch
  2. mousemove doesn't track

patrickhlauke.github.io/touch/tests/event-listener.html

(IE11/WinPhone 8.1 Update no optimization for width=device-width)
300ms delay happens only just before click event is fired - compare to touch events where the delay happens after touchend but before all mouse compatibility events

patrickhlauke.github.io/touch/tests/event-listener.html

(IE/Win8 has double-tap to zoom, so problem on desktop too)

patrickhlauke.github.io/touch/tests/event-listener.html

(Microsoft Edge/Win10 has double-tap to zoom, so problem on desktop too)

pointer / mouse events and delay

...
[300ms delay]
click
...

300ms delay just before click event

“how can we make it feel responsive like a native app?”

we could try a similar approach to touch events...

won't work: preventDefault() stops mouse compatibility events, but click is not considered mouse compatibility event

patrickhlauke.github.io/touch/tests/event-listener.html

a more declarative approach
with touch-action

touch-action

what action should the browser handle?


touch-action: auto /* default */
touch-action: none
touch-action: pan-x
touch-action: pan-y
touch-action: manipulation /* pan/zoom */

touch-action: pan-x pan-y  /* can be combined */

www.w3.org/TR/pointerevents/#the-touch-action-css-property

only determines default touch action, does not stop compatibility mouse events nor click

Pointer Events Level 2

expanded set of values (useful for pull-to-refresh, carousels, etc)


touch-action: pan-left
touch-action: pan-right
touch-action: pan-up
touch-action: pan-down

touch-action: pan-left pan-down /* can be combined */
touch-action: pan-x pan-down    /* can be combined */

w3c.github.io/pointerevents/#the-touch-action-css-property

new values only determine allowed pan direction at the start of the interaction;
once panning starts, user can also move in opposite direction along same axis

compat.spec.whatwg.org adds extra value pinch-zoom

patrickhlauke.github.io/touch/touch-action-scrolling

although it's called "touch-action", it applies to any pointer type that pans/zooms
(e.g. Samsung Note + stylus, Chrome <58/Android + mouse)

w3c.github.io/pointerevents/#declaring-candidate-regions-for-default-touch-behaviors

touch-action:none

(suppress all default browser behaviour)

IE11/WinPhone8.1 shows no click delay with touch-action:none

patrickhlauke.github.io/touch/tests/event-listener_touch[...]

Microsoft Edge/Windows 10 shows no click delay after tap on touchscreen with touch-action:none

patrickhlauke.github.io/touch/tests/event-listener_touch[...]

touch-action:none kills scrolling, long-press, pinch/zoom

touch-action:manipulation

(suppress double-tap-to-zoom)

IE11/WinPhone8.1 shows no click delay with touch-action:manipulation

patrickhlauke.github.io/touch/tests/event-listener_touch[...]

Microsoft Edge/Windows 10 shows no click delay after tap on touchscreen with touch-action:manipulation

patrickhlauke.github.io/touch/tests/event-listener_touch[...]

caniuse.com: Can I use touch-action?

browsers working to remove double-tap to zoom delay
(when page not zoomable)

IE11/Windows Phone 8.1u1 support for user-scalable=no and click delay suppression

<meta name="viewport" content="user-scalable=no">

patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html

Windows Phone 8's 'Allow zooming on all web content' setting to override unzoomable viewports
IE11/Windows Phone 8.1u1 set to 'Allow zooming' - 300ms delay just before click event returns even when user-scalable=no

"Allow zooming on all web content" reintroduces delay

patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html

Windows 10 Mobile lacks 'Allow zooming on all web content' setting to override unzoomable viewports

"Allow zooming..." option currently missing in Win 10 Mobile

New web platform features announcement: On Windows 10 Mobile devices, the user can now zoom into the page regardless of zoom settings defined by meta viewport

Windows 10 build 15007 on Mobile ignores unscalable viewports

no delay until user zooms for first time...

IE11/Windows Phone 8.1u1 - width=device-width has no effect on the 300ms delay

width=device-width still has delay in IE11/WinPhone 8.1 Update

patrickhlauke.github.io/touch/tests/event-listener_width-device-width.html

Microsoft Edge removes the 300ms delay for width=device-width

Microsoft Edge removes delay for width=device-width

patrickhlauke.github.io/touch/tests/event-listener_width-device-width.html

patrickhlauke.github.io/touch/tests/results/#suppressing-300ms-delay

  1. delayed event dispatch
  2. mousemove doesn't track
Particle party just with mouse event handlers has same issues under pointer events as with touch events - only a single mousemove is fired on first touching

patrickhlauke.github.io/touch/particle/2

mousemove/pointermove fire, but browser scroll action takes over

you can "fake" it with
touch-action:none
and listen for mouse events...

By suppressing default touch action on the particle party canvas, pointer events now 'track' - mousemove is fired continuously as finger moves across canvas

patrickhlauke.github.io/touch/particle/2a

(does not work in Microsoft Edge/Windows 10 Mobile
due to touch events support)

better: just listen to pointermove...

no need for separate mouse or touch event listeners

/* touch events: separate handling */

foo.addEventListener('touchmove', ... , false);
foo.addEventListener('mousemove', ... , false);


/* pointer events: single listener for mouse, stylus, touch */

foo.addEventListener('pointermove', ... , false);

no need for separate mouse or touch code to get x / y coords

/* Reminder: Touch Events need separate code for x / y coords */

function positionHandler(e) {
    if ((e.clientX)&&(e.clientY)) {
        posX = e.clientX; posY = e.clientY;
    } else if (e.targetTouches) {
        posX = e.targetTouches[0].clientX;
        posY = e.targetTouches[0].clientY;
        ...
    }
}

canvas.addEventListener('mousemove', positionHandler, false );
canvas.addEventListener('touchmove', positionHandler, false );
/* Pointer Events extend Mouse Events */

foo.addEventListener('pointermove', function(e) {
    ...
    posX = e.clientX;
    posY = e.clientY;
    ...
}, false);

www.w3.org/TR/pointerevents/#pointerevent-interface

3D Rotator by Creative Punch

coded to only use mouse events

3D Rotator modified to use Pointer Events

minimal code changes, as Pointer Events extend mouse events

(4) Basic mouse-driven fake slider

Exercise/demo: convert mouse-driven code to use Pointer Events

(6) Basic pointer events-driven fake slider

but you can distinguish
mouse or touch or stylus

foo.addEventListener('pointermove', function(e) {
    ...
    switch(e.pointerType) {
        case 'mouse':
              ...
              break;
        case 'pen':
              ...
              break;
        case 'touch':
              ...
              break;
        default: /* future-proof */
    }
    ...
} , false);

/* in IE11/Edge, pointerType returns a string
   in IE10, the return type is long */

MSPOINTER_TYPE_TOUCH: 0x00000002
MSPOINTER_TYPE_PEN:   0x00000003
MSPOINTER_TYPE_MOUSE: 0x00000004

MSDN: IE Dev Center - API reference - pointerType property

for single pointer interactions, check isPrimary

/* only do something if it's a primary pointer
   (e.g. the first finger on the touchscreen) */

foo.addEventListener('pointermove', function(e) {
    if (e.isPrimary) {
      ...
    }
} , false);

what about multitouch?

/* PointerEvents don't have the handy TouchList objects,
   so we have to replicate something similar... */

var points = [];
switch (e.type) {
    case 'pointerdown':
          /* add to the array */
          break;
    case 'pointermove':
          /* update the relevant array entry's x and y */
          break;
    case 'pointerup':
    case 'pointercancel':
          /* remove the relevant array entry */
          break;
}
Multiple pointers can be primary - first finger on a touchscreen and mouse used simultaneously

patrickhlauke.github.io/touch/tracker/multi-touch-tracker-pointer.html

(note multiple isPrimary pointers, per pointer type)

Touch and mouse used simultaneously, with extra information being displayed for each pointer (type, coordinates, whether it's a primary pointer, force, width, height

patrickhlauke.github.io/touch/tracker/multi-touch-tracker-pointer-hud.html

simultaneous use of inputs is hardware-dependent
(e.g. Surface 3 "palm blocking" prevents concurrent touch/stylus/mouse, but not
touch/external mouse/external stylus)

YouTube: Pointer Events support multiple stylus/pen inputs simultaneously

/* like iOS/Safari, IE/Edge have higher-level gestures,
   but these are not part of the W3C Pointer Events spec */

foo.addEventListener("MSGestureStart", ... , false)
foo.addEventListener("MSGestureEnd", ... , false)
foo.addEventListener("MSGestureChange", ... , false)
foo.addEventListener("MSInertiaStart", ... , false)
foo.addEventListener("MSGestureTap", ... , false)
foo.addEventListener("MSGestureHold", ... , false)

/* not supported in other browsers, but potentially
   useful for IE/Edge exclusive content
   (e.g. UWP packaged/hosted/hybrid web app);
   for web, replicate these from basic principles again... */

MSDN IE10 Developer Guide: Gesture object and events

extended capabilities
(if supported by hardware)

/* Pointer Events - pressure */

interface PointerEvent : MouseEvent {
    readonly attribute long pointerId;
    readonly attribute long width;
    readonly attribute long height;
    readonly attribute float pressure;
    readonly attribute float tangentialPressure;
    readonly attribute long tiltX;
    readonly attribute long tiltY;
    readonly attribute long twist;
    readonly attribute DOMString pointerType;
    readonly attribute boolean isPrimary;
}

pressure: value in range 01. if no hardware support,
0.5 in active button state, 0 otherwise

patrickhlauke.github.io/touch/tracker/...

YouTube: Touch tracker with Surface 3 pen

hovering stylus

/* Pointer Events - contact geometry */

interface PointerEvent : MouseEvent {
    readonly attribute long pointerId;
    readonly attribute long width;
    readonly attribute long height;
    readonly attribute float pressure;
    readonly attribute float tangentialPressure;
    readonly attribute long tiltX;
    readonly attribute long tiltY;
    readonly attribute long twist;
    readonly attribute DOMString pointerType;
    readonly attribute boolean isPrimary;
}

if hardware can't detect contact geometry, either 0 or "best guess"
(e.g. for mouse/stylus, return width / height of 1)

In Microsoft Edge on Surface 3, touch pointers have accurate width/height based on contact geometry, while mouse and stylus have width/height of 1

patrickhlauke.github.io/touch/tracker/tracker-width-height.html

YouTube: Pointer Events width and height ...

On a non-pressure-sensitive touchscreen with no contact geometry sensor - e.g. Nokia Lumia 520 - touch pointers have pressure 0.5, as they're in active button state, but width and height is 0

patrickhlauke.github.io/touch/tracker/multi-touch-tracker-pointer-hud.html

(Nokia Lumia 520: pressure is 0.5 - active state - and width / height is 0)

/* Pointer Events - tilt */

interface PointerEvent : MouseEvent {
    readonly attribute long pointerId;
    readonly attribute long width;
    readonly attribute long height;
    readonly attribute float pressure;
    readonly attribute float tangentialPressure;
    readonly attribute long tiltX;
    readonly attribute long tiltY;
    readonly attribute long twist;
    readonly attribute DOMString pointerType;
    readonly attribute boolean isPrimary;
}

tiltX/tiltY: value in degrees -9090.
returns 0 if hardware does not support tilt

patrickhlauke.github.io/touch/pen-tracker

YouTube: ...pen with tilt, pressure and hover support

patrickhlauke.github.io/touch/pen-tracker

YouTube: Chrome on Samsung Galaxy Note 4 - Pointer Events and stylus

pointermove fires if any property changes,
not just x/y position
(width, height, tiltX, tiltY, pressure)

pointer capture
(implicit vs explicit)

Pointer Events Level 2: Implicit Pointer Capture

/* events related to pointer capture */

foo.addEventListener("gotpointercapture", ... , false)
foo.addEventListener("lostpointercapture", ... , false)

/* methods related to pointer capture */

element.setPointerCapture(pointerId)
element.releasePointerCapture(pointerId)

if (element.hasPointerCapture(pointerId)) { ... }

/* example of how to capture a pointer explicitly */

element.addEventListener('pointerdown', function(e) {
  this.setPointerCapture(e.pointerId);
}, false }

/* capture automatically released on pointerup / pointercancel
   or explicitly with releasePointerCapture() */

patrickhlauke.github.io/touch/tests/pointercapture.html

setPointerCapture
can simplify mouse input

e.g. draggable interfaces/sliders, use instead of
"bind mousemove to document/body on mousedown"

(6) Basic pointer events-driven fake slider

Exercise/demo: simplify code to use pointer capture

(7) Basic pointer events-driven fake slider with setPointerCapture

pointer events as the future?

transitional event handling
(until all browsers support pointer events)

/* cover all cases (hat-tip Stu Cox) */

if (window.PointerEvent) {

    /* bind to Pointer Events: pointerdown, pointerup, etc */

} else {

    /* bind to mouse events: mousedown, mouseup, etc */

    if ('ontouchstart' in window) {
        /* bind to Touch Events: touchstart, touchend, etc */
    }
}

/* bind to keyboard / click  */

polyfills for pointer events
(code for tomorrow, today)

GitHub - jQuery Pointer Events Polyfill (PEP)

/* adding PEP from the jQuery CDN */

<script src="https://code.jquery.com/pep/0.4.3/pep.js"></script>


/* doesn't parse CSS, needs custom touch-action attribute
   (could inject this via custom JavaScript on load/mutation?) */

<button touch-action="none">...</div>

/* navigator.maxTouchPoints always returns 0
   (real value would require hardware integration) */
A pointer events demo running in iOS/Safari thanks to PEP

patrickhlauke.github.io/touch/pep-draw

Event listener with PEP, running in iOS9/Safari

patrickhlauke.github.io/touch/pep/listener/event-listener.html

events fired on tap with PEP


pointerover > pointerenter > pointerdown >
touchstart >
pointerup > pointerout > pointerleave >
touchend >
mouseover > mouseenter > mousemove >
mousedown > mouseup >
click

essentially, relevant Pointer Events before touchstart and touchend

3D Rotator modified to use Pointer Events

minimal code changes, as Pointer Events extend mouse events

3D Rotator modified to use Pointer Events (and PEP)

(7) Basic pointer events-driven fake slider with setPointerCapture

Exercise/demo: make slider work in non-Pointer-Events browsers

(8) Basic pointer events-driven fake slider with setPointerCapture and PEP

utility libraries
(because life is too short to hand-code gesture support)

Hammer.js

Hammer.js

patrickhlauke.github.io/touch/swipe

/* Hammer's high-level events example */

var element = document.getElementById('test_el');

var hammertime = new Hammer(element);

hammertime.on("swipe",
  function(event) {
    /* handle horizontal swipes */
});

patrickhlauke.github.io/touch/hammer/swipe

debugging/testing

Large number of mobile phone/tablet test devices

nothing beats having real devices...

BrowserStack (includes Physical Devices)

Android Google Nexus 6 physical device running in BrowserStack

Chrome DevTools / Remote Debugging Android Devices

Chrome DevTools / Using the Console / Monitor events

Touch Events can be explicitly enabled in Chrome's chrome://flags to test for sites/code that does the naive 'if it supports touch events i won't listen for click' assumptions

enable touch events even without a touchscreen
(to test "naive" 'ontouchstart' in window code)

chrome://flags/#touch-events

Chrome DevTools / Device Mode & Mobile Emulation

correctly emulates touch and pointer events

Chrome DevTools / Device Mode & Mobile Emulation

correctly emulates pointer events (with pointerType=="touch")

New 'Touch > Force enabled' option in the 'Sensors' section of Chrome Developer Tools

emulate a touch interface (even without mobile emulation)

Firefox Developer Tools > Responsive Design Mode

correctly emulates touch events

no touch emulation, nor touch events + pointer events (like on real Windows 10 Mobile) emulation, in Edge/F12 Tools

no touch emulation in Safari "Responsive Design Mode"

bonus material

no concept of hover
on touch devices
iOS fakes it, Samsung Galaxy Note II/Pro + stylus, ...

Diagrammatic flow of iOS' faked hover support - stopping event sequence if there's a 'change' on the page

iOS Developer Library - Safari Web Content Guide - Handling Events

while this works ok, Safari is the only browser doing it...

Samsung Galaxy Note Pro with stylus has a near-field hover effect that fires mousemove events even when not touching the screen

YouTube: Samsung Galaxy Note Pro 12.2 stylus hover

requires advanced stylus, which is uncommon...

chrome://flags setting to enable support for hover capable touchscreens

chrome://flags/#enable-touch-hover

requires hover-capable touchscreen, which is uncommon...

MSDN: Hover touch support

Channel9 MSDN - IE11: Using Hover Menus with Touch

MSDN: Using aria-haspopup to simulate hover on touch-enabled devices

non-standard use of the aria-haspopup attribute...

avoid hover interfaces?

arguable ... it's a user experience decision

complement for touch!

same considerations as for keyboard users...

Input Device Capabilities (Draft Community Group Report)

Input Device Capabilities

myButton.addEventListener('touchstart', addHighlight, false);
myButton.addEventListener('touchend', removeHighlight, false);

myButton.addEventListener('mousedown', addHighlight, false);
myButton.addEventListener('mouseup', removeHighlight, false);

myButton.addEventListener('click', someFunction, false);

/* addHighlight/removeHighlight will fire twice for touch
  (first for touch, then for mouse compatibility events);
  can't use e.preventDefault(), as we want to rely on 'click' */
myButton.addEventListener('touchstart', addHighlight, false);
myButton.addEventListener('touchend', removeHighlight, false);

myButton.addEventListener('mousedown', function(e) {
  if (!e.sourceCapabilities.firesTouchEvents) {
    addHighlight(e);
  } }, false);
myButton.addEventListener('mouseup', function(e) {
  if (!e.sourceCapabilities.firesTouchEvents) {
    removeHighlight(e);
  } }, false);

myButton.addEventListener('click', someFunction, false);

/* same function for touch and mouse, but if mouse check if source
   input fires touch events (i.e. this is a mouse compat event) */

Passive Event Listeners (EventListenerOptions)

chrome://flags setting to enable passive event listener override

chrome://flags/#passive-listener-default

Leap Motion Controller

Leap Motion

/* rough proof of concept that fires programmatic
   Pointer Events for each "pointable" (finger / tool) */

e = new PointerEvent('pointermove', {
    pointerId: ... , width: ... , height: ... ,
    pressure: ... , tiltX: ... , tiltY: ... ,
    pointerType: 'leap', isPrimary: false,
    clientX: ... , clientY: ... ,
    screenX: ... , screenY: ...
});

/* dispatch event to whatever element is at the coordinates
   (no support for pointer capture at this stage) */

document.elementFromPoint(x, y).dispatchEvent(e);

YouTube: Leap Motion generating programmatic Pointer Events

patrickhlauke.github.io/touch/leapmotion/tracker

further reading...

get in touch

@patrick_h_lauke
github.com/patrickhlauke/touch
patrickhlauke.github.io/getting-touchy-presentation
speakerdeck.com/patrickhlauke
slideshare.net/redux
paciellogroup.com
splintered.co.uk

Creative Commons: Attribution Non-Commercial Share-Alike

youtube.com/watch?v=AZKpByV5764