History of the Fullscreen API

This is the story of the Fullscreen API, one of prefixes, capitalization, and event targets…

March–October 2007: “Shouldn’t the video API include a way to toggle full screen on/off?” asks Mihai Sucan, a mere month after Opera’s debut of the video element. Much is said about the security implications of such an API, nothing happens, and years pass…

November 2009–March 2010: Eric Carlson (Apple) adds an API for fullscreen video to WebKit:

partial interface HTMLVideoElement {
  readonly attribute boolean webkitSupportsFullscreen;
  readonly attribute boolean webkitDisplayingFullscreen;
  void webkitEnterFullScreen();
  void webkitExitFullScreen();

The webkitbeginfullscreen and webkitendfullscreen event are fired on the video element.

The inconsistent capitalization is noticed and fixed by adding webkitEnterFullscreen() and webkitExitFullscreen() aliases. The original names are kept for backwards compatibility.

Revisions: 50893, 54143, 55946

June 2010: Robert O’Callahan (Mozilla) begins working on a proposal to allow any element, not just video, to go fullscreen.

August 2010–January 2011: Jer Noble (Apple) implements something similar to the then-current Mozilla proposal in WebKit:

partial interface Element {
  const unsigned short ALLOW_KEYBOARD_INPUT = 1;
  void webkitRequestFullScreen(unsigned short flags);
partial interface Document {
  readonly attribute boolean webkitIsFullScreen;
  readonly attribute boolean webkitFullScreenKeyboardInputAllowed;
  readonly attribute Element? webkitCurrentFullScreenElement;
  void webkitCancelFullScreen();

The webkitfullscreenchange event is fired on the current fullscreen element and bubbles.

Revisions: 66251, 75277

September–December 2011: Chris Pearce (Mozilla) begins implementing the proposal in Gecko. Meanwhile, Anne van Kesteren (then Opera, now Mozilla) starts working on a spec at the W3C. The fullscreen element stack is introduced, which makes nested fullscreen (e.g. video in a presentation) possible. When exiting from nested fullscreen, it now becomes un-obvious which element(s) to notify, so the event target is changed to the document.
Revisions: 1c36dc1c92c5, 332cd2925b28

These spec changes are implemented in Gecko, but the old names, and mozFullScreen, are kept:

partial interface Element {
  void mozRequestFullScreen();
partial interface Document {
  readonly attribute boolean mozFullScreen;
  readonly attribute boolean mozFullScreenEnabled;
  readonly attribute Element? mozFullScreenElement;
  void mozCancelFullScreen();

The mozfullscreenchange event is fired on the document and bubbles.
Bug: 545812

March 2012: Jer Noble implements the W3C spec in WebKit:

partial interface Element {
  void webkitRequestFullscreen();
partial interface Document {
  readonly attribute boolean webkitFullscreenEnabled;
  readonly attribute Element? webkitFullscreenElement;
  void webkitExitFullscreen();

Notably, the webkitfullscreenchange event target is the fullscreen element, not the document. Because the event bubbles, listening on the document does work, though.
Revision: 111028

September 2012: The Fullscreen API moves to the WHATWG. The W3C version has been unmaintained ever since.

November 2012: Opera 12.10 is released with support for the spec as it was in February, without prefixes. Opera 12 is the last major version based on Presto, so this unprefixed implementation does not last long.

May 2013: Vincent Scheib (Google) questions why the events bubble. Since it is just an accident of history, they are made to not bubble in the spec.
Commit: 5eb5af63369385c888a5146dda57d462cb9a41da

October 2013: Internet Explorer 11 is released with an ms-prefixed fullscreen API:

partial interface Element {
  void msRequestFullscreen();
partial interface Document {
  readonly attribute boolean msFullscreenEnabled;
  readonly attribute Element? msFullscreenElement;
  void msExitFullscreen();

The MSFullscreenChange event is fired on the document… and bubbles! Most likely the implementation predates the spec change.

Now: This is a bit of a mess! The long history of changes in capitalization and event targets has given JavaScript library authors ample opportunity to write code that will fail once the unprefixed API is made available. I’m trying to implement and ship the unprefixed Fullscreen API in Blink, and some bumps in the road are likely.

Media playback restrictions in Blink

Blink and WebKit have a setting for requiring a “user gesture” to play or pause an audio or video element, which is enabled in Opera for Android, Chrome for Android, the default Android browser, Safari for iOS and probably other browsers. This makes some sense, since mobile devices are used in public and in bed, where unsolicited sound from random Web sites could be a nuisance. Also, autoplaying video ads would waste bandwidth.

The trouble is that this gets in the way of reasonable use cases like games or playlists, and developers are not impressed. We’ve discussed this a lot internally at Opera, and as an experiment we’ve removed the restrictions in Opera beta for Android.

However, I’ve also found a workaround for current browsers. As of WebKit r108831, all restrictions are removed in the first successful load() or play() call. Any user gesture is accepted, so one can listen to all input events and remove the restrictions as soon as the user clicks, touches or uses the keyboard. One does not need to start playback at that point, but can wait until a later time. For example, one could “liberate” a number of audio elements for later use in a game.

I’ve prepared a demo of the workaround. It works in current versions of Opera, Chrome and Safari. While it does not work in the default Android browser prior to KitKat, even there it could be adapted to e.g. autoplay background music by calling play() instead of load() in the input event handler.

Given this, I think that either a user gesture should be required for every play() or the restrictions should be removed completely. My tentative suggestion is the latter.

One month in Blink

On October 1, I began working in Opera’s new Web Technology team, tasked with improving the Web platform by contributing to the Chromium and Blink projects. Most of my effort has been on the Blink side, and it has been a very pleasant experience so far. Since I’ve worked so much with <video> before, I jumped right into that area and looked for things to work on. Chromium/Blink is a big code base which will take time to get intimately familiar with, but I was still able to make small improvements in a few areas.

Event handler attributes

My colleague Henrik reported that onplay and friends were missing on HTMLMediaElement, and I thought it surely would be easy to fix. However, I am easily distracted, so instead of just adding them I decided to implement GlobalEventHandlers and WindowEventHandlers, which ended up taking me 11 commits and even requiring IDL code generator (Perl!) changes. Now onplay is supported, but you probably don’t want to use it – use addEventListener(‘play’, func) if uncertain!

While in the neighborhood, I also made the onerror attribute on <body> and <frameset> work per spec. I was a bit surprised to find this broken, presumably it isn’t that important for site compat…

Removing features

Although I’m quite happy with Blink code, I was surprised to find how much non-standard stuff there is, including prefixed APIs, undocumented quirks and things that have been removed from the spec. Fortunately, the Blink project is supportive of removing these things, with care, and I’ve now picked most of the low-hanging fruit on HTMLMediaElement, getting rid of a weird npr.org quirk, webkitPreservesPitch, webkitHasClosedCaptions, webkitClosedCaptionsVisible, startTime and initialTime.

Counting features

When there’s a risk that removing a feature is going to break the Web, the first step is to figure out by how much. UseCounter counts (opt-in) which features are used on each page view, so I went ahead and added/improved counters for various things that would be nice to eventually remove: a bunch of prefixed HTMLMediaElement APIs, the TextTrackCue constructor, the beforeload event and the two (!) prefixed fullscreen APIs. There’s also the media attribute on <source>, which is still in the spec, but getting usage data will help us decide whether to remove it.

Fixing bugs

It’s a beautiful thing when you can fix a bug just by removing code, and I did so this month by removing the width and height properties from the <video> intrinsic size logic. After my fix intrinsic size is still not per spec, but at least it’s less wrong.

In my only non-trivial Chromium contribution, I made CookieMonster wait for disk flush when deleting cookies, so that cookies are really gone when the UI says they are. This involved fixing a flaky test, which was fun.

Loading text tracks

I began looking into <track> and WebVTT and soon stumbled upon a FIXME in TextTrackLoader, which is what feeds data to the WebVTT parser. Unable to resist the bait, I refactored TextTrackLoader to use RawResource and removed TextTrackResource. Finally, I fixed some other minor issues and fiddled a bit with the TextTrack IDL.

One year ago

Also this month, one year has passed since my final commit in Core (Presto), which was on October 12, 2012. I was a little bit sad when I noticed this anniversary, but at the same I’m really happy about working on Chromium and Blink. Hopefully the next month will be even better!

Web Technology

The Web Technology team (source)

VP8 has landed

Today, WebM/VP8 was announced at Google IO. The last month I have been working on adding support for Opera and we have now released labs builds for Windows, Mac and Linux. We have also published an article for web developers who want to start using WebM <video>. Here’s what it looks like, the Sintel trailer on YouTube playing in Opera with WebM <video>. No Flash!

Previously I’ve posted on Opera Core Concerns, but this time I want to share some personal reflections. (In other words: any views expressed are my own, not those of Opera Software.) What follows is the (quite geeky) history of me and the various codecs and organizations I’ve come into contact with over the past 8 years.

Back in gymnasiet I was a compulsive MP3 collector and was very picky about my bitrates. 128 kbps sounded (still sounds) horrible and it pained me, so when I learned about the Vorbis audio codec I was very excited. Not only was it technically superior, it was also completely free. I re-ripped all of my CDs as Vorbis, told all my friends to do the same and started listening to Machinae Supremacy simply because they offered Ogg downloads. I was a fanboy.

In June 2002 On2 released VP3 to the world. It was my summer holidays and I spent most days inside on an extra slow dial-up connection. I clearly remember that upon reading the news I literally bounced out of my chair and threw my hands in the air out of joy. (Remember, I was 17.) The first thing I did when there was code I could compile and run (packaged by Xiph I believe) was to encode and watch Star Trek: First Contact. The example decoder could neither pause or play in fullscreen, so instead I changed my screen resolution and just watched.

At the time I couldn’t do much to help out, but I wanted to be part of this cool community. One of the first pieces of C code I ever wrote was oggsplit, a not-so-useful tool for splitting multiplexed Ogg streams into separate files. I never used it much, but was quite proud to see it in Xiph’s ogg-tools package.

Fast foward. In the summer of 2006 I began working as a summer intern at Opera Software in Linköping, where I wrote an example plugin for video playback on Opera Devices SDK. I picked GStreamer as the backend and by the time I was done I must have watched A New Computer ~1000 times.

In February 2007 Opera proposed <video> and released a proof-of concept Ogg Vorbis+Theora build. I had no part in this, so it came as somewhat of a surprise. Initially thought that they had used my plugin, but that turned out to not be the case – it was libogg, libvorbis and libtheora integrated directly into the browser. The most exciting part was the strong stance for open standards, something that I obviously agree with.

What happened after that is pretty well known: the <video> tag makes obvious sense, so it quickly got implemented in other browsers. When I joined Opera’s core department (part time) in the summer of 2008, <video> hadn’t been touched much for over a year, so I was tasked with bringing it back to life. Loving both audio/video and the web, it would be hard to find a more suitable and fun job. I ended up porting my then 2-year old plugin and thus Opera is now using GStreamer internally. The Codec Wars™ were always a pain, but we did finally release Opera 10.50 with support for Ogg Vorbis+Theora.

After Google announced that they were buying On2 there was lots of speculation that they would release VP8. I have certainly hoped it would happen, but it seemed a bit too good to be true. Therefore, my reaction when it was confirmed was similar to when VP3 was released – bouncing like a 17-year old. That Vorbis is the chosen audio codec for WebM only makes things better. How lucky I am, that this time I get to actually be part of the release event. It’s been immensely fun working on this, in secrecy, then seeing everything happen in a maelstrom of releases, tweets and blog posts today. Håkon is at Google IO running my code on stage, but just a few weeks ago he was in Opera’s Beijing office, watching sunflowers in one of the first Opera VP8 builds:

While not yet 100% bug-free, VP8 in Opera is well on its way and will be in an official release soon. Today is a good day for open video and the open web. Many thanks to everyone who have worked to make this possible. Live long and prosper, WebM!

Vector Graphics Stylized Stroke Fonts

Finally, my thesis is completed and published! It is the product of work I did at Opera Software’s Beijing office last autumn and I have spent a lot of time writing the thesis report and preparing the final presentation. In short, I’ve implemented a fancy type of stroke fonts by building on well-known vector graphics concepts and modern web standards. This type of font is especially well suited for Chinese characters, which is one of the reasons why I wanted to work in the Beijing office. (Admittedly, the main reason is that I love Beijing.) I am very satisfied with the results.

Chinese character 阮 (family name Ruǎn/Nguyễn)
Chinese character 阮 (family name Ruǎn/Nguyễn)

Personally, I think the most interesting part is the stroking algorithm and its possible use in HTML 5 Canvas and SVG. However, web standards move slowly so it’s perhaps more likely it will be put to use in some other context first. Do read the report if you are interested in vector graphics and/or fonts.

Disclaimer: The views expressed are my own and not those of Opera Software.

I’m sorry. Life is good!

I’m sorry that I haven’t updated my blog in such a long time. I’m sorry that I’ve written too much in Chinese. I’m sorry that I am in Beijing when all of my friends are not. I’m sorry that I’m not going to translate that article about me until I write my memoirs.

Since my last post I’ve been back to Sweden to get myself a new visa and was back in Beijing on August 15. Even though I still have one year of left on my computer science degree I am already doing my thesis project here at Opera Software‘s Beijing office. The topic may or not be secret and involves many things that I really enjoy doing. I should be done by January, because that’s when I’m going to continue my studies in Linköping.

Next week is a national holiday in China, at which time my angel/girlfriend is coming from Việt Nam to see me. I cannot imagine a better way to spend the week than roaming around Beijing, going to all the places we used to like. 太幸福了! What’s more, I’m going to visit her in Hà Nội (the capital of Việt Nam) in the first two weeks of January before going back to Sweden. Life is good!

Interview with me in Shanghai IT Times 2007-06-15

Onwards to victory!

Victory is near! I’m getting on a plane to Beijing in less than three weeks. I’ve got my visa but I haven’t looked for a place to live yet, which is a source of concern for my mother and a source of excitement for myself. The plan is to stay at a hotel the first night and then try to find something. I’ve located BLCU on Google Maps. If you zoom out you can see that it’s to the north-west of the Forbidden City (the big rectangluar structure).

There will be a entrance exam on the first day, so I’ve been studying a little in my spare time in the hope of being able to get into a non-beginners group. Part of that “studying” has been watching Chinese (Mandarin) television dramas. I’ve seen a few good Taiwanese dramas (It Started With a Kiss!), but it seems as if all the popular mainland dramas are martial arts and/or period dramas which have failed to impress me. It appears that some of the voices are dubbed which makes it all seem a bit “off” (dubbed in the same same language as the actor speaks, just like old Hong Kong movies). I wish I were German, then I wouldn’t notice. (I wonder if it’s easier for a ventriloquist to fool a German audience…)

My summer internship at Opera turned out super-awsome and I’m going to keep working part-time the whole year while I’m in Beijing. Getting experience working in China would be awsome, so I hope I’ll have a chance to spend some time at the Opera office in Beijing (although I’ll mostly be working at home, wherever that is). I discovered that Beijing is “bigger” than Linköping, so it might take a long time to get to the office by bus/subway, since it’s not in the same district as BLCU.

I’ve been doing some reading on Wikipedia and found out all sorts of useful things: There’s a chopstick tax to discorage the use of disposable chopsticks. Beijing has population of 8.5 million (Linköping: 136,000). The girl:boy birth ratio is 100:117 due to policital/cultural reasons. Beijing taxis have license plates that begin with 北B. The average height of the Chinese is approximately 170 and 160 cm for men and women respectively. China holds the world record in capital punishment. Finally, you can’t see the Great Wall of China from the moon!

Work for me! Yay!

This morning I signed on to work at Opera Software’s Linköping office for the summer. Since the web is my niche, I’m expecting it to be very fun and a learning experience. It also meant signing my first non-disclosure agreement which, apart from prohibiting me to disclose secret information, prohibits my working on or promoting competing products during my stay at Opera and for two years afterwards. I’ve never contributed to Firefox before and probably wouldn’t have, but I still have mixed feelings about it. I don’t have to worry about being able to pay my rent though, which I otherwise would have (although some might say I could always have been a waiter). In any event, real work for me! Yay!