Before HTML5, browsers could only natively display one type of multimedia.

<img src="omgcat.gif">
        

Plugins proliferated:

• 1991: Apple Quicktime (.mov)
• 1994: MIDI (background music)
• 1995: RealAudio (.ra, .ram)
• 1997: RealVideo (H.263)
• 1998: Windows Media Player
• 1999: Quicktime for Windows (.mov, .mp3, .mp4, .swf)
• 2002: Macromedia Shockwave Flash (.swf)
• 2007: Microsoft Silverlight (flash, vid, etc.)

To embed a file for a plugin to open, you'd use:

Introduced in Netscape 2.0, a simple tag.

Introduced in HTML 4.01 spec, this tag supports more complex configuration.

The embed tag should specify src and type attributes.

<embed type="application/x-shockwave-flash"
 src="zombie_walk.swf"/>

The object tag should have one or more param child tags that give info to the browser on what to embed.

<object width="300" height="150">
  <param name="movie" value="zombie_walk.swf"/>
  <param name="bgcolor" value="white"/>
</object>

The object tag can include content that is executed if the browser doesn't understand the object tag. This content could be text, another object tag, or an embed tag.

<object classid="clsid:02BF25D5-8C17-4B23-BC80-D3488ABDDC6B"
 codebase="http://www.apple.com/qtactivex/qtplugin.cab"
 height="400" width="500">
 <param name="src" value="springs.mov" >
 <param name="autoplay" value="true" >

 <embed 
  type="video/quicktime"
  pluginspage="http://www.apple.com/quicktime/download/"
  height="400" width="400"
  src="springs.mov" autoplay="true" 
 />
</object>

The code from the previous slide produces:

The (old) Youtube embed code includes an object and an embed tag.

<object width="480" height="385">
  <param name="movie"
     value="http://www.youtube.com/v/ddm5bbeqHVA"/>
  <param name="allowFullScreen"
     value="true"/>
  <param name="allowscriptaccess"
     value="always"/>

  <embed src="http://www.youtube.com/v/ddm5bbeqHVA"
         type="application/x-shockwave-flash"
         allowscriptaccess="always" 
         allowfullscreen="true" width="480" height="385"/>

</object>

The code from the previous slide produces:

• Bad user experience (pop-ups)
• Didn't play nice with eachother (crash!)
• Opened up security holes (malware)
• Didn't work everywhere (Flash on iPads)

In HTML5, you can embed audio or video using native HTML tags audio and video, and if the browser supports the tags, it will give users controls to play the file.

Why is native better?

• No plugins needed
• Better performance
• Native, accessible controls

The audio element is used for embedding an audio player inside a page for a particular audio file.

You can use various attributes to customize the player: preload, autoplay, loop, and controls.

<audio src="argentina.ogg"
  controls="true" preload="true"></audio>

The video element embeds a video player for a particular video file.

You can use these attributes to customize the player: poster, preload, autoplay, loop, and controls.

<video src="chrome_japan.ogv" controls width="390"></video>

An audio track is compressed, stored, and decoded according to a codec. Most audio codecs support two channels of sound, some support more.

The most relevant audio codecs are:

• MP3: Patent-encumbered.
• AAC (Advanced Audio Coding): Patent-encumbered. Used in Apple products.
• Ogg Vorbis: Free, open-source, patent-free

There is no official codec for browsers to support, so they vary:

HTML5 Audio Support

A video file, like an ".avi" file, is really a container for multiple related files that describe a video, like:

• a video track
• one or more audio tracks with synchronization markers
• metadata (title, album art, etc).

The most popular video containers are:

• MPEG4: .mp4, .m4v
• Ogg: .ogv
• WebM: .webm

A video track is compressed and stored in a particular way, and a "codec" describes how to turn the stored data into the series of images that you see.

The most relevant video codecs are:

• H.264: Patent-encumbered.
• Theora: Royalty-free.
• VP8: Patented but licensed royalty-free.

Each browser supports a different combo of formats and codecs, due to patents issues. This is the current situation:

HTML5 Video Support

There are now many online services, desktop apps, and command-line libraries for encoding, like:

• Miro Video Converter (app)
• Media Converter(online)
• FireFogg (Firefox plugin)
• HandBrake (open-source, cross-platform)
• Command-line

As we saw, different browsers support different formats. Thankfully, HTML5 lets us specify multiple sources for the video and audio elements, so browsers can use whichever works for them.

<video height="200" controls="">
  <source src="chrome-japan.webm" type="video/webm"/>
  <source src="chrome-japan.mp4" type="video/mp4"/>
  <source src="chrome_japan.ogv" type="video/ogg"/>
</video>

There are some browsers that don't support these new elements. In that case, you can provide a fallback option via a browser plug-in, like Java or Flash, and put the fallback inside the element. For example, here's a Flash fallback:

<video src="video.ogv" controls>  
    <object data="flvplayer.swf" type="application/x-shockwave-flash">  
      <param value="flvplayer.swf" name="movie"/>  
    </object>  
</video>  

The track tag can be used to specify subtitles, captions, chapters, descriptions, and metadata:

<video src="video-file.mp4"  controls>
  <track src="en.vtt" kind="subtitles" srclang="en" label="English subtitles" default>
  <track src="de.vtt" kind="subtitles" srclang="de" label="German subtitles">
  <track src="ch-en.vtt" kind="chapters" srclang="en" label="English chapter listing" default>
</video>

Video Track · HTML5 Rocks Demo · Elephants Dream

The WebVTT or SRT file format describes timed information.

Tags and annotations can be used to specify class names, timestamps, positioning:

1
00:00:10.500 --> 00:00:13.000 A:start
Elephants Dream

2
00:00:15.000 --> 00:00:18.000 A:end L:10%
And thus it <b>begins</b>... 
      

• CaptionMaker
• Amara
• Overstream
• Subtitle Horse
• DotSub
• Subtitle Workshop

Many of these attributes can be set via JS, and trigger events on change:

• duration
• autoplay
• loop
• controls
• muted
• src
• currentSrc
• networkState
• preload
• buffered
• readyState
• volume
• playbackRate
• paused
• ended
• width
• height
• poster

var video = document.getElementsByTagName("video")[0];
alert(video.currentTime);
video.playbackRate = video.playbackRate * 2;
      

MDN Video Attributes reference

• loadstart
• progress
• suspend
• abort
• error
• emptied
• stalled
• loadedmetadata
• loadeddata
• canplay
• canplaythrough
• playing
• waiting
• seeking
• seeked
• ended
• durationchange
• timeupdate
• play
• pause
• ratechange
• volumechange

<div id="time">0</div>
<script>
var video = document.getElementsByTagName("video")[0];
video.addEventListener('timechange', function() {
  document.getElementById("time").innerHTML = video.currentTime;
}, false);
</script>
      

• load()
• play()
• pause()
• canPlayType()
• addTextTrack()

<button id="play" title="play" onclick="playVideo()">play</button>
<script>
function playVideo() {
  var video = document.getElementsByTagName("video")[0];
  video.play();
}
</script>
      

This example does everything!

• video.js
• MediaElementJS
• JWPlayer
• LeanBack
• js_videosub
• Playr
• PopcornJS
• Captionator

• YouTube
• HeyWatch
• Transloadit
• ZenCoder
• encoding.com
• PandaStream
• T3Media

• Webcam input (getUserMedia)
• Streaming (Peer Connections, WebSockets)
• Audio API (Mozilla's or Google's)

• Chrome Experiments