Frequently Asked Questions (FAQ)

1. What is DO-IT Video Search?
2. Why was DO-IT Video Search created?
3. How is DO-IT Video Search funded?
4. How does it work?
5. Is this approach scalable to larger video collections?
6. Why are you using a Flash video player?
7. What Flash video player are you using?
8. Is the Flash video player accessible to screen reader users?
9. Is the Flash video being streamed?
10. Why are you using closed captions instead of open captions?
11. Are the videos audio described?
12. Can I customize the interface?
13. What is your procedure for making your videos accessible?
14. Is the DO-IT Video Search source code available?
15. What is on your To-Do List?
16. Who do I contact with feedback or suggestions?
17. Who do I contact for support in using JW FLV PLayer or any of the other products mentioned in this FAQ?
18. Are there other examples of video search applications?
19. What are some other resources on multimedia accessibility?

What is DO-IT Video Search?

DO-IT Video Search is a web-based application that allows users to search the full text of DO-IT's video library. Over the years, DO-IT has produced more than 30 videos on a wide range of topics related to people with disabilities accessing curriculum, using technology, and pursuing challenging careers. our videos are full-text searchable because they're closed captioned.

Why was DO-IT Video Search created?

We had three main motivations for creating this application:

1. To provide a means by which users can search the full text of DO-IT's video library, and quickly jump to video scenes where their search words are being spoken.
2. To demonstrate a secondary benefit of closed captions. If accessibility alone isn't enough to motivate organizations to caption their media, our hope is that the added benefit of being able to search the full text of video archives will be added motivation.
3. To provide a test platform for exploring accessible media delivery. See our To-Do List for our future plans in this area.

How is DO-IT Video Search funded?

DO-IT Video Search was developed and continues to be updated and maintained with funding from the National Science Foundation (award #HRD-0227995 and HRD-0833504). The contents do not necessarily represent the policies of the U.S. federal government, and you should not assume their endorsement.

How does it work?

All of DO-IT's videos are closed captioned for the deaf and hard of hearing. Closed captions come in a variety of formats depending on the type of media being used, but most of these are text or XML files that can easily be parsed by a simple script, and used for searching. For our purposes, we store each individual caption and its associated start and end time as records in a MySQL database. The user interface is written in PHP. When a user searches for a keyword or phrase, a MySQL full-text search query is executed, and results are delivered to the user. Results are grouped by video with the most relevant videos (according to MySQL) listed first. Each item in the results is caption text, including a link that starts playing that video in the embedded media player at the indicated time where that text is being spoken. For the nuts and bolts of the search algorithm, we depend entirely on MySQL full-text search, which we have found to be adequate. However, we are testing a variety of search queries and working to make adjustments where we can.

Is this approach scalable to larger video collections?

Admittedly, our database is quite small. We currently have nearly 40 videos, and approximately 5000 captions. We have not tested our approach on a larger collection. For comparison, see our list of other examples of video search applications.

Why are you using a Flash video player?

Historically, DO-IT videos have been delivered on-line in both Real and Windows Media formats. This changed in 2006 to Quicktime and Windows Media to account for the growing strength of Quicktime (and iTunes) in the video player market. However, none of these media players had all the features that we needed for our video search interface. We needed a player that could be embedded into the web page along with search results, and transcript, and other closely associated content; and the embedded player had to be capable of receiving start times, had to be readily available for all operating systems; and had to be accessible to users with disabilities when embedded. According to a WebAIM test of Media Player Accessibility, the three major media players were not particularly accessible when delivered as embedded players within a web page. This article is now several versions old, but our own informal tests have found that little has changed with regard to embedded player accessibility.

Adobe Flash provides a viable solution. Macromedia (prior to being acquired by Adobe) began adding accessibility features to Flash in the early 2000's, including the ability for authors to add alternate text and keyboard support to objects such as buttons on a media player's controller bar. There are several individuals and groups who have been developing accessible Flash media players, and even if these failed to meet our needs we could always create our own, which isn't so easy an option for the other media types. Also, Adobe claimed as early as 2005 that the Flash Player was installed on more than 98% of Internet-connected desktops (in PDF, requires Adobe Acrobat Reader). Many independent sources confirm that Flash is much more widely installed than any other player.

For those few users who don't have the Flash player installed, or have older versions of screen readers that don't support Flash accessibility, or who simply prefer other video formats, we continue to provide Quicktime and Windows videos as options. However, only the Flash Player is currently used within the web interface.

What Flash video player are you using?

Our intent is to use DO-IT Video Search as a test lab for a variety of media delivery options. Users can therefore choose from a selection of media players on the Preferences page. However, we currently only support one player, the JW FLV Media Player, version 3.16, named with the initials of its developer Jeroen Wijering. Jeroen's player is highly customizable through a wide variety of user-defined variables and a powerful Javascript API. It also has an accessible interface for screen readers that support Flash accessibility, plus support for closed captions and closed audio description (see Question 11 for more on our support for audio description).

One problem with Flash is that most browsers don't support keyboard access to embedded objects unless the user first gives that object focus by clicking on it with a mouse. This is a problem that affects all known browsers except Internet Explorer 6 and higher. Obviously this is a problem for users who are unable to use a mouse, unless their assistive technology is able to bypass this barrier. This was another reason we selected the JW FLV Media Player. Its Javascript API allows us to create HTML buttons that can control the Flash movie from outside the Flash movie. These alternative player buttons may also be useful to other users, including users with low vision, or users who simply like them better than the standard controls. By default both sets of controls are available on the screen, but this can be changed on the Preferences page.

Is the Flash video player accessible to screen reader users?

JW FLV Media Player is reasonably accessible to screen reader users whose assistive technology is current enough to support Flash accessibility (e.g., Window-Eyes 4.2 or higher, JAWS 4.5 or higher). There are a few quirks, however, such as control buttons that have confusing or misleading alternate text. Consult the Help page for tips on operating the media player with a screen reader.

We have also provided an alternative set of large font HTML buttons for controlling the basic functions of the movie player. Each of these buttons is assigned an access key so they can be activated from anywhere on the web page. See the Help page for details.

By default, both sets of media player controls are shown, but either or both of them can be disabled on the Preferences page.

Is the Flash video being streamed?

Traditionally there have been two techniques for delivering Flash video: Streaming and progressive download. Streaming requires a streaming server such as Adobe's Flash Media Server. This product can be expensive for some budgets, as can the commercially hosted solutions. The University of Washington's media services group does not support Flash streaming either, so we are currently unable to provide true streaming. The alternative, progressive download, allows video to be played as it's being downloaded via HTTP, and requires no special technology. However, it does not support direct access to any point in the video until the video has been downloaded up to that point. For our purposes, this would mean that if a search result points to a caption that appears 10 minutes into the video, the user would have to wait until 10 minutes of video had been downloaded before the user could access that video content. This is clearly unacceptable for our needs.

An alternative is PHP streaming, in which a small PHP script is used to deliver a new Flash video header, starting at any designated keypoint within a larger video. This technique has evolved since it was originally proposed by Stefan Ricther in 2005, and the entire development cycle can be tracked at FlashComGuru.com, in the articles Streaming flv video via PHP, take two and Flash Video Streaming via PHP - Optimized by xmoov.com. One of the tools that has emerged from the PHP Streaming movement is the PHP Streaming Extension for the JW FLV Player, which is the tool we are currently using to pseudo-stream our Flash video.

In order for PHP Streaming to work, the Flash video (FLV) file must be encoded with keyframes at the same time as caption start times, or at least frequently enough so that the video is sure to include a keyframe close to the time at which any caption will begin. Additionally, the FLV file must be injected with meta data indicating the position of each keypoint. This is easily done either with FLV Meta Data Injector (FLVMDI) or Captionate, both from the Manitu Group. The latter of these products has the added benefit of providing a complete solution for adding closed captions to Flash videos.

Why are you using closed captions instead of open captions?

Those of you who are familiar with DO-IT videos know that we have a long history of producing all our videos with open captions. Open captions are mixed into the video and are always visible, whereas closed captions exist independently of the video and can therefore be toggled on and off. All our videos are originally produced for broadcast and for distribution on hard media such as VHS and DVD. We continue to add open captions to these video for many reasons:

• DO-IT's videos are often used in presentations and lectures. Open captions provide access, even for users (including teachers, presenters, etc.) who don't know how to enable closed captions on their TV's, or whose projection equipment doesn't support closed captions.
• Open captions can benefit audience members who might not otherwise know they would benefit, e.g., people who are unable to hear the audio due to their current situation (a noisy room, a public computer lab with no headphones); people for whom English is not their native language; people who process information best if it's presented in multiple modes.
• If videos are encoded for playback in players or devices that don't support closed captions, open captions allow the video to still be accessible.
• Open captioned videos are easier to distribute. We only have to distribute one file (the video file), as opposed to multiple files for closed captioned video, which must be linked correctly on the receiving end in order to work.

For the above reasons open captions continue to be viable in general. However, there are at least two cases in which closed captions are a preferred means of delivering captions:

• Open captions are part of the video image. They don't actually exist as text, and therefore can not be edited or searched.
• Open captions degrade at low bandwidths. Our traffic data as of 2006 shows that when our visitors are presented with a choice of high, medium or low bandwidth, over 28 percent choose low bandwidth. These users receive a video with open captions that are blurred to the point of being virtually unreadable, as the following screen shot illustrates.

Are the videos audio described?

Yes, all of DO-IT's videos include audio description. Audio description is a narrative voiceover that describes content (including key actions or screen text) that is otherwise presented only visually, and is therefore not accessible to someone who is unable to see it. Since the added narration may be distracting to some users, we have produced two versions of all our videos, one with audio description, and one without. By default, we show the audio described version of all videos in the DO-IT Video Search application. However, this can be changed on the Preferences page.

One of the reasons we selected JW FLV Media Player (described in the earlier question regarding our choice of media players) is its support for closed audio description, to our knowledge the only Flash media player that has such support. With this feature, audio description can be delivered in a distinct MP3 audio file, and can be toggled on and off by a button on the player's control bar, similar to the way closed captions work. We find this to be a nice feature because it allows us to deliver one video that meets the diverse needs of all audience members, rather than having a separate video for users who require audio description.

Since closed audio description is new to us, it is only available on videos produced in 2008 or later. The following is a complete list:

• Equal Access: Universal Design of Computer Labs
• Working Together: Computers and People with Learning Disabilities
• Working Together: Computers and People with Mobility Impairments
• Part of Me, Not All of Me

To sample audio description, you must have "Closed Audio Description" checked in your Preferences. Also, your choice of media players determines how closed audio description is rendered (none are perfect yet). JW Media Player 3.16 provides an audio description toggle button on the player control bar, identified with the letter "A". However, this version does not allow audio description volume to be controlled separately from the program audio. As a result, the audio description is difficult to hear over the background audio. With the release of JW Media Player 4.2, audio description has its own independent volume control (internally, not yet controllable by users). However, there is no audio description button so audio description can not be turned off.

Can I customize the interface?

Yes. You can change many of the default settings of the DO-IT Video Search website on the Preferences page. Doing so requires that your browser be set to accept cookies.

What is your procedure for making your videos accessible?

The following is a step-by-step guide through the process by which our videos are produced:

1. Write the scripts. All our videos are created from scripts. Often this includes unscripted content (e.g., interviews, footage filmed on location at the University of Washington), but even this content is integrated into the script. Starting with a script results in a more polished presentation and an easier video editing process later. Also, the script content can easily be converted into captions, eliminating the extra time and cost of transcribing the video post-production.
2. Plan for accessibility during production. Since all our videos include open captions, we are careful when filming to avoid critical content appearing near the bottom of the frame, where captions will later be displayed. Similarly, since all our videos include audio description, we are careful when scripting to pace the video in ways that ensure that there is enough space for narration to be mixed in.
3. Produce and encode the videos. Our videos are produced by University of Washington Television (UWTV), and the final deliverable includes media for broadcast television, DVD, and the Web. For the latter, videos are encoded into Quicktime, Windows Media, and Flash Video (FLV) formats. The FLV files are encoded with keyframes at frequent intervals, necessary for PHP streaming and linking to particular points within the video. Captions and audio description are added during production by third party vendors, but since our videos are initially scripted we save on the cost of caption production. Videos prior to 2008 were open captioned, and caption data was no longer available for retrofitting the original pre-captioned versions of our videos with closed captions. For these videos, we used Automatic Sync Technologies, a company whose CaptionSync service enabled us to quickly (automatically, using speech recognition) convert our existing scripts to closed captions in all the required caption formats. For more recent videos, our videos are closed captioned during production, and are delivered in the required caption formats including Quicktime Text (Quicktime), SAMI (Windows Media), and Timed Text, the W3C standard format supported by JW FLV Player.
4. Inject the FLV file with meta data using Captionate. This step is described in more detail in Question 9, above.
5. Prepare for Video Search. Our final step is to save all materials to our web and streaming media servers, then tell the Video Search application where to find these materials by completing a simple behind-the-scenes web form developed for DO-IT staff. This information, including each caption and its start and stop times (extracted from the caption files) is stored to a MySQL database, and is immediately ready for searching.

Is the DO-IT Video Search source code available?

The PHP source code is not presently available since it is highly specialized for the unique characteristics of DO-IT's video collection. With more work (and funding), we may consider making it available in the future. Meanwhile, the trickier parts involve using JW FLV Player and PhilFlash extension, both of which are well-documented elsewhere. Feel free to consult our HTML and Javascript files using your web browser to see how we've implemented these resources.

What is on your To-Do List?

We're currently working on the following tasks:

• Fix the known bugs:
  1. Occasionally, closed audio description starts playing in the JW FLV Player automatically even when the video does not start playing automatically. This is an intermittent problem and the cause has not yet been ascertained. It is only a minor problem, since once the user presses Play the audio description automatically gets re-synchronized with the video.
  2. Occasionally, linking within a video causes the media player to hang. This is an intermittent problem, but we're working to identify precisely when and why this happens, and fix it.
• Develop an accessible interface in which the user can interact simultaneously with the full text transcript and media player. For example, text from the transcript may be highlighted as it is spoken in the video; or conversely, users can select any text from the transcript to play the video at that point.
• Add true Flash streaming capabilities. The University of Washington does not currently provide centralized support in this area, but we are exploring our options.
• Add support for other video players, including JW FLV Media Player 4.0, National Center on Accessible Media's CCPlayer (which accompanies CC For Flash), and Adobe's FLVPlackBack and FLVPlaybackCaptioning components (for more information see the Adobe Flash CS3 Accessibility Design Guidelines).
• Monitor developments with the WHATWG's proposed <video> element, and add support for this element if ultimately feasible and reasonably supported by browsers. The proposed <video> element is well-described in the Mozilla Foundation's Position Paper for the W3C Video On The Web Workshop.

Who do I contact with feedback or suggestions?

We welcome and encourage your feedback, especially on accessibility and usability of DO-IT Video Search. Please send feedback and suggestions to Terry Thompson, Technology Accessibility Specialist at DO-IT.

Who do I contact for support in using JW FLV PLayer or any of the other products mentioned in this FAQ?

Although we have used a variety of software tools to create the DO-IT Video Search application, we are not the best source for obtaining support on these tools.

For support related to JW FLV Media Player, the user community is very active and supportive. Consult the JW Media Player Support Forum. This forum may also be the best source of support for third-party utilities that are used in conjunction with JW FLV Media Player, including the PHP Streaming Extension and many others.

For support with any of the other products mentioned in this FAQ, consult the product websites for user forums or contact information.

Are there other examples of video search applications?

Most of the Web's major video portals, including YouTube.com, video.aol.com, and others, provide the ability to search video, but base that search on titles, descriptions, and/or key words rather than full text. A few companies, including Docsoft and EveryZing.com, are attempting to harness the growing power of speech recognition technology to automatically transcribe and caption video. However, a close look at the transcripts produced by EveryZing.com will reveal the current limitations of this technology.

Also, Automatic Sync Technologies (AST) has developed a video search system that is available for purchase. AST specializes in automatic generation of closed captions from an existing transcript, so captions naturally provide the searchable data for AST's system. More information is available on AST's Streaming Media Video Search page.

The most high profile example of search that uses captions is Google Video, which encourages users to add captions to their videos in Google's own closed caption format, and for videos that are captioned Google supports direct linking within the video, just as DO-IT Video Search does, albeit without full attention to accessibility of the interface. For additional information, including a link to an example showing within-video linking, see Closed Captioning Search Options on the Official Google Video Blog.

What are some other resources on multimedia accessibility?

For additional information about captioning, audio description, or other issues related to multimedia, consult the DO-IT Knowledge Base. A good entry point is the AccessIT article How do I make multimedia accessible?.