So far, we've been focused on the use of HD video in ePub documents but what about SD? Standard definition video is still quite common, especially for older works produced for television. Whereas HD has an aspect ratio of 1.78:1 or 16:9, SD has an aspect ratio of 1.33:1 or 4:3.
As we have seen, iOS devices (iPad, iPhone and iPod touch), introduce black bars (letterbox, pillarbox and windowbox) and cropping/scaling in order to maintain the aspect ratio of the source video. Without these, the video we see on these devices would be stretched or squished which is not what the author intended. This is due to the fact that iOS devices do not have 16:9 screens. The iPad is 1024x768 which is 1:33:1 or 4:3. The iPhone and iPod touch are 480x320 (older) or 960x640 (newer, retina displays) which is 1.5:1 or 3:2. In other words, iOS devices have screens with aspect ratios that are the same as or very close to standard definition TV dimensions.
I've created the following screencast to illustrate the many ways that the iBooks application on these iOS devices will present Standard Definition (SD) video. There's a lot to account for with different orientations (portrait vs landscape) and whether the reader chooses to view video embedded in an EPUB-based eBook in situ, proportional full screen or non-proportional full screen. It's best to see all of this in action.
If your source is SD, QuickTime X Player will maintain that aspect ratio in all of its export options. This is ideal for video embedded in EPUB-based eBooks. Playback at 640x480 minimizes the use of black bars, cropping and scaling, the iPod bit rate keeps a good balance between file size and fidelity in most cases and the MPEG-4 Baseline Profile assures that the video will play wherever the iBooks.app is found.
Sunday, April 3, 2011
New Developments in Video Encoded for iBooks.app
Last September, I looked at the need to make sure that video embedded in an ePub document would play and play well on all of the devices where the iBooks.app is found (iPhone, iPod touch and iPad). Since then, new developments not only make this topic more important but also warrant a review of the principles involved.
New developments include the fact that the ePub standard is now at version three and formally specifies the embedding of audio and video files in ePub documents using the same HTML 5 video and audio tags that the iBooks.app has long supported. Thus, there will likely be more eReaders supporting rich media beside iBooks. When those new eReaders do arrive, we'll need to look at them closely. For now, the iBooks.app is still the only game in town for rich media in ePub documents. As well, we have new hardware and operating systems to consider and that brings up "backward compatibility" as an increasingly important factor.
So, let's take another look at this topic and bring our earlier conclusions up to date.
Hardware Targeting The iBooks.app, now at version 1.2.1, requires iOS 4 or later so that reduces the range of hardware that it will run on to iPhone 3G, iPhone 3GS, iPhone 4, iPad plus the 2nd or later generation iPod touch. The most recent of these (iPhone 4, iPad and 4G iPod touch) sport the significantly more powerful A4 processor and, thus, are capable of decoding more advanced versions of the MPEG-4 H.264 standard. Specifically, these newer models can handle H.264 Main Profile whereas the older models can only handle the Baseline Profile. Simply put, baseline will play everywhere iBooks runs whereas video encoded using the Main Profile will only play on newer iOS devices. Thus, aiming for the greatest reach means that we need to stick with Baseline Profile for maximum backward compatibility.
Viewport Targeting Thanks to an article by video wizard Jan Ozer entitled "Encoding for the iPad", I've learned that, thanks to the video scaling function on the iPad, encoding HD (16:9) video at 640x360 pixels looks great for most content regardless of viewport size. The exception to this is video with fine detail such as smallish text. That gets very blurry but since we're talking about ePub here, that shouldn't be a problem. Save the text for the body of your eBook and use static images for fine visual detail.
Bit-rate & File Size Targeting An ePub document is self-contained. Generally speaking, there are no external dependencies for video or audio. Thus, video, audio, image and text files are all found inside the ePub file. Actually, an ePub document is a special kind of Zip archive that eReader apps know how to handle. Because of this self-sufficiency, bit-rate is not as critical as it would be if ePub video required an Internet download as a web page does. On the other hand, video file size is a function of bit-rate and, so, has a lot to do with the file size of your ePub document, how long it takes to download and how long your audience will have to wait before they can start reading. A frame size of 640x360 and a bit-rate of 1.4 Mbps (mega-bits per second) using the MPEG-4 Baseline Profile should produce pristine video for most HD (16:9) sourced content.
The Golden Mean Of course there are good use cases that diverge from this golden mean. The iPad's 1024x768 screen, for example, will accommodate 720p (1280x720) by scaling it down to 1024x576. The iPad will also support MPEG-4 Main Profile and very high bit rates for optimal quality playback. However, unlike other media synched to iOS devices via the iTunes.app, the video content of self-contained ePub documents is not evaluated, Thus, it is quite possible to create an eBook containing video or audio that will not play properly or at all. It is critical that ePub documents be tested on the smallest, least capable iOS devices as well as the iPad and iPhone 4 if your goal is to have the video in your ePub documents accessible wherever iBooks is found.
Reviewing Our Recommended Methods Previously, I had recommended using the iPhone presets that are found in the output options of many applications such as QuickTime Player, Screenflow, Miro and Handbrake. This is still a viable strategy though it is not optimal. The iPhone preset in QuickTime Player, for example, currently encodes a 480x300 video at 880 Kbps using the Baseline Profile. It will scale and play everywhere that iBooks runs but its not the best that we can do. Actually, the iPod preset comes closer at: 1.463 Mbps, 640x360, Baseline@3.0 when the source is 1280x720. This is much better for 16:9 source.
Scalability Testing As mentioned above, testing is critical. Not only do we have to test on multiple devices but we also need to test using all of the orientation and scaling options. Here is an illustrated list to guide your testing:
The iPod touch and the iPhone all share the same screen size (viewport). Here is the portrait view:
Here, we see a proportional scaling of the video constrained by the width of the screen in this orientation. Note the "expander" button in the upper right corner of the center image. Tapping that button fills the screen without regard for aspect ratio (proportion) like this:
Notice that the expander button has now become a collapse button.
Next, we look at the landscape orientation where the same controls are operative but the visual effect is, of course, different.
Next, we look at the iPad where video can be played in-situ, full-screen (proportional) and full screen (non-proportional). We start with the portrait in-situ view.
Here, the iPad advantage is that video can be played in context with other book elements such as text. Tapping on the expander button will bring us to the proportional full screen view which looks like this:
Notice that the collapse button on the video control bar (bottom) is the same in both images. Tapping this element will return to the in-situ video where you may continue watching or stop the video and resume reading text. The expander buttons at the top have a different function. The one on the leftmost image expands to the non-proportional full-screen image on the right. The one on the rightmost image returns the full screen proportional video on the left.
All of these views need to be checked in order to assure that your audience is seeing the video as you intend. Fortunately, the algorithms that Apple uses to scale video in these widely different environments is very good. If you start with high quality source at 1280x720 and then export using the iPod preset, you will likely be satisfied with all of these views.
New developments include the fact that the ePub standard is now at version three and formally specifies the embedding of audio and video files in ePub documents using the same HTML 5 video and audio tags that the iBooks.app has long supported. Thus, there will likely be more eReaders supporting rich media beside iBooks. When those new eReaders do arrive, we'll need to look at them closely. For now, the iBooks.app is still the only game in town for rich media in ePub documents. As well, we have new hardware and operating systems to consider and that brings up "backward compatibility" as an increasingly important factor.
So, let's take another look at this topic and bring our earlier conclusions up to date.
Hardware Targeting The iBooks.app, now at version 1.2.1, requires iOS 4 or later so that reduces the range of hardware that it will run on to iPhone 3G, iPhone 3GS, iPhone 4, iPad plus the 2nd or later generation iPod touch. The most recent of these (iPhone 4, iPad and 4G iPod touch) sport the significantly more powerful A4 processor and, thus, are capable of decoding more advanced versions of the MPEG-4 H.264 standard. Specifically, these newer models can handle H.264 Main Profile whereas the older models can only handle the Baseline Profile. Simply put, baseline will play everywhere iBooks runs whereas video encoded using the Main Profile will only play on newer iOS devices. Thus, aiming for the greatest reach means that we need to stick with Baseline Profile for maximum backward compatibility.
Viewport Targeting Thanks to an article by video wizard Jan Ozer entitled "Encoding for the iPad", I've learned that, thanks to the video scaling function on the iPad, encoding HD (16:9) video at 640x360 pixels looks great for most content regardless of viewport size. The exception to this is video with fine detail such as smallish text. That gets very blurry but since we're talking about ePub here, that shouldn't be a problem. Save the text for the body of your eBook and use static images for fine visual detail.
Bit-rate & File Size Targeting An ePub document is self-contained. Generally speaking, there are no external dependencies for video or audio. Thus, video, audio, image and text files are all found inside the ePub file. Actually, an ePub document is a special kind of Zip archive that eReader apps know how to handle. Because of this self-sufficiency, bit-rate is not as critical as it would be if ePub video required an Internet download as a web page does. On the other hand, video file size is a function of bit-rate and, so, has a lot to do with the file size of your ePub document, how long it takes to download and how long your audience will have to wait before they can start reading. A frame size of 640x360 and a bit-rate of 1.4 Mbps (mega-bits per second) using the MPEG-4 Baseline Profile should produce pristine video for most HD (16:9) sourced content.
The Golden Mean Of course there are good use cases that diverge from this golden mean. The iPad's 1024x768 screen, for example, will accommodate 720p (1280x720) by scaling it down to 1024x576. The iPad will also support MPEG-4 Main Profile and very high bit rates for optimal quality playback. However, unlike other media synched to iOS devices via the iTunes.app, the video content of self-contained ePub documents is not evaluated, Thus, it is quite possible to create an eBook containing video or audio that will not play properly or at all. It is critical that ePub documents be tested on the smallest, least capable iOS devices as well as the iPad and iPhone 4 if your goal is to have the video in your ePub documents accessible wherever iBooks is found.
Reviewing Our Recommended Methods Previously, I had recommended using the iPhone presets that are found in the output options of many applications such as QuickTime Player, Screenflow, Miro and Handbrake. This is still a viable strategy though it is not optimal. The iPhone preset in QuickTime Player, for example, currently encodes a 480x300 video at 880 Kbps using the Baseline Profile. It will scale and play everywhere that iBooks runs but its not the best that we can do. Actually, the iPod preset comes closer at: 1.463 Mbps, 640x360, Baseline@3.0 when the source is 1280x720. This is much better for 16:9 source.
Scalability Testing As mentioned above, testing is critical. Not only do we have to test on multiple devices but we also need to test using all of the orientation and scaling options. Here is an illustrated list to guide your testing:
The iPod touch and the iPhone all share the same screen size (viewport). Here is the portrait view:
Here, we see a proportional scaling of the video constrained by the width of the screen in this orientation. Note the "expander" button in the upper right corner of the center image. Tapping that button fills the screen without regard for aspect ratio (proportion) like this:
Notice that the expander button has now become a collapse button.
Next, we look at the landscape orientation where the same controls are operative but the visual effect is, of course, different.
Next, we look at the iPad where video can be played in-situ, full-screen (proportional) and full screen (non-proportional). We start with the portrait in-situ view.
Here, the iPad advantage is that video can be played in context with other book elements such as text. Tapping on the expander button will bring us to the proportional full screen view which looks like this:
Notice that the collapse button on the video control bar (bottom) is the same in both images. Tapping this element will return to the in-situ video where you may continue watching or stop the video and resume reading text. The expander buttons at the top have a different function. The one on the leftmost image expands to the non-proportional full-screen image on the right. The one on the rightmost image returns the full screen proportional video on the left.
All of these views need to be checked in order to assure that your audience is seeing the video as you intend. Fortunately, the algorithms that Apple uses to scale video in these widely different environments is very good. If you start with high quality source at 1280x720 and then export using the iPod preset, you will likely be satisfied with all of these views.
Subscribe to:
Posts (Atom)