Optimizing iPad Video for iBooks Author and Pages

Including video in an eBook, especially an eTextbook, has become essential and expected. If a still image is worth a thousand words, then the potential explanatory power of an image moving at 30 frames per second must be exponentially greater. Getting from potential to actual value, however, requires compelling content and clear, efficient video.

The preeminent tools for creating eBooks and eTextbooks containing video are iBooks Author and Pages, both from Apple. The iBooks Author app comes bundled with MacOS X which is free and Pages comes bundled at no extra cost with all new Macs. There are no other apps on any platform that match both the affordability and ease of use of Pages and iBooks Author.

However, there have been issues with the way these apps have handled eBook video in the past. Some of those issues have been documented here. This post is a re-examination of those issues to assess the current state of video in eBooks produced by these two apps. Have things gotten better, remained unchanged or gotten worse? That is the question.

The iBooks Author (iBA) application has had difficulty handling video efficiently. In version 1.0, authors had to meet very strict and vague encoding requirements in order to avoid rejection of a video file. In succeeding versions, Apple addressed the many issues arising from this situation by incorporating a video optimizing routine that would re-encode any video brought into an iBA project via the Media widget. Early versions of this new functionality solved the problems that authors were having with video encoding for their eBook projects but they did so by making those video files unnecessarily large. It was quite common for a video file to become three times larger than the original after being “optimized” in iBooks Author.

Jan Ozer developed a clever way to circumvent the iBA video optimizer using custom presets for the Handbrake video transcoder and he offered how-to workshops on that tactic. In June of 2014, however, Apple updated iBA and rendered this workaround ineffective.

The current version of iBA (2.2) was released in October of 2014. The status of the video optimizer has never been referenced in any iBooks Author version history or Read Me file. Our knowledge of it has only come about by comparing input to output and then inferring what goes on within the innards this black box.

Even less well known is the history of video handling in the Pages application. The current version of Pages (5.5.2) has an export to ePub function which silently and invisibly invokes a video optimizer that, like iBA, calls upon system-level routines such as AVConvert to optimize your media files.” Pages seems to have followed iBooks Author with respect to handling the inclusion of video in eBooks without requiring authors to become expert in the art of video compression.

So with the increasing importance of video in eBooks and especially in eTextbooks, it is important to stay abreast of the ways in which Apple’s iBooks Author produces *.ibooks files containing video and how Apple’s Pages produces *.epub files containing video even if Apple isn’t very forthcoming on the topic.

METHODOLOGY

Both Pages and iBooks Author output eBooks (*.epub or *.ibooks files) that share the same architecture. Each eBook is actually a collection of various files and folders in an enclosing folder that has been archived using the Zip standard. The suffix of the resulting *.zip file is then changed to *.epub or *.ibooks which enables eReaders to know that they can be rendered as eBooks. The free ePub Zip/Unzip application handles this chore very nicely.

It’s being able to “look inside” that enables us to see what these video optimizers have done to the video we provided, comparing input to output, before to after. An inexpensive ($0.99) app named MediaInfo provides us with detailed information on each version of the video we wish to examine. Less information can be gotten for free by using QuickTime X Player’s Get Info function.

So our methodology is to bring video files into our iBA and Pages projects, export as *.ibooks or *.epub, then deconstruct the resulting eBook by unzipping it, finding the video files, developing information about those files and, finally, comparing and analyzing the differences.

DETAILED RESULTS

Raw data, including test videos, with detailed analyses may be found in the following eTextbook created with iBooks Author. Download this *.ibooks file and read it with the iBooks app on an iPad 2 or newer or a MacOS X 10.9 or newer computer. This volume will illustrate by examples precisely how the current (2.2) version of iBooks Author handles ingested video. It will also document what Pages does to ingested video.

EXECUTIVE SUMMARY

The iBooks Author (iBA) application has improved video handling to the point where corrective action after export is no longer warranted or recommended. The media optimizer in iBA 2.2 will increase file size only slightly over what was ingested. However, optimizing video with Handbrake prior to ingestion produced significantly smaller video files of equivalent quality and this is recommended as current best practice. The eTextbook cited above contains a link to the custom Handbrake preset used in these investigations with instructions on how to import it into the Handbrake app and use it.

The Pages app, unfortunately, has not improved as much. Video files ingested by Pages 5.5.2 are typically inflated to three times their original size or more. The best practice response to this unfortunate situation is to use the deconstruction techniques used in this investigation to replace the inflated video file with a version encoded by Handbrake using the aforementioned preset. Stepwise instructions on how to do this are found in the eTextbook cited above.

Standard Definition Video in Books Using the .epub or .ibooks Format

So far, we've been focused on the use of HD video in ePub documents but what about SD? Standard definition video is still quite common, especially for older works produced for television. Whereas HD has an aspect ratio of 1.78:1 or 16:9, SD has an aspect ratio of 1.33:1 or 4:3.

As we have seen, iOS devices (iPad, iPhone and iPod touch), introduce black bars (letterbox, pillarbox and windowbox) and cropping/scaling in order to maintain the aspect ratio of the source video. Without these, the video we see on these devices would be stretched or squished which is not what the author intended. This is due to the fact that iOS devices do not have 16:9 screens. The iPad is 1024x768 which is 1:33:1 or 4:3. The iPhone and iPod touch are 480x320 (older) or 960x640 (newer, retina displays) which is 1.5:1 or 3:2. In other words, iOS devices have screens with aspect ratios that are the same as or very close to standard definition TV dimensions.

I've created the following screencast to illustrate the many ways that the iBooks application on these iOS devices will present Standard Definition (SD) video. There's a lot to account for with different orientations (portrait vs landscape) and whether the reader chooses to view video embedded in an EPUB-based eBook in situ, proportional full screen or non-proportional full screen. It's best to see all of this in action.



If your source is SD, QuickTime X Player will maintain that aspect ratio in all of its export options. This is ideal for video embedded in EPUB-based eBooks. Playback at 640x480 minimizes the use of black bars, cropping and scaling, the iPod bit rate keeps a good balance between file size and fidelity in most cases and the MPEG-4 Baseline Profile assures that the video will play wherever the iBooks.app is found.

New Developments in Video Encoded for iBooks.app

Last September, I looked at the need to make sure that video embedded in an ePub document would play and play well on all of the devices where the iBooks.app is found (iPhone, iPod touch and iPad). Since then, new developments not only make this topic more important but also warrant a review of the principles involved.

New developments include the fact that the ePub standard is now at version three and formally specifies the embedding of audio and video files in ePub documents using the same HTML 5 video and audio tags that the iBooks.app has long supported. Thus, there will likely be more eReaders supporting rich media beside iBooks. When those new eReaders do arrive, we'll need to look at them closely. For now, the iBooks.app is still the only game in town for rich media in ePub documents. As well, we have new hardware and operating systems to consider and that brings up "backward compatibility" as an increasingly important factor.

So, let's take another look at this topic and bring our earlier conclusions up to date.

Hardware Targeting The iBooks.app, now at version 1.2.1, requires iOS 4 or later so that reduces the range of hardware that it will run on to iPhone 3G, iPhone 3GS, iPhone 4, iPad plus the 2^nd or later generation iPod touch. The most recent of these (iPhone 4, iPad and 4G iPod touch) sport the significantly more powerful A4 processor and, thus, are capable of decoding more advanced versions of the MPEG-4 H.264 standard. Specifically, these newer models can handle H.264 Main Profile whereas the older models can only handle the Baseline Profile. Simply put, baseline will play everywhere iBooks runs whereas video encoded using the Main Profile will only play on newer iOS devices. Thus, aiming for the greatest reach means that we need to stick with Baseline Profile for maximum backward compatibility.

Viewport Targeting Thanks to an article by video wizard Jan Ozer entitled "Encoding for the iPad", I've learned that, thanks to the video scaling function on the iPad, encoding HD (16:9) video at 640x360 pixels looks great for most content regardless of viewport size. The exception to this is video with fine detail such as smallish text. That gets very blurry but since we're talking about ePub here, that shouldn't be a problem. Save the text for the body of your eBook and use static images for fine visual detail.

Bit-rate & File Size Targeting An ePub document is self-contained. Generally speaking, there are no external dependencies for video or audio. Thus, video, audio, image and text files are all found inside the ePub file. Actually, an ePub document is a special kind of Zip archive that eReader apps know how to handle. Because of this self-sufficiency, bit-rate is not as critical as it would be if ePub video required an Internet download as a web page does. On the other hand, video file size is a function of bit-rate and, so, has a lot to do with the file size of your ePub document, how long it takes to download and how long your audience will have to wait before they can start reading. A frame size of 640x360 and a bit-rate of 1.4 Mbps (mega-bits per second) using the MPEG-4 Baseline Profile should produce pristine video for most HD (16:9) sourced content.

The Golden Mean Of course there are good use cases that diverge from this golden mean. The iPad's 1024x768 screen, for example, will accommodate 720p (1280x720) by scaling it down to 1024x576. The iPad will also support MPEG-4 Main Profile and very high bit rates for optimal quality playback. However, unlike other media synched to iOS devices via the iTunes.app, the video content of self-contained ePub documents is not evaluated, Thus, it is quite possible to create an eBook containing video or audio that will not play properly or at all. It is critical that ePub documents be tested on the smallest, least capable iOS devices as well as the iPad and iPhone 4 if your goal is to have the video in your ePub documents accessible wherever iBooks is found.

Reviewing Our Recommended Methods Previously, I had recommended using the iPhone presets that are found in the output options of many applications such as QuickTime Player, Screenflow, Miro and Handbrake. This is still a viable strategy though it is not optimal. The iPhone preset in QuickTime Player, for example, currently encodes a 480x300 video at 880 Kbps using the Baseline Profile. It will scale and play everywhere that iBooks runs but its not the best that we can do. Actually, the iPod preset comes closer at: 1.463 Mbps, 640x360, Baseline@3.0 when the source is 1280x720. This is much better for 16:9 source.

Scalability Testing As mentioned above, testing is critical. Not only do we have to test on multiple devices but we also need to test using all of the orientation and scaling options. Here is an illustrated list to guide your testing:

The iPod touch and the iPhone all share the same screen size (viewport). Here is the portrait view:



Here, we see a proportional scaling of the video constrained by the width of the screen in this orientation. Note the "expander" button in the upper right corner of the center image. Tapping that button fills the screen without regard for aspect ratio (proportion) like this:



Notice that the expander button has now become a collapse button.

Next, we look at the landscape orientation where the same controls are operative but the visual effect is, of course, different.



Next, we look at the iPad where video can be played in-situ, full-screen (proportional) and full screen (non-proportional). We start with the portrait in-situ view.



Here, the iPad advantage is that video can be played in context with other book elements such as text. Tapping on the expander button will bring us to the proportional full screen view which looks like this:



Notice that the collapse button on the video control bar (bottom) is the same in both images. Tapping this element will return to the in-situ video where you may continue watching or stop the video and resume reading text. The expander buttons at the top have a different function. The one on the leftmost image expands to the non-proportional full-screen image on the right. The one on the rightmost image returns the full screen proportional video on the left.

All of these views need to be checked in order to assure that your audience is seeing the video as you intend. Fortunately, the algorithms that Apple uses to scale video in these widely different environments is very good. If you start with high quality source at 1280x720 and then export using the iPod preset, you will likely be satisfied with all of these views.

The question of whether eBooks, eTexts, eHandouts and so on can be published as podcasts is an important one for educators, especially if their students are using the iTunes application, if they are using iTunes U to help deliver courseware or if they are involved at all in mLearning. Of course, we are talking about eBooks that are free because there are ample means for distributing eBooks that are not free. In the case of the new iPad, there is the iBookstore although I have not yet found a school textbook for sale there surely that's just a matter of time.

The iBookstore is currently only available via the iBooks.app on the iPad but we will see an iBooks.app for the iPhone and iPod touch as soon as iPhone OS 4.0 is released this summer. Then, too, there are many titles found in the iBookstore that are in the public domain and free. It appears that Apple has taken good advantage of the work of the Gutenberg project. All of these eBooks are in the ePub format which appears to be the de facto standard.

Thus, we're not so concerned about books that are free, in the public domain and have been digitized by Project Gutenberg or books that are not free. What we are concerned with are those free, ePub-based text books, handouts, reprints, and other documents that university faculty might want to assign to their students to read on one of these mobile devices or even on a larger, not-so-mobile computer.

Right now, we can give a qualified "yes" to this question. Here's how it works. Simply include .epub files in the RSS feed that you create for a podcast channel using same conventions that you've been using for .mp3, .mp4, mov, .m4a, .m4b, .m4v and .pdf files. The only thing that will be different is the suffix on attached files. That will be .epub.

Here's what will happen. When you subscribe to this channel using iTunes.app 9.1.1 or newer, the .epub files will become available for download according to the prefs that you've set in iTunes.app. Once downloaded, you will see these .epub files referenced in both the Podcast Library and the Books Library of the iTunes.app. Synchronizing with an iPad, those .epub documents will be placed on the shelf in the iBooks app and identified as a podcast in the list view. Ditto for iPhones and iPod touch devices with iPhone OS 4.0 (summer 2010). All of the features of the iBooks application such as bookmarking will be available as your students read these ePub-based documents.

Important caveat. Currently, iTunes U does not accept the .epub suffix in an RSS feed so if you try to subscribe to a podcast channel via the iTunes U RSS Tab feature, it will fail. However, I expect that this restriction will be lifted once all Apple mobile devices have their own iBooks application. That should be this summer.

To see this in action, subscribe to my eBook Test Channel.