So far, we've been focused on the use of HD video in ePub documents but what about SD? Standard definition video is still quite common, especially for older works produced for television. Whereas HD has an aspect ratio of 1.78:1 or 16:9, SD has an aspect ratio of 1.33:1 or 4:3.
As we have seen, iOS devices (iPad, iPhone and iPod touch), introduce black bars (letterbox, pillarbox and windowbox) and cropping/scaling in order to maintain the aspect ratio of the source video. Without these, the video we see on these devices would be stretched or squished which is not what the author intended. This is due to the fact that iOS devices do not have 16:9 screens. The iPad is 1024x768 which is 1:33:1 or 4:3. The iPhone and iPod touch are 480x320 (older) or 960x640 (newer, retina displays) which is 1.5:1 or 3:2. In other words, iOS devices have screens with aspect ratios that are the same as or very close to standard definition TV dimensions.
I've created the following screencast to illustrate the many ways that the iBooks application on these iOS devices will present Standard Definition (SD) video. There's a lot to account for with different orientations (portrait vs landscape) and whether the reader chooses to view video embedded in an EPUB-based eBook in situ, proportional full screen or non-proportional full screen. It's best to see all of this in action.
If your source is SD, QuickTime X Player will maintain that aspect ratio in all of its export options. This is ideal for video embedded in EPUB-based eBooks. Playback at 640x480 minimizes the use of black bars, cropping and scaling, the iPod bit rate keeps a good balance between file size and fidelity in most cases and the MPEG-4 Baseline Profile assures that the video will play wherever the iBooks.app is found.
Sunday, April 3, 2011
New Developments in Video Encoded for iBooks.app
Last September, I looked at the need to make sure that video embedded in an ePub document would play and play well on all of the devices where the iBooks.app is found (iPhone, iPod touch and iPad). Since then, new developments not only make this topic more important but also warrant a review of the principles involved.
New developments include the fact that the ePub standard is now at version three and formally specifies the embedding of audio and video files in ePub documents using the same HTML 5 video and audio tags that the iBooks.app has long supported. Thus, there will likely be more eReaders supporting rich media beside iBooks. When those new eReaders do arrive, we'll need to look at them closely. For now, the iBooks.app is still the only game in town for rich media in ePub documents. As well, we have new hardware and operating systems to consider and that brings up "backward compatibility" as an increasingly important factor.
So, let's take another look at this topic and bring our earlier conclusions up to date.
Hardware Targeting The iBooks.app, now at version 1.2.1, requires iOS 4 or later so that reduces the range of hardware that it will run on to iPhone 3G, iPhone 3GS, iPhone 4, iPad plus the 2nd or later generation iPod touch. The most recent of these (iPhone 4, iPad and 4G iPod touch) sport the significantly more powerful A4 processor and, thus, are capable of decoding more advanced versions of the MPEG-4 H.264 standard. Specifically, these newer models can handle H.264 Main Profile whereas the older models can only handle the Baseline Profile. Simply put, baseline will play everywhere iBooks runs whereas video encoded using the Main Profile will only play on newer iOS devices. Thus, aiming for the greatest reach means that we need to stick with Baseline Profile for maximum backward compatibility.
Viewport Targeting Thanks to an article by video wizard Jan Ozer entitled "Encoding for the iPad", I've learned that, thanks to the video scaling function on the iPad, encoding HD (16:9) video at 640x360 pixels looks great for most content regardless of viewport size. The exception to this is video with fine detail such as smallish text. That gets very blurry but since we're talking about ePub here, that shouldn't be a problem. Save the text for the body of your eBook and use static images for fine visual detail.
Bit-rate & File Size Targeting An ePub document is self-contained. Generally speaking, there are no external dependencies for video or audio. Thus, video, audio, image and text files are all found inside the ePub file. Actually, an ePub document is a special kind of Zip archive that eReader apps know how to handle. Because of this self-sufficiency, bit-rate is not as critical as it would be if ePub video required an Internet download as a web page does. On the other hand, video file size is a function of bit-rate and, so, has a lot to do with the file size of your ePub document, how long it takes to download and how long your audience will have to wait before they can start reading. A frame size of 640x360 and a bit-rate of 1.4 Mbps (mega-bits per second) using the MPEG-4 Baseline Profile should produce pristine video for most HD (16:9) sourced content.
The Golden Mean Of course there are good use cases that diverge from this golden mean. The iPad's 1024x768 screen, for example, will accommodate 720p (1280x720) by scaling it down to 1024x576. The iPad will also support MPEG-4 Main Profile and very high bit rates for optimal quality playback. However, unlike other media synched to iOS devices via the iTunes.app, the video content of self-contained ePub documents is not evaluated, Thus, it is quite possible to create an eBook containing video or audio that will not play properly or at all. It is critical that ePub documents be tested on the smallest, least capable iOS devices as well as the iPad and iPhone 4 if your goal is to have the video in your ePub documents accessible wherever iBooks is found.
Reviewing Our Recommended Methods Previously, I had recommended using the iPhone presets that are found in the output options of many applications such as QuickTime Player, Screenflow, Miro and Handbrake. This is still a viable strategy though it is not optimal. The iPhone preset in QuickTime Player, for example, currently encodes a 480x300 video at 880 Kbps using the Baseline Profile. It will scale and play everywhere that iBooks runs but its not the best that we can do. Actually, the iPod preset comes closer at: 1.463 Mbps, 640x360, Baseline@3.0 when the source is 1280x720. This is much better for 16:9 source.
Scalability Testing As mentioned above, testing is critical. Not only do we have to test on multiple devices but we also need to test using all of the orientation and scaling options. Here is an illustrated list to guide your testing:
The iPod touch and the iPhone all share the same screen size (viewport). Here is the portrait view:
Here, we see a proportional scaling of the video constrained by the width of the screen in this orientation. Note the "expander" button in the upper right corner of the center image. Tapping that button fills the screen without regard for aspect ratio (proportion) like this:
Notice that the expander button has now become a collapse button.
Next, we look at the landscape orientation where the same controls are operative but the visual effect is, of course, different.
Next, we look at the iPad where video can be played in-situ, full-screen (proportional) and full screen (non-proportional). We start with the portrait in-situ view.
Here, the iPad advantage is that video can be played in context with other book elements such as text. Tapping on the expander button will bring us to the proportional full screen view which looks like this:
Notice that the collapse button on the video control bar (bottom) is the same in both images. Tapping this element will return to the in-situ video where you may continue watching or stop the video and resume reading text. The expander buttons at the top have a different function. The one on the leftmost image expands to the non-proportional full-screen image on the right. The one on the rightmost image returns the full screen proportional video on the left.
All of these views need to be checked in order to assure that your audience is seeing the video as you intend. Fortunately, the algorithms that Apple uses to scale video in these widely different environments is very good. If you start with high quality source at 1280x720 and then export using the iPod preset, you will likely be satisfied with all of these views.
New developments include the fact that the ePub standard is now at version three and formally specifies the embedding of audio and video files in ePub documents using the same HTML 5 video and audio tags that the iBooks.app has long supported. Thus, there will likely be more eReaders supporting rich media beside iBooks. When those new eReaders do arrive, we'll need to look at them closely. For now, the iBooks.app is still the only game in town for rich media in ePub documents. As well, we have new hardware and operating systems to consider and that brings up "backward compatibility" as an increasingly important factor.
So, let's take another look at this topic and bring our earlier conclusions up to date.
Hardware Targeting The iBooks.app, now at version 1.2.1, requires iOS 4 or later so that reduces the range of hardware that it will run on to iPhone 3G, iPhone 3GS, iPhone 4, iPad plus the 2nd or later generation iPod touch. The most recent of these (iPhone 4, iPad and 4G iPod touch) sport the significantly more powerful A4 processor and, thus, are capable of decoding more advanced versions of the MPEG-4 H.264 standard. Specifically, these newer models can handle H.264 Main Profile whereas the older models can only handle the Baseline Profile. Simply put, baseline will play everywhere iBooks runs whereas video encoded using the Main Profile will only play on newer iOS devices. Thus, aiming for the greatest reach means that we need to stick with Baseline Profile for maximum backward compatibility.
Viewport Targeting Thanks to an article by video wizard Jan Ozer entitled "Encoding for the iPad", I've learned that, thanks to the video scaling function on the iPad, encoding HD (16:9) video at 640x360 pixels looks great for most content regardless of viewport size. The exception to this is video with fine detail such as smallish text. That gets very blurry but since we're talking about ePub here, that shouldn't be a problem. Save the text for the body of your eBook and use static images for fine visual detail.
Bit-rate & File Size Targeting An ePub document is self-contained. Generally speaking, there are no external dependencies for video or audio. Thus, video, audio, image and text files are all found inside the ePub file. Actually, an ePub document is a special kind of Zip archive that eReader apps know how to handle. Because of this self-sufficiency, bit-rate is not as critical as it would be if ePub video required an Internet download as a web page does. On the other hand, video file size is a function of bit-rate and, so, has a lot to do with the file size of your ePub document, how long it takes to download and how long your audience will have to wait before they can start reading. A frame size of 640x360 and a bit-rate of 1.4 Mbps (mega-bits per second) using the MPEG-4 Baseline Profile should produce pristine video for most HD (16:9) sourced content.
The Golden Mean Of course there are good use cases that diverge from this golden mean. The iPad's 1024x768 screen, for example, will accommodate 720p (1280x720) by scaling it down to 1024x576. The iPad will also support MPEG-4 Main Profile and very high bit rates for optimal quality playback. However, unlike other media synched to iOS devices via the iTunes.app, the video content of self-contained ePub documents is not evaluated, Thus, it is quite possible to create an eBook containing video or audio that will not play properly or at all. It is critical that ePub documents be tested on the smallest, least capable iOS devices as well as the iPad and iPhone 4 if your goal is to have the video in your ePub documents accessible wherever iBooks is found.
Reviewing Our Recommended Methods Previously, I had recommended using the iPhone presets that are found in the output options of many applications such as QuickTime Player, Screenflow, Miro and Handbrake. This is still a viable strategy though it is not optimal. The iPhone preset in QuickTime Player, for example, currently encodes a 480x300 video at 880 Kbps using the Baseline Profile. It will scale and play everywhere that iBooks runs but its not the best that we can do. Actually, the iPod preset comes closer at: 1.463 Mbps, 640x360, Baseline@3.0 when the source is 1280x720. This is much better for 16:9 source.
Scalability Testing As mentioned above, testing is critical. Not only do we have to test on multiple devices but we also need to test using all of the orientation and scaling options. Here is an illustrated list to guide your testing:
The iPod touch and the iPhone all share the same screen size (viewport). Here is the portrait view:
Here, we see a proportional scaling of the video constrained by the width of the screen in this orientation. Note the "expander" button in the upper right corner of the center image. Tapping that button fills the screen without regard for aspect ratio (proportion) like this:
Notice that the expander button has now become a collapse button.
Next, we look at the landscape orientation where the same controls are operative but the visual effect is, of course, different.
Next, we look at the iPad where video can be played in-situ, full-screen (proportional) and full screen (non-proportional). We start with the portrait in-situ view.
Here, the iPad advantage is that video can be played in context with other book elements such as text. Tapping on the expander button will bring us to the proportional full screen view which looks like this:
Notice that the collapse button on the video control bar (bottom) is the same in both images. Tapping this element will return to the in-situ video where you may continue watching or stop the video and resume reading text. The expander buttons at the top have a different function. The one on the leftmost image expands to the non-proportional full-screen image on the right. The one on the rightmost image returns the full screen proportional video on the left.
All of these views need to be checked in order to assure that your audience is seeing the video as you intend. Fortunately, the algorithms that Apple uses to scale video in these widely different environments is very good. If you start with high quality source at 1280x720 and then export using the iPod preset, you will likely be satisfied with all of these views.
Friday, December 24, 2010
The Prospects for EPUB Version 3
As we know, the EPUB standard is being revised to version 3. It's on a fast track (approval expected Q3 2011) and the 14 point charter is ambitious. In addition to danger, change also presents opportunity.
Apple is among those who have an interest in shaping that change and they have been overtly busy with the iBooks.app for iPad, iPhone and iPod touch as well as the iBookstore that supports that app. Apple has also been busy in the background. Among the many things that Apple is doing that are just out of view for most of the general public is what I believe can only be characterized as a concerted effort to influence the evolution of version three of the EPUB standard.
Although Apple claims to follow the EPUB standard, they have recently introduced features supported in the iBooks app that do not conform to the current standard. Conforming EPUB files play just fine in iBooks but these new, super, EPUBs do things in iBooks that haven't been possible before, especially on mobile devices.
Using the HTML 5 [video] and [audio] tag was the first, simple but dramatic move. This has been followed by other features that require rather sophisticated Javascript coding such as the recent implementation of "fully illustrated books" in iBooks 1.2 that open graphic and tabular information in a new window that overrides the conventional ePub "flowed" text format in favor of full-screen display, even in landscape mode.
Deconstructing these super ePub files reveals important insights into both the iBooks app and Apple's EPUB strategy. Standard ePub files are but Zip archives containing text and image files. Those contained files evidence a striking similarity with the code of the web. Thus, eReaders are akin to web browsers, albeit very specialized ones. Apple has simply added Javascript and new HTML 5 constructs to the CSS, XHTML and so on found in conventional ePub files and interpreted by conventional eReaders. Rather than web-like, Apple's ePub files are web files, period. Similarly, the iBooks app is more of a modern HTML 5 web browser than an ePub eBook reader.
It has been pointed out that Apple isn't currently sharing how-to information on these techniques with medium to small publishers and self-publishers. I suspect that there's more that's not being shared. It may well be that Apple isn't yet sharing the tools to easily implement these features except with the chosen few. Why do I think that? Here is why:
Apple developers have access to tools and documentation that others do not. One of the tools recently released to Apple developers is the iAd Producer application. It looks like this:
What iAd Producer does is provide one with a drag and drop UI to assemble an interactive, animated display, an iAd. This iAd is actually an HTML 5 mini-web site. That is, a folder containing HTML, CSS, images, media and Javascript that any modern web browser will display properly because it is standards-based. This tool is only available to Apple developers.
Might there be a similar tool that is only available to the larger publishers? I don't know for sure but confirmation of the existence of such a tool would be unsurprising to me. It would be trivial. I think, to adapt what we see as iAd developer to an application that could be called iBook Developer. Nowadays, web and mobile technology are all converging toward HTML 5.
So, why is Apple out in front on this? Alan Kay, former Apple Fellow, said it this way in 1971: "The best way to predict the future is to invent it." By putting out living examples of EPUB that human readers respond positively to, Apple is inventing the future. It is, thereby, also influencing the development of this key standard. Those who are working on the standard simply cannot ignore these events.
Apple is engaged in a very smart campaign that will benefit all those who create and read digital books as well as benefit Apple. Yet another example of "doing well by doing good."
Apple is among those who have an interest in shaping that change and they have been overtly busy with the iBooks.app for iPad, iPhone and iPod touch as well as the iBookstore that supports that app. Apple has also been busy in the background. Among the many things that Apple is doing that are just out of view for most of the general public is what I believe can only be characterized as a concerted effort to influence the evolution of version three of the EPUB standard.
Although Apple claims to follow the EPUB standard, they have recently introduced features supported in the iBooks app that do not conform to the current standard. Conforming EPUB files play just fine in iBooks but these new, super, EPUBs do things in iBooks that haven't been possible before, especially on mobile devices.
Using the HTML 5 [video] and [audio] tag was the first, simple but dramatic move. This has been followed by other features that require rather sophisticated Javascript coding such as the recent implementation of "fully illustrated books" in iBooks 1.2 that open graphic and tabular information in a new window that overrides the conventional ePub "flowed" text format in favor of full-screen display, even in landscape mode.
Deconstructing these super ePub files reveals important insights into both the iBooks app and Apple's EPUB strategy. Standard ePub files are but Zip archives containing text and image files. Those contained files evidence a striking similarity with the code of the web. Thus, eReaders are akin to web browsers, albeit very specialized ones. Apple has simply added Javascript and new HTML 5 constructs to the CSS, XHTML and so on found in conventional ePub files and interpreted by conventional eReaders. Rather than web-like, Apple's ePub files are web files, period. Similarly, the iBooks app is more of a modern HTML 5 web browser than an ePub eBook reader.
It has been pointed out that Apple isn't currently sharing how-to information on these techniques with medium to small publishers and self-publishers. I suspect that there's more that's not being shared. It may well be that Apple isn't yet sharing the tools to easily implement these features except with the chosen few. Why do I think that? Here is why:
Apple developers have access to tools and documentation that others do not. One of the tools recently released to Apple developers is the iAd Producer application. It looks like this:
What iAd Producer does is provide one with a drag and drop UI to assemble an interactive, animated display, an iAd. This iAd is actually an HTML 5 mini-web site. That is, a folder containing HTML, CSS, images, media and Javascript that any modern web browser will display properly because it is standards-based. This tool is only available to Apple developers.
Might there be a similar tool that is only available to the larger publishers? I don't know for sure but confirmation of the existence of such a tool would be unsurprising to me. It would be trivial. I think, to adapt what we see as iAd developer to an application that could be called iBook Developer. Nowadays, web and mobile technology are all converging toward HTML 5.
So, why is Apple out in front on this? Alan Kay, former Apple Fellow, said it this way in 1971: "The best way to predict the future is to invent it." By putting out living examples of EPUB that human readers respond positively to, Apple is inventing the future. It is, thereby, also influencing the development of this key standard. Those who are working on the standard simply cannot ignore these events.
Apple is engaged in a very smart campaign that will benefit all those who create and read digital books as well as benefit Apple. Yet another example of "doing well by doing good."
Sunday, December 12, 2010
Internet Archive eBook Reader Released
The Internet Archive has a new online eBook Reader with some very nice features such as the ability to share eBooks on your web site or via your blog. Here's an example:
Of course you can also download the eBook in ePub and other formats for reading on mobile and other devices, Here's a screenshot:
Read more about this at The Open Library Blog.
Of course you can also download the eBook in ePub and other formats for reading on mobile and other devices, Here's a screenshot:
Read more about this at The Open Library Blog.
Sunday, October 31, 2010
I've started sampling some of the ePub eBooks available via iTunes U. One of the technically more interesting ones so far is "The Restless Universe" from The Open University. What caught my eye in this textbook was the implementation of a simple interactive Q&A which uses a hyperlink to show the answer, presumably after the learner has written down what they think the answer is, workbook style. Many would consider this to be essential for eTextbooks.
The following screen shot illustrates this feature:
I believe that this is easily implemented in the Pages application (part of Apple's iWork suite of applications c. $77 education price) and will attempt to test that in the coming days.
Here are three iTunes U sites that were ready with ePubs at launch:
- Connexions - Open Textbooks
- The Open University - OpenLearn Course Material
- Oxford - Shakespeare's First Folio
What is currently supported:
For content providers:
˜ Uploading ePub files to a hosted iTunes U site
˜ Adding ePub via RSS to a feed-based iTunes U site
NOTE: The latter option is the most commonly used method in the University System of Georgia. Content is uploaded to the USG Podcasting Server which generates an RSS feed that can be used to populate a collection (aka course or channel) in iTunes U. This method enables faculty to disseminate content in other venues.
Here is an examples page showing the various options for sharing an entire channel or a single episode (aka lecture, lesson or item).
For end-users:
˜ Downloading ePub files from iTunes U to an iTunes client on Mac and PC
˜ Syncing downloaded ePub files from iTunes to iBooks on an iPod touch, iPhone, or iPad
˜ Downloading ePub files, over the air, from iTunes U to iBooks on an iPad
Coming soon:
˜ Downloading ePub files, over the air, from iTunes U to iBooks on an iPod touch or iPhone (this will come later with iOS 4.2)
The following screen shot illustrates this feature:
I believe that this is easily implemented in the Pages application (part of Apple's iWork suite of applications c. $77 education price) and will attempt to test that in the coming days.
Here are three iTunes U sites that were ready with ePubs at launch:
- Connexions - Open Textbooks
- The Open University - OpenLearn Course Material
- Oxford - Shakespeare's First Folio
What is currently supported:
For content providers:
˜ Uploading ePub files to a hosted iTunes U site
˜ Adding ePub via RSS to a feed-based iTunes U site
NOTE: The latter option is the most commonly used method in the University System of Georgia. Content is uploaded to the USG Podcasting Server which generates an RSS feed that can be used to populate a collection (aka course or channel) in iTunes U. This method enables faculty to disseminate content in other venues.
Here is an examples page showing the various options for sharing an entire channel or a single episode (aka lecture, lesson or item).
For end-users:
˜ Downloading ePub files from iTunes U to an iTunes client on Mac and PC
˜ Syncing downloaded ePub files from iTunes to iBooks on an iPod touch, iPhone, or iPad
˜ Downloading ePub files, over the air, from iTunes U to iBooks on an iPad
Coming soon:
˜ Downloading ePub files, over the air, from iTunes U to iBooks on an iPod touch or iPhone (this will come later with iOS 4.2)
Friday, October 29, 2010
On October 29, 2010, iTunes U administrators received the message shown in the screen shot below. It announces the fact that ePub-based eBooks may now be included in iTunes U collections (aka courses or channels) and items (aka lessons or episodes).
This is a very significant development as I argued back in April 2010 that it would be. Support for ePub in iTunes U provides faculty with an excellent vehicle for disseminating instructional materials in the form of ePub documents. These can range in size from 300-400 page textbooks to much smaller documents such as handouts or even syllabi.
We should also expect to see an upsurge in interest in applications that facilitate the creation of ePub documents and there are already many options to choose from. Apple's pages application added support for ePub as reported here back in August 2010. Pages and the iWork suite of which it is a part will undoubtedly become must have software.
This is a very significant development as I argued back in April 2010 that it would be. Support for ePub in iTunes U provides faculty with an excellent vehicle for disseminating instructional materials in the form of ePub documents. These can range in size from 300-400 page textbooks to much smaller documents such as handouts or even syllabi.
We should also expect to see an upsurge in interest in applications that facilitate the creation of ePub documents and there are already many options to choose from. Apple's pages application added support for ePub as reported here back in August 2010. Pages and the iWork suite of which it is a part will undoubtedly become must have software.
Thursday, October 28, 2010
My presentation was very well received and we were able to capture both the slides and presenter audio/video to produce this 54 minute screencast. I hope that you find it interesting and useful. One of three versions will be auto-selected for you depending upon the device that you are using (computer or mobile device with a good web browser and MPEG-4 video capability). The desktop computer version is quite large (1.13 GB) so may take a while to load enough data to play without interruption to the end if your bandwidth is limited.
This is also a good example of screencasting using the terrific ScreenFlow application from Telestream. The slideshow was captured on the presenter's MacOS X 10.6.x MacBook Pro laptop and the presenter audio and video were captured by a Canon DV cam and wireless mic (thanks to Blake Bridges of the Georgia Digital Innovation Group).
This is also a good example of screencasting using the terrific ScreenFlow application from Telestream. The slideshow was captured on the presenter's MacOS X 10.6.x MacBook Pro laptop and the presenter audio and video were captured by a Canon DV cam and wireless mic (thanks to Blake Bridges of the Georgia Digital Innovation Group).
Subscribe to:
Posts (Atom)