Optimizing iPad Video for iBooks Author and Pages

Including video in an eBook, especially an eTextbook, has become essential and expected. If a still image is worth a thousand words, then the potential explanatory power of an image moving at 30 frames per second must be exponentially greater. Getting from potential to actual value, however, requires compelling content and clear, efficient video.

The preeminent tools for creating eBooks and eTextbooks containing video are iBooks Author and Pages, both from Apple. The iBooks Author app comes bundled with MacOS X which is free and Pages comes bundled at no extra cost with all new Macs. There are no other apps on any platform that match both the affordability and ease of use of Pages and iBooks Author.

However, there have been issues with the way these apps have handled eBook video in the past. Some of those issues have been documented here. This post is a re-examination of those issues to assess the current state of video in eBooks produced by these two apps. Have things gotten better, remained unchanged or gotten worse? That is the question.

The iBooks Author (iBA) application has had difficulty handling video efficiently. In version 1.0, authors had to meet very strict and vague encoding requirements in order to avoid rejection of a video file. In succeeding versions, Apple addressed the many issues arising from this situation by incorporating a video optimizing routine that would re-encode any video brought into an iBA project via the Media widget. Early versions of this new functionality solved the problems that authors were having with video encoding for their eBook projects but they did so by making those video files unnecessarily large. It was quite common for a video file to become three times larger than the original after being “optimized” in iBooks Author.

Jan Ozer developed a clever way to circumvent the iBA video optimizer using custom presets for the Handbrake video transcoder and he offered how-to workshops on that tactic. In June of 2014, however, Apple updated iBA and rendered this workaround ineffective.

The current version of iBA (2.2) was released in October of 2014. The status of the video optimizer has never been referenced in any iBooks Author version history or Read Me file. Our knowledge of it has only come about by comparing input to output and then inferring what goes on within the innards this black box.

Even less well known is the history of video handling in the Pages application. The current version of Pages (5.5.2) has an export to ePub function which silently and invisibly invokes a video optimizer that, like iBA, calls upon system-level routines such as AVConvert to optimize your media files.” Pages seems to have followed iBooks Author with respect to handling the inclusion of video in eBooks without requiring authors to become expert in the art of video compression.

So with the increasing importance of video in eBooks and especially in eTextbooks, it is important to stay abreast of the ways in which Apple’s iBooks Author produces *.ibooks files containing video and how Apple’s Pages produces *.epub files containing video even if Apple isn’t very forthcoming on the topic.

METHODOLOGY

Both Pages and iBooks Author output eBooks (*.epub or *.ibooks files) that share the same architecture. Each eBook is actually a collection of various files and folders in an enclosing folder that has been archived using the Zip standard. The suffix of the resulting *.zip file is then changed to *.epub or *.ibooks which enables eReaders to know that they can be rendered as eBooks. The free ePub Zip/Unzip application handles this chore very nicely.

It’s being able to “look inside” that enables us to see what these video optimizers have done to the video we provided, comparing input to output, before to after. An inexpensive ($0.99) app named MediaInfo provides us with detailed information on each version of the video we wish to examine. Less information can be gotten for free by using QuickTime X Player’s Get Info function.

So our methodology is to bring video files into our iBA and Pages projects, export as *.ibooks or *.epub, then deconstruct the resulting eBook by unzipping it, finding the video files, developing information about those files and, finally, comparing and analyzing the differences.

DETAILED RESULTS

Raw data, including test videos, with detailed analyses may be found in the following eTextbook created with iBooks Author. Download this *.ibooks file and read it with the iBooks app on an iPad 2 or newer or a MacOS X 10.9 or newer computer. This volume will illustrate by examples precisely how the current (2.2) version of iBooks Author handles ingested video. It will also document what Pages does to ingested video.

EXECUTIVE SUMMARY

The iBooks Author (iBA) application has improved video handling to the point where corrective action after export is no longer warranted or recommended. The media optimizer in iBA 2.2 will increase file size only slightly over what was ingested. However, optimizing video with Handbrake prior to ingestion produced significantly smaller video files of equivalent quality and this is recommended as current best practice. The eTextbook cited above contains a link to the custom Handbrake preset used in these investigations with instructions on how to import it into the Handbrake app and use it.

The Pages app, unfortunately, has not improved as much. Video files ingested by Pages 5.5.2 are typically inflated to three times their original size or more. The best practice response to this unfortunate situation is to use the deconstruction techniques used in this investigation to replace the inflated video file with a version encoded by Handbrake using the aforementioned preset. Stepwise instructions on how to do this are found in the eTextbook cited above.

Video Bloat in Pages ePub Exports and How to Fix It

Pages 5.5.1 for MacOS X has an excellent ePub export option. That export option includes elements of ePub 3 such as embedded audio and video so Pages, being free with MacOS X, is a very attractive application for teachers at all levels who want to create eTextbooks containing rich media. However, there is a problem with Pages ePub video that this blog post will identify, document and provide solutions for. First, the problem statement.
Starting with Apple’s ePub template, we create a test project. To follow along, download the “ePub best practices” document from Apple’s “Creating ePub files with Pages” support document, open it in Pages and do File > Save as template … which makes this template available to us now and in the future.
After we have the template saved, we add a 75 MB 1280 X 720 video with a duration of 5 minutes and 36 seconds (download) to the still open Pages document. It now looks like this:

The next step is to export this document as an ePub (File > Export To > ePub …). As we look at the output of this process in the Finder, we see that the ePub is about 280 MB despite the fact that the parent file, the Pages project, is only 80 MB in size. Here’s what that looks like:

As far as I know, there is no application that can add useful data to source video. So, what is going on? Could this be an instance of unnecessary …

… video bloat? The following delves into how to investigate that possibility. The first step in that investigation is to download ePub Zip Unzip so that we may peer inside the ePub file structure, find the video file and compare it with the original that was initially fed to Pages. This app has a very simple user interface that looks like this:

The reality of an ePub file is that it is actually what is called a package file, a folder containing a variety of files that has been archived into a single file using the Zip compression algorithm. What ePub Zip Unzip does in this first step is simply to unzip the file into the folder it started out as. Once we have unzipped the ePub file, we can open this folder and examine its contents. Specifically, we’re interested in the video media because this is the only thing that we added to the Pages project beyond the template itself. Here is what we see:

Clearly, we are looking at an instance of file bloat. The video file has grown more than 200 MB beyond its original size. This is not likely to be an improvement. To determine why this is so, we turn to another free, open source app called MediaInfo. Note that QuickTime X Player will also provide the most essential of this data but MediaInfo can provide additional data that is very useful to advanced users. Here’s what MediaInfo says about the original file:

… and the video file exported by Pages:

Here’s a table summarizing this analysis:

The change in suffix from .mp4 to .m4v is inconsequential but the change in bit rate is key. The bit rate is 3.5 times greater than the source file! This is precisely what has inflated the file size. The formula for file size is bit rate times duration. So, will our readers benefit from this larger, longer to download file? Let’s see.

Link to full size version of this video: click here (opens in a new window)

Quite frankly, I see no improvement. Certainly not an improvement justifying an additional 200 MB in file size.

So that’s the problem. Now, how do we fix it? We already have some terrific clues. We have the ePub on the operating table and opened up so we can see and operate upon all of the internals. Why not try to replace the bloated video with the more efficient one that we started with. Here’s how:

First, we move the better file into the same location as the bloated version. Note that they have different names. This complicates things just a little bit. So copy the good video into the same folder as the bad video, like this:

This is a simple drag and drop operation in the Finder. Next, we copy the full name of the bad video like so:

Move the bad (bloated) file to the trash like so:

Select and paste the name of the bad (bloated) file renaming the good file like so:

Accept the change in the file suffix like so:

Zip the folder back into an .ePub file like so:

Enjoy the success of having reduced the file size of your ePub.

Test the results in the iBooks.app to confirm that success.

Standard Definition Video in Books Using the .epub or .ibooks Format

So far, we've been focused on the use of HD video in ePub documents but what about SD? Standard definition video is still quite common, especially for older works produced for television. Whereas HD has an aspect ratio of 1.78:1 or 16:9, SD has an aspect ratio of 1.33:1 or 4:3.

As we have seen, iOS devices (iPad, iPhone and iPod touch), introduce black bars (letterbox, pillarbox and windowbox) and cropping/scaling in order to maintain the aspect ratio of the source video. Without these, the video we see on these devices would be stretched or squished which is not what the author intended. This is due to the fact that iOS devices do not have 16:9 screens. The iPad is 1024x768 which is 1:33:1 or 4:3. The iPhone and iPod touch are 480x320 (older) or 960x640 (newer, retina displays) which is 1.5:1 or 3:2. In other words, iOS devices have screens with aspect ratios that are the same as or very close to standard definition TV dimensions.

I've created the following screencast to illustrate the many ways that the iBooks application on these iOS devices will present Standard Definition (SD) video. There's a lot to account for with different orientations (portrait vs landscape) and whether the reader chooses to view video embedded in an EPUB-based eBook in situ, proportional full screen or non-proportional full screen. It's best to see all of this in action.



If your source is SD, QuickTime X Player will maintain that aspect ratio in all of its export options. This is ideal for video embedded in EPUB-based eBooks. Playback at 640x480 minimizes the use of black bars, cropping and scaling, the iPod bit rate keeps a good balance between file size and fidelity in most cases and the MPEG-4 Baseline Profile assures that the video will play wherever the iBooks.app is found.

New Developments in Video Encoded for iBooks.app

Last September, I looked at the need to make sure that video embedded in an ePub document would play and play well on all of the devices where the iBooks.app is found (iPhone, iPod touch and iPad). Since then, new developments not only make this topic more important but also warrant a review of the principles involved.

New developments include the fact that the ePub standard is now at version three and formally specifies the embedding of audio and video files in ePub documents using the same HTML 5 video and audio tags that the iBooks.app has long supported. Thus, there will likely be more eReaders supporting rich media beside iBooks. When those new eReaders do arrive, we'll need to look at them closely. For now, the iBooks.app is still the only game in town for rich media in ePub documents. As well, we have new hardware and operating systems to consider and that brings up "backward compatibility" as an increasingly important factor.

So, let's take another look at this topic and bring our earlier conclusions up to date.

Hardware Targeting The iBooks.app, now at version 1.2.1, requires iOS 4 or later so that reduces the range of hardware that it will run on to iPhone 3G, iPhone 3GS, iPhone 4, iPad plus the 2^nd or later generation iPod touch. The most recent of these (iPhone 4, iPad and 4G iPod touch) sport the significantly more powerful A4 processor and, thus, are capable of decoding more advanced versions of the MPEG-4 H.264 standard. Specifically, these newer models can handle H.264 Main Profile whereas the older models can only handle the Baseline Profile. Simply put, baseline will play everywhere iBooks runs whereas video encoded using the Main Profile will only play on newer iOS devices. Thus, aiming for the greatest reach means that we need to stick with Baseline Profile for maximum backward compatibility.

Viewport Targeting Thanks to an article by video wizard Jan Ozer entitled "Encoding for the iPad", I've learned that, thanks to the video scaling function on the iPad, encoding HD (16:9) video at 640x360 pixels looks great for most content regardless of viewport size. The exception to this is video with fine detail such as smallish text. That gets very blurry but since we're talking about ePub here, that shouldn't be a problem. Save the text for the body of your eBook and use static images for fine visual detail.

Bit-rate & File Size Targeting An ePub document is self-contained. Generally speaking, there are no external dependencies for video or audio. Thus, video, audio, image and text files are all found inside the ePub file. Actually, an ePub document is a special kind of Zip archive that eReader apps know how to handle. Because of this self-sufficiency, bit-rate is not as critical as it would be if ePub video required an Internet download as a web page does. On the other hand, video file size is a function of bit-rate and, so, has a lot to do with the file size of your ePub document, how long it takes to download and how long your audience will have to wait before they can start reading. A frame size of 640x360 and a bit-rate of 1.4 Mbps (mega-bits per second) using the MPEG-4 Baseline Profile should produce pristine video for most HD (16:9) sourced content.

The Golden Mean Of course there are good use cases that diverge from this golden mean. The iPad's 1024x768 screen, for example, will accommodate 720p (1280x720) by scaling it down to 1024x576. The iPad will also support MPEG-4 Main Profile and very high bit rates for optimal quality playback. However, unlike other media synched to iOS devices via the iTunes.app, the video content of self-contained ePub documents is not evaluated, Thus, it is quite possible to create an eBook containing video or audio that will not play properly or at all. It is critical that ePub documents be tested on the smallest, least capable iOS devices as well as the iPad and iPhone 4 if your goal is to have the video in your ePub documents accessible wherever iBooks is found.

Reviewing Our Recommended Methods Previously, I had recommended using the iPhone presets that are found in the output options of many applications such as QuickTime Player, Screenflow, Miro and Handbrake. This is still a viable strategy though it is not optimal. The iPhone preset in QuickTime Player, for example, currently encodes a 480x300 video at 880 Kbps using the Baseline Profile. It will scale and play everywhere that iBooks runs but its not the best that we can do. Actually, the iPod preset comes closer at: 1.463 Mbps, 640x360, Baseline@3.0 when the source is 1280x720. This is much better for 16:9 source.

Scalability Testing As mentioned above, testing is critical. Not only do we have to test on multiple devices but we also need to test using all of the orientation and scaling options. Here is an illustrated list to guide your testing:

The iPod touch and the iPhone all share the same screen size (viewport). Here is the portrait view:



Here, we see a proportional scaling of the video constrained by the width of the screen in this orientation. Note the "expander" button in the upper right corner of the center image. Tapping that button fills the screen without regard for aspect ratio (proportion) like this:



Notice that the expander button has now become a collapse button.

Next, we look at the landscape orientation where the same controls are operative but the visual effect is, of course, different.



Next, we look at the iPad where video can be played in-situ, full-screen (proportional) and full screen (non-proportional). We start with the portrait in-situ view.



Here, the iPad advantage is that video can be played in context with other book elements such as text. Tapping on the expander button will bring us to the proportional full screen view which looks like this:



Notice that the collapse button on the video control bar (bottom) is the same in both images. Tapping this element will return to the in-situ video where you may continue watching or stop the video and resume reading text. The expander buttons at the top have a different function. The one on the leftmost image expands to the non-proportional full-screen image on the right. The one on the rightmost image returns the full screen proportional video on the left.

All of these views need to be checked in order to assure that your audience is seeing the video as you intend. Fortunately, the algorithms that Apple uses to scale video in these widely different environments is very good. If you start with high quality source at 1280x720 and then export using the iPod preset, you will likely be satisfied with all of these views.

The Prospects for EPUB Version 3

As we know, the EPUB standard is being revised to version 3. It's on a fast track (approval expected Q3 2011) and the 14 point charter is ambitious. In addition to danger, change also presents opportunity.

Apple is among those who have an interest in shaping that change and they have been overtly busy with the iBooks.app for iPad, iPhone and iPod touch as well as the iBookstore that supports that app. Apple has also been busy in the background. Among the many things that Apple is doing that are just out of view for most of the general public is what I believe can only be characterized as a concerted effort to influence the evolution of version three of the EPUB standard.

Although Apple claims to follow the EPUB standard, they have recently introduced features supported in the iBooks app that do not conform to the current standard. Conforming EPUB files play just fine in iBooks but these new, super, EPUBs do things in iBooks that haven't been possible before, especially on mobile devices.

Using the HTML 5 [video] and [audio] tag was the first, simple but dramatic move. This has been followed by other features that require rather sophisticated Javascript coding such as the recent implementation of "fully illustrated books" in iBooks 1.2 that open graphic and tabular information in a new window that overrides the conventional ePub "flowed" text format in favor of full-screen display, even in landscape mode.

Deconstructing these super ePub files reveals important insights into both the iBooks app and Apple's EPUB strategy. Standard ePub files are but Zip archives containing text and image files. Those contained files evidence a striking similarity with the code of the web. Thus, eReaders are akin to web browsers, albeit very specialized ones. Apple has simply added Javascript and new HTML 5 constructs to the CSS, XHTML and so on found in conventional ePub files and interpreted by conventional eReaders. Rather than web-like, Apple's ePub files are web files, period. Similarly, the iBooks app is more of a modern HTML 5 web browser than an ePub eBook reader.

It has been pointed out that Apple isn't currently sharing how-to information on these techniques with medium to small publishers and self-publishers. I suspect that there's more that's not being shared. It may well be that Apple isn't yet sharing the tools to easily implement these features except with the chosen few. Why do I think that? Here is why:

Apple developers have access to tools and documentation that others do not. One of the tools recently released to Apple developers is the iAd Producer application. It looks like this:



What iAd Producer does is provide one with a drag and drop UI to assemble an interactive, animated display, an iAd. This iAd is actually an HTML 5 mini-web site. That is, a folder containing HTML, CSS, images, media and Javascript that any modern web browser will display properly because it is standards-based. This tool is only available to Apple developers.

Might there be a similar tool that is only available to the larger publishers? I don't know for sure but confirmation of the existence of such a tool would be unsurprising to me. It would be trivial. I think, to adapt what we see as iAd developer to an application that could be called iBook Developer. Nowadays, web and mobile technology are all converging toward HTML 5.

So, why is Apple out in front on this? Alan Kay, former Apple Fellow, said it this way in 1971: "The best way to predict the future is to invent it." By putting out living examples of EPUB that human readers respond positively to, Apple is inventing the future. It is, thereby, also influencing the development of this key standard. Those who are working on the standard simply cannot ignore these events.

Apple is engaged in a very smart campaign that will benefit all those who create and read digital books as well as benefit Apple. Yet another example of "doing well by doing good."

I've started sampling some of the ePub eBooks available via iTunes U. One of the technically more interesting ones so far is "The Restless Universe" from The Open University. What caught my eye in this textbook was the implementation of a simple interactive Q&A which uses a hyperlink to show the answer, presumably after the learner has written down what they think the answer is, workbook style. Many would consider this to be essential for eTextbooks.

The following screen shot illustrates this feature:




I believe that this is easily implemented in the Pages application (part of Apple's iWork suite of applications c. $77 education price) and will attempt to test that in the coming days.

Here are three iTunes U sites that were ready with ePubs at launch:

- Connexions - Open Textbooks
- The Open University - OpenLearn Course Material
- Oxford - Shakespeare's First Folio

What is currently supported:

For content providers:
˜ Uploading ePub files to a hosted iTunes U site
˜ Adding ePub via RSS to a feed-based iTunes U site

NOTE: The latter option is the most commonly used method in the University System of Georgia. Content is uploaded to the USG Podcasting Server which generates an RSS feed that can be used to populate a collection (aka course or channel) in iTunes U. This method enables faculty to disseminate content in other venues.

Here is an examples page showing the various options for sharing an entire channel or a single episode (aka lecture, lesson or item).

For end-users:
˜ Downloading ePub files from iTunes U to an iTunes client on Mac and PC
˜ Syncing downloaded ePub files from iTunes to iBooks on an iPod touch, iPhone, or iPad
˜ Downloading ePub files, over the air, from iTunes U to iBooks on an iPad


Coming soon:
˜ Downloading ePub files, over the air, from iTunes U to iBooks on an iPod touch or iPhone (this will come later with iOS 4.2)

On October 29, 2010, iTunes U administrators received the message shown in the screen shot below. It announces the fact that ePub-based eBooks may now be included in iTunes U collections (aka courses or channels) and items (aka lessons or episodes).

This is a very significant development as I argued back in April 2010 that it would be. Support for ePub in iTunes U provides faculty with an excellent vehicle for disseminating instructional materials in the form of ePub documents. These can range in size from 300-400 page textbooks to much smaller documents such as handouts or even syllabi.

We should also expect to see an upsurge in interest in applications that facilitate the creation of ePub documents and there are already many options to choose from. Apple's pages application added support for ePub as reported here back in August 2010. Pages and the iWork suite of which it is a part will undoubtedly become must have software.

My presentation was very well received and we were able to capture both the slides and presenter audio/video to produce this 54 minute screencast. I hope that you find it interesting and useful. One of three versions will be auto-selected for you depending upon the device that you are using (computer or mobile device with a good web browser and MPEG-4 video capability). The desktop computer version is quite large (1.13 GB) so may take a while to load enough data to play without interruption to the end if your bandwidth is limited.






This is also a good example of screencasting using the terrific ScreenFlow application from Telestream. The slideshow was captured on the presenter's MacOS X 10.6.x MacBook Pro laptop and the presenter audio and video were captured by a Canon DV cam and wireless mic (thanks to Blake Bridges of the Georgia Digital Innovation Group).

An ePub document need not be a full length novel or textbook. It could well be a set of class notes, a handout, a monograph, or any other instructional support document. So, rapid and easy publishing of .epub files is of great interest to academics these days now that students have a wide variety of hardware and software that is optimized for reading the ePub format.

All one needs is access to a server with an RSS feed capability. In Georgia, all university system faculty have access to the USG Podcasting Server which is very easy to use and has the ability to dynamically generate RSS feeds. However, any standard web server can be used if you can create and maintain your own RSS feed, a simple text document that follows the RSS 2.0 Standard.

The following screencast will take you through the process of uploading an .epub file to the USG Podcasting Server, copying the subscription address to share with students and then follow the student path of subscribing to the RSS feed with the iTunes application and synchronizing those eBooks with a mobile device running Apple's iBooks application.





Note: If a student doesn't have an Apple device running the iBooks application, there are other options that we'll examine in an upcoming blog post covering the many ways there are to read an .epub file.

Now that we can create ePub-based eBooks quickly and easily, our thoughts naturally turn to finding ways and means of publishing those digital books and other ePub documents. There are several options to consider so I'll take them on one at a time in successive blog posts. This post will deal with establishing an online catalog that enables remote access to your book collection. Depending upon who you share the address and optional password with, this could be just for your personal convenience or it could be where your students go to discover and acquire ePub documents (books, monographs, handouts, etc.) that you have made available to them.

A tool that can help us achieve this end is the free, cross-platform application called "Calibre." Calibre has many attributes such as managing your eBook library, converting eBooks, syncing to eBook reading devices, viewing eBooks and providing you with a content server for online access to your book collection. It is this last capability that we'll be focusing on here.

Calibre's content server is based upon the Open Publication Distribution System (OPDS). Many eReader applications such as Stanza are capable of browsing an ODPS catalog by categories such as newest, title, ratings, publishers, series, authors, tags, etc. and then downloading selected items. The catalog can be password protected. Here's a screenshot of the server settings:

One thing that you'll want to be sure of is that you network administration has arranged for access via HTTP port 8080 or whatever other port you have configured the server to "listen" to for requests. There may be a firewall in place that would prevent that. If that is not possible, just use port 80 which is the default port for HTTP. If you can surf the web, port 80 is open and, so, this should work.

Here's a screen shot showing Stanza on iPad browsing an OPDS catalog:

It's also possible for Caliber to send eBooks and magazines via email. Here's the setup for that:

Following up on my last post, I wanted to talk about how a typical faculty person might quickly and easily create an ePub-based eBook just by selecting text and invoking an automated process on their desktop or laptop computer if that faculty person is fortunate enough to have a modern Macintosh computer.
I also wanted to show rather than describe how easy this can be so here's my first video screencast for this blog. I hope that each frame is worth 10,000 words to you.




You may want to download this ePub-based eBook, "side load" it into the iTunes application and synch to a device running the iBooks application (iPad, iPhone or iPod touch).

For the next episode, I'll try to show how a more complex eBook project would be done using this same Automator service so stay tuned.

Ibis Reader is a very capable on-line eBook reader that uses HTML 5, an emerging new standard for web sites and web-based services. Thus, Ibis Reader requires an HTML 5-compliant web browser. Fortunately, most modern browsers on desktop, laptop and even "smart" mobile devices are largely HTML 5-compliant. HTML 5 enables the Ibis Reader to blur the distinction between cloud-based and native applications, especially on certain mobile devices where network connectivity may not always be fast enough or might not be available at all.

To follow along, point mobileSafari (or other webkit-based mobile browser) at: http://m.ibisreader.com and then, on Apple iOS devices, create a web clip so that this web app is presented in full screen mode (eliminates browser UI elements) and is easier to access later on. If you are using an iPhone, iPad, iPod touch or an Android phone, you'll be asked to permit the creation of a local database of up to 50 MB in size. Desktop/laptop readers will want to use: http://ibisreader.com.

Ibis Reader provides an encouraging answer to those would-be eBook authors who ask, "If I create an eBook in ePub format, who will be able to read it?" That answer is, "Anyone with Internet access and an HTML 5 web browser." Of course, many will have other software and hardware options as well and they may well prefer them for one reason or another but Ibis Reader has the widest reach of all and that may be critical for many authors. Being a web application, Ibis Reader can be used on any desktop, laptop or netbook computer and on many mobile devices including the wildly popular Apple lineup of iPad, iPhone and iPod touch and any of the Android phones.

Among the many benefits of HTML 5 is Local Storage which benefits iPad, iPhone, iPod touch and Android phones by enabling the download of entire eBooks to a local SQL Lite database on the device. This makes it possible for these devices to retrieve and present the Ibis Reader web app plus the full text of one or more eBooks without being connected to a network at all. Previously, this is what differentiated native apps from web apps. The web app could not run without a network whereas the native app could. This is now no longer the case.

Native apps still have some advantages over web apps in the areas of rich media and interactivity. However, even these may go away as the work beginning the next iteration of the ePub standard gets underway. More on this in a later post.

The question of whether eBooks, eTexts, eHandouts and so on can be published as podcasts is an important one for educators, especially if their students are using the iTunes application, if they are using iTunes U to help deliver courseware or if they are involved at all in mLearning. Of course, we are talking about eBooks that are free because there are ample means for distributing eBooks that are not free. In the case of the new iPad, there is the iBookstore although I have not yet found a school textbook for sale there surely that's just a matter of time.

The iBookstore is currently only available via the iBooks.app on the iPad but we will see an iBooks.app for the iPhone and iPod touch as soon as iPhone OS 4.0 is released this summer. Then, too, there are many titles found in the iBookstore that are in the public domain and free. It appears that Apple has taken good advantage of the work of the Gutenberg project. All of these eBooks are in the ePub format which appears to be the de facto standard.

Thus, we're not so concerned about books that are free, in the public domain and have been digitized by Project Gutenberg or books that are not free. What we are concerned with are those free, ePub-based text books, handouts, reprints, and other documents that university faculty might want to assign to their students to read on one of these mobile devices or even on a larger, not-so-mobile computer.

Right now, we can give a qualified "yes" to this question. Here's how it works. Simply include .epub files in the RSS feed that you create for a podcast channel using same conventions that you've been using for .mp3, .mp4, mov, .m4a, .m4b, .m4v and .pdf files. The only thing that will be different is the suffix on attached files. That will be .epub.

Here's what will happen. When you subscribe to this channel using iTunes.app 9.1.1 or newer, the .epub files will become available for download according to the prefs that you've set in iTunes.app. Once downloaded, you will see these .epub files referenced in both the Podcast Library and the Books Library of the iTunes.app. Synchronizing with an iPad, those .epub documents will be placed on the shelf in the iBooks app and identified as a podcast in the list view. Ditto for iPhones and iPod touch devices with iPhone OS 4.0 (summer 2010). All of the features of the iBooks application such as bookmarking will be available as your students read these ePub-based documents.

Important caveat. Currently, iTunes U does not accept the .epub suffix in an RSS feed so if you try to subscribe to a podcast channel via the iTunes U RSS Tab feature, it will fail. However, I expect that this restriction will be lifted once all Apple mobile devices have their own iBooks application. That should be this summer.

To see this in action, subscribe to my eBook Test Channel.

With the ePub standard for eBooks rather firmly in place, the discussion has turned to what happens next. Many are curious about how interactive digital publications (eBook, eText, eMag, ePamphlet, etc.) might be implemented. Will they be implemented as iPhone OS applications that can only be experienced on Apple mobile devices (iPad, iPhone or iPod touch)? Will they be implemented with some future iteration of the ePub standard? Or will they be implemented in some other way?

The fact that the ePub standard really doesn't support interactivity leads some to settle upon applications as the way to go. Concerned with being restricted to Apple mobile devices and the much higher costs of participation of the app model, others have talked-up the possibilities of extending the ePub standard to include greater interactivity. DRM, by the way, can be applied to either of these options but experience has already shown that these protections can be defeated just as easily.

There is a third alternative that isn't getting much attention right now. I'll call it "Hyper Lit" for want of a better label. It's already here and living amongst us. We are not talking about this new alternative yet simply because we don't recognize it as a potential solution to this challenge. HyperLit is hidden in plain sight. That technology is nothing more than HTML 5 poured into new containers.

Take a look at the developer docs for iTunes LP and iTunes Extras at: http://www.apple.com/itunes/lp-and-extras/ It's nothing more than HTML, CSS and Javascript in a folder that has been zipped and given a new suffix, either .itlp or .ite. The evidence that anyone can create this kind of interactive experience is here: http://ituneslp.net/ Note that .itlp and .ite files can be side-loaded into iTunes with drag & drop and that action totally circumvents the iTunes Store. No money changes hands.

Now imagine these containers holding interactive educational content. That package of content might have an identifying suffix such as .itlo (iTunes Learning Object) or .itlm (iTunes Learning Module). Further down the road, we can imagine these new media types being a part of an RSS feed (podcast channel episodes) handled by aggregators such as the iTunes application.

The application model will appeal to those who own or can rent the significant means of production required. It provides great flexibility in arranging interactive experiences and the illusion of protection from piracy. It also adds a measure of exclusivity which helps customers part with their money more easily. We can call this the monetized model for digital interactive reading experiences. Note that apps can be free to the consumer where the costs of production have been paid by some other, possibly eleemosynary, entity.

The ePub-based eBook model will appeal to those who concentrate upon providing a linear experience. Commercial interests will add DRM to "protect" the content from piracy and others will forego that option and rely instead on Creative Commons licensing or similar mechanisms. Depending upon the presence of DRM, we can call this either the monetized model for digital linear reading experiences or the open model for digital linear reading experiences. Since the cost of production using the ePub standard is very low, ePub-based models will be widely used.

The fledgeling HyperLit model will appeal to those who want to provide an open interactive reading experience. HyperLit documents will provide faculty and students with ways and means to talk about complex ideas with text, audio, video and animation, cite online sources and even initiate online dialog amongst the readers of a document. After all, a HyperLit document is basically a locally viewed web site using the file:/// scheme.

Thus, the digital reading experience will likely be available to us in several flavors, the traditional linear experience and new, interactive ways of engaging content. How these new interactive approaches to the reading experience get sorted out will be interesting to watch, especially in educational circles. There are lots of contenders. Not all will survive but it is equally likely that no one option will predominate. The need for open content on the one hand and the need for economic viability on the other are too great.

As for the reading experience itself, there are some interesting views already coming to the fore. Although focused exclusively on the novel, I found these two opinion pieces by Michael Grothaus to be interesting:

Dear John Makinson and Penguin, please don't "reinvent" books.

A tale of two mediums: Despite the iPad, traditional books aren't going anywhere.

With the advent of the iPad and other mobile devices that support eBook readers, the eBook is enjoying a renaissance. The most widely adopted eBook format is the ePub open standard. There is now a well-established ePub eco-system comprised of software tools for creating and maintaining eBooks as well as a wide variety of devices for reading them. Of great interest in academia is the fact that much of the software needed to create, transform, convert and manage .epub files is free. This will become ever more important as educators rely more and more on digital content.

Thus, I can create .epub files using free, cross-platform software and students can consume that content using a variety of mobile and not-so-mobile devices. Using an iPad, for example, my students would simply drag the .epub files that I provide into the Books library of the iTunes application and then synch with the iPad. I could simply post those .epub files to my Faculty Web Server account for download.

However, as an academic using iTunes U, I am very eager to learn whether and how eBooks can be distributed using the same podcasting model as is now used by iTunes U to distribute audio, video and PDF files. The advantages of offering content via podcasting over simple file services are many, including being able to more easily restrict access, use subscription to schedule delivery of content over the semester timeline and extending the benefits of content aggregation (automated organization) to my students.

Although we frequently think about eBooks as novel-length tomes, the format will also support shorter works such as a textbook chapters, assignments, journal articles and the like. An eText and supplementary content could be delivered in serialized form throughout the semester. The problem, right now, is that neither the iTunes application nor iTunes U will handle files carrying the .epub suffix. This is a decision that only Apple can take but we can certainly lobby for it and I think that we should. I mention this to anyone at Apple who will listen and I have also signed up for a free developer account so that I can submit "enhancement requests" via bugreport.apple.com

With the discovery that DRM'd ePub volumes attempt to disallow quoting via copy and paste, one hoped-for advantage of eTextbooks over their physical cousins seems to be in doubt, at least for commercially published eTextBooks that are sure to be 'protected' with Digital Rights Management (DRM) software. On the recently released iPad, text copied from a DRM's ePub volume cannot be copied and pasted directly into a writing application such as Pages.

Of course, innovative folks have already conjured up a workaround.reported here

... that kinda, sorta works. But why should this be necessary? Whose priorities trump all others?

Traditionally, writing a paper and quoting from various sources inevitably involved transcription, just like Monks of old.  There simply wasn’t another way.

With the dawn of the digital era, it became possible to use a digital scanner and Optical Scanning Recognition (OCR) software to produce machine readable text and illustrations that could be re-purposed in academic and other writing.  Better but still not a boon to students without access to expensive gear. They were still mired in ancient scholarly bogs.

Now that we have eBooks and eTexts and eJournals and wonderful, magical devices to read them on, one would think that the scholarly life would ease-up a bit but noooooooo, the commercial interests will have none of that.  The doctrine of presumptive guilt means that scholars will continue to be consigned to ancient monastic transcription rather than the liberation of copy and paste.  That, of course, will come at the cost of less time invested in developing their higher order thinking skills.

Quoting excerpts from a book for the purpose of criticism or for scholarly purposes has long been protected as a fair use in US and international law. DRM places an unnecessary and unwarranted tax upon the exercise of these rights, especially in this digital era where there are better options than tedious and error-prone transcription.

The tragedy here is that DRM doesn't prevent the kind of copyright infringement that threatens a publisher's profitability. Those wholesale pirates will have their way regardless. Only scholarship will suffer.