Choosing the right ebook file formats

There are many types of paper and paper-size  to choose from when printing a paper book. The same myriad of choices applies to digital books except instead of texture and paper-weight, we’re concerned with what device and software the reader may choose to read your book.

If I had all the time in the world and I was producing a fairly simple digital book which didn’t require format specific features I suggest ensuring my digital book was produced in all of these formats:

  1. epub – an open format which means it will hopefully be around for years to come. An epub file will open in many ebook reader software and hardware such as an iPad or Android device. You can choose epub 2 or epub 3 depending on your target audience – stick with epub 2 for the widest support.
  2. Amazon kindle format (mobi and kf8) – because amazon rules the market it makes sense to ensure it works on both the kindle software and the kindle device
  3. Adobe PDF – Almost every device on the planet will open a PDF file so its a safe bet and is also open (see point 1). Further-more PDF is built to handle print, which could be a useful feature for some readers
  4. Apple ibook (IBA) – if I wanted to sell via the apple marketplace then i’d want to use their format.

Tip: if I was to distribute any of my own books I’d plump for giving ALL of the above to the reader in one bundle. This way they get the choice of their preferred format. I guess an issue for a small number of readers is they’ll receive a zipped bundle and have no clue what to do next. I didn’t say it was fool-proof.

Why is there more than one format I hear you cry? The market for selling ebooks is fierce and each major provider wants you to lock-in to their ecosystem, hence a fragmented market and a high number of file formats.

The key to producing our book fairly painlessly for each format is to plan early and use a workflow that doesn’t rely on any one particular format. The nerds of this world call this being ‘device agnostic’.  Essentially write the book using whatever your preferred writing tool is and then have a workflow that makes it straight forward to produce for the context and constraints you have.

If I wanted to sell my digital book and I happened to know that all my potential customers used Amazon then I would likely produce this format first. If I wanted the book to open in  say five years time I’d go for epub and PDF.

A final word on file format features

At the time of writing in early 2014, the format with the most advanced features that are implemented by a reading device is the apple ibooks format. It has some pretty impressive features for adding interactivity that may be essential for the success of our book. In which case your workflow for ibooks will split off and tackle those issues. Just be careful that you don’t go so far down that road that you can’t make a good experience for the other hobbled formats. By this i mean that readers and/or software for displaying epub, kindle and PDF don’t yet support any of the advanced feature support you’d like. Things change with each new reader software release though so the future could brighten up any day. If you need to use a feature e.g. video support, check if its supported in your primary testing kit.

GDS Service Manual – the ebook

Back in July Jukesie asked if I could knock out the GDS Service Manual for his commute and as a handy guide to refer back to.

I completely forgot I did this ebook and meant to stick it online somewhere so here it is…kinda. It was a bit of a rush so PLEASE do let me know about missing/broken bits.

I have the EPUB version (ipad, tablets of all flavours etc) and Kindle kicking around. As I am still on holiday the epub will have to do for now as I don’t have access to my FTP and WordPress has a 2mb limit (aargh).

Soon I will post the epub and Kindle properly, not from my phone abroad when I’m meant to be offline !

Note that I purposefully left out the YouTube videos as myself and others may not regularly have WiFi so I figured this was a fair compromise. If there is interest in the video I can put one out.

I hand coded this file so there may be errors which are likely mine!

I hope you’ll find it useful as I know Matt Jukes and I have.

 

 

 

ARLIS Conference: Self-publishing in education

Yesterday I was invited to speak at the Arts and Library Society annual conference. I spoke for 40mins about how staff and students have been self-publishing and the need for libraries to add ‘self-publishing’ to their long list of topics requiring support.

I see self-publishing as an opportunity for libraries to really shine in their support for staff in particular. To quote Craig Mod:

The way books are written has changed.
The canvas for books has changed.
The post-published life of a book has changed.

I enjoyed the talk and hope that my hopping around style of delivery still gave a sense of the topic.

I was lucky to also see a few of the other talks and it really opened my eyes to publishing in the arts and museum arena. I think i’ll be hanging around some of the museum curators to see if I can get my hands stuck into a new pet project!

The slides can be found on Google Docs (the PPT is busted) below and if anybody needs me to add some context just get in touch.

David Hopkins

David Hopkins @hopkinsdavid is a learning technologist, biker, and popular blogger. When you want to know about something about e-learning, check David’s blog first – don’t waste your time. We talk about David’s new book, QR Codes in Education, the writing process, making ebooks, and our lofty dream to make more books to which end the score is David 1 – 0 Zak.

Length: 42mins

[soundcloud params=”auto_play=false&show_comments=false”]https://soundcloud.com/zakmensah/002-david-hopkins[/soundcloud]

Show notes

http://www.guykawasaki.com/ape/ is a great beginners guide to self-publishing and covers many of the issues around the current ecosystem and pricing.

http://craigmod.com/journal/post_artifact/ Craig Mod produced this must read essay about the new ecosystem for authoring. Start with this essay then read every other essay too!

http://calibre-ebook.com/ The first tool that many of us will use to make an ebook. Use it and ensure you test widely.

http://www.zakmensah.co.uk/2012/10/02/ebook-testing-kit/ My testing tips and tools for ensuring a good reader experience.

Hayley Atkinson

Zak and Hayley Atkinson talk about self-publishing students and the challenges of making your own books.

22mins

[soundcloud params=”auto_play=false&show_comments=false”]https://soundcloud.com/zakmensah/001-hayley-atkinson[/soundcloud]

Show notes and links

http://ebooksinlearning.wordpress.com/
The blog that Hayley uses to showcase some of the recent projects at Leeds on using ebooks with students.

https://twitter.com/ebookslearning
The best way to get in touch with Hayley is via her twitter account.

http://www.apple.com/uk/ibooks-author/
Hayley and the students are using Apple ibooks author to self-publishing their own books.

http://calibre-ebook.com/
For non Apple users, Calibre is probably the best tool to get you started with making your own books.

http://validator.idpf.org/
If using the epub format make sure you check it validates as this is required by many of the ebook online shops and helps reduce the chances of errors for the reader.

http://creativecommons.org/licenses/
Using Creative Commons licensed materials is a free and legal way to use third-party text, images, video and audio. Always link back to the source so that others can also share the material and why not consider sharing some of your own materials using a Creative Commons license.

Terese Bird

Terese Bird portrait

As with all good people you discover online, I came across Terese Bird by way of her high quality work around all things digital media. Terese kindly offered to be interviewed about her typical day and interest in digital books. Thank you Terese for taking the time to answer my questions and being my first interview subject!

Tell us about yourself

I am a learning technologist with the Institute of Learning Innovation at the University of Leicester. We primarily do research into learning and learning innovation, disseminate findings, and construct practical teaching and learning models which apply the findings, for our own university and beyond. I assist with the research in ways such as figuring out how to implement new or tricky technological trials or solutions, setting up and analysing data from online surveys, and conducting focus groups and interviews. I present findings in workshops and conference presentations, and write about project findings whenever I can — through blogging, project reports, and other writing projects.

What’s a typical day like for you?

I don’t think I have a typical day. This morning I had to buy milk, coffee and tea on my way in to work, in preparation for day 1 of a two-day workshop on Learning Design – the 7Cs of learning design, which is a framework developed by our director, Professor Grainne Conole. Once at work and seeing to the refreshments, room, and technology, I greeted guests — 12 medical educators from one of our local hospitals, plus a few visiting scholars at ILI — and got everyone settled into the session. While Grainne taught her sessions, I helped discussions, took photos and tweeted about the session on our Twitter account @learninginn. I spent lunchtime setting up laptops and testing microphone and playback facilities on the demonstrator computer, in readiness for the session I later taught on Rich Media Learning Materials — the strengths and weaknesses of and tips to create text, audio, and video learning materials especially usable on any mobile device. Once the workshop ended at 4pm, I spent the rest of the workday, and much of my evening, setting up meetings and replying to emails.

How did you get interested in digital publishing?

I got interested in digital publishing in 2009 because it was a focus of research in the JISC-funded DUCKLING project our team worked on. My job was to figure out how to load locally-authored learning materials onto Sony e-readers (the first e-reader available in the UK), to be shipped to distance masters students around the world. I stumbled upon the software Calibre and just fiddled with it until I had a method of nicely converting Word documents into first lrf, then epub. This was pretty ground-breaking stuff in 2009, and it helped me to win the ALT Highly Commended Learning Technologist of the Year 2010.  (Sorry for the plug but I’m kinda proud of that.)

What are your go-to tools or software for writing?

My go-to tool for my own writing is either directly into WordPress, or Microsoft Word. Hey, it works. My go-tool for an ebook is iBooks Author. It’s free and it offers good-looking templates, because I cannot design my way out of a paper bag. It saves as a pdf and as .ibooks, which might be all I need. If I need epub, then I copy and paste the text and images into Pages and export as epub. If I had a copy of Indesign I would probably just use that to do everything, but Indesign isn’t cheap and it doesn’t strike me as simple to use, so for now I’ll stick with easy and cheap (iBooks Author and Pages).

Where would you like to see digital publishing going?

I would like to see digital publishing offer financial rewards primarily to the author, and to the publisher as long as s/he offers added value. I would like good textbooks to be available in rich, innovative formats at low prices for students. I would like to see teachers and academics being empowered through academic publishing, and I would not like to see publishers trying to take over teachers’ roles as is beginning to happen now.

As for the technology of digital publishing, I don’t really think it’s necessary for there to be one uniform way for all mobile devices to be able to read an ebook. I don’t mind there being an Apple and an Amazon and a Samsung and a Google. Companies need to earn money because people need to earn money.

If the Napster model had prevailed over iTunes, musicians would be earning less money these days. I wish publishers could find a way to really add value, to support authors and allow digital innovation all at the same time.

In a few years we’ll probably have people crashing into each other on the pavement because they’ll be wearing Google Glass and trying to read ebooks and walk down the street at the same time. Or even worse – trying to read ebooks and drive at the same time! You read it here first!

Hello World: travels in virtuality – the ebook

At the weekend I wrapped up a project turning ‘Hello World: travels in virtuality ‘ by Sue Thomas into both Kindle and epub files.

The book was originally released in dead-tree’s format back in 2004 and digital publishing has allowed us to breath a new lease of life into the book and be easily available to the world!

If you want to hear more about how I made the book let me know and i’ll write something but you can just as easily read Ahab! by Craig Mod which covers all the ground and points you to a helpful sample project.  One of the key resources in Craig’s article is to the Kindle previewer which will display the ebook file in all generations of Kindle, saving you having to purchase all of the devices. My top tip is to start with the newest versions and work back until you get bored.

The Kindle version is now available to purchase for only £2.05.

04 Publishing jargon

On my ebook publishing journey I have begun to stumble across processes, stages and phrases that use traditional print terminology. I will note them here for my own sanity and maybe that of others as I figure where it makes sense we should use existing and established phrases.

  • Galley proof – used to describe review copies of a print book
  • Uncorrected proof – commonly used in place of ‘galley proof’ for digital books
  • Name of punctuation and general typography – a list down the right of key technical terms…. a tilde is not a squiggle!

03 Under the cover of the MRC ebook

Collection of mobile devices with the mrc ebook
Photo Credit: Nigel Goldsmith

Last week the Medical Research Council released it’s new annual report in various digital formats. Tribehut (that’s me yeeeah) planned, designed and built the ebook versions.

I have been banging on about making ebooks for some time and this caught the attention of Matt Jukes. Ever the ‘digital’ experimentalist, Matt asked if I would be interested in making the digital versions of their annual report ‘Advancing Medicine, changing lives’. I jumped at the opportunity.

With budgets being squeezed and a general ban on many print outputs, the public sector has been forced to turn to ‘digital’ outputs. For those of us who love ‘digital’ now is the time to stand up and to demonstrate that digital might actually be good.

I felt this project could act as a perfect demonstration of just how good digital can be and I wasn’t planning on dropping the ball.

Planning and design

One of the key aspects of a ensuring a successful project workflow is to get one defining voice on the client side, in this case Matt, and to get the content at the very beginning of the project. Assumptions about content will not only waste time but lead you down the wrong path as you simply cannot make judgements without seeing all the pieces.  Thankfully the team at the MRC had already produced the PDF version so this wasn’t an issue. I was sent the PDF along with a short brief which gave me enough to assess the project requirements and commit to the project. Note – ensure that the ‘final’ content is really that, I have been burnt before with being sent an old ‘final’ version which stops a project dead.

The original beautiful design was created with ‘print’ very firmly in mind. From the PDF (I call this the blueprint), I discussed with Matt the sticking points in regard to any potential limitations, constraints and show-stoppers in relation to the technologies we’d be employing. Those decisions not only helped to manage expectations but also enabled me to make appropriate design choices.

For example, most ereaders and reading apps tend to over-ride many of the designers choices, such as font choice. So my suggestion was to leave the font choice to the reader and/or device. Also, the PDF version has some graphical flourishes in the margins which visually link articles across a page spread, but this is lost in the ebook – the viewport (what you can see at any given time) is not fixed as it is with print. For better or worse, an ebook is primary about text and so we agreed to keep the pretty graphics for chapter starts and within sub-sections where they are context specific.

Regarding what devices we needed to target, we had no previous data, so I couldn’t be sure what devices the audience had. Thus the ebook needed to work across as many devices, apps and configurations as is reasonably possible. Furthermore, being an annual report it required a minimum 1 year shelf life. Rather than making an assumption that everybody would be using either the Apple iPad or an Amazon Kindle I planned to make it work very well across the board. This means building a well constructed epub and Kindle mobi file with a sprinkling of technical features that are unique to the iPad or Kindle. Luckily ereaders ignore what they don’t support so there wasn’t too much to worry about regarding this.

I am a fan of rapidly building something as soon as possible, test it and then refine.
At this point both sides were ready and excited to move to the code phase.

Coding

Until Matt and the team could ‘see and use’ the ebook it would be difficult to complete the project so I pre-warned them that the previews were just that (we ended up with 25 iterations with 13 being the first MRC saw).

At this point I requested all of the PDF assets, which included all source imagery and colour palettes for the build.

I needed a final EPUB2 file and an accompanying Kindle mobi file. I chose to stick with EPUB2 over the newer EPUB3 as it isn’t widely supported yet and also I didn’t have any use for the specific EPUB3 features. No reason to reduce compatibility for the sake of being an early adopter – showing off experimental features was too high risk in this scenario.

For this build I used an HTML editor called Coda (but you could use anything as long as it has find+replace with REGex support as a bonus).

I took all of the text from the PDF and dumped each section of content into a plain html panel within Coda.  I then re-built all of the sections, adding tags etc until I was happy that I had all of my content with placeholders for the images. Some of this can be automated to reduce heavy coding BUT be aware that automation may disappear your content…so I just used a few regex commands to mass add things like paragraph tags.

Next I built a global template epub page with 1 sentence of text and ensured that the template worked.

From the working template I added each section and then added the other required files for an epub such as the mimetype, xml and content.opf which includes links to all the assets used and metadata.

An epub file has a strict bunch of required files so pay attention at this point of the build.

The kindle file is built using the Amazon conversion tool “KindleGen” from the EPUB file. It has a few of its own quirks so I made sure I had read the documentation.

At this point I had Baldur Bjarnason kindly look over my files to sanity check my decisions. He found and squashed a few bugs so was well worth an extra pair of eyes.

Testing and refining

At this point I would like to stress the huge value in documenting your changes in a simple changelog. It is too easy to forget what changes you made 3 versions earlier and things break because of this. I kept brief notes in one simple text file with all code changes included and rationale. This was also useful for when Matt and the team made change requests so we could all see who, when and what got changed.

Screengrab of notes

I used a bunch of test devices, which I recently wrote about to ensure the EPUB worked as widely as possible. During this phase changes included:

  • the cover image was simplified to work at small preview thumbnail sizes.
  • all chapters got pretty graphics from Vin Kumar, MRC designer.
  • graphic file sizes were optimised, particularly to help mobile readers.
  • section page breaks were forced for the Apple iPad to stop sections starting in odd places.
  • hyphenation was adjusted as the defaults were hideous.
  • validate, validate, validate,

With each change came improvement to the final reading experience.

The KindleGen app comes with a previewer so that you can simulate your test file in any of the kindle hardware versions without buying older devices. Sweet!

Screengrab of kindle gen previewer

Which shows you an accurate preview

Kindle preview

Decisions or compromise

In the end we managed to reach a great place for the final EPUB. Yet not all reading experiences will be equal. The iPad and the Kindle both have fantastic results (even the images were adjusted to work in low contrast for kindle).

As some reading apps like to overly control the reading experience, details like the colour section headings aren’t included in the nexus 7 apps Akido which likes to make all text 1 colour, over-riding my choices. However these things are beyond our control so all we can do is include them and let the reader app chose to include or ignore them. Maybe some day the app will update with these features included. A great example of this was towards the end of testing, the Adobe Digital Editions app upgraded to v2, making all the previously black headings appear as intended – the colour of the section it belonged to.

Screengrab of Adobe Digital Editions

Our ‘digital’ world is always evolving so even these compromises are really just decision points. If everything was static or we didn’t have much choice I would be even more upset, so I sleep happy.

Launch

When I pushed the final two files to Matt I got the jitters as I was very excited to let the world see our hard work.

I hope that this project will encourage others to consider the ebook format along side PDF outputs and that this post has shed some light on the process.

I would like to say that I got to work with some truly great people on this project and would like to thank Matt Jukes, Vin Kumar, Matt Durant, Baldur Bjarnason, Nigel Goldsmith, Roberta Perli and Stephen Gray for feedback and use of devices.

Matt has written about his experience of the project too, so do let us know what you think.

Below is what an EPUB looks like under the cover.

Breakdown of an EPUB

If you enjoyed this post then check out my 100 things about digital publishing series.

Follow me on twitter @zakmensah