This startup from Vancouver offers an interesting front to back publishing platform. Named Leanpub, it has something to offer writers, publishers, self-publishers and readers. I don’t know much about it, but we may have to try it. It seems especially useful for bloggers turning their posts in to books and for technical writers on fast-changing subjects. Let me know if you have tried it.

Somehow I missed the announcement of iBooks Author last week. TechCrunch published a list of “subtle details” from Apple’s announcing event. Engadget published a helpful hands-on illustrating the interface with lots of photos and a video. Huffington Post has excerpts from the end-user license agreement and legal analysis. ArsTechnica has a good collection of reactions to the restrictions. Mashable has a brief comparison chart identifying the competition.

In short, the tool is free, and with it, you can create media-heavy ebooks. If you want to sell the books you create with the tool, you can only sell the books through Apple. And, oh yeah, these aren’t epubs.

The ArsTechnica article points out that although the format is not strictly epub2-compliant, “there shouldn’t be any technical limitation to exporting a …compliant ePub document if none of the interactive features are used. Unfortunately, iBooks Author only exports PDFs and text.”

GigaOM has reported on a webcomic author who has already used the tool to create and distribute (for free!) a collection of his work. Download it from Richard Stevens’ website. You’ll see the file type is .ibooks. So… basically, even if you’re creating a free book, you can distribute the book however you like, but it’s still limited to iBooks rendering, is that right? Certainly, this is a useful, boundary-breaking tool, and folks like John Gruber have done a great job explaining why it’s in Apple’s interest to pursue a proprietary format. I have to say, though, this does not sound like the answer.

We haven’t test-driven iBooks Author yet. Please let us know if you have!

I was proudly updating a techie friend recently about the progress I’ve been making in my latest ebook conversion for Kevin. I’d mentioned it a few weeks prior. His response was not “That’s awesome!” but rather “Why does it take so long?” Not exactly what I was hoping for, but definitely a fair question.

It took me about a week to remind myself how InDesign works (it had been at least a year since our last InD project), to learn how InDesign book documents (.indb) work, and to get up to speed with InDesign CS5.5’s new epub export features. I highly recommend the Lynda tutorials for that set of information. I also leaned on Liz Castro’s EPUB Straight to the Point and Rufus Deuchler’s How to Create an eBook with Adobe InDesign CS5 (still helpful post-CS5.5). And since the cardinal rule of working in InDesign is to apply styles consistently, I found India Amos’ post on how she names and manages styles to be very helpful.

For my setup, I created a Document preset with a single page (facing pages turned off) with a master text frame. Although it doesn’t make any difference Rufus Deuchler recommends a page size (live area) of 600×800 pixels; Terry White uses 51x66p). (MobileRead has a great e-reader matrix with display dimensions.)

I created a separate InDesign document for each chapter in the book. Chapter by chapter, I imported my text from Word (previously described here). After importing the first chapter, I started to create my character and paragraph styles. When creating subsequent chapter, I imported the styles from the previously created chapter. All chapters/documents were pulled into a single InD book file. I struggled a bit to remain mindful of my style source and to keep my styles synced.

Once I finally got my InD book and styles all set up and my, I started to experiment with generating EPUBs. It took me another week to troubleshoot the errors I found during testing and validation.

The first issue I came up against was related to images. I initially inserted very high-res images that resulted in a ginormous epub file. I replaced and relinked them in InDesign, but upon re-export, InDesign seemed to disregard my export settings. Instead of exporting at 150dpi, they came out at 72dpi, giving me the opposite problem of pixelated images. I posed the problem to the helpful folks who participate in the #eprdctn conversation, and @markerrett suggested replacing the images after export. This means unpacking the epub, navigating to the images folder, and replacing all of the images with images of the same name, created to the desired spec. Worked like a charm.

I wrestled with embedded fonts. The first problem I had was with the descenders on the last line of text getting cut off. After taking Liz Castro’s advice and adding a font declaration to the META-INF folder, the descender problem was fixed. However, I was still getting validation errors relating to fonts (maybe because some were true-type instead of open-type?). Because the typography in this book was not complicated (one font is fine), I decided to re-export the epub WITHOUT any fonts embedded. Problem solved averted.

One annoying but not crippling issue with EPUB created by InD is an excess of style sheet entries. I found some discussion of the issue at the Adobe forums, but I chose not to pursue “fixing” my CSS since it didn’t produce any errors and didn’t add significant size to my file.

Finally, iTunes Producer didn’t like the spaces in the titles of my html files (which were a product of a naming scheme encouraged by the Lynda tutorial on working with book documents in InD). So, I renamed the files and then updated the toc.ncx and content.opf files.

Even after removing the spaces from the html file names, Stanza would not display the document properly. There were several blank pages between each page of content, and, even more perplexing, the first two html files were omitted completely from display. I’m still not sure what was causing the trouble in Stanza – suggestions welcome!

I did desk-top testing in Adobe Digital Editions, Ibis Reader, and in the EPUBReader FireFox extension. I also tested on an iPhone and iPad, in iBooks, Kobo, and Stanza.

Here’s a skeletal outline of my workflow for this book:

PDF –> EPUB:
PDF –> Word (preserving character/paragraph styling as much as possible)
for each chapter
Word –> InDb
separate doc for each chapter, collected in a single InDesign book
style set up and application in InD
map styles to tags
images placed in-line with the text (no captions)
book meta-data added to style source document
export epub with CSS, no fonts embedded
test on devices and with validator
troubleshoot
troubleshoot
epub complete, intense feelings of accomplishment

EPUB –> Kindle:
combine all html files (from epub export) into a single html file
un-smarten quotes and add html entities for special characters
add kindle page-break code before new chapters
add table of contents and related internal links
remove extraneous epub files (leaving only html, css, and images)
upload to kdp for final azw wrapping

Resources

Lynda.com — InDesign CS5.5 New Features

Colin Flemming — CS 5.5 ePub Videos

Liz Castro — EPUB Straight to the Point

Rufus Deuchler — How to Create an eBook with Adobe InDesign CS5

India Amos — How stylish are you?
Description of a book designer’s succinct and intuitive style naming scheme

Adobe — Export content for EPUB (CS5.5)
Brief but helpful description of all the epub export options in CS5.5

This week’s #eprdctn hour, moderated by @BookDesignGirl, just concluded. The topic was eReader devices and apps and their unique code rendering issues. These are some gems worth remembering:

Misc

@amarie – uses TweetChat to focus on conversations with a certain hashtag

Rendering Issues

@MatthewDiener – “for Kindle/Mobi, @jtallent’s Kindle Formatting points out most of the rendering issues”

@BookDesignGirl – “Edges of the images in my .jpg’s were blurry and the [epub] spec said to change to .gif it that was happening. It did the trick.”

@ BookDesignGirl – “Tally: @camilcloutier uses center tags @NickRuffilo uses CSS not center tags & I’ve used empty span tags to make text center on iBooks!”

@pablod – Previews HTML components of an ePub file in Mobile Safari in the iOS emulator. Note: only good for previewing single pages (not full epub docs) – not a substitute for previewing on an actual iOS device, but in a pinch, it can help.

On embedding fonts

@ BookDesignGirl – “I’ve started embedding fonts. Used http://tinyurl.com/6cw5hoz for current project & it worked fine (knock on wood).”

@pablod – “I had to pull out a div with embedded fonts which was causing a rendering issue at page break for a UL within the div in iBooks.”

@tinahender – “As a a rule, I do not embed fonts b/c licensing issues are not clear. But if a client asked, I would look into it further.”

@pablod – “Alternately, Google.com/webfonts gets around the pesky font licensing issue. Also, theleagueofmoveabletype.com”

@biladew – “embed fonts only when necessary. Otherwise generic font faces should give you what you need (serif, sans-serif, etc)”

@BookDesignGirl – “We don’t embed b/c of licensing/potential validation issues but did for this project. Times font didn’t work for tweens”

@aarontroia – “go to @ebookarchitects bit.ly/lDSNWv and download Font Test, its a really good font embedding example”

On InDesign-based workflows and conversion tools

@MatthewDiener – “ID=>Aptara=>ePUB=>Kindle so far”

@BookDesignGirl – “We do InDesign to ePub (either conversion house or in-house) -> edit ePub -> convert to .mobi -> note conversion issues -> go back to epub and edit in Dreamweaver -> .mobi (repeat)”

@TonyRobertsMN – “Yes, export to ePUB then modify. Nothing Indesign produces on it’s own satisfies me. I edit the HTML manually, then make the .mobi”

@JenniferOmner – “epub -> .mobi -> note conversion issues -> back to epub file for Mobi & edit in Oxyen -> .mobi”

@markerrett – “I did InD to ePub then use Calibre to .mobi it”

@BookDesignGirl – “I convert my epubs with Kindle Previewer.”

On KindleGen

@aarontroia and @jtallent – both recommend KindleGen over Calibre or Sigil

@jtallent – “easy to do in command line. “path_to_kindlegen.exe path_to_opf””

@MatthewDiener – “Once you figure out how to use it from command line/Terminal it is an easy tool, and edit/rerender simple”

@MatthewDiener – “If you need help with KindleGen, instructions on using it are included in this document. http://bit.ly/opHs8W”

@BookDesignGirl – “I assumed that KindlePreviewer uses KindleGen. And then you have the preview right there.”

Zip tips

@tinahender, @aarontroia, and @PenguinPowered – all recommend @danrodneytweets’s epub zip and unzip scripts (also here)

@markerrett – “Plus BBEdit now lets you open HTML inside the ePub with the need to Unzip then zip again.”

ERROR: wtw-intl.epub/OEBPS/toc.ncx(110): ‘heading_id_17′: fragment identifier is not defined in ‘OEBPS/Text/wtw-intl.html’

I got this error (actually, like 200 of them) from the EPUB validator that ThreePress very generously hosts, and also from iTunes Producer when trying to upload a file for the iBookStore.

I tried FlightCrew too, another tool I’m super happy to have, but got a similarly unhelpful error message:
file: wtw-intl.epub/OEBPS/toc.ncx
line: 110
message: This element’s “src” attribute value is “Text/wtw-intl.html#heading_id_17″, but an element with an ID the fragment is referring to does not exist in that file.

OK, that’s English, but after double-checking the files and values it mentioned, I still couldn’t identify the problem.

After banging my head against the wall for 30 minutes, I decided to run Sigil‘s check and discovered a few seemingly unrelated errors. So I closed a couple paragraph tags that were left open (ah, the perils of hand-coding), and corrected a couple of typos in a couple of internal anchored references. I then re-ran the file through EPUB Check and, presto change-o, errors gone. Nothing to do with the table of contents file (toc.ncx), and nothing to do with that particular id value. Yeah, that totally makes sense

I’ve previously mentioned the epubcheck wiki and the mobileread epub forum as helpful sources of info for error troubleshooting, but I wonder if there are others.

Can anyone recommend any other great diagnostic tools or resources for EPUB errors?

We’re working on another EPUB conversion now. For this one, we’re moving from PDF to EPUB via InDesign. Currently, the whole book exists as a single PDF. The text in the PDF includes a small but not insignificant amount of character styling (bold, italic, etc). The best workflow I’ve found so far seems to be:

1. Break the single PDF into multiple PDFs by chapter. I’ll be using a Book organization in InDesign, which means creating a separate InDesign document for each chapter. For that reason, I’d like to import the text chapter-by-chapter.

2. Copy-Paste the text for each chapter into a separate Word document. Remove extraneous carriage returns. There are two main methods for getting text into InDesign. You can copy-paste directly into a text frame, but for big sections of text, this is not recommended. The better method (I think) is to use InDesign’s Place command (ctrl+D). To do this, text needs to be in a place-able format like TXT, DOC, or RTF. Because I’m working with formatted text, DOC or RTF are my best options. After a few rounds of experimentation, I find Word to offer the best tools for maintaining character formatting and eliminating extraneous line breaks. Depending on what happens upon paste (or paste special – I’ve experimented with all of the options and it seems like I get a different result each time), I may use AutoFormat to fix extra carriage returns while maintaining paragraph breaks.

3. ‘Place’ text in InDesign. When you place text, you can “Show options”. The only adjustment to the defaults I’ve made is to omit page breaks. I’ve left several defaults that don’t apply to my document and it doesn’t seem to break anything. This is what my Place options looks like:

So far, these three steps seem to work pretty well. It seems like there’s a fair amount of variation in the way my pasted text lands in Word (especially in terms of the presence or absence of extra carriage returns). Also, although the first chapter I worked on seemed to transmit fairly clean text, in the second and third chapters I’m finding many instances of words running together (“inhalesthem” instead of “inhales them”). I suspect this is related to the carriage return issue. Although I’m not always seeing extra carriage returns, it seems like their absence is sometimes accompanied by a missing space.

If anyone has a better workflow for getting styled text from PDF to InDesign, please share in the comments!

Ebook conversion from PDF is possible but problematic. Often, PDFs carry print-specific elements that aren’t useful in the context of reflowable text, like page numbers and repeating chapter titles. In the case of our most recent conversion, our PDF was fraught with various printers crop-marks, the file name, and creation date on every page. It’s hard for conversion programs (like Calibre and the converter built into Amazon’s KDP web-based uploader) to recognize and strip away unwanted elements like that. Layout-specific elements within PDFs, like sidebars, call-outs, and anything tabular, can also befuddle converters.

Layout programs like InDesign offer increasingly sophisticated ebook output options that allow the user to specify reflow-friendly display instructions for elements like these (eg: don’t include page numbers in the epub output, place sidebars at the end of the section and style them differently, etc).

I recently converted a book for which I only had the PDF –no InDesign files, nor Word files to port to InDesign.

Even though I knew the Calibre developer discourages conversion from PDF, I took a swing. The program produced a fairly literal conversion (including crop-marks) but with lots of strange line-breaks and no images (where did they go? I don’t know). I also attempted uploading a copy of the PDF to KDP for conversion, an although it impressively removed the crop marks and other repeating page elements, images were again partially left out (partially because some textual elements from images were preserved, like chart text, which was probably an artifact of the Adobe OCR process).

Copy/special-pasting directly from the PDF to Word did a decent job of preserving italics and bold text, but also preserved unwanted repeating page elements. It also included artificial line-breaks (which would interrupt reflowability).

Finally, I tried exporting HTML from the PDF using Adobe Acrobat. This produced a set of files (html, css, and images) fairly similar in display quality to the Calibre conversion, the KDP conversion, and the Word copy/paste. All of these alternate formats, based on the PDF, would need quite a bit of fine-tuning. I’d need to create style classes, inspect parts of the text with unique formatting, re-insert images optimized for display in the ebook environment.

I decided editing the HTML directly would give me the most precise control over everything. However, the HTML that Adobe exported was U G L Y. Tons of extraneous style application and markup of unwanted elements (line-breaks, crop-marks, etc). Not to mention it was really hard to read — just one continuous flow of code.

These are the steps I performed to clean-up and process the HTML:

1) HTML Tidy, to nest elements and ID errors (eg: unclosed tags)

2) Remove unwanted image elements (crop-marks, mangled charts and illos)

3) Remove unwanted textual elements (page numbers)

4) Convert in-line styling to CSS styles

5) Re-insert images (744px wide, jogs, 72dpi)

6) Insert links for TOC and endnotes

7) Cleanup CSS elements which Kindle strictly regulates (justification, paragraph indents, list-styling, blockquotes)

A similar process could be done from Word directly. Word has a pretty neat auto-format feature that performs a similar task to HTML Tidy. It’ll remove hard line-breaks and assign headings and styles pretty consistently so that you can more easily make changes. Instead of creating CSS styles, you’d be creating custom styles in Word. Instead of doing GREP searches in Text Wrangler, you’d be doing wildcard searches in Word. Working in Word is probably easier to read for more people. (I previewed as I went by viewing the file in Firefox, which allowed me to diagnose problem areas using the Firebug extension.) You could then upload your Word file to KDP for conversion or you could port it to InDesign for epub export. I like to think that I avoided a lot of bloat-related trouble-shooting by working directly on the HTML, but honestly, Kindle is a lot more forgiving than iBooks, so it’s a little hard to tell. Natasha Fondren presents a good argument for hand-coding ebooks. Ditto.

I’ll need to work on the iBooks conversion next to test that theory.

Formatting guides/helps for preparing Kindle ebooks:

Kindle Formatting, by Joshua Tallent
website and guide are both super helpful

Natasha Fondren’s blog, Avdventures in Writing on the Road
see especially the Kindle Formatting category

As Kevin already mentioned, his book Bicycle Haiku recently became available for the Kindle. And now, I’m happy to report that it’s also available in the iBookstore. In both cases, we prepared the e-books in-house and uploaded them for distribution through self-publishing programs (Amazon’s Kindle Direct Publishing, and Apple’s iTunes Connect, respectively).

Although there were some differences in the file preparation processes, the most significant distinction between the services so far is the application process and the length of the wait from when you submit your file to when it actually shows up for sale in the store. With Kindle Direct Publishing (KDP), account creation is instantaneous. You need to provide tax and payment information, but that’s about it. With iTunes Connect, in addition to tax and payment info, you also need to apply with at least one ISBN for a title you intend to distribute. They also specify that you must be able to deliver valid EPUBs. Optionally, you may submit your prepared EPUB with your application, and I did so. It was about two weeks before we were notified of our acceptance into the program.

In terms of delivery time, the file we submitted to KDP was available within a day. With iBooks, it was fully 15 business days (5 business days longer than Apple says it might take) before our book appeared in the iBookstore. The upload mechanics are also slightly different from one service to the next (KPD’s upload is done in-browser, iTunes Connect makes use of a separate uploader program which you install locally), but the distinction is trivial next to the huge disparity in wait time.

Besides the inconvenience of the wait, this delay on Apple’s side means that while we’ve already accumulated several weeks of sales data from Amazon, we’re still in the early days of our iBooks sales, making a side-by-side performance comparison difficult. If you want to release titles on both platforms in parallel, it would seem you should initiate with Apple and confirm the release there and then release to Amazon — this would at least minimize the gap in release time.

Avi Solomon offers his ultra short course on how to publish and sell your first ebook, in the form of a set of 16 slides on Slideshare.

It’s a good orientation. Details are left to you.

It assumes you have a “book” written and edited. Puts it into the standard epub format. Final result will be an ebook on Amazon’s kindle or the Nook (but not in the App Store).

That’s the ratio one self-publisher found was needed to extract paid sales from free books.

Technical writer Keir Thomas reports on his experiments in screen publishing. He wrote a book (a resource guide for the programming language Ubuntu) and then made a PDF. He posted the PDF online available for free. The hosting site and the PDF itself contained advertisments for paper versions of the book (printed on demand by Amazon’s CreateSpace) priced at $12.99. Hundreds of thousands of people downloaded the free book, which Thomas treated as “free” marketing. Some downloaders bought the book. Here are the results of his sales:

I’ve lost count how many people have downloaded the eBook but the last time I audited the figures, which was around six months after the book’s release, it’d seen around 500,000 downloads. I suspect that number has doubled since then.

Since going on sale at the start of 2009, the book has made me $9,000. Bearing in mind the book took three months to produce, that’s a salary of $3,000 per month, although costs such as hosting have to be deducted, and I also spent quite a few days marketing the book once published.

I estimate I have to give away 446 copies of the eBook for every sale of the print edition. Without Amazon S3, I simply couldn’t have done it.

The book’s been a success in terms of providing a free educational resource for the Ubuntu community, and I’m very proud of this achievement. But as a commercial endeavour, I wouldn’t recommend anybody take a self-publishing route.

Thomas is now trying a new experiment, selling Kindle versions of two new books at 99 cents.