Windows 11 Field Guide: Putting the Book on a Diet

Well, I finally did it. Inspired by a reader question in Ask Paul, I discovered that none of the documented ways to get the Windows 11 Field Guide into a Kindle e-reader or the Kindle mobile app worked properly. Or at all.

It was time to put the book, which weighed in at over 350 MB in PDF or EPUB form, on a diet.

Past time, really: Back in February, I wrote about this issue and spent a lot of time—wasted a lot of time, it now seems—trying to fix it using various methods. But size has long been an issue for the Windows 11 Field Guide, a book that I originally hoped would be about 500 pages long; the current version is 1,150 pages long in PDF form today. Heck, this was a problem for the Windows 10 Field Guide, a book that remains a relatively svelte 489 pages.

Without getting into a history of my downsizing efforts, I’ll just point out that this effort ended up being a lot like a person trying to lose weight: You try everything and nothing seems to work. And then, suddenly, something does work. And now you have the answer.

In this case, the answer feels painfully obvious: I simply reduced the quality of every single image in the book. In my defense, I feel like this is something that Leanpub should do automatically: I already use JPEG images instead of bigger but higher-quality PNG images, and years ago I saw some nice size reductions (in terms of on-disk MBs and book length) by making the images in the book visually smaller than before. That is, almost none of the images in the book are displayed at 100 percent of their largest possible width, visually (from margin to margin), and the full-screen images are all set to 70 percent. Smaller/zoomed-in images are … well, even smaller.

Anyway, Leanpub doesn’t do anything to optimize the collection of Markdown and image files I send them to collate into the complete book, and that does have me wondering if there is a better and/or efficient service I might use instead. But instead of lashing out at Leanpub, I will for now simply accept that this is a problem I could have fixed long ago. And have, at least, now finally fixed.

To reduce the quality of the images, I first experimented with, and then used, Affinity Photo 2. Like Photoshop, it has a way to automate, or batch, image operations across a range of files. Photoshop’s version is, in some ways, more sophisticated—you can, for example, point it at a folder in the file system and optionally include all subfolders, too—but I am more comfortable with Affinity Photo and I needed to babysit the process: This is too important to get wrong.

The experiments mostly involved lowering the quality level on a folder full of images, a sort of “how long can you go?” thing, without distorting the visual quality. This was a bit time-consuming: I would select a folder of images, set the image quality down by 10 percent—so to 90 percent quality on the first pass—use those images in a test chapter, sync the changes to Leanpub over Git, publish a subset preview of the book that included only those images, wait for the churning and bubbling on their end to conclude, download the PDF, and then scan the results, zooming in and out to see whether there were artifacts or other visual issues.

The first two passes, at 90 percent and 80 percent image quality, were indiscernible in that I couldn’t see any differences in the images at all. What I did notice was that the file size of that single chapter in PDF form was going down each time. And that was very interesting. So I threw caution to the wind and skipped head, setting the quality level of the images to 50 percent. Sync, preview subset publish, churn … voilà! The resulting chapter was dramatically small from a file size perspective. And the images, though smaller on-disk, much smaller, still looked very nice.

So I backed up the entire Windows 11 Field Guide images folder. This folder, which has 22H2, 23H2, and 24H2 sub-folders, each with its own chapter-based sub-folders, takes up 724 MB on disk and contains over 2,600 JPEG image files.

Then, I got to work with Affinity Photo, using its Batch Job feature to load images into a job that would convert them, in place (in the same folder, overwriting the original) to versions at 50 percent image quality. I started with the 24H2 subfolder because it’s relatively small—it has just two subfolders for now, for Introduction and Phone Link—and subset preview published once more just to be sure. It looked great and was significantly smaller than the original. And then I worked my way through 23H2 and 22H2, using bigger and bigger sets of files across more and more subfolders as I went, and my confidence grew.

There are 41 subfolders of images in 23H2 and 92 in 22H2. About 1,000 images in the former and over 2,500 in the latter. Some of the older images have been replaced by newer versions elsewhere, but I did them all just in case. Two or three times during this process, I used Git to sync the downsized files with the master copy of the book in GitHub. And I did full-book previews to see how the changes impacted the file sizes.

Leanpub only lets me preview the book in PDF form. But after converting all the images from the 23H2 and 24H2 chapters, I could see that the book had shrunk from about 377 MB on disk to 216 MB. That was very exciting, and though I kept worrying I’d find images that looked off, that didn’t happen. And so I pressed on.

In the end, the image file reductions reduced the size of the book’s images folder from 724 MB to 206 MB, a reduction of over 71 percent. This is good for me because I sync this folder (and the rest of the book) to all my computers, and that’s less data in the cloud and less data to sync. It’s also good for Leanpub, which struggles to produce our book, which I believe is one of the biggest, if not the biggest (by length and disk size), in its catalog.

But it would also be good for readers, and that’s what matters the most. And so I kicked off the first full-book preview with all the images downsized, curious where it would land. How small would the 377 MB PDF file be after these reductions?

107 MB.

That’s—wait for it—the same 71 percent size reduction that I achieved with the raw files, which shows you that Leanpub does f#$k-all in this area. But whatever: A 71 percent reduction is massive from a user experience perspective. This may be the single most important change I’ve made in years.

The problem with a 377 MB PDF file—or, a 344 MB EPUB file—is that it’s unwieldy. Files this big can stress the sad little processors in a Kindle e-reader, but they’re slow to load in a desktop web browser (in PDF form) or in the Kindle (or iBooks, or whatever) app in a phone or tablet too. Worse, these files are too big to use Send to Kindle, which gets external documents into your Kindle library so they can be accessed on any device, or the Share feature in your mobile device: Amazon limits Send to Kindle to files under 200 MB. The Windows 11 Field Guide was simply too big before.

But now it’s fine.

To find out how fine, I needed to publish it publicly. This is when Leanpub creates an EPUB version of the book, the version one uses for Kindle or other e-readers. And so I couldn’t see how much smaller it would be in the crucial Kindle ecosystem until it went live.

So I published it. Waited while that happened. And then eagerly downloaded the resulting EPUB version of the book to see where we landed.

The 344 MB EPUB file is now just 86.3 MB.

86.3 MB! That’s a reduction of almost exactly 75 percent. This book is now just one-quarter of its original size. Or weight. Or whatever.

86.3 MB is much easier to deal with, whether you’re using the Kindle app on your phone or anything else. It’s also well under the 200 MB Send to Kindle limit, which means you can now re-download the book from your Leanpub library on your PC in EPUB form and use Send to Kindle to put it in your Kindle library. It also means you could just download the book directly to your phone or tablet (again, in EPUB) and then use the Files app (or similar) to “Open in” or “Share” to the Kindle app or your reader of choice. Where it would fail before, hard, now it should just work.

This is very exciting. Very exciting.

So exciting that I will now do the same for the Windows 10 Field Guide and Windows Everywhere. And going forward, I’ll ensure that all new image files are uploaded to Leanpub at 50 percent image quality.

There may be other changes. I noted that I had reduced the visual size of full-screen images in the book from 100 percent width (margin to margin) to 70 percent, and I’m wondering about maybe making those bigger (wider). That will make the book bigger on-disk and longer from a page count perspective, so maybe that’s not necessary of desirable. I will experiment.

But for now, this is fantastic. Thanks to helix2301 for finally pushing me to suck it up and do the right thing. I am so sorry this took so long, but I’m also ecstatic that it got there. Finally.