Browsed by
Category: Git

Automation for Authors

Automation for Authors

Or: Writing with Python

I really love Scrivener. I may have mentioned this before. And I really love Leanpub. And for the most part, they work really well together. But there’s always room for improvement. And tonight I have added yet another layer of sophistication to my workflow. I’m pretty excited to share this with you.

The Workflow

Here’s a high level overview of how this all works:

  1. Write in Scrivener
  2. Export text files from Scrivener
  3. Check text files into a git repository shared with Leanpub
  4. Tell Leanpub to publish the book

Scrivener: Writing and Exporting

In Scrivener, you write all your text in little individual sections that you can move around at will. This is incredibly handy. If you decide the chapter on configurations should be in Part I: Beginning instead of Part II: Refinement you just drag it to the appropriate folder and you’re good to go. You can then export all these snippets as text files, and Scrivener will keep your file structure intact, like so:

A list of topics in Scrivener

Which turns into this on export:

The exported files from Scrivener

But where did those numbers in the file names come from? They came from here:

Scrivener Export Options

So now we’ve got a directory full of directories full of text! Let’s send it to Leanpub, right? Well, we need to do a little work first.

Prep for Print: The Leanpub Stuff

If you just give the Leanpub book generator a pile of text files it doesn’t know what order they should be in. So you provide it an extra file, book.txt, that includes all of your text files and the order in which they should be put into the book. The current book.txt file for Painless Git looks like this:

1 Front Matter/1 frontmatter.txt
1 Front Matter/2 Book Status.txt
1 Front Matter/3 Preface.txt
1 Front Matter/4 Introduction.txt
1 Front Matter/5 Structure.txt
2 Main Matter/1 mainmatter.txt
2 Main Matter/2 Beginning/01 Part 1.txt
2 Main Matter/2 Beginning/02 A Brief History of Git.txt
2 Main Matter/2 Beginning/03 Installing.txt
2 Main Matter/2 Beginning/04 First Steps.txt
2 Main Matter/2 Beginning/05 Commit.txt
2 Main Matter/2 Beginning/06 Interlude Tools.txt
2 Main Matter/2 Beginning/07 Branching.txt
2 Main Matter/2 Beginning/08 Configure.txt
2 Main Matter/2 Beginning/09 Sharing.txt
2 Main Matter/2 Beginning/10 Oops.txt
2 Main Matter/3 Refinement/01 Part 2.txt
2 Main Matter/3 Refinement/02 Good Git Habits.txt
2 Main Matter/3 Refinement/03 Small Commits.txt
2 Main Matter/3 Refinement/04 Hygiene.txt

When I wrote Painless Vim and Painless Tmux I kept this book.txt file up to date by hand. It was just another text file in Scrivener. So when I added a new section to the book I would go into book.txt and add each new section. The benefit to doing it that was was that I could work on a new section for a while before it got added to the book. The downside, of course, is human error. Sometimes I would add a new file in the wrong place, so chapter two came before chapter one. Or I would update the order in Scrivener and not update it in book.txt, which meant that when I published a new version it didn’t include my new chapter, and I had to fix book.txt, then publish again immediately. Whatever the case, sometimes book.txt got out of sync with reality.

It’s so easy to get files lined up in Scrivener and once you’ve done it in Scriv you shouldn’t have to do it again in book.txt. If you’re a developer the words “write a script” are already foremost in your mind, aren’t they? What if we could hook into git’s workflow to generate book.txt just before we commit?

Oh right, we can.

Python and Git Hooks

So now you need to create book.txt programmatically from your actual file structure. You’ve got numbered everything, so you can just recurse through the directory, find each leaf node and then prepend its path, and you’ve got a file path, right?

You just did too much work. Python is made to make things easy. os.walk will find every file in your directory and all subdirectories. I wrote a short python script that finds all the files, puts them in order, and then calls git add book.txt. Make that executable and save it as /painless_git/.git/hooks/pre-commit and you’re in business, my friend.

So now I no longer have to manage book.txt by hand. Using the features in Scrivener and in git, I’ve made it that much more painless to write Painless Git.

I’ve also come up with a good excuse for why I haven’t added any actual text to Painless Git today.

Introducing Painless Git

Introducing Painless Git

I have a three week break between summer and fall terms in my MBA program, so naturally I decided to write a new Painless book. I’m excited to announce Painless Git, the third [1] book in the Painless series.

The most common request I have received is for a book on something people actually use. And I’m well positioned to write this one. Unlike Painless Vim, which I wrote to help me learn vim, or Painless Tmux, which I wrote to help me learn Tmux, I’m actually already pretty good with git.

At my last job I helped move the entire 600+ developers in our organization from subversion to git, and spent months training individual teams on the new system. Then I switched jobs and moved our team here from subversion to git, and trained them on it. In my time as the “git guy” I’ve seen just about every git mess people can cause and found my way back out of most of them.

So Why Are you Writing This Book Now?

For years I’ve resisted writing Painless Git. I kept coming up with excuses like “I want to write novels!” Or “My wife just had another baby!” Or “I’m getting an MBA and it’s really hard!”

But then a wonderful thing happened: I have been put in charge of a brand new team of developers and guess what? It’s time to teach them git. Faced with this prospect I thought “I could dig up all my old presentations and cheat sheets and teach this team git over the course of a few weeks, or I could spend the next year or so finally writing Painless Git. As I’ve already demonstrated, it doesn’t take much to push me over the edge.

I’m very exited to share this one. Painless Git is a distillation of the tips, tricks, methods, and patterns that I’ve been teaching to teams from one part of Salt Lake City all the way to a suburb twenty miles outside of Salt Lake City. These are patterns and recommendations that have been battle hardened and stood up to old SVN pros and brand new interns alike.

  1. and final. I mean it this time!  ↩