Browsed by
Tag: Painless Git

Moving to Markua

Moving to Markua

I decided to move Painless Git from “Leanpub-Flavored Markdown” to Leanpub’s newer, fancier Markdown variant Markua. This wasn’t a decision I made lightly. I’ve built up a fairly slick workflow around LFM and publishing from Scrivener to Leanpub and out to the world. But in the end the benefits far outweighed the costs. Especially since the effort-cost was fairly low.

What were the Benefits?

Glad you asked, imaginary question asker! Let’s go down the list:

Future Proofing

Leanpub is putting more effort into Markua than Leanpub-flavored Markdown, which makes sense. Markua is very focused on making it easy to write books and full-on courses using nothing but a markup language. They have put a lot of thought into streamlining all the “fiddly bits” of writing, things like including images, or text figures, or code snippets, or what have you. All of these includes have features in common, and Markua reduces the differences to where including a JavaScript file is really no different from including a picture.

Easier Publishing Through Better Markup

Using Markua means I no longer have to maintain three different meta-information files:

  • A book.txt to tell Leanpub which files were in the main, published book.
  • A sample.txt to create an abbreviated sample version, to give readers a taste but not the whole thing.
  • Optionally, a preview.txt file, which would just generate a tiny little preview for you to see if your formatting was working correctly.

My fancy Python/Git hybrid solution that I wrote about last year handled the book.txt file, but I had to maintain the other two by hand, which means that I didn’t. I never really did small previews, and I never had a sample version.

That whole system is no longer necessary! I now have one book.txt file and it has one line in it:

manuscript.txt

Since I can now use markings in the text itself to dictate which sections are included in the published book, or in the sample book, or neither, or both, I simply use Scrivener’s impressively powerful “Compile” feature to output a single manuscript.txt file.

A directory containing my manuscript and that one-line book.txt now lives in a shared folder. Leanpub can grab that file when I hit the “publish” button and it all just…works. All that stuff I used to have to do with numbering folders and files and auto-generating a book.txt file? Gone. All that file juggling to keep my images out of the manuscript directory while python was doing stuff, but putting them back when python was done? A thing of the past. Things just got a whole lot easier.

How Hard Was the Hard Part?

But this was an effort, right? There was some work involved in the change, right?

Well, yeah. But not really a ton. This is my list of tasks that I put together while working on the conversion:

Change my italics

In most Markdown variants, _this_ and *this* both make italic text. I’ve used both more or less interchangeably, but I tend to find underscores easier to type. In Markua, _this_ will be the markup for underlined text instead of italics. So I had to change all _underscore italics_ to *star italics*. I do like me some underlines, but over-using them is a bad plan. As of this writing Markua still seems to be treating _this_ as italics, but I don’t want to suddenly have that change on me.

Change “Part” Markers

Easy:

  • Old: -# Part name.
  • New: # Part name #.

The manual says this will change again soon, but I can live with that. There are only four “parts” in my book anyway.

Update Chapter Headings

I include ID tags for chapters so I can cross-link between parts of my book. I had to convert those id tags from # Chapter Name {#chapter-name} to:

{id: chapter-name, book: true, sample: false}
#Chapter Name

Which looks more complicated, but is far more useful. (And I’ve automated away the complexity, but we’ll get to that.) This new version is a large part of why I decided to switch to Markua, as I mentioned above. Handling it this way makes the management of the book so much easier, and allows me to keep unwritten sections out of the published book without doing any fancy trickery with exports and whatnot.

Fix Code Blocks

The old style:

{lang="java"}
~~~~~~~~
class Stupid{
public static void main (String[] args){

}
}
~~~~~~~~

Was never super popular, even in markdown variants that supported it. The new style is basically “fenced code blocks” as used by every Markdown variant:


```java class Stupid{ public static void main (String[] args){ } }

“`

Markua also has other plans to make the whole “text figure” system more consistent with images and imported code and all manner of cool stuff.

Miscellany

Actually, that’s all I had to change. All my other markup, like asides and tips totally still work. Best of all, footnotes still work as they always have. ✅

Markua adds a lot of enhancements to asides, but they’re non-breaking, so I can add them in as I get around to revising.
In addition to normal footnotes, they’ve added an endnote notation that, should I ever want to create endnotes, I can now do that. That’s right, I can two different kinds of witty little bits o’ text.

Updating TextExpander Snippets

I use TextExpander to automate some of the repetitive parts of books, like markdown links and chapter headings. The main change was to my “Create a new chapter” snippet, as previously mentioned.

In the update process I discovered that TextExpander has cool little macros that let you choose the state of the book and sample thing without all that tedious typing, saving me of four or possibly even five keystrokes. Fancy.

Display the TextExpander UI for creating a Markua Title.

Conclusion: If You’re Using Leanpub, You Should Be Using Markua

Overall the process took about four hours, and now I’m back to writing. Aside from the italics thing my writing has stayed the same. I use the same TextExpander snippets that I used to use, they just create Markua style output now.

And my publishing workflow is drastically easier. Here are the steps it used to take:

  1. Export the files, using Scrivener’s awesome defaults.
  2. Check the files into git
  3. Commit
  4. Push
  5. Ensure that book.txt was updated correctly by my Python script. (It always was, but I’m paranoid, even of my own code)
  6. Hit the publish button on Leanpub.

Versus the new workflow:

  1. Compile the manuscript using Scrivener’s awesome compile feature. Scrivener remembers all my settings so this is literally a single button press.
  2. Hit the publish button on Leanpub.

And now I can start looking forward to all the other things that Markua can do , once the text is complete…

Painless Git, Part One, Almost Done!

Painless Git, Part One, Almost Done!

Part One is really coming along!

Sure, Part One is nine chapters long and Part Two is currently slated to be around seventeen chapters, but hey, progress is progress. 

I’m having a lot of fun writing Painless Git, and I think that even if Part Two is longer, it’ll be even more fun, both to write and to read. 

Automation for Authors

Automation for Authors

Or: Writing with Python

I really love Scrivener. I may have mentioned this before. And I really love Leanpub. And for the most part, they work really well together. But there’s always room for improvement. And tonight I have added yet another layer of sophistication to my workflow. I’m pretty excited to share this with you.

The Workflow

Here’s a high level overview of how this all works:

  1. Write in Scrivener
  2. Export text files from Scrivener
  3. Check text files into a git repository shared with Leanpub
  4. Tell Leanpub to publish the book

Scrivener: Writing and Exporting

In Scrivener, you write all your text in little individual sections that you can move around at will. This is incredibly handy. If you decide the chapter on configurations should be in Part I: Beginning instead of Part II: Refinement you just drag it to the appropriate folder and you’re good to go. You can then export all these snippets as text files, and Scrivener will keep your file structure intact, like so:

A list of topics in Scrivener

Which turns into this on export:

The exported files from Scrivener

But where did those numbers in the file names come from? They came from here:

Scrivener Export Options

So now we’ve got a directory full of directories full of text! Let’s send it to Leanpub, right? Well, we need to do a little work first.

Prep for Print: The Leanpub Stuff

If you just give the Leanpub book generator a pile of text files it doesn’t know what order they should be in. So you provide it an extra file, book.txt, that includes all of your text files and the order in which they should be put into the book. The current book.txt file for Painless Git looks like this:

1 Front Matter/1 frontmatter.txt
1 Front Matter/2 Book Status.txt
1 Front Matter/3 Preface.txt
1 Front Matter/4 Introduction.txt
1 Front Matter/5 Structure.txt
2 Main Matter/1 mainmatter.txt
2 Main Matter/2 Beginning/01 Part 1.txt
2 Main Matter/2 Beginning/02 A Brief History of Git.txt
2 Main Matter/2 Beginning/03 Installing.txt
2 Main Matter/2 Beginning/04 First Steps.txt
2 Main Matter/2 Beginning/05 Commit.txt
2 Main Matter/2 Beginning/06 Interlude Tools.txt
2 Main Matter/2 Beginning/07 Branching.txt
2 Main Matter/2 Beginning/08 Configure.txt
2 Main Matter/2 Beginning/09 Sharing.txt
2 Main Matter/2 Beginning/10 Oops.txt
2 Main Matter/3 Refinement/01 Part 2.txt
2 Main Matter/3 Refinement/02 Good Git Habits.txt
2 Main Matter/3 Refinement/03 Small Commits.txt
2 Main Matter/3 Refinement/04 Hygiene.txt
...

When I wrote Painless Vim and Painless Tmux I kept this book.txt file up to date by hand. It was just another text file in Scrivener. So when I added a new section to the book I would go into book.txt and add each new section. The benefit to doing it that was was that I could work on a new section for a while before it got added to the book. The downside, of course, is human error. Sometimes I would add a new file in the wrong place, so chapter two came before chapter one. Or I would update the order in Scrivener and not update it in book.txt, which meant that when I published a new version it didn’t include my new chapter, and I had to fix book.txt, then publish again immediately. Whatever the case, sometimes book.txt got out of sync with reality.

It’s so easy to get files lined up in Scrivener and once you’ve done it in Scriv you shouldn’t have to do it again in book.txt. If you’re a developer the words “write a script” are already foremost in your mind, aren’t they? What if we could hook into git’s workflow to generate book.txt just before we commit?

Oh right, we can.

Python and Git Hooks

So now you need to create book.txt programmatically from your actual file structure. You’ve got numbered everything, so you can just recurse through the directory, find each leaf node and then prepend its path, and you’ve got a file path, right?

You just did too much work. Python is made to make things easy. os.walk will find every file in your directory and all subdirectories. I wrote a short python script that finds all the files, puts them in order, and then calls git add book.txt. Make that executable and save it as /painless_git/.git/hooks/pre-commit and you’re in business, my friend.

So now I no longer have to manage book.txt by hand. Using the features in Scrivener and in git, I’ve made it that much more painless to write Painless Git.

I’ve also come up with a good excuse for why I haven’t added any actual text to Painless Git today.

Introducing Painless Git

Introducing Painless Git

I have a three week break between summer and fall terms in my MBA program, so naturally I decided to write a new Painless book. I’m excited to announce Painless Git, the third [1] book in the Painless series.

The most common request I have received is for a book on something people actually use. And I’m well positioned to write this one. Unlike Painless Vim, which I wrote to help me learn vim, or Painless Tmux, which I wrote to help me learn Tmux, I’m actually already pretty good with git.

At my last job I helped move the entire 600+ developers in our organization from subversion to git, and spent months training individual teams on the new system. Then I switched jobs and moved our team here from subversion to git, and trained them on it. In my time as the “git guy” I’ve seen just about every git mess people can cause and found my way back out of most of them.

So Why Are you Writing This Book Now?

For years I’ve resisted writing Painless Git. I kept coming up with excuses like “I want to write novels!” Or “My wife just had another baby!” Or “I’m getting an MBA and it’s really hard!”

But then a wonderful thing happened: I have been put in charge of a brand new team of developers and guess what? It’s time to teach them git. Faced with this prospect I thought “I could dig up all my old presentations and cheat sheets and teach this team git over the course of a few weeks, or I could spend the next year or so finally writing Painless Git. As I’ve already demonstrated, it doesn’t take much to push me over the edge.

I’m very exited to share this one. Painless Git is a distillation of the tips, tricks, methods, and patterns that I’ve been teaching to teams from one part of Salt Lake City all the way to a suburb twenty miles outside of Salt Lake City. These are patterns and recommendations that have been battle hardened and stood up to old SVN pros and brand new interns alike.


  1. and final. I mean it this time!  ↩