Latest News >> 2008-09-29

I read Obie’s most recent post about his intense passion for Loverboy’s quintessential anthem, “Lovin’ Every Minute Of It”. I find the early 1980’s music is inspiring and uplifting and definitely suited to such important things as corporate culture, recruiting, and motivating the troops to do better. Yes, nothing gets a worker working better than a little Loverboy right in their ear.

2008-09-25

Don’t forget folks, the FU NYC show will be in a few hours (7pm-9pm). I’ll be icecasting this one at http://zedshaw.com:8000/fu_nyc and as usual you can use VLC, mplayer or many other players to play the stream.

2008-09-17

I got into music school last week and I’m going to study guitar exclusively for the next year. This is something I’ve always wanted to do, but just never had the chance. Either I wasn’t good enough (being self-taught for so many years) or I just didn’t have the money. After being laid off and getting a small package I decided to practice my ass off on the guitar, do a few live shows to get ready, and then audition for a school in the city.

2008-09-04

The Freehackers Union NYC show went insanely well. I managed to pull off a full live internet feed of the audio to people in FU and the show at the same time. We had about 10 newbies show up to give their first five minutes and 11 listeners on IRC/icecast. Some people showed up just to hang out, so we relaxed the rules and let them stay to build an audience. Overall, there were some cool projects presented and everyone had a good time.

Idiopidae: code is code; prose is prose.

Idiopidae is my attempt at finally releasing something that makes it easier for technical documentation authors to write.

The purpose of Idiopidae is to keep the code in the code, and the prose in the prose, and then merge the two together based on very light comments in the source.

You can see the original HTML of this file as well as the final output to compare the two. You can also look at a version that’s done before wiki rendering which shows how Idiopidae automagically figures out that the sample.page file is a text file and properly formats for that output.

Concepts

Idiopidae works on the idea that in your “prose” file you’ll put include statements as comments, and in your “code” file you’ll put export statements to mark off regions of code that need to be named.

When you run the idio Python script on your prose file, it follows the include statements and loads the file and section you specify into an output result. It will also format it with the Pygments library to produce nice typsetting (currently defaults to HTML).

This file you’re reading right now is simply a Textile prose file that includes and describes Idiopidae’s source. The process for creating it was:

  > cd doc
  > webgen
  > idio output/index.html  > output/test.html

The source is available from a Bazaar repository at:

http://www.zedshaw.com/repository/zapps

As currently just a demo of Zapps but it will be moved into its own project folder soon as it is good enough to use and distribute.

The Runtime

It’s best if we start with the runtime.py file, which is responsible for using the IdiopidaeParser to process the files. It starts off with your typical boilerplate code but I like the with statement so I include some future stuff:

### @include "runtime.py" 1
### @end

Next we need to keep track of stuff:

### @include "runtime.py" "Builder Class" html
### @end

Now, there’s three methods that the parser uses heavily during the parsing phase to chunk up a document into the proper structure for later analysis:

### @include "runtime.py" "Main Methods: include, export, append" html
### @end

These aren’t used by callers so much as by the IdiopidaeParser and the Composer. These methods then use:

### @include "runtime.py" "next_statement" html
### @end

To swap into the next statement and:

### @include "runtime.py" "append_current_export" html
### @end

To append each export to a list of exports found.

The process we’re describing involves the IdiopidaeParser using the Builder under the direction of the Composer:

### @include "runtime.py" "Composer Class" html
### @end

It is built with a simple loop in the idio file that acts as a binary for users to run:

### @include "idio" 1 html
### @end

First we have how a file is loaded and parsed by the composer:

### @include "runtime.py" "How Files Are Loaded" html
### @end

which is actually used by the process method:

### @include "runtime.py" "How Files Are Processed" html
### @end

This is the most complex method since it is where all the real work is being done. It loads the file we want to compose, and goes through all the sections. Any section that’s an export is just printed out, but any section that’s an import is processed as another call to include and format to get the text:

### @include "runtime.py"  "Formatting Lines With or Without Numbers" html
### @end

The include method is actually very simple:

### @include "runtime.py" "How Other Sections Are Included" html
### @end

And that’s all of idiopidae except the parser, which we’ll go over next.

The Parser

The parser is the key to how Idiopidae works and it uses the Zapps that I adopted recently. It shows you can easily crank out little parsers for little languages that are fast enough for real work.

Since most people don’t get parsers, you could do good to use bzr to grab the code and study how this file is translated into the idiopidae.py file.

Every parser generator has three main components: code stuff, tokens, and grammar rules. For Idiopidae there’s not much code stuff than the import of the runtime:

### @include "idiopidae.g" starter text
### @end

Then we just start off the parser declaration, which will be turned into a class named idiopidae.IdiopidaeParser that you can run:

### @include "idiopidae.g" grammar text
### @end

Now, we need to have a bunch of tokens which we want to either discard as just visual aids for the user, or keep as input data:

### @include "idiopidae.g" tokens text
### @end

You can’t tell from the above list what it is dropped and what is kept, for that you have to look in the grammar. The trick is we define all the base “words” or tokens and then we use the grammar to sift through them to pull out what is considered Junk or a Statement:

### @include "idiopidae.g" rules text
### @end

More on reading this later.