Lamson Project Ideas

I really like working on Lamson. It’s a fun project because nobody else is currently doing anything this slick for email. I get the impression that people think it’s either cool (but have no use for it) or that it’s lame because email is lame. Web is hot baby, not email.

Sure, the web might get you laid, but email is a hell of a lot more fun. You don’t have to worry about dipshits complaining about “typography” and “design” because they got a Mac for christmas. There’s no CSS, browser wars, real-time availability, scalability problems, or a billion programming languages and frameworks. With SMTP everything is solidified and in desperate need of a facelift, so you can take your time and enjoy writing something useful.

With that in mind, I’ve decided to lay out some ideas I’ve come up with while I hack on Lamson. I may never get around to implementing half of these ideas, and about half of them are probably retarded, but man they could be fun. If you’re looking for something to use Lamson on, feel free to rip off some of these.

The Mailing List Sample

First off, the Lamson source comes with a very simple mailing list example I worked up to test things out. I’m actually running that mailing list on the lamson site and it works, but barely. It does have some cool features which are hard to pull off with other mailing list systems, but it needs much more development to be fully useful.

The most useful (or evil) part of the mailinglist sample is that you don’t have to register a list to make one, or to subscribe. Lamson’s regex routing is dynamic, so rather than edit an alias file and setup a ton of directories for each list, it can make them on the fly. It sees that the list doesn’t exist, makes the user confirm, and then makes it and subscribes them in one go.

However, this will need some work to be useful. It’ll need a list of the available lists, a “similarity search” in case someone spelled it wrong, better email responses, better handling of MIME encodings, and a good archiving system.

If you’re looking for something to hack on without too much investment, simply taking the Lamson mailing list sample and adding these features to it would be a good start. In fact, if you went through this great list of required features you could have a good starting point for a killer mailing list. At least that’s my plan.

My goal with the mailing list sample is to potentially host free mailing lists for open source projects or anyone that needs them. Right now it’s a major pain in the ass to get a mailing list going, so I think it could be a useful service as well as a good test case for Lamson.

The Spam Experiment

Over the last week I’ve been running a very simple spam filter in my Lamson server based on SpamBayes that’s been getting good results. So far it filters out all email that is obviously not for me, and then runs the rest through the filter to sift out the spam vs. ham. It’s about 80% effective so far, and getting better as I continue to train it.

The next step with my spam blocking experiment is to include it in the Lamson framework so that filtering and training for spam is dead easy. The goal would be to make using SpamBayes to filter an entire site a trivial bit of code so that all Lamson systems get it for free.

Twitter Stuff

I’ve got the foundation, but no code, for a project I’m calling “sylvester”. It’s pretty simple, but could be a fun test of Lamson interecting with a REST interface. It would be a service where you send an email to WORD@sylvester.lamsonproject.org and sylvester does a search on twitter for that WORD. Then once a day (or hour, depending on how hot it is) you get all the latest tweets for that search word.

Another idea is to simply let people get other people’s tweets over email. There’s a few startups doing this, so it’d be funny to see how easily something like this could be made in Lamson with the python-twitter project.

Smart Address Obfuscator

Today someone asked me for this, and most of the available systems seriously suck. The idea is simple: let people get a random email address and use it to then throw away when they don’t want it. There’s a few sites that do this, but they are a pain in the ass to use or post everything you receive to a web page.

The problem with this idea is that it could be used by spammers and evil people to do evil things. That’s where the spam filter and mailing lists come in. If there’s a Lamson based mailing list service that let’s members flag messages as spam to train the filter, then it’d be possible to reuse that trained data to detect accounts being used for spam.

If you have a good spam detector trained by an army of people from some other real service, then the obfuscator can prevent most of the abuse.

If they are doing illegal things then there’s nothing you could do. Either you’d have to go with “nothing is stored” or “everything is stored for a week and we love the FBI”. I kind of like the second one since it would deter abuse, but the first one is easier to maintain.

Writing an obfuscator in Lamson should be easy, since it is similar to the simplified mailing list system. You could even make it low effort to start a new address. They send to new@hideme.lamsonproject.org and it responds with their ready to use address just for them.

I’d Rather Have Rizzo

Well, people seemed to love I Want Sandy but it got shut down. So fuck Sandy. She’s a prude anyway. I want Rizzo instead.

What IRHR does is accept email from users asking to be reminded of some event in the future. There’s a few sites that do this, with different reminder languages depending on their usability. Mostly you tell it to remind you of things at a certain time on a certain day, and then it gets complex if you need repeating events.

The trick for this project has less to do with the email, and more the language used and the concept of time. You’d want to keep the parser for such a thing loose and easy to use, since using it over email would suck if it was too “programmer”.

Micro-Subscribe Mailing Lists

Why are the only options in a mailing list either:

1. Subscribe to every tiny little thing said even if you don’t care.
2. Get a digest of the day that’s a pain in the ass to read.

What I’d like for many of the large lists is a “micro-subscribe” or “micro-list” system. Each day I get an email of the day’s active threads. Each thread has a simple web link that I can click, and then I’ll start receiving emails for that thread only. Maybe I get the past messages too, who knows. Maybe there’s a page I click on and it does the actual micro-subscribe with a nice browser interface. Hell, maybe I reply with only the threads I want and the micro-list figures it out from there.

The point is, on the really large mailing lists, you don’t want everything, but you do want to get a few threads that seem to show up repeatedly.

In Lamson this would most likely need the web component, but you could get a basic version of this written if you read the threads (by some ID) out of replies from the user. The real hard part would be the CPU and disk needed to crank through all messages posted and figure out who should receive them or not. But hell, that’s what a database is for right?

Super Awesome Blog Comments

I don’t have comments on my blog because people who comment on blogs are typically dickheads. Either I’d have to spend my days filtering out the douchebags or not have them, so I don’t have them.

Part of the problem with blog comments is that people can come by and shit all over your blog and then run away. You don’t know who they are, and they don’t have to stay to defend their statements or continue to contribute.

With that in mind, what I’d like is a mail-to-blog comment system that “micro-subscribes” anyone who comments to the blog post until commenting is closed. There would be a tiny little mailing list for each blog post (created on the fly) and when people send an email to the address their comment is posted to the blog and sent to everyone else who’s commented. When they comment they are automatically subscribed to that blog post’s micro-list and are now able to reply to continue commenting.

The purpose of using email, apart from making it a little harder to comment, is that it’s as close as you get to a real identity online. Making a new one is a pain in the ass (well, unless the obfuscator becomes popular) so they are more inclined to use one they already have. Combining the identity from email with the required subscribe to the rest of the comments means that people who comment on your blog are committed, and if they say something stupid they have to deal with the replies.

I’m sure this would annoy the hell out of commenters, but that’s mostly the point. It benefits the people who really want to contribute to a conversation, and deters the people who want to graffiti a site with one-liners. Throw in the spam filtering and you’ve got a great blog commenting system.

The Exchange Savior

This one is simple: MS Exchange™ dominates the corporate world, but it is used like a giant file share by the users. They attach 40MB presentations and send them to hundreds of people without even thinking. This is then actually duplicated to each user for no real reason (at least it did last time I used Exchange). That means your single 40MB email sent to 100 people uses 100*40MB of space.

What happens next is every moron who received it then modifies it and uses Exchange as a revision control tool. They append numbers to the file name and send revisions back-and-forth until it’s just right. Hopefully they don’t fill up their Inbox quota while this is happening or they can’t get their job done.

Solving this in Lamson would involve the following:

1. Every email goes through Lamson before it hits Exchange.
2. Lamson takes any attachment and pulls it off, and then stores it by the file’s SHA hash on a web server. If the file already exists by the content SHA then it avoids making a new one.
3. The attachment is then replaced with a link to the web server, and people use that to get it.
4. The social network of recipients is tracked for that file, and as recipients make changes, Lamson can track the file’s alterations over time and point people at past versions.
5. End result is recipients could receive a link to new versions and a link to a page showing the revision history for the file including who modified it.
6. Finally, Lamson can work with the website to indicate the access permissions for the files based on who is being included in the file’s social network. As the file is sent to more people, Lamson tells the ActiveDirectory to add them to gain access. If you want Joe to get the file, send him an email with the link or include him in the next revision’s list and he’ll get access.

The goal of a system like this is to pull all the files off of Exchange and onto a web server with revision history, user activity, and security that doesn’t get in the way of people doing their jobs. If someone is given access that shouldn’t have it, everyone would know because they would be listed in the information web page for the file. You could go and see that Marry sent it to Joe and shouldn’t have, then you can go talk to them.

Now, imagine the above system, but throw in all the other things I’ve discussed so far:

• Mailing lists for each file are created on the fly so that people can talk about the work they’re doing without any fuss. These are micro-lists so people can subscribe to the parts of the discussion that interest them.
• Spam filters are used not really to block spam, but to detect fraud. I did some work with this and spam filters are very good at detecting fraud. Shockingly good.
• People could also schedule meetings and things through the email, or have recurring events about that file registered with a simple reply to the list.
• Throw in other goodies like virus scanning and reporting to management on the employee behavior and you’ve got a wicked great tool.

These features could be cream, because really if you can strip out attachments, rip out viruses, and store duplicate files on a web server then you’ve got something useful already.

That’s It For Now

I think that’s enough ideas for now. I’ll be posting things like this as I come up with them and feel people might be interested. If you work on any of these then send me an email so I can help by either contributing code or by making Lamson work better for your project.