thinking in geek tagline’s are so web2.0

Crappystrano

Note: After I started this series Jeremy Miller has posted a very relevant article about source code hygiene issues.

(This is the third part in a series - you should probably read part one and two first.)

Today I’ll present a sanitised copy of my deployment script. I’ve based the design of deploy.cmd on Capistrano¹ . The script is nicknamed “Crappystrano”. Here’s the output (pretty htmlized source is here):

>deploy.cmd

Crappystrano - the shit script which (sort of) deploys stuff

deploy.cmd staging|live [branchpath]

Here’s breakdown of the steps² - skipping some error checking³. :

1. Do we have the required dependencies⁴ on the path? If we don’t copy them to the path if they’re just 1 exe - otherwise exit with an error.
2. Check the command line parameters. We must have the environment we’re releasing to? Are we releasing from the trunk or from a SVN branch?
3. Construct a release time stamp. This is YYYYMMDDHHMMSSMS.
4. Perform a “svn export” from the trunk/branch to %tmp%\staging|live%timestamp%. [E]
5. Use msbuild to build the solution. [E]
6. Rearrange the folder structure so we only get the artefacts we need in the release package.
7. Zip up the release package using the environment name + time stamp.
8. move the release package back to the root directory of the project and delete the export from %tmp%.
9. Make a SVN tag (svn copy trunk/branch /tags/environment-timestamp).

I’ve uploaded a zip file with the script and folder structure + a little readme. This is not the best way of doing things - it’s just a very lightweight way. Comments, questions, feedback etc… are gratefully received.

1. http://www.capify.org/ - I know some people are going to say - just use Capistrano. There are two reasons for me not to: 1. I’m the only person in my team familiar with Ruby. 2. Even if you argue that they don’t need to understand ruby to run a cap deployment recipe - they still have to have it installed. It’s just yet another dependency which I’m not willing to take currently. [↩]
2. This is all simplified as the actual script supports a SaaS product which has multiple instances - one per client [↩]
3. noted with [E] [↩]
4. gnu zip.exe, svn [↩]

A deployment process

I know I said my next post would be my deployment script but after posting - I realised that I should do a 1000 foot overview of what I’m trying to achieve.

For me - a successful deployment has a number of components:

• Build
  1. Code successfully builds 
  2. tests pass.
• Versioning
  1. I know exactly which revision was deployed from subversion.
  2. I know what environment it was deployed to.
• Deployment Safety

  1. On my server I have the ability to rollback a deployment.

• Support
  1. I can compare the current deployed version to previous releases when I need to do post-deployment production debugging.

To achieve this I use a number of conventions which I’ve borrowed and adapted from Capistrano.

My Subversion repository is structured so that every release get’s it’s own tag - including the environment which the release was made to¹ :

The tags are kind of self explanatory - the format is:

[environment]-YYYYMMDDHHMMSSMS 

This is much easier than trying to remember that version 2.54.33.2 was released on Tuesday to client X.

In a deployment environment (staging or live) - each release is unpacked into it’s own folder.²

After a deployment package has been unpacked on the server there will be a new folder for the release. Making a release active involves a couple of (currently) non-automated steps:

1. Updating the database connection details (this is a matter of copying a database.config into the release folder³ ). 
2. running a DB upgrade
3. changing the webroot in IIS

People who are paying attention will notice that this process is not necessarily atomic. Depending on the DB changes - it’s possible that the old/new code cannot run on the same database - in this case I drop a app_offline.htm - into the previous release folder before I run the DB upgrade.

1. The folders in /tags are client names. This is because our SaaS app is released per-client - not multi tenanted. i.e. Each client is running a different version at any given time - and is upgraded to the latest trunk as necessary. [↩]
2. The occluded bits are the client name - this is for convenience in managing release packages (zip files) as the server has a folder for each client [↩]
3. this is the only thing on the server which is not versioned in subversion. My web.config has the following: <connectionStrings configSource=”database.config”/> - this file just contains the connection details for the application. [↩]

[ControllerAction] and sane defaults

[Ed: For anyone who’s living under a bush Microsoft have released a CTP of the new ASP.NET MVC framework. ]

I’m reluctant to get involved in this - but I feel like there’s another side to this story which is worth considering.

Since it was first presented at altnetconf - the decision to require users¹ to have to specify which methods on controllers are available to be routed to ([ControllerAction]) and which view (RenderView()) to render has come in for a lot of criticism.

Lots of the brightest and best minds in the .net world are screaming about this in various forums. Claiming that it violates:

• the DRY principle
• convention over configuration
• best practice in other frameworks
• commonly accepted laws of physics

Not one of the people complaining is going to use this framework vanilla. They are going to add their own layer of behavior over the top. It’s been designed to make it easy (trivial in this case) to extend and change the policy.

Sane defaults are really important for frameworks. Yes they get in the way of power users. Yes it normally takes a power user 10 minutes to work around them and add that piece of code to their toolbox from then on.

I don’t know how many people were following the emergent rails community back in 2005 when the Google Web Accelerator was released - but many of them (including the smartest² ) got a salient lesson in GET idempotence. (i.e. GET Order?id=23&action=delete - should never delete anything).

Here was a situation where the framework (rails) came with not so safe defaults and bad example code which encouraged doing the wrong thing. These days - rails has absorbed this lesson so completely that it could be described as a REST server and in some ways leads developers into following best practices for web apps by default.

I really don’t understand what the huge deal is here? This seems as clear cut to me as the built in RequestValidation in ASP.NET or automatically escaping SqlParameters for SQL injection. Sure - if your a power user it can be a pain - but it takes 10 seconds to turn off and then your responsible when your site get p0wn’d.

It’s not like ControllerBase is sealed.

1. by default - but easily modifiable [↩]
2. or is that loudest? [↩]

Wordpress 2.3.1

<blog.service.announcement>

I’ve just upgraded from 2.2.x(?) to the latest and greatest. I think everything is back to normal - but if you notice anything unusual - please let me know.

</blog.service.announcement>

Automated deployment == CI for Operations

Deployment is a subject that is near and dear to my heart. I’m almost always the guy who gets to debug production problems, deployment problems and generally wears a developer + infrastructure hat.¹

When it comes to web applications - most teams don’t bother automating their deployment processes. This is something that I think is in the process of changing² - but is not yet a generally accepted practice on a par with TDD, user stories or iterations.

At a minimum - it should be possible to:

1. Issue a single command which creates a release package for your project. (i.e. built, tested, tagged in subversion and zipped up). For me - this tool/command is called: deploy.cmd and is just a batch file.
2. Issue another command in your deployment environment which will configure the new release. (i.e. update IIS root, upgrade the database, update connection strings + any other configuration required).

In my current project I’ve built 1. For me 2 does not exist as a single command line (It’s currently a combination of several tools - this should be changed!).

In my next post I’ll outline my very simple solution for this. Just to get the conversation started.

1. Ten years ago - part of my job was building deployment packages for desktop applications (this is pre MSI). Identifying all the dependencies of an app and scripting them into an install package and testing the deployment was at times almost a full time job. [↩]
2. there have been a couple of good threads about this on the altnetconf mailing list [↩]

Enterprise plaforms - teaching new dogs old tricks

Oren is talking about building a substrate for an enterprise platform. This has been something I’ve wanted to do since I first got my hands on c# and the rest of the .net framework in 2001. I’ve never done it because it always seemed that a one size fits all solution would be unworkable.

At the time - the best web architecture I knew of was the ArsDigita Community System (still available and maintained as OpenACS). What Oren is describing sounds similar in many ways.

The ACS architecture is based on the days when Oracle and TCL were good options for building web based systems…. but the underlying design is still very very good (e.g. it was the first place I every saw the Party pattern). They were focused on designing good data models and separating concerns.

They have the following available in their kernel:

• Parties, Users and (nestable) groups
• A permissions system which supports per “object” (think row) permissions.
• A package manager which allows you to install (and remove) modules of functionality into an existing system (including schema changes, new routes etc). e.g. adding a e-commerce, blogging or threaded discussion module is a matter of clicking in a web ui.
• Scheduled tasks

This thing was and is pretty scalable (it runs the photo.net community).

It’s over 10 years old still a source of good ideas. I recommend taking a poke around their documentation and seeing what you can learn.

(I remember reading once that Philip Greenspun - the original ACS designer was asked by Microsoft to build a new version while the .net framework 1.0 was in development as a showpiece. For some reason he didn’t but I always felt that it was a shame this never happened).

Multiple instances of memcached on win32

It seems like the win32 crowd is discovering memcached. I’ve got a couple of support queries about it over the last month or two and I thought I’d post an interesting one here:

“How can you run multiple instances of memcached on a single machine?”

There’s a couple of reasons why you’d want to do this. The most obvious is that you’ve got multiple clients being serviced on a single machine and you want to absolutely ensure that data is not leaking from one client to another.

If you want to setup multiple instances the once you’ve got the win32 port downloaded and unziped somewhere you need to use the following magic incantation:

sc create memcached2 binPath= “c:\memcached\memcached.exe -p 11111 -d runservice”

You can pass any other params you wish (e.g. amount of memory to use). To run multiple instances just install multiple services with different names/ports.

Note: “sc” is the command line tool for working with the win32 service control manager. It allows you to stop/start/create/delete and more - windows services. Check out it’s command line help by typing “sc” at the command line.

Eric Evans on over-design and intrinsic complexity

Just watched a great presentation by Eric Evans (of Domain Driven Design fame) called: Strategic Domain-Driven Design - Effective Modelling for Larger Projects. I highly recommend it.

Here’s a couple of great quotes (It’s worth noting that he’s talking about large projects):

Not everything is going to be well designed.

You may have a part [of a system] which is not complex - or is just straight forward data collection. You’ve got a lot of screens , a lot of fields. You’ve got to put it into the data base and do a simple report on it.

Don’t spent a lot of effort modelling these things. Just find some simple technology that’s good at that stuff.

The idea that the entire system is going to be based on an object oriented model is I think is a highly questionable assumption.

It’s a really interesting perspective - and a great antidote to some of the discussion at the moment which almost treats DDD as a silver bullet.

Anyone who’s feeling guilty because they don’t have their solutions structured into Core, Model, Services, Presentation, UI layers - or even worse is using a SqlDataSource control in their app - should watch it. It’s an excellent dose of pragmatism.

ASP.NET MVC

I got a chance over the weekend to watch the videos from altnetconf on the new Microsoft ASP.NET MVC framework. It’s a really exciting development. Anyway - as I’ve listened I’ve got myself a list of questions about features - I figured I’d put them here so I can see how they get on.

1. In views ScottGu shows how to write views using <%= %> style code. Now that was all good when asp classic was a new and good idea. These days some (lots?) of people think that HtmlEncode’ing by default is a good idea. It makes attacks like XSS and CSRF more difficult and is the default for much of the WebForms controls. It’d be nice to see support for this integrated into the WebForms compiler or maybe a rails like <%h= %> option.
2. Scott demos a new extension method: .UpdateFrom(NameValueCollection) which allows you to update a class from a NameValueCollection. This is super sweet - especially with the built in reflection caching he mentions. I’d like to know if this extension methos has support for whitelist/blacklist of incoming property names/values? (Like the MonoRail DataBindAttribute.Exclude/Allow properties). IMO - this is an important security consideration.
3. Can you configure the routing rules based on HostHeader+Url or is it just Url?)
4. In the example he uses - the Layout is specified by the View (i.e. MasterPage=”") - this is an improper separation of concerns - it should be upto the Controller to specify the “layout” (masterpage).

Anyway - it’s really just a list of things I’ve thought of and am curious about. Overall I’m amazingly excited and really looking forward to getting my hands on some bits.

Naming things

[Ed: I’ve been on holiday and both ends of it have been sandwiched with more work than my team can handle - they got 4 websites live last week alone! I’m slowly clawing my workload back down - yesterday I built a functional equivalent of the Wufoo form builder. Anyway - back to your scheduled program]

One massive advantage of my current job is that I work two desk’s down from a team of copy writers. Over the last couple of years I’ve come to realise what an asset this is. There’s a famous quote along the lines that there are only two unsolved problems in computer science. Cache invalidation and naming things.

Often when I’m working on a feature I get stuck with what to call a class, interface, feature, UI element etc. Previously when this happened I’d talk it through with another dev/project manager/product owner. Sometimes this helps and they’re able to provide just the word your looking for. Other times it doesn’t.

About a year ago - I realised that the copy writers whole job is finding words to fit concepts that other people describe.

Now I just walk 10 steps and describe the type of word (phrase/whatever) I’m looking for and they start giving me alternatives. I pick one I’m happy with and I’m on my way.

e.g. Yesterday - I needed to describe an attribute of a Question in this form builder thing (the wufoo like beast mentioned above). The properties of this type of question are that there is a correct answer/s. If you don’t answer the question correctly - then “something” happens. I was flailing around trying to find a succinct way of describing this in my model (IsFailableQuestion, PassFail etc.. ) after an unsuccessful conversation with another developer and the product manager - I asked one of the copywriters and they said: ummm… killer question?Perfect!

(If this seems like a strange naming choice then it’s because of the specifics of the domain that this makes sense - no - no one ever dies).