Hinchcliffe started by saying that “no small system can stand sustained contact with a much larger system without being fundamentally changed”, and argued that the since the web is now much bigger than enterprise systems,it has a significant impact on enterprise architectures.

Quoting Dare Obasanjo, Hinchcliffe said that “my web site is bigger than your enterprise”. The biggest IT centres are not in the enterprise, they are web sites. Web is today a software platform bigger than any operating system, with half a billion hosts and “has all your customers, all your competitors and all the ideas and innovation”. However, Hinchcliffe thinks that there are “only a few proven strategies for long-term competitive advantage” and the community has only now figured out how to use it.

As the requirements for modern software architecture, Hinchcliffe pointed out:

• highly distributed and federated
• have a social architecture
• built from cutting edge platfoms and parts
• have to scale globally
• set with expectations that are very high for functionality and low for the cost to develop/own new apps
• created with productivity-oriented development tools

Enterprise dependencies are now used live with federated online systems, not by linking to libraries. Hinchcliffe gave an example of clickatell.com, a world leader in mobile messaging which is used as an online service through a HTTP API, not integrated traditionally. Because of that, a big new challenge for software companies is that they need to become very good at operations as well as developing software.

The value in software systems has, according to Hinchcliffe, moved up the stack. It’s no longer about the processes but about the data. Facebook, Youtube and Google are examples of leading online systems which are based around data, not processes. To get there, software architecture has to be designed for participation. All top online leaders have the architecture designed for participation. Twitter launched with an API straight away, and the majority of its uses comes from the API. Youtube is the biggest video repository in the world but they haven’t built up the library themselves, they got there through an architecture of participation. Companies like Harley Davidson and IKEA get customers to create the data, organise communities and provide support. The architecture is there to enable that. The World Bank has a developer network.

Another interesting thing is that the expectations of high productivity make software companies make some strange choices, at least from the classic enterprise perspective. Facebook is built in PHP. Twitter is build with Rails. Most enterprise architects would not choose those platforms to build systems as big as Facebook or Twitter. However, these systems are still market leaders. There is an increasing move to assembly and integration over development of the new code. Architectures have to be designed to support rapid change. Flickr adds changes to production sometimes every thirty minutes. Hinchcliffe said that Flickr is continuously in beta. [Fair enough, but none of those systems actually takes money from people and works with financial accounts. I'm not so convinced that we could keep a transaction processing system in “constant beta”].

As key advances that are changing the landscape and will change the landscape in the future, Hinchcliffe pointed out:

• cloud computing
• non-relational databases
• new productivity oriented development platforms

As the key factors for success, he pointed out

• peer production and crowdsourcing
• owning your classes of data on the network
• using new distribution models to leverage the web as your platform
• social systems
• opening supply chains and 3rd party sourcing (programmableweb.com)

I will be covering QCon London 2009 in detail on this blog. Click here for other news and reviews from the conference

• MVC approach in general helps a lot to make web UIs testable as it allows us to test lots of things below the surface of the UI efficiently, including the controller workflow, and just roll back transactions to make tests repeatable
• Comperensive testing above the UI surface (eg through web automation tools) often doesn’t pay off as scripts take too much time to execute and maintain.
• Automation of stuff below the surface leaves us with more time to do exploratory testing, but some basic automated tests still make sense to do through the web ui as well (eg face saving tests)
• Rather than testing the formatting through the UI, a better approach is to move complex formatting away from view templates into helper classes which can be unit tested. This also reduce the stuff you need to test through the Web surface.

ASP.NET MVC specific ideas

• The default (web forms) view engine allows you to put arbitrary code in the view, but view rendering cannot be easily unit tested, so this requires discipline and ensuring that view pages contain only display and formatting code. Going back to the one of already mentioned ideas, complex formatting should be moved to helper classes and not be in the view page as well.
• ASP.NET MVC doesn’t really provide any special support for unit testing out of the box. There is no equivalent to Monorail’s test HTTP contexts. You can still mock out HTTP objects yourself. However, controller methods that don’t depend on HTTP contexts or sessions can be easily unit tested as normal classes.
• Using helpers to generate forms makes views more resilient to change as routes are automatically generated based on class properties. So when you refactor controller method names, form targets and properties get updated automatically.
• Similar approach in the controllers is to use the ExpressionHelper from the optional extension package Micrososft.Web.MVC, which enables us to get calculate routes with a lambda expression, generating the URL for a particular action. Using this rather than hard-coding strings makes controllers more resilient to change, as lambda expressions are strongly typed and refactoring friendly.
• Although IView has the render method with a Writer argument, because the default WebForms view engine is using HttpContext.Current it ignores the writer and cannot be mocked out. Other view engines do make use of the TextWriter so the view output could be captured for testing purposes from a unit test.
• The MVC contrib package has some more additional tools to test routes from unit test tools.

We did manage to put together a very simple web application that demonstrates some of these techniques during the workshop, and you can download the source code for that from the Open space code subversion repository.

I really enjoyed participating in the Openspace Code workshops and I’m really looking forward to the next one. Apparently, they are going to be organised every two months. Here are some links to other write-ups:

• Alan Dean
• Rob Cooper
• Chris O’Dell
• Jamie Thomson

Photo by Alan Dean

I use WordPress for my blog, and so far I am relatively satisfied with it. As a very popular online software package, it does get attacked a lot and security updates are released every once in a while. My site was hacked last year and the bastards dumped a bunch of hidden porn site links in about twenty articles, which took me a few days to clear up. So I learned the hard way that when the admin console suggests an upgrade, I should take its advice. I also added a cron task to check for a few keywords in the database and alert me if someone starts advertising limb enlargement devices for free, and since then I had no real problems. That is, until my site became a hot spot for south-east Asian smut aficionados overnight.

My first guess was that someone was simply spamming me with fake referrer headers, since there was absolutely no reason why my web site would actually appear in Google’s search results for adult movies, Korean or with a different geographic origin. Web sites use request referrer headers to identify where the visitors are coming from. A web browser will send the address of the site where you click on a link to the linked web site, if you have not turned that off. It is not a 100% reliable mechanism to identify visitor sources, as some people turn that feature off and some browsers have bugs and send rubbish, but in general it works OK. With the recent surge in the number of blogs, a new kind of spamming started to take place online. Spammers send fake requests to web sites, putting the address of the web site they are advertising into the referrer header. The rationale behind it is, I guess, to make the site owners to click on the referrer link to see who is sending people to their web site.

But there were quite a few of those requests, much more than with typical spam. The visitors were led to the search web page, and did not look at any other page after that, which could be explained by the fact that they were probably disappointed to find only clips of an ugly bald guy talking about agile acceptance testing instead of their favourite underground adult stars. However, with the web page they downloaded images, css and javascript files, which spammers typically don’t do. I did not know which article actually brought the unexpected guests since only the search page was affected. The database lookup did not help either — luckily this time it seemed that the site was not hacked.

I tried out the query on Google, just for fun, to be absolutely amazed that my site was the third on the list. Sure enough, my search page was there. I simply had to click on that to see what happens, and a few seconds later I was looking at a spam web site. My web logs showed a hit from Google again, but I was not looking at my site. Clicking on the “cached” link on Google led to the same outcome. I grabbed the page using wget, which definitely would not jump out directly, and there I found the words “korean underground adult movies”, but only after the “There are no results for…” phrase. More interesting, after that, there was a HTML image tag with “-1.com” as the source, and an onError event redirecting people to the spam web site. When the page loaded, the browser could not find -1.com to load the picture, and fired the onError event, sending the visitors from my web site to some place they could probably watch something more to their liking. Not a bad trick at all!

God knows how they got Google to index my web page with both their keywords and the redirection tag as a search phrase, but they did. And it’s not only my blog, there’s a few thousand other sites with the same problem. Search on google for “onerror freeimagew” to see them. The results containing </title> in the site name will probably redirect you automatically to the spam site.

The problem was that my blog just dumped out whatever people put into the search form when it could not find any relevant posts. The input string was properly sanitised before it was sent to the database, and WordPress generally cleans up all user submitted comments from hostile content, but it looks like they did not think of someone using the search form to hack the web site. In any case, I just changed the theme search.php file to print “Sorry, no posts matched your criteria” when there are no results, and that fixed the problem. A proper solution would be to strip out HTML tags from the search but I was too lazy to look for all the places where the phrase could be set.

In any case, this is one more example how important it is to filter and sanitise everything put in by web site users, regardless of how safe it may seem, and never ever printing it back on the web site without checking for potential problems.

Image credits: Sonja Gjenero

For those of you that have just joined this tutorial, this is the fourth iteration of the EvilLink web application, that I am building as a way to demonstrate features of the Castle Project application framework and to present some of the best practices for web application development using the Castle Project. Previous articles in this tutorials are:

• ActiveRecord basics and unit testing
• Monorail basics
• Saving time and effort with advanced Monorail features

So far, we have developed a skeleton of the web application that has user operations and link management functionality. Now we write some unit tests for the controllers.

A quick note before we begin. If you are in UK this week, you might want to come to the free session on developing Ajax applications with Monorail that will take place on Thursday (12/06) afternoon at Skills Matter offices in London. This will give us a chance to discuss any questions about this application as well face to face.

Isolating controllers

The first thing we need to do in order to make our controllers unit testable is to isolate them from any external dependencies, including the System.Web.Http objects. Most of the code that would normally be bound to those objects is handled by Monorail anyway. The only place where System.Web objects are used directly is in the UserContext session wrapper. So let’s replace the UserContext object with an interface and a default implementation. Then we can provide a dummy or mock implementation for unit testing.

using System.Web;
using EvilLink.Database;
namespace EvilLink.Controllers
{
    public interface IUserContext {
        int? CurrentUserId { get; set;}
        User CurrentUser { get; }
    }
    public class WebUserContext:IUserContext 
    {
        public int? CurrentUserId
        {
            get
            { 
              return System.Web.HttpContext.Current.Session["currentUserId"]
               as int?; 
            }
            set
            { 
              System.Web.HttpContext.Current.Session["currentUserId"] = value;
            }
        }
        public User CurrentUser
        {
            get { return User.Find(CurrentUserId); }
        }
    }
}

Now we have to propagate this change to the controllers. Change the field type to IUserContext and create two constructors. The no-parameter constructor will automatically set the default implementation so that our application continues to work as before, but we can also set the context from the outside using the constructor with a parameter.

public class UserOpsController : SmartDispatcherController
{
    private IUserContext context;        
    public UserOpsController(){
        context=new WebUserContext();
    }
    public UserOpsController(IUserContext ctx)
    {
        context = ctx;
    }

This is how we would mock out any external services that controllers may be connecting to or other stuff that prevents unit testing. At the moment, the controllers are still tied to the default implementation with a compile-time dependency. In the next article of this tutorial we look into better ways to handle that, using the Windsor dependency injection container.

Completely detaching from the real Web

Now create a new project in the solution, called EvilLink.Controllers.Test. Add references to Castle.ActiveRecord, Castle.Monorail.Framework, Castle.Monorail.TestSupport, NHibernate and nunit.framework DLLs. Also add references to EvilLink.Controllers, EvilLink.Database and EvilLink.Database.Test projects that we have developed so far. This new project will host our controller unit tests.

The Castle.Monorail.TestSupport library has almost all the support we need to unit test controllers. It provides us with a mock HTTP environment implementation that allows controllers to execute and be inspected in the code. Class BaseControllerTest does all the heavy lifting, we just need to subclass it and call PrepareController to initialize the controller. Then we can execute any of the controller methods directly and verify the effects on the domain or check controller properties to verify messages or objects prepared for rendering. All subclasses of SmartDispatcherController have a public Context property that allows us to inspect the Rails engine context as well.

There is just one slight problem with the BaseControllerTest. All its interesting methods are protected, so the test class has to subclass BaseControllerTest. It would be ideal to reuse the unit testing support for the data access layer that we developed in the first article, but C# does not support multiple inheritance. A workaround for that is to extend BaseControllerTest once, expose appropriate methods as public, and then embed that into another test class that initialises the data access system. Here is MRTestBase class that provides all the functionality of ARTestBase and also allows us to initialise controllers for testing:

using EvilLink.Database.Test;
using Castle.MonoRail.TestSupport;
using Castle.MonoRail.Framework;
namespace EvilLink.Controllers.Test
{
    public class ConcreteControllerTest:BaseControllerTest{
        // just expose PrepareController as a public method
        public void InitController(Controller controller)
          {           
            PrepareController(controller);
          }
    }
    public class MRTestBase:ARTestBase
    {
        private ConcreteControllerTest context = new ConcreteControllerTest();
        // expose InitController
        public void InitController(Controller controller)
        {
            context.InitController(controller);
        }
    }
}

We can use a dummy implementation of the user context, not tied to the real HttpSession object, to verify the session storage (this would be a good place to use a mock factory).

using EvilLink.Database;
namespace EvilLink.Controllers.Test
{
    public class DummyUserContext:IUserContext
    {
        private int? currentuserid;
        public int? CurrentUserId
        {
            get { return currentuserid; }
            set { currentuserid = value; }
        }
        public User CurrentUser
        {
            get { return currentuserid == null ? 
                null : User.Find(currentuserid); }
        }
    }
}

Writing NUnit tests

We can now write a couple of unit tests inspecting the workflow and session behaviour of the user operations controller. In the setup, we initialise context objects, create a user to play with and prepare the controller. Notice that we just call the constructor and then pass the object to PrepareController and that is it — no black magic involved. We can call the login method and then inspect the user context to check whether the expected user object was stored in the session. We can also try to log in with an incorrect password and verify the message coming out (hint: PropertyBag is a public property of the controller, so we can inspect it directly). The SelectedViewName property will give us a way to check which view was selected for rendering. We can verify that the controller initiated redirection by checking Context.Response.WasRedirected property, but I don’t know of a good way to verify redirection URL at the moment. Jimmy Bogard explored some options for that like changing the redirect method. Apart from that, we can easily test everything else:

using EvilLink.Database.Test;
using NUnit.Framework;
using EvilLink.Database;
using Castle.MonoRail.TestSupport;
 
namespace EvilLink.Controllers.Test
{
    [TestFixture]
    public class TestUserOps:MRTestBase
    {
        DummyUserContext wuc;
        UserOpsController ops;
        User  mike;
        [SetUp]
        public override void SetUp()
        {
            base.SetUp();
            wuc = new DummyUserContext();
            ops = new UserOpsController(wuc);
            InitController(ops);            
            mike = new User("mike", "mike", "mpass", "m@m.com");
            mike.SaveAndFlush();
        }
        [Test]
        public void TestLogin()
        {
            ops.Login("mike", "mpass", "/home");          
            Assert.AreEqual(mike, wuc.CurrentUser);           
        }
        [Test]
        public void TestLoginIncorrectPassword()
        {
            ops.Login("mike", "wrongpass", "/home");
            Assert.AreEqual(null, wuc.CurrentUser);           
            Assert.AreEqual("Username or password incorrect, please try again",
                   ops.PropertyBag["error"]);
            Assert.AreEqual(ops.Name+"\\index", ops.SelectedViewName);
            Assert.AreEqual("/home", ops.PropertyBag["goBackUrl"]);            
        }
        [Test]
        public void TestRegisterUser()
        {
            ops.SubmitRegistration(
              new User("tom", "tsmith", "tpass", "t@t.com"));
            Assert.AreEqual(
             "You have registered successfully. You can now log in.",
                ops.Context.Flash["message"]);
            Assert.AreEqual(true, ops.Context.Response.WasRedirected);
            User u=User.FindAllByProperty("Username", "tsmith")[0];
            Assert.AreEqual("tom", u.Name);
            Assert.AreEqual("t@t.com", u.Email);
        }
    }
}

Build the project, fire up your NUnit test runner, and you should see the tests pass:

Some links for the end

• Source code for this iteration
• More detailed information about BaseControllerTest
• Older library for testing, that works by simulating complete http requests and responses.

Stuff to remember

• BaseControllerTest class from Castle.Monorail.TestSupport provides a framework to unit test controllers.
• Just create the controller as any other object and pass it to the PrepareController method of BaseControllerTest to be able to call its methods directly
• SmartDispatcherController exposes almost everything interesting to verify as public properties, including SelectedView, PropertyBag and Context.

Next exercise

In the next article of this tutorial, we look into injecting services and configuration into controllers using the Windsor dependency injection framework. That article should be online soon. (Subscribe to RSS updates to get notified about that). As I mentioned above, if you are in UK, come to the free session on developing Ajax applications with Monorail that will take place on Thursday 12th to learn more about what Castle can do for you.