You only preach what you've just discovered

WPF 4's DataGrid - bound template columns, anybody?

2011-07-04T14:14:00.003+02:00

I recently had occasion to use the WPF DataGrid component introduced in .NET 4. And here's something really stupid: all the built-in primitive column types (text, checkbox etc.) derive from a DataGridBoundColumn base type. This allows you to specify a binding for the cell content - very logical and WPFy.

The DataGridTemplateColumn is the sole exception. I reflected the code and noticed that they're using a ContentControl internally to present the cell - so how hard would it be to simply apply the binding to the Content property? I'll tell you how hard - it's one line of code.

It's seems like they've done more work and added an extra base class simply to make the template column less useful.

Stupid.

Configuration-driven IoC why?

2011-07-04T14:04:00.003+02:00

I've recently gone off configuration-driven IoC in a big way. It certainly has it uses (particularly in web applications, I've found, and especially if you're selling them on). However it makes no sense at all for desktop apps. Usually you find that if you change the configuration, you're redeploying the app anyway - so why not have the configuration done centrally in code? I've never seen a non-programmer change configuration, so this costs you nothing and saves you a ton of supporting infrastructure.

Also, config-driven code tends to lead to the phenomenon of too much loose coupling. Since you have to obey the container's rules about initialization, you can't make compile-time assertions about dependencies, which can lead to a sort of "fire away and hope it's there" style of programming. And plenty of mysterious errors in apparently unrelated parts of the application.

Just my two cents, I'm not feeling particularly vehement about this one.

Global events - follow-up and sample

2010-07-10T16:19:00.003+02:00

After having been asked several times for an example of the new architecture I spoke about in my previous post, I've posted a sample at CodeProject - The Service Tree Model.

It's a short sample, and it's missing the all-important wiring-builder, but you can get an idea of how the configuration is generated and how the app uses it.

Global events considered harmful

2010-06-23T12:27:00.008+02:00

The “global event architecture” is what could loosely be described as an architectural pattern used principally in composite applications (such as those built using the Smart Client Software Factory or, lately, Prism). It is characterized by loosely-coupled components communicating by means of broadcast events, typically marshaled through an event broker. The events are “broadcast” events in three senses: there is little or no expectation by the sender of what will be receiving the event; a given event may originate in multiple components; and the receipt of events is usually) application-wide, i.e. not confined to any particular context. Also characteristic of this pattern is that senders are connected to recipients implicitly, by the name or the type of the event.

In my article, "Global events considered harmful", I attack the global event architecture often found in composite applications. Having previously worked on three major applications that used the Smart Client Software Factory architecture or derivatives, I recently was privileged to be able to design a completely new architecture suitable for WPF applications.

By far the largest source of problems in the composite applications I had worked on is what I call the "global event architecture". The specific problems I identified were:

Implicit contracts are formed by the events and their arguments.
The contracts tend to overlap in function and meaning.
State is scattered throughout the application.
It is difficult to get a picture of the interdependencies of components, as they are implicitly coupled by the events.
It can be difficult to localize the events (e.g. to the scope of a single dialog or view)
Hanging subscriptions can cause serious application errors, and are hard to find and solve.
Event receipt/publication requirements in one module may force code to be written in one or more apparently unrelated modules, introducing nasty cross-cutting concerns and delocalized code.

The new architecture solved these problems, and proved to be far easier to develop on and maintain. It is based around the idea of having a "wiring builder". The wiring builder essentially writes a script to run just one application-level controller. The controller constructs containers, binds component properties (including exported commands and command dependencies), and manages workflow where necessary. From the wiring builder, we can generate dependency diagrams between components, as well as workflow diagrams indicating the transitions between application states.

The bulk of the application consists of many simple components and services, each with a single function. The components are written so that their dependencies can easily be mocked; this means that contracts between components are well-defined and testable, and also that it is easy to develop the components in isolation. This also improves reusability.

This new architecture I call the service-tree model, and it has the following advantages:

Having a wiring manager means that applications can quickly be reconfigured in form and function, without the necessity of a binary release.
The wiring manager can give you a view of dependencies and workflows that corresponds to what you would see in a functional spec.
It's much easier to develop the individual components, because they do not rely on external events to drive their state.
The code has much better locality.
The application configuration is centralized.
State contracts are explicit.

If the article seems one-sided, it's because experience with the new architecture has left me absolutely convinced that I'll never again use a non-local event. Hopefully you'll have the same experience.

Read: Global events considered harmful.

CrossThreadAction

2010-05-13T17:20:00.010+02:00

Here's another handy class in the spirit of WeakEventHandler. It's useful when you want to run work on a background thread, but wish to provide a callback that will be run on your own (usually UI) thread. Apparently Silverlight users will find this particularly useful, there being no Application.Current.Dispatcher in Silverlight, but as an almost-exclusively WPF user, I use it as helpful shorthand for "call me back on my own thread".

The class:


public class CrossThreadAction<TData> : DependencyObject
{
    Action<TData> target;

    public CrossThreadAction(Action<TData> target)
    {
        this.target = target;
    }

    private void Invoke(TData data)
    {
        Dispatcher.BeginInvoke(target, data);
    }

    public static implicit operator Action<TData>(
        CrossThreadAction<TData> cta)
    {
        return cta.Invoke;
    }
}

Use it as follows:


private void StartWork()
{
    // using a server method with signature
    //   void DoWork(Action<string> callback);
    server.DoWork(new CrossThreadAction<string>(FinishWork));
}

private void FinishWork(string result)
{
    textBlock1.Text = result;
}

WPF users may make it slightly less heavy by inheriting from DispatcherObject instead of DependencyObject. Also, beware of storing an instance in an untyped variable; generally it's a good idea to cast it to Action<TData> as soon as you've created it.

Electoral Reform in Britain

2010-05-07T12:05:00.001+02:00

What’s wrong with the system

As anybody who has followed this election will have realised, it’s staggeringly unfair that the Liberal Democrats, with 23% of the popular vote, only got 8% of the seats. That’s not democratic, and it entrenches the two-party system. Do you often find it hard to distinguish between the respective policies of Labour and the Tories? You’re not alone.

The Lib Dems want a proportional representation system. However such systems have their own problems, the biggest being that MPs would be even less accountable to their constituencies than at present. Think safe-seat manoeuvres on steroids. I don’t think that will fly in a nation where constituents are used to being able to chuck out MPs they don’t like. MPs who are unpopular with the electorate will get in by virtue of their party’s popularity rather than their own. As Italians will attest, this leads to MPs being able to shrug off the most outrageous of scandals, provided only that they can make good with their party.

A sensible solution

Much better would be to implement democracy at a local level. Suppose that each constituency elected a small parliamentary council, proportionally representative of the candidates (a Single Transferrable Vote system would do very well for these councils). That parliamentary council would then elect a representative to sit in parliament, but – and here’s where the power really comes to the people – the council would have the power to instruct its representative on which way to vote. That means that votes in parliament would be on the basis of what the constituencies want, rather than what the party wants. Real democracy!

Why this will never happen

The problem is this: the parties simply won’t wear it. Being unable to whip their MPs into line, party policies would become not so much policies as guidelines. An MP might join a party to benefit from collective think-tanks on the issues, but the real power would be taken away from the parties and given instead to the constituencies. You like it, I like it, politicians hate it. After all, what’s the use in being Prime Minister if you can’t tell parliament what to do?

Nice thought, though

Credit for this idea goes, as far as I can tell, to Sir Antony Jay and Jonathan Lynn. Wouldn’t it be nice if politicians really did act in the interests of the people who elected them?

Slow file deletion on Windows 7 RTM

2009-11-16T10:24:00.003+02:00

The picture below says it all:

Of course, it didn't actually take 23 hours. It actually took about 45 minutes - that's around 13 items per second. More than half of which was taken up by calculating how long it was going to take (estimating the quality of the calculation is left to the reader).

Now I understand the need for transactional semantics during a delete, the necessity of shared filed locking, and the possibility of file system watchers getting involved in their own idiosyncratic ways.

But 13 items per second? On a 10,000 RPM drive? Seriously, guys.

The WeakEventHandler

2009-11-13T15:48:00.009+02:00

When your Observer class hooks an event on an Observable, it implicitly creates a strong reference from Observable to Observer. That means the Observer won't be collected until the Observable is. That means memory leaks at best; latent bugs due to out-of-date observers at worst. So I've written a little WeakEventHandler class that behaves like a weak delegate: it allows the Observer to be collected before the Observable is. Best of all, the syntax is almost identical to the original.

Instead of writing

observable.Event += new EventHandler<MyEventArgs>(this.HandleEvent);

you'd write

observable.Event += new WeakEventHandler<MyEventArgs>(this.HandleEvent);

and the delegate behaves as if it were weak. However there is one downside: the proxy object that is created to pass the event to the observer hangs around. This can cause problems if the event is attached to by a great many short-lived observers. Fortunately there is a solution:

observable.Event += new WeakEventHandler<MyEventArgs>(this.HandleEvent,
    handler => observable.Event -= handler);

The implementation of WeakEventHandler is given below.

delegate void EventDetacher<TArgs>(EventHandler<TArgs> handler)
    where TArgs : EventArgs;


public class WeakEventHandler <TArgs>
    where TArgs : EventArgs
{
    MethodInfo method;
    WeakReference target;
    EventDetacher<TArgs> detacher;


    public WeakEventHandler(EventHandler<TArgs> realHandler, EventDetacher<TArgs> detacher)
    {
        this.method = realHandler.Method;
        this.target = new WeakReference(realHandler.Target);
        this.detacher = detacher;
    }


    public WeakEventHandler(EventHandler<TArgs> realHandler)
    {
        this.method = realHandler.Method;
        this.target = new WeakReference(realHandler.Target);
        this.detacher = null;
    }


    private void HandleEvent(object sender, TArgs args)
    {
        object realObject = target.Target;
        if (realObject == null)
        {
            if (detacher != null)
                detacher(this);
        }
        else
        {
            method.Invoke(realObject, new object[] { sender, args });
        }
    }

    public static implicit operator EventHandler<TArgs>(WeakEventHandler<TArgs> proxy)
    {
        return proxy.HandleEvent;
    }
}

Science is not about belief

2009-09-20T17:35:00.004+02:00

Quantum mechanics is a triumph of the scientific method over common sense. It's weird. Mathematically weird, and it makes deeply strange predictions. Albert Einstein found it difficult to accept ("[God] does not play dice"), and spent many of his later years trying to disprove the theory. Richard Feynman remarked, "I think I can safely say that nobody understands quantum mechanics."

And yet it has the irritating habit of being right all the time. And when I said right, I mean it makes predictions that are accurate to upward of 20 decimal places. That's predictive power.

The moral of the story is that you don't have to believe science to accept that it is correct. Science can, and does, do violence to our common sense, our preconceptions, and our intuitions about the way the world works. Yet it is always correct (within certain parameters); otherwise it wouldn't be science.

Many people think that science "constantly changes its mind", and that that somehow makes it less worthy of acceptance than, say, religious dogma (which never changes). In fact the opposite is true. Every time a scientific theory is discarded, it's because a better one has come along which explains the evidence more accurately, or over a wider range of phenomena. That's exactly what happened to Newtonian gravity. It wasn't that Newton was wrong; rather Einstein was more right. Science is constantly improving, and that's precisely why it is so effective as a naturalistic philosophy.

And then there are those poor souls who claim things like "evolution is a matter of faith, just as much as creation is!" Rubbish. Hogswallop. I can't think of a word strong enough to describe just how wrong this statement is. It's an evil perversion of the truth: science is the opposite of faith.

Scientists don't throw the die and pray for a six. They throw the die over and over, measure the outcomes, and conclude that each number is just as likely to occur as the others. Or if the die is weighted, they will discover this fact. Then they think of a hypothesis to explain the phenomenon: "since each surface of the die has an equal area, and since the density is constant, the shape guarantees equal probability of outcomes." Once thought of, the hypothesis must be tested. What happens if we reduce the area of one side of the die? What if we add more sides, but ensure that they have equal area?

I have no idea if the above hypothesis is correct. Probably it isn't. But it illustrates the way scientists proceed. The results of their experiments are often surprising. Sometimes years of accumulated common sense have to be thrown away in a heartbeat. It can be emotionally wrenching, especially if you have a particular attachment to an idea that now turns out to be wrong.

This is the ruthless honesty of scientists: they never deny reality, no matter how hard it is to accept.

Government is one big monopoly...

2009-09-18T18:51:00.001+02:00

...which would be okay if there weren't legislative barriers to competing with it.

My favourite math joke...

2009-09-11T09:58:00.000+02:00

e-to-the-x and constant-function were walking along happily in Mathland, when suddenly constant-function took fright and tried to hide behind e-to-the-x’s skirts. “What’s wrong?” asked e-to-the-x, “why are you hiding?” Constant-function replied, “because I just saw differential operator, and you know what that means. If he gets near me I’ll be differentiated and vanish!” “Ah, well, I’m not afraid!” declared e-to-the-x. “What do I have to worry about? I’m e-to-the-x!” Boldly she marched up to differential operator. “Hello, I’m e-to-the-x!” she announced assertively. “Pleased to meet you,” replied differential operator – “I’m d-by-dy!”

The post hoc visitor

2009-08-31T11:56:00.002+02:00

The visitor pattern is an excellent way to separate concerns, by moving operations on a hierarchy of objects out of the hierarchy itself. Unfortunately it suffers from two major flaws:

You need to declare and implement the Visit method on each class in the hierarchy, which is not only wasteful of code, but also impossible if you don't have control over the class hierarchy (e.g. it's in an external module).

You need to implement a receiver method for every node on every visitor, even if the visitor handles a whole sub-hierarchy in the same way.

What to do? Well, Python handles this extremely well. You can inject virtual methods into the class hierarchy, even after instances have been created, as follows:


ASTNode.VisitOptimizer = classmethod(lambda t: None)
ValueNode.VisitOptimizer = classmethod(lambda t: t.Value)
ExpressionNode.VisitOptimizer = classmethod(optimizeExpression)

Calling node.VisitOptimizer will dispatch to the appropriate method.

Can this be done in C#? Perhaps in version 4: the jury is out. What about extension methods, I hear you ask? Don't bother. Extension methods match the declared type rather than the actual type, for various good if unfortunate reasons. In versions prior to 4, the best method I've come up with is to write a facilitator class that uses reflection to find the best (i.e. most closely-typed) method in the visitor, and calls that. Needless to say, this is heavy.

Anyone got a better idea?

Creationism... and Sourceforge.

2009-07-04T15:28:00.000+02:00

PS - The new Sourceforge site looks really cool.

What democracy means

2009-06-18T14:15:00.003+02:00

Earlier today, I posted a comment on Youtube where I said that for me, economic liberty is not a question of efficiency or of selfishness. Instead, it's a moral issue. I don't believe that the majority has any right to dictate what I do with my money. I will voluntarily donate to charity (which I think is a good idea by the way), but the point is that it is voluntary. By what right, I asked, does any other person hold claim to my productive output?

Now, rickelmonoggin responded, saying that what I had said was that I do not believe in democracy. That gave me pause. On reflection, it's true. We believe in different versions of democracy. His version may be loosely termed "majority rule". My version is individual liberty.

I believe the term democracy is equally applicable to both, and here's why. The word derives from the Greek demos kratos, meaning literally "people power". I think individual people should have power, rickelmonoggin believes the majority should have power. The latter view is immoral in my opinion. Taking from one person to give to another without voluntary consent is theft, no matter whether it is done by an individual or a government. I hold that no group of individuals has any more rights than a single individual in that group. Might does not make right, in other words.

So what does my vision of democracy look like? Let's take the example of roads. I'd support a system where if you don't pay the road tax, you don't get to drive on the roads. That's fair. If you don't pay the electricity bill, you don't get electricity. You have no right to demand that other people pay for services you enjoy. If they willingly contribute to funding those services, that's absolutely fine. That way, the amount of funding allocated to services is determined by the market of individual choices, rather than by a sub-committee sitting in a small room somewhere in Washington.

Perhaps an instinctive reaction to this philosophy is that it is cruel and selfish. I disagree; I think to say this is to say that man is a mean, selfish little animal, when it is perfectly clear that we're not. The richest men in the world, Bill Gates and Warren Buffett, are also the largest contributors to charity. They don't have to contribute, they do so voluntarily. To me, that's a fine example of the essential good nature of humanity.

Let's look at the alternative philosophy. In the philosophy of majority rule, one implicitly accepts the principle that a group of individuals may demand unearned payment from a smaller group of individuals, provided the first group is big enough to establish a majority. That is a cruel and unjust position to take. By that logic, it's perfectly acceptable to enslave others, provided most people agree that it's a good idea.

One of the distinguishing features of the United States is that it has a constitution that prevents precisely this sort of abuse of power. The constitution, in fact, exists to limit majority rule. It says that all men have inalienable rights that no decree of congress may abrogate. The only problem, as I see it, is that it doesn't go far enough.

So what do you think? Is majority rule fair, just and moral? Or is it just an excuse to pick the pockets of the wealthy minority?

Science: a call to arms

2009-06-15T10:07:00.002+02:00

We live in a world made by science. From putting on the coffee pot in the morning to switching out the lights at night, everything we do involves some or other product of science. Yet science does not ask that we pay homage to it. It does not require us to attend service or pray at its altar. We do not even have to obtain a minimum understanding of science before we make use of its products.

Science does not assert truth without reason and evidence. Science does not dictate how we live our lives. The essence of science is a devotion to discovering what is real. It tolerates dissent, and absorbs change when it is clear that the existing understanding is flawed. The professional opinion of one scientist is worth more than the views of a thousand twittering idiots.

And yet, for the very reason that it is so tolerant, science does not respond aggressively when it is pissed on by uninformed morons. Science makes no philosophical comments. It is polite to its detractors. The irony is that virtually all of the anti-science, anti-reason brigade make active use of technology to opine that science is flawed, untrue or "just a theory".

It's time for that to change. It's time for us to teach the troglodytes just how powerful a scientific theory is. It's time to show the world that not all opinions are equal. It's time for the talking heads to show a little goddamned respect to science. They owe it.

This is a call for scientists to stop appeasing the unwashed masses. Stop allowing them to claim that their ideas are just as valid as scientific theories. Tell them the truth: that when it comes to reality, science is the only game in town. All the rest is noise. When they whine about their personal beliefs, tell them to put up or shut up.

Amazing lack of grace

2009-05-30T23:35:00.001+02:00

wwww.conservapedia.com. Jesus Christ. That's all I have to say. Have things really gotten this bad? Is America turning its back on rationality? You tell me.

Fool me can't get fooled again

2009-04-06T11:40:00.002+02:00

I'm not American. But hell, it's a free internet, isn't it? So I'm going to Opine. Barrack Obama is better than George W. Bush, and here's why. He's a slimy politician. That means he says one thing and does another (what most of us would call "lying"). Okay, perhaps he's a bit more sincere than most, but power corrupts, as they say... you can't climb the greasy pole without getting a bit oily.

So why do I prefer him? Because if I'm going to be deceived, lied to, misdirected, and politicked generally, then I'd rather it be done by someone with the brains to carry out the scheme successfully. There GWB falls flat. If I were an American, I'd want someone on my side who was shrewd, devious, capable of pulling the wool over "them"'s eyes. Someone like Bill Clinton, in fact.

Sure, we can admire GWB for his straight-talking, shoot-from-the-hip style, even if we disagree with what he says (and I most vehemently do). But as president? Pull the other one. Oh wait, you already did...

Transport after oil

2009-03-06T15:37:00.002+02:00

When the oil runs out:

I can imagine electric cars. Powered ultimately by a combination of coal and nuclear energy. Wind/water/solar will certainly proliferate as rising energy costs make them more viable, but until the coal runs out, that's where the energy is.

I forsee nuclear shipping. A ship is quite big enough to play host to a reactor, and it's a bit easier politically to put something that might blow up on a vessel that's not likely to be nearby when it does. Anyway, reactors are getting smaller and safer.

I cannot see myself flying. Unless there's a radical breakthrough in propulsion and/or solar technology. As things stand, solar panels are too inefficient, cell batteries are too heavy and other types of power plants are just too damned big.

Time to sell those airline stocks?

What's currency got to do with the price of gold?

2009-03-05T10:26:00.002+02:00

There are still some loons out there who preach a return to the gold standard. Yes, loons: crazy and/or simple-minded individuals. Here's why the epithet is appropriate.

The argument for the gold standard is essentially this: currency should be backed by something with intrinsic value; gold has intrinsic value; therefore currency should be backed by gold. Let's take the second premise first.

Gold has no intrinsic value. A hunk of gold in your basement cannot feed, clothe or house you. It is only beneficiated (given value) by the uses to which it is put. Those uses include electronics, but primarily gold serves a decorative function, as in jewelery, souvenirs and ornamentation. Artifacts of fashion, in fact. Enduring fashion, to be sure, but whimsical nonetheless.

Now for the first premise: is a floating currency not already backed by something of intrinsic value? Of course it is. The intrinsic value here is the ability to purchase the goods and produce of the economies that make use of the currency. Backing the dollar is the American economy. Would you trade that for the gold in Fort Knox?

(On a side note, I believe this is one of the rare instances where Ayn Rand didn't carry her philosophy through to its logical conclusion. In arguing for the gold standard, she implies that gold has productive power, much as Marx argues that factories do. Wrong, as she herself pointed out - it's the productive powers of the individuals that count, not the materials they use.)

But wait, there's more. Coupling a currency to the value of gold is inherently dangerous, not just because that value is subject to the vagaries of fashion, but precisely because the irrational belief persists that gold has intrinsic value. What happens to the price of gold when the stock market crashes? It tends to go up. Investors buy into gold (and gold producers) just as the demand for consumption of (as opposed to investment demand for) gold goes down. Safe haven or mirage in the desert? What happens when the economy recovers, and forward contracts are fixed at economically unsustainable prices? You have a huge investment surplus undermining an overpriced supply-chain. Something's gotta give.

No big deal?

2008-03-08T15:50:00.004+02:00

Have you ever done something "obvious" and later discovered that it's the topic of a famous paper, or something that gets put in the "advanced" section of the textbooks?

I just read the wikipedia article on tail recursion, and got a surprise when I read the last section.

I've always implemented lists in C along these lines:


void insert(list_t** list, int idx, list_t* item)
{
  while(*list && idx--)
    list = &(*list)->next;
  item->next = *list;
  *list = item;
}

Now to me, this is pretty obvious stuff. But it turns out to have a name - "tail recursion modulo cons" - and credit for its invention goes to David H. D. Warren. Go figure.

I have to ask, though - are our standards getting higher so fast that an advanced concept from twenty years ago is now not worth mentioning?

Are you a follower or a leader?

2008-02-22T13:22:00.001+02:00

I'm neither. Certainly not a follower, because I'm independently-minded and naturally suspicious of authority. Nor am I a leader, because I'm not interested in shepherding my followers (and I don't expect them to agree with or obey me).

On the other hand, I almost always seem to end up setting the direction, and others usually follow my lead. Doesn't that make me a leader, and them followers? Perhaps, but then we have to restate my original question as: are you a boss or an employee?

But that was how you interpreted the question in the first place, wasn't it?

;-)

Efficient singleton initialization with Interlocked

2008-02-22T12:25:00.005+02:00

We had (are still having) an interesting discussion over at Sven-Torben Janus's blog. The discussion is about singletons, but it reminded me of a pattern I've used which has more general application. This is how it might be used to initialize a singleton:


using System.Threading;

public class Singleton
{
  private Singleton() { /* ... */ }

  const int NotInitialized = 0;
  const int Initializing = 1;
  const int Initialized = 2;
  static int state;
  static Singleton instance;

  public static Singleton Instance
  {
    get
    {
      if (instance != null)
        return instance;
      while (true)
      {
        switch (Interlocked.CompareExchange(ref state, Initializing, NotInitialized))
        {
        case NotInitialized:
          Interlocked.Exchange(ref instance, new Singleton());
          Interlocked.Exchange(ref state, Initialized);
          return instance;
        case Initializing:
          Thread.Sleep(0);  // yield and loop until initialization is finished
          break;
        case Initialized:
          return instance;  // does this need to be interlocked?
        }
      }
    }
  }
}

This is a much cheaper pattern than the Java-style double-checked locking pattern (which doesn't work on the CLR). However, I wouldn't use it to initialize singletons, because the CLR already guarantees lazy first-time initialization for static members! Which means that all you need do is this:


public class Singleton
{
  static Singleton instance = new Singleton();
  public static Singleton Instance { get { return instance; } }
}

However, the pattern is still useful. I use it for service initialization, for example where I have to run a costly database query before I can start responding to service requests:


public void GuardInitialized()
{
  switch (Interlocked.CompareExchange(ref state, Initializing, NotInitialized))
  {
    case NotInitialized:
      ThreadPool.QueueUserWorkItem(this.CostlyInitializationFunction);
      throw new FaultException(new ServerNotReady());
    case Initializing:
      throw new FaultException(new ServerNotReady());
    case Initialized:
      break;
  }
}

As always, your mileage may vary.

Update: Some actual timing tests revealed this this wasn't so efficient after all. In fact, until I added the "if (instance != null) return instance;" check to the singleton example, it was actually slower than lock() or memory barrier techniques! With the check, it's faster, but only by 5-10%. The JITter doesn't recognized the interlocked operations as intrinsic and inline them. I suspect that the unexpectedly slow performance is due to pinning and CLR synchronization overhead. Would anyone in the know like to comment?

Further update: I exchanged a few mails with Rico Mariani, and learned that the Monitor primitive actually implements a lock-promotion protocol. Which means that unless there's actually contention for the lock, it's effectively the same thing as an InterlockedCompareExchange operation. So for all practical purposes, lock() and Interlocked.CompareExchange are equivalent. The only time you'll see a real performance difference is when you're writing code where contention is actually a significant factor. So there you go :-).

Formatting code blocks in blogs

2008-02-18T12:08:00.004+02:00

Trying to add nice code samples to blogs can be a bit frustrating. So I wrote a little bit of Javascript to deal with the problem. Feel free to leech (it's in the source for this page). If you make any interesting improvements, let me know!

Shared APIs, part 1 of 4: The why and the how

2008-02-15T17:41:00.009+02:00

So you're writing a three- (or more) tier application. This is probably the most common application architecture, so you're in good (or at least well-populated) company. What I'm going to advocate now is the "shared API" pattern.

Let's get the most obvious objection out of the way first. "But," you say, "surely in a service-orientated architecture we want loose coupling? Doesn't a shared API break that principle?"

The answer is yes and no. Yes, a shared API does create tighter coupling at compile time, but provided the service contract doesn't change (and it shouldn't, after a public release), you are still going to be loosely-coupled at run-time. That is to say, you don't have to update the client software every time you rebuild the service layer, and vice versa.

This is really the best of both worlds. You want tight coupling at compile time, to prevent your domain model from being (a) duplicated and hence (b) susceptible to client/server drift. Conversely, at run-time you really want the coupling to be loose, to prevent your users from getting frustrated with having to upgrade their software all the time.

There are other objections to shared APIs, but I believe that they stem principally from bad design and implementation. The basic idea of a shared API is that you have to define your service contract up-front; so arguing against it on the grounds that it makes life difficult for programmers when they want to just make a little change to the domain model or service contract is, in fact, rather perverse. After all, isn't defining your domain model first one of the most rudimentary design principles?

So now the question becomes, what does a shared-API architecture look like? I'll use WCF to illustrate. There are really two parts to a shared API: the service/data contract and the model.

Let's start with the model. You can do it the other way around, but that sometimes leads to problems later on. Step one is to create a set of interfaces (or abstract classes if you like) that represent your system:

The purpose of this exercise is to get the model right. At this point we are not considering efficiency or implementation constraints. The only concern is with the informational content of and structural relationships between the entities.

Entity relationships can be subtle things. Everybody's heard of "one-to-one", "one-to-many" and "many-to-many", but there's much more to the story. One important question to answer is that of ownership. An object may link to a set of sub-objects, but it doesn't necessarily "own" them. If a sub-object might be shared between two parents, it's almost certainly not an "owning" relationship. The other question is that of visibility. Is the child aware of its parent? (Why?) Does the parent know about its children? Answering these questions is often more of an art than a science, and volumes could be written about the different approaches that are taken.

At any rate, there are generally three types of relationship: parent contains an array of children, parent points to child(ren), and child points to parent. Each implies different ownership and visibility semantics.

Having satisfied ourselves that the domain model is correct, we now turn our attention to the use cases. Each step in a use case could potentially involve the invocation of a service or services. The question that must be answered is: "what data must be passed in which direction at this point?" This is often easy, as in the case of data that is specified to appear on a particular screen, but it can be very much more complicated - as in, what are the services required to update a form or document?

Transactions must also be considered at this point, not only in terms of the guarantees your system must make, but also in terms of the architecture you intend to use. For example, if you use MSDTC to manage your transactions, then outbound transaction flow must be enabled on all the client machines. This could be an administrative and/or security problem, in which case you might be better off changing your service model to explicitly include Begin/Commit/Rollback operations.

Once you've identified the services you require, and consolidated them as appropriate ("as simple as possible, but not simpler"), you're ready to write your service contract. There is generally an intersection between your domain model and your service contract. Try not to focus too much on this! Remember that the service model is a completely different thing from the domain model. Services operate in a specific context, whereas domain entities are abstract and generalized.

Here's an example of a service contract for the domain I described above:

Next time I'll talk about the implementation aspects of shared APIs as they relate to their client and server consumers, and also about the limitations of object orientated languages in this respect. Those familiar with WCF will probably have noticed some problems with the code above, and can therefore anticipate what is coming...

Slicing objects; or, when you only want to pass the piece the method wants

2008-02-15T17:36:00.002+02:00

I humbly refer you to this post by Sven-Torben Janus, and add only the following helpful hint: System.Runtime.Serialization.FormatterServices.GetUninitializedObject().

Short-lived object manager

2007-12-10T10:46:00.000+02:00

I don't know if I'm re-inventing the wheel here, but I refactored a couple of server-side objects to arrive at this base class:


using System;
using System.Collections.Generic;
using System.Threading;

class ShortLived : IDisposable
{
    Guid key;
    DateTime lastAccess = DateTime.Now;
    int disposed = 0;
    static Timer gcThread = null;
    static Dictionary<Guid, ShortLived> objects = new Dictionary<Guid, ShortLived>();
    static readonly TimeSpan Lifetime = TimeSpan.FromMinutes(5);
    static readonly TimeSpan GcInterval = TimeSpan.FromMinutes(1);
    static object syncRoot = new object();

    public Guid Key
    {
        get
        {
            return key;
        }
    }

    public static object SyncRoot
    {
        get
        {
            return syncRoot;
        }
    }

    public static T GetObject<T>(Guid key) where T : ShortLived
    {
        ShortLived result = null;
        if (objects.TryGetValue(key, out result))
            return result as T;
        return default(T);
    }

    public static IEnumerable<T> GetObjects<T>() where T : ShortLived
    {
        List<T> results = new List<T>();
        lock (SyncRoot)
        {
            foreach (ShortLived obj in objects.Values)
            {
                if (obj is T)
                    results.Add((T)obj);
            }
        }
        return results;
    }

    void IDisposable.Dispose()
    {
        Dispose(Interlocked.CompareExchange(ref disposed, 1, 0) == 1);
    }

    protected virtual void Dispose(bool disposed)
    {
    }

    protected ShortLived()
    {
        key = Guid.NewGuid();
        Init();
    }

    protected ShortLived(Guid key)
    {
        this.key = key;
        Init();
    }

    private void Init()
    {
        lock (SyncRoot)
            objects.Add(key, this);
        Timer newCollector = new Timer(GarbageCollect, null, GcInterval, GcInterval);
        if (Interlocked.CompareExchange(ref gcThread, newCollector, null) == null)
            gcThread.Change(GcInterval, GcInterval);
        else
            newCollector.Dispose();
    }

    protected void KeepAlive()
    {
        lastAccess = DateTime.Now;
    }

    private static void GarbageCollect(object context)
    {
        List<Guid> expired = new List<Guid>();
        DateTime now = DateTime.Now;
        lock (SyncRoot)
        {
            foreach (KeyValuePair<Guid, ShortLived> kv in objects)
            {
                if (now - kv.Value.lastAccess > Lifetime)
                    expired.Add(kv.Key);
            }
            foreach (Guid key in expired)
            {
                IDisposable obj = objects[key];
                objects.Remove(key);
                obj.Dispose();
            }
            if (objects.Count == 0)
            {
                gcThread.Change(Timeout.Infinite, Timeout.Infinite);
                Interlocked.Exchange(ref gcThread, null);
            }
        }
    }
}

To use, just inherit from ShortLived, and call KeepAlive whenever the object is accessed. Don't store references to your object; store a reference to its Key property, and use that to look it up when you want it again.

WCF Services alongside legacy ASMX

2007-12-05T15:27:00.000+02:00

I guess the situation I found myself in recently is pretty common. So we have an intranet application which uses .NET 1.1 webservices hosted in IIS 6. Now we want to add a new batch of WCF services, without clobbering the environment too much. Transaction support is definitely a must, and obviously we need to be able to determine the client's credentials. There's a lot of legacy code that refers to HttpContext.Current.User.Identity.Name. Sounds familiar?

Right, so the first thing you discover is that only http bindings are supported when the service is hosted in IIS. So you try basicHttpBinding, but that doesn't support transactions. That leaves wsHttpBinding. Now you get an error along the lines of "the service requires Anonymous access but it is not configured in IIS..." After a bit of digging, you find out that IIS is set up for NTLM authentication only. wsHttpBinding supports this, but only at a transport level. Which means https. So you hack up a certificate for the server and get everything working, but... now transactions can't be flowed because the DTC doesn't have WS-AtomicTransaction support enabled. More research, and it looks like getting this to happen involves a certificate for every single client.

It was about that time that I noticed that girl scout was 6 storeys... I mean I decided to look for alternatives. To cut a long story short, here's my web.config:

<configuration>
  <system.serviceModel>
    <serviceHostingEnvironment aspNetCompatibilityEnabled="true"/>
    <bindings>
      <customBinding>
        <binding name="iisat">
          <transactionFlow transactionProtocol="OleTransactions"/>
          <textMessageEncoding/>
          <httpTransport authenticationScheme="Ntlm"/>
        </binding>
      </customBinding>
    </bindings>
    <services>
      <service name="(censored)">
        <endpoint
          address=""
          binding="customBinding"
          bindingConfiguration="iisat"
          contract="(censored)" />
      </service>
    </services>
  </system.serviceModel>
</configuration>

You're also going to want to apply the AspNetCompatibilityRequirementsAttribute to your service implementations. That gets you access to HttpContext. Also you need to ensure that the DTC is configured for network access on all participating machines (including the clients). This is a bit of a pain, but the relevant configuration is at Control Panel/Administrative Tools/Component Services. Under Component Services/Computers, right-click on My Computer and choose properties. There's an MSDTC tab. Click "Security Configuration" and check the relevant tick boxes. Bear in mind that the client generally initiates and therefore hosts the transaction.

HTH.

WCF: Contract-level FaultContract

2007-11-29T19:35:00.000+02:00

The FaultContract attribute can only be applied to operations. This is great for requiring the programmer to be explicit when defining the contract. However, sometimes we want to indicate that every operation in a contract may raise a set of "standard" faults. It's a real pain to mark each operation with the appropriate fault contracts; furthermore it can make the contract very messy, and it's very easy to forget to specify the standard faults.

So what we really want is a contract-level fault attribute; something like "ContractFault(typeof(Message))". Or perhaps even better, a "StandardFaults" attribute.

It turns out that this can be done, but only in a way that according to the MSDN "may result in undefined behavior". Oh well. Undefined behavior, here we come.

The way to do it is to define an attribute that implements IContractBehavior. This attribute will allow us to insert behavior both when we're starting the service host, and when we're creating a client proxy. Handily, this approach also affects the WSDL metadata that is exported, so non-WCF clients can work with our services too. Here's what the StandardFaultsAttribute looks like:


[AttributeUsage(AttributeTargets.Interface, AllowMultiple = false, Inherited = true)]
public class StandardFaultsAttribute : Attribute, IContractBehavior
{
    // this is a list of our standard fault detail classes.
    static Type[] Faults = new Type[]
    {
        typeof(AuthFailure),
        typeof(UnexpectedException),
        typeof(UserFriendlyError)
    };

    public void AddBindingParameters(
        ContractDescription contractDescription,
        ServiceEndpoint endpoint,
        BindingParameterCollection bindingParameters)
    {
    }

    public void ApplyClientBehavior(
        ContractDescription contractDescription,
        ServiceEndpoint endpoint,
        ClientRuntime clientRuntime)
    {
    }

    public void ApplyDispatchBehavior(
        ContractDescription contractDescription,
        ServiceEndpoint endpoint,
        DispatchRuntime dispatchRuntime)
    {
    }

    public void Validate(
        ContractDescription contractDescription,
        ServiceEndpoint endpoint)
    {
        foreach (OperationDescription op in contractDescription.Operations)
        {
            foreach (Type fault in Faults)
            {
                op.Faults.Add(MakeFault(fault));
            }
        }
    }

    private FaultDescription MakeFault(Type detailType)
    {
        string action = detailType.Name;
        DescriptionAttribute description = (DescriptionAttribute)
            Attribute.GetCustomAttribute(detailType, typeof(DescriptionAttribute));
        if (description != null)
            action = description.Description;
        FaultDescription fd = new FaultDescription(action);
        fd.DetailType = detailType;
        fd.Name = detailType.Name;
        return fd;
    }
}

There are a couple of things to note about this implementation. First, we actually have to instantiate a new FaultDescription object for each fault on each operation. There's no way to share FaultDescriptions between operations. Second, we use the DescriptionAttribute of the detail class (if it's there) to set the Action for the fault. This gives relatively friendly FaultException<> messages. Third, the "undefined behavior" I mentioned is the act of modifying the contractDescription in the Validate method. We're technically not allowed to do that. But it works.

To use our new attribute, we simply apply it to the service contract:


[ServiceContract, StandardFaults]
public interface IMyService
{
    // ....
}

And we're done.

Some observations on good and evil

2007-10-03T17:11:00.000+02:00

Some quick notes before I forget. These issues were interesting to me once upon a time.

Good and evil can only be defined with respect to a moral philosophy. One can choose to act in accordance with a given morality (good) or against it (evil). The basis of Christian morality lies in that wonderful and wonderfully ambiguous statement "God is good". Of course, the nature of God (and therefore of goodness) is defined by church canon. To my way of thinking, this makes Christian goodness a dangerously arbitrary standard, and one that is highly susceptible to manipulation.

Is there a first-principles derivation of morality available? Kant seemed to think so. I tend to disagree. My own position is unclear, but I'm inclined towards adopting a gene-centric attitude. Societal mores develop along lines that are fitted to genetic and memetic survival. Of course, I am in a position to reason out the consequences of an action in a way that, with any luck, can overcome the limitations of impulsive or instinctive evolved morality. In other words, it sometimes makes sense to be evil. The a priori good guys don't always win. The ultimate winners redefine the concept of good.

In summary: I see morality as a problem in the domain of game theory. By Christian standards, I am quite evil. I'm fine with that.

"It's a philosophical choice", said Dr Smith.

Free games

2007-02-06T13:01:00.000+02:00

I wrote versions of Scrabble and Sudoku in C# for fun. Both have interesting AIs (computer opponents), which is why I did them. Download at http://www.informationlogistics.co.za/games.

Some thoughts on database design

2005-09-27T11:44:00.000+02:00

I occasionally like to read Joel on Software, a sensible site which ought to be required reading for software company managers. However, today I found some things in the Aardvark spec (PDF) which I strongly disagree with.

Microsoft Access has done some terrible damage to the world of database design. By far the most pernicious effect has been the AutoNumber phenomenon. I've lost track of the number of databases I've seen with foolishness like this:


 tblCountry (
   CountryID autonumber primary key,
   CountryCode char(2),
   CountryName varchar(50))

What's wrong with this design? Let me give you a hint. Here are two rows from an actual database I've worked with:


 65, 'DE', 'Germany'
 218, 'DE', 'GERMANY, FEDERAL REPUBLIC OF'

Oh dear, oh dear. A better design for this table is of course:


 tblCountry (
   CountryCode char(2) primary key,
   CountryName varchar(50))

Books have been written about designing tables, but here is a simple rule: start with the fields you need, and see if you can find a candidate key amongst them. If you can't, chances are your table doesn't need a primary key in the first place. Here's an example of such a table:


 InvoiceEntries (
   InvoiceNumber int references Invoices (InvoiceNumber),
   ItemCode char(4) references Items (ItemCode),
   Quantity int);

If you really, really need to be able to uniquely identify each tuple (row), then an autonumber PK is an option. But in general I try to avoid them at all costs. Here's the reason: autonumber PKs are meaningless. What does CountryID=143 signify? What country is that? If you can't look at the rows of your table and make sense of all the data in one reading, you're in a bad place as a DBA.

The second bit of lunacy, also attributable to Microsoft in a small way, is the habit of extending Hungarian notation to table naming. I'm talking about tables called tblCountry, tblCustomer, and so forth. Here's another real-life example of what can happen:

Let's say you start off with a table called tblCustomer. Several months or years later you discover that in fact you have two types of customers, corporate and private. At this stage you have several thousand lines of legacy code dealing with tblCustomer, so changing the table isn't really an option.

Fortunately, databases have wonderful things called views and triggers that let you deal with just such a situation in an elegant fashion. What you need to do is create two new tables, say tblCorporateCustomer and tblPrivateCustomer, and then replace tblCustomer with a view onto the union of these new tables. You write triggers to support the legacy operations of inserting, updating and deleting on tblCustomer. This enables your legacy code to continue working seamlessly while you go on to do great things with the new tables.

There's just one fly in the ointment. tblCustomer is no longer a table, it's a view. What's the new guy on the team going to make of a view called tblCustomer? Exactly what purpose does that "tbl" prefix serve? What's the point of differentiating between tables and views in the first place? Confusion to the new guy? Job security for the greybeards? You tell me.

Triggers: instead-of a superset of before?

2005-07-05T15:27:00.000+02:00

Towards the end of last year I had a flaming argument with a friend over the merits of SQL Server. One of his counterarguments was that instead of triggers are an adequate substitute for before triggers. Here's an example of why this is not the case.

Consider the following fairly typical design. We have an Travel table with a cascade update foreign key on Employees and Agents. There is a TravelEmployee table, cascade delete FK against Travel. When TravelEmployee records are deleted, a log table is updated by a trigger, using certain information from the Travel table (travel agent, for example).

The problem is that occasionally an entire Travel record is deleted, and when this happens, the corresponding TravelEmployee records are deleted, but the log table is not updated. Surprise! What's happening is that the trigger on TravelEmployee is doing something like this:


insert into TravelLog (EmployeeCode, TravelAgent, Action)
select EmployeeCode, TravelAgent, 'Cancelled' as Action
from deleted inner join Travel on deleted.TicketNumber = Travel.TicketNumber

Of course, by the time this trigger is executed, the record from Travel that contains the TravelAgent field is long gone, so the join produces nothing. The correct solution to this problem is to change the trigger to a before trigger. But SQL Server does not support this. How about an instead of trigger, then? Ah, but TravelEmployee has a cascade-update foreign key, and so SQL Server rightly refuses to create an instead of trigger on the grounds that it cannot then guarantee referential integrity.

Oops. There are a variety of less-than-satisfactory solutions to this problem. What we ended up doing was dropping the foreign key constraint on TravelEmployee and maintaining referential integrity by other means. Another possible solution is to create a new table containing deleted Travel records, which can then be used by the above trigger. There are a few more solutions, which I'm sure you can think of. All of these solutions have flaws.

It's cold comfort, but I was quite right: instead of is no substitute for before.

Using databases in C#

2005-05-13T13:54:00.000+02:00

Having used the excellent pqxx library to access PostgreSQL databases via C++, I was somewhat less than overwhelmed when confronted with the .NET framework's data access libraries. The latter are really just a trivial evolution of ADO, of which I have few fond memories. But, I remembered, any system lacking sufficiently elegant APIs needs must beget patterns. So here is a handy one to tuck into your copy-paste-coding repository. In C#:


// using System.Data.SqlClient;
SqlConnection dbCon = new SqlConnection("server=localhost;database=example;");
dbCon.Open();

lock (dbCon) using (SqlCommand cmd = dbCon.CreateCommand())
{
 cmd.CommandText = "select card_number, surname, name from employees;";
 using(SqlDataReader csr = cmd.ExecuteReader()) while (csr.Read())
 {
   employeeMap.Add(
     csr.GetInt32(0),
     new Employee(csr.GetInt32(0), csr.GetString(1), csr.GetString(2)));
 }
}

This is just about the most readable way to process a query on a long-running connection, while maintaining the necessary API contracts. First, observe that the connection object is locked for the duration of the query, thus ensuring that no more than one SqlCommand is created at a time. Furthermore, both the command and the reader are declared in using blocks, ensuring the correct disposal of both objects. And the whole thing manages to only indent twice. Not too bad.

Of course, when you start using transactions, things get more complicated, and I still haven't found a wholly satisfactory way of laying out transactional code so that it is easy to read, purposeful and not cluttered with adjunct requirements.

I wish the designers of the System.Data APIs had read Python's DB-API spec, flawed though it is. Ideally, we want a connection entity, from which we can spawn many transaction entities, which in turn allow us to maintain multiple cursors. Is that too much to ask?

AOP

2005-04-14T13:09:00.000+02:00

It occurred to me that AOP corrects many of the deficiencies in Java that cause me to consider it less robust than C++: most importantly, its inability to guarantee pre- and postconditions after the fashion of RAII. I haven't got time to do a formal comparison, so I would be interested to know anybody's opinion on the following topics:

AOP techniques that can't be represented adequately in "pure" C++ (and by adequate I mean with the same degree of isolation).

C++ techniques that can't be represented adequately in AspectJ.

Dispatch Tables in Python

2005-04-11T13:22:00.000+02:00

I mentioned dispatch tables last time around. Dispatch tables are basically dictionaries of functions. Here's a concrete if trivial example of how to use them, written in Python.


class Document:
  def __init__(self):
    self.dtable = {
      "color": Document.setColor,
      "text": Document.addText
    }
  def setColor(self, rgb):
     # set the pen colour for the document
     pass
  def addText(self, text):
     # add some text to the document
     pass
  def loadXml(self, node):
    while node:
      self.dtable[node.nodeName](self, node.data)
      node = node.nextSibling

As you can see, the loadXml function itself knows nothing about what XML nodes might occur, or what functions must handle them. This is important because it allows code to handle new nodes (or change the handling of old nodes) to be added in, say, a subclass without changing the original class at all. All that is needed is to modify the dispatch table (dtable). This is a powerful technique, and much used in Lisp and C. It has several advantages over a more conventional if-then-else approach:

It's more elegant. In most languages (with the curious exception of Java), this technique takes less lines of code and is easier to work with. Fewer lines of code equals fewer mistakes.

It's more robust. A mistake in one case won't topple the other cases. There is more separation between the handlers for various types than with some other techniques; also, the dispatch mechanism need know nothing specific about the handlers.

It's future-proof. The mapping of metadata to interpretation is independent, in most cases, of the dispatch mechanism. Changes to input data formats, for example, can be handled with minimal intrusions into existing code.

It's faster. Surprised? A dictionary look-up is ideally O(1); a chain of ifs is O(n).

Python and VB

2005-03-03T16:57:00.000+02:00

So I'm working on a project where VB.NET is mandated by the customer. The reason, presumably, is that it's dead easy to drag any poor schmuck off the streets and have him churn out VB code for minimum wage.

Leaving the short-sightedness of this aside, I'd like to talk about what a shock it was having to write VB code after having spent some glorious months in Python. To cut to the chase, and without wishing to wallow in warrantless waffle, the kind of shock that it was was this: bad.

Compare, cherished chums, certain canonical code cases:

' VB
Dim doc As new System.Xml.XmlDocument
doc.Load ("test.xml")
Dim pages As System.Xml.XmlNodeList
pages = doc.GetElementsByTagName ("page")
If pages.Count <> 1 Then
 throw new Exception ("Invalid XML")
End If
Me.images = new System.Collections.ArrayList
Dim imgentry As System.Xml.XmlNode
Dim attr As System.Xml.XmlNode
For Each imgentry In pages(0).ChildNodes
  If imgentry.Name = "image" Then
    Dim imginfo As New ImageInfo
    Me.images.Add(imginfo)
    For Each attr In imgentry.ChildNodes
      If Attr.Name = "title" Then
        imginfo.title = Attr.FirstChild.Value
      End If
      If Attr.Name = "filename" Then
        imginfo.filename = Attr.FirstChild.Value
      End If
    Next
  End If
Next


# Python
from xml.dom import minidom
doc = dom.parse("test.xml")
pages = doc.getElementsByTagName("page")
if len(pages) != 1:
  raise Exception, "Invalid XML"
self.images = []
for imgentry in pages[0].childNodes:
  if imgentry.nodeName == "image":
    imginfo = ImageInfo()
    self.images.append(imginfo)
    for attr in imgentry.childNodes:
      if attr == "title":
        imginfo.title = attr.firstChild.nodeValue
      if attr == "filename":
        imginfo.filename = attr.firstChild.nodeValue

Now, if anything, this example is a little unfair to Python. Why? Because a reasonable Python programmer would take one look at the code above, laugh his ass off and rewrite it with a dispatch table and list comprehension. The same approach would, I'm fairly confident, double the number of lines of VB code required, while halving the number of Python lines.

By the way, a dispatch table is a far more robust solution to the above problem. I might discuss them in a future post, although there's plenty of reference material out there on the web.

At any rate, it should be evident from the above that even a literal translation of VB code into Python results in substantial savings in lines of code, and therefore in time and errors. Don't even start with the dynamic-versus-static-typing debate: the fact is that the end result is largely the same in both cases, and the work required differs substantially in favour of Python. In my pragmatic little book, that's QED.

SQL Server Bad Joke Of The Day

2005-02-24T00:29:00.000+02:00

select
case cast(cast(cast('2004-11-27 13:14:00' as datetime) as float) as int)

when cast(cast(cast('2004-11-27 13:14:00' as datetime) as int) as int)

then 'happy panda'

else 'sad panda'

end

Go ahead. Try it.

I'd love to hear from anyone who doesn't think this is a serious problem.

Am I left-wing or right-wing?

2004-11-04T12:20:00.000+02:00

Well, let's see. The left favours political liberty and economic totalitarianism. The right favours economic liberty and political totalitarianism. I favour political liberty and economic liberty. I am therefore fundamentally opposed to both wings.

I stongly recommend that readers read the views of Friedrich Hayek in this regard.

A Perilous Lack of Vision

2004-06-29T10:43:00.000+02:00

Warren Buffett has made passing reference to the unenviable position mutual-fund managers find themselves in. They are constrained by the need to show short-term (often quarterly) results, which often means that they must prematurely exit from investments that, for various reasons, have taken a turn for the worse. Conversely, they must often invest in what are clearly short-term upswings in order to make up the numbers. This results in a kind of flocking behaviour ("follow the leader"), which leads to the observed erratic, manic-depressive behaviour in the stock market that has little basis in economic reality.

This type of flocking is not limited to investors, of course. Fundamentally, it represents a kind of risk-aversion that is endemic in modern capitalist systems. This is the type of thinking that goes, "I may not achieve the best results in the long term, but at least I'll never suffer a major setback in the short term. Or at any rate, I'll be no worse off than my competitors". There is nothing wrong in principle with an avoidance of risk: I myself am one of the most risk-averse people I know. However, it becomes a problem when it begins to strangle growth and the achievement of one's potential.

The Operational Efficiency Chimera

The operational efficiency of a business can be measured by dividing its overhead costs by its gross profit. The ratio can be improved either by increasing gross profitability, or by reducing overhead costs (usually by means of retrenchments -- although executives can occasionally be convinced to give up the corporate jet, too). Since increasing profitability is usually extremely difficult, it's always tempting to focus exclusively on eliminating overhead. In moderation, this is a good exercise; but cut too deeply into the fat, and you start to hit the muscle and the bone. An amusing if not entirely valid extension to this analogy is that the fat often underlies the muscle.

It is logical then that good managers should also be asking themselves: what can I do to increase gross profitability? The answer to this varies from industry to industry. In the 'commodity' industries (steel, agriculture, general manufacturing), the answer is unfortunately "not much". Improved processes are about the only thing that will do the trick. The conclusion we should draw is that these industries should invest heavily in R&D, which can be viewed as a mini-industry in itself. R&D is more like a service industry.

Services are Different

In service industries, we run into an accounting convention that I believe is detrimental to efficient management. The problem is that service industries rely almost entirely on human capital to produce the goods. However, the cost of salaries is still recorded as an overhead cost. This makes accounting sense, but it can lead to bad decisions when managers try to treat services the same way they treat manufacturing. Basically, overhead is seen as "bad", and since services typically operate at low or even zero gross marginal cost, there doesn't seem to be much improvement to be made at that end of the system. Thus managers are compelled to cut overhead, even though this 'overhead' should properly be treated as a direct cost of production!

This brings me to the crux of my argument. The manufacturing model of accounting is inappropriate for service-type industries. The salary costs of those actually performing the work are in fact a direct cost of sales. However, the problem lies not in the accounting but in the way the figures are used to make decisions: humans providing services are assets, not expenses, and they should be treated as such.

The Human Asset

The remarkable thing about these human assets is that they do not depreciate. In fact, handled correctly, your people actually increase in value. Yet we do not see nearly enough investment in human assets -- certainly not below executive level. The trend I have noticed is that positions are hired rather than people. The candidate should be able to do X, Y and Z, and that is the end of that. This is a huge mistake. A person hired to do a job that involves thinking is an investment. You do not want such a person simply to perform mandated functions: you want them to contribute to the value of your business.

This argument is eminently reasonable, and I have never heard anyone disagree with it. Yet there is a reason why this theoretical ideal is not reflected in practice. You guessed it: risk-aversion. The problem with competent people is that you come to rely upon them. This is a risk (one might reason). What if the person leaves? What if I have to compensate him in an extraordinary fashion for his extraordinary service? Far better, surely, to rely only upon fungible minimum-standard employees, and not put my business at risk?

This line of thinking is, if you'll forgive me, rubbish. It's completely wrong-headed. Yes, competent employees are a risk. But that's what business is all about! You take a risk by investing, and you reap the rewards. If you don't invest, you'll never see a return. Imagine what we would say if a steel mill owner decided only to purchase the cheapest furnaces, on the grounds that the premier models are expensive to run and expensive to rebuild if they crack. False economy? You bet.

In Conclusion...

So what am I saying? In service industries (and, importantly, service divisions of companies in other industries), people form the bulk of the assets that are missing from the balance sheet. The money that is paid to them should be viewed as a cost of production rather than overhead cost. As in any other business, the job of the managers of a service company is to select and maintain the appropriate assets for the work in question. That means finding skilled people, and nurturing them. The more competent the better! It is absolutely a false economy to try to minimize the necessary skill levels. Trying to create fungible employees is actually counter-productive. When you attempt to treat skilled employees as interchangeable units, you are harming your business.

There is vast untapped productivity in the human asset. Ability should be allowed to flourish rather than supressed into conformity. Managing a services company is extremely difficult, because it's a lot harder to measure the performance of people than it is to make similar measurements of machines. It takes a different kind of manager, certainly, and a different kind of approach.

Unfortunately, it's taking a long time for us to become aware of the fact that providing services is not like producing steel. The attempts to fit services into the old manufacturing patterns represent a perilous lack of vision. Too much attention is paid to things that are irrelevant (fixed assets), and too little is paid to things that most certainly are relevant (human capital). This is an instance of experience letting us down: we're trying to apply a pattern that we're familiar with to a new subject, and it doesn't fit. That is why service industries, and software in particular, are so inefficient and risky. We're squeezing them in the wrong places.

What if we were to start listing our employees on the balance sheet as assets? What if their salaries were written up as costs of sales?

If you're in services, do as Apple says, and "think different".

Garbage Collection, RAII, and 'using'

2004-06-21T10:14:00.000+02:00

If you fall exactly within the median range of computer programmers, then you've probably heard that "all modern languages use garbage collection", and you've taken to the idea with glee: you like the idea of having your garbage collected for you. However, you're probably also a little concerned that perhaps garbage collection is just a teensy bit less efficient than 'manual' memory management, although you'd never admit it in public (in case someone started forcing you to use malloc() and free() again).

On the other hand, you've probably not heard of RAII. If you have, you may know that it stands for "resource acquisition is initialization", although you're still not quite clear what that means. If it means anything.

If you do know about RAII, then you were probably extremely wary of the C# programming language - that is, until you discovered the 'using' statement. It's all good now, right?

To enlighten those who are at this point saying, "huh?", and to avoid (I hope!) a number of straw-man arguments, here's a brief summary of each of the terms in the title.

Garbage collection

Garbage collection is often referred to as "automatic memory management", although this can be a little misleading, as we'll see. The purpose of a garbage collector is to 'collect' blocks of memory that won't be used again and recycle them. The trick, of course, is knowing which blocks won't be used. There are several strategies for doing this, which I won't go into here. Generally, however, what happens is that allocated memory tends to stay allocated for a little longer than it otherwise would. This can lead to several efficiencies in the memory allocation scheme: most particularly, the malloc() function (or its equivalent) can be simplified enormously, as it no longer has to consider fragmentation problems. Generally speaking, this means that garbage collectors are more efficient for programs that allocate many small chunks of memory. Of course, the downside is that at any given moment, more memory is in use than is actually necessary - but then we're no longer trying to squeeze code, data and scratch space into 16k, are we?

Any actions carried out by an object at the point at which the garbage collector releases the memory allocated to it I will refer to as "finalization".

RAII

RAII is an idiom that has been floating around in C++ circles for some time now. The essential idea is that a resource (memory, file handle, database connection...) is acquired when an object is initialized (i.e. in the object's constructor). Conversely, the resource is released when the object is destroyed. The point of destruction is the interesting bit, and we'll have more on that later. However, suffice it to say that in C++, 'auto' or stack-allocated objects are destroyed as soon as they go "out of scope", meaning the variable that was bound to them when they were created is no longer accessible.

Generally when C++ programmers talk about RAII they're talking about wrapping up a resource in an object in such a way as to make certain guarantees about the way in which the resource will be used. In particular, they are used for exception safety: C++ does not have a 'finally' block. Bjarne Stroustrup explains why here.

There is an interesting twist to this story. It's possible to implement garbage collection using the RAII idiom (amongst other things). In fact, if you look here, you'll find an example of just such a thing. More commonly C++ programmers use a simpler reference-counting implementation often referred to as a "smart pointer". You can find a good one here. The good thing about a reference-counted implementation is that you can accurately predict when the object in question will be destroyed. What about cycles, you say? Consider weak references. I've found that they clarify my thinking on a number of issues revolving around ownership, lifecycles, and authority.

You can find out more about RAII here.

The 'using' statement

Here is where things get interesting. In C#, as in Java, the garbage collector makes no guarantees about when an object will be collected. C# supports 'destructors', but they're more accurately referred to as finalizers, since they are associated with the release of the memory rather than with the end of an object's useful lifecycle. This means that you can't use C# 'destructors' to manage resources as you can in C++. However, the C# team were aware of this, and they were also aware that Java-like try/finally constructs were bad: they involve a lot of typing, they make the code less readable, they detract from the purpose or intent of the code, and (and this is a big and) it's very easy for programmers -- even experienced ones -- to forget to use them.

The C# solution was to introduce a new construct: the using statement. This is essentially just shorthand for try/finally. When the scope of the using statement ends, the IDispose interface of the object that was 'used' is automatically called. This approach has two weaknesses. The first is minor: it limits the lifetime of the disposable object to the scope of a single statement. This is often not what we want. More serious, however, is the second objection: it still doesn't solve the problem of end-user forgetfulness. We're all human. We forget. We don't read the documentation. We often don't realise that we need to do anything other than just use the object we want to use. There's no safety net.

Some inferences

Here is a list of some of the things we can conclude from the preceding paragraphs:

Garbage collection frees the programmer from having to use 'free'.

Garbage collection is often more efficient than malloc/free.

Garbage collection defers the point at which the memory is released, often indefinitely.

What happens when the memory is released is called finalization.

RAII relies on the timely destruction of objects to manage resources.

RAII leads to easy-to-read ("intentional") code, with no need for try/finally or using statements.

RAII makes exception safety very much easier.

The using statement is really just try/finally in disguise.

The end-user still needs to be told when to use 'using'.

Now we can say something interesting about the relationships between these concepts. Garbage collection (GC) solves the problem of memory management, whereas RAII solves the more general problem of resource management. GC uses the concept of "finalization"; RAII uses "destruction". "Using" is an attempt to bring the elegance and power of RAII into a GC context, but it's not much better than try/finally.

The problem, finally

Ideally, we would like to use both GC and RAII. Why can't we? Well, mostly because we can't be sure when a GC finalizer will run (if at all). This rules out the use of RAII in contexts where it's most useful -- mutex locks, for example. But why should we assume that the concepts of finalization and destruction are necessarily co-dependent? Destruction is the controlled, predictable end-of-life of an object. Finalization is the unpredictable point of memory-release. Surely these concepts are orthogonal?

Let's run with this a little further. We want controlled destruction, but we want to avoid getting the end-user involved in the details of when this should happen. One way of achieving these ends (in a language like Java or Python where [almost] all variables are by-reference) is through reference-counting. Reference-counted objects are almost garbage collected (except when they're involved in cyclical reference graphs), and they allow deterministic destruction. Since RAII objects tend not to refer to anything except the resource they represent, the problem of cycles is not really a significant one. Were objects with destructors implicitly subject to reference counting, and the destructor called when the reference count dropped to zero (the memory could be freed and the finalizer called at some later point), we'd have achieved the end of having both garbage collection and RAII in a single box.

The great thing about this solution is that it allows the programmers who design libraries to ensure that their objects won't be abused [too much]. The end-user doesn't have to remember that this particular object should have Dispose() or Close() (or whatever it is) called -- he just writes what comes naturally:

string[] getConfig() {

    File f = new File("config.txt");

    return f.readLines();

}

...and all is taken care of.

There are some wonderful things that can be done once this technique is admitted; there are also some gritty technical problems that have to be overcome. But that's a discussion for later.