(Ab)Use of the C-word (cloud)

May 15th, 2009

The more cloud architectures gather buzz, the more I am seeing all sorts of misuse of the c-word. From describing grid computing to confusing it with SaaS, people seem to see clouds everywhere, even in the clearest of skies!

The “Palme d’Or” should probably be given to a business acquaintance who’s been able to use the word 3 times in the same 5mns presentation for 3 different meanings:

  • the Spring cloud === the Spring context
  • the ActiveMQ cloud === the ActiveMQ management of queues
  • the VM cloud === the in-memory execution of a process

You’ve got to love them clouds!

I wonder, even as I am myself piloting in the storm, how long it will be before I get really tired of the c-word!

Sphere: Related Content

Software Craftmanship Conference 2009 report

March 4th, 2009

The SC2009 was one of these conferences where you would leave with more questions than answers. And it feels good! All in all, I felt humbled by the quality of the sessions as much as that of the participants. Evidently, everyone there was very pleased to be present and you could feel it in the bustling about and around!

The conference was hosted and catered in the BBC Worldwide buildings and as an architecture nerd, I have to say I loved the building and the interior decoration; it must be a fantastic place to work in, but it is hardly the point of this post, I simply digress.

First, you might want to head to Twitter / #sc2009 to have a look at how participants and speakers lived the event in real time.

Also, a broader coverage of all the blogs and reports for the conference will certainly pop-up on Jason Gorman’s blog (some day, when he will have got round to doing it). And more specifically, you’ll find a vox populis video of participants, including that of a guy who’s just realised he took a terrible accent in front of the camera ;-)

The complete program of the conference is available on the conference’s website and, gathering from participants to other session, the content was pretty interesting too; I wonder what kind of sessions were rejected by the selection committee.

Programming in the small

Some time ago, I had the opportunity to have a conversation around a pint with Ivan Moore and Mike Hill, where we were debating and mostly agreeing on how code should be formatted. Before that conversation, I felt like an alien insisting on a 400 characters line length in eclipse in place of the 80 characters that is the general, wrong, consensus.

This session was drawing from their own experience on formatting, writing and refactoring code, and how to challenge accepted “best practices” of code writing. It was very fun to refactor the bad code examples they provided!

Specification Workshops

This session was subtitled “The missing Link for ATDD & Example-Driven Development” and showed how you would come round to actually specify acceptance test cases using examples brainstorming: gather a lot of people that have knowledge in the business rules you want to specify and have them give examples of the rules. Discuss.

Definitely something I would try, but I’ll probably read Gojko Adzic’s book before to gain more insight on how to do it and not confuse it with iteration planning…

Responsibility-driven Design with Mock Objects

This session, however fun, didn’t quite cut it for me. First the facilitators started by “pitying” people who don’t do TDD; that could be a whole separate post in itself, but that kind of statement kinda makes me boil inside… and I am not even talking about all the Java/.Net snickering!

Plus, we did only scratch the surface of the purpose of the session, that is, learning about responsibility-driven design; we mocked objects alright, but we spent most of the time refactoring the code we had written…

However, I learned a lot from this session: they used Ruby as the supporting language for the demonstration… and I have to say I was rather impressed with a few things they did with it! Should I really, really, learn Ruby though?

Empirical Experiences of Refactoring in Open Source

In this session, Steve Counsel explained how he has analysed a few open-source projects to see how refactorings were applied throughout a project lifecycle. Although very interesting it was somewhat predictable: the most complex refactorings (e.g. remove duplication, extract superclass…) are less implemented than the simpler ones (e.g. rename variable, rename method…) by a factor of 10!

Thing is, even though this research stroke a chord with me as I am very keen on refactorings (even automated refactoring), it seemed like few conclusions were actually drawn from it; Steve did not tell us whether he had controlled for feature additions or development team turnover… furthermore, because the case studied were open source projects, I am not sure how much it would apply to corporate systems or commercial products. I guess I will have to find out!

Test-Driven Development of Asynchronous Systems

This session was possibly the most underwhelming one of the day; partly because it was the last one, mostly because it didn’t stimulate as much thoughts as the other ones.

Granted, Nat Pryce has implemented a very clever way of testing asynchronous processes during the integration tests (basically, having process probes that can tell whether the system has reached a stable position before executing test assertions)! I think if I ever face this issue, I would be very likely to use the same approach; or purchase a product that would do the trick! Wink-wink… :)

The take-home lesson learned

As I said before, I met a lot of very interesting people who care about software as much as I do, and the overall experience made me buzzing with thoughts.

But the one thing that I got out of it and that I will definitely be even more stringent about, from now on, is: refactor, refactor, refactor!

Sphere: Related Content

How not to present

January 20th, 2009

I am a fan of Garr Reynolds’ Presentation Zen for delivering good presentations to audiences, and I wish more people were, but this piece from McSweeney’s blog pretty much sums up any corporate presentation I have seen this far… in style!

Sphere: Related Content

Get your place to a FREE software development conference.

December 2nd, 2008

OK, it’s probably a bit pushy to highlight the fact that the Software Craftmanship 2009 conference is free, but it’s the best I can do in terms of attention-grabbing headlines.

Now that you have actually started to read, I can go on and say that if you’re developer worth any of their salt and you can make it to London on 26th February, this is THE conference to attend to. You’ll get a chance to improve your craft by learning from the masters (as the conference is chaired by Jason Gorman, you can be pretty sure the talks will be very exciting). Plus, my guess is you will even be able to show off your skills if you want to!

What more, you can even submit your own session proposal… so go on, make yourself heard! Or simply register, there won’t be enough seats for everyone…

Sphere: Related Content

Developers performance and random variation

September 8th, 2008

Is it possible to rank the relative performance of developers in a team without falling foul of random variation?
Ben Goldacre wrote a very interesting piece (as usual) in The Guardian this week-end (Sat 6 September edition) about the silliness of national “studies” which failed to take a vetted statistical sample before coming to any conclusion. You can read the piece on Ben Goldacre’s blog here.
However, this is most interesting when we discover random variation at work in other domains than research and it stroke me as particularly applicable to evaluating developers performance in a team.

Individual performance

In order to evaluate performance, we must define measure points in all the work produced by the individual. These metrics will include quantity-based indicators like the number of lines of code written, the number of bugs solved, or the number of bugs introduced (for negative performance) but they also would need to account for quality-based indicators as percentage of code written that is covered by tests, package and class cohesion or number of dependencies introduced.
The problem with the latter set of metrics is that it is very difficult to predict what impact on the overall system quality they will have. Nonetheless, the practice is to keep them as sweet as possible.

Individual performance, therefore can be measured absolutely from the code produced and can be compared to previous performance for the same individual. However, is it enough to use these metrics? How has the developer coped with the work that he had been assigned? Has he been able to complete the task in the most efficient way?

Feature-relative performance

Given 2 new features A and B to develop (or say bugs A and B to fix) with respective complexity Ca and Cb, how does an individual developer implement the code required to support them?
In an ideal world, every developer would code the same way on Monday at 9am, that he is on Wednesday at 12.30pm or Friday at 5pm… but there is not a single way to solve a problem, there is not a single algorithm that sorts a table, nor is there a single spelling convention for naming methods, and developers will choose the one that suits their state of mind at the very moment they need to implement it; and that doesn’t even encompass the boredom parameter which will see developers implement the famous Hello World! in every fashion possible just to make sure they keep themselves entertained!

With each feature having a different impact on the system and the developer being equally likely to implement the same feature in two different ways at different times, what is the value of those metrics when comparing them feature by feature?
And what is the value of comparing them between developers?

Team-relative performance

And here we introduce random variation: when you are trying to evaluate the relative performance of members of a team, how can you make sure you have comparable metrics to compare them with?
Let’s see with an example where we consider 2 developers of same competence that we want to evaluate against each other:

Iteration #1
Developer A has developed features F1 and F2 of complexity C1 and C2 and has achieved metrics M1 and M2. Developer B has developed features F3 and F4 of respective complexities C3 and C4. Because they are of same competence, we can assume that they have been allocated equivalent tasks and that C1+C2 ≈ C3+C4.

Iteration #2
Developer A is given to develop features F5 and F6 (complexities C5 and C6) and to fix bug B1 (Cb1) found on F3; he completes F5 and fixes B1, but doesn’t complete F6 by himself. Developer B completes features F7 and F8 (C7 and C8), fixes bugs B2 and B3 found in F4 (Cb2 and Cb3) and helps developer A complete F6.
Again, the tasks have been allocated on the assumption that C5+C5+Cb1 ≈ C7+C8+Cb2+Cb3.

Notice how I didn’t mention metrics in Iteration #2; that is where the problems actually begin. While we can easily measure our metrics for F5, F7 and F8, and allocate them to the performance measurement of each developer, there is an issue with F6 to determine how to share the performance between the developers.
Moreover the bugs solves incur new metrics being calculated on F3 and F4 as composite metrics of the pre-existing code with the fix code; therefore M3′=M3+ΔMb1 and M4′=M4+ΔMb2+ΔMb3.
It is likely at this point that we want to allocate only the delta performance to each developers for bugs. We probably want to remove this delta to the previously allocated metric for the feature.

We can also note that at the end of the Iteration #2, we could either conclude that the complexity estimation for the tasks was wrong or that developer B is more skilled than developer A.
By experience, I can safely say that neither conclusion can be drawn automatically.

As a wrap-up of this small theoretical example, we try to determine the composite performance of each developer:

  • Developer A: Mx = M1+M2+M5+∂M6+ΔMb1
  • Developer B: My = M3+M4+∂M6+ΔMb1-ΔMb1+ΔMb2-ΔMb2
    My = M3+M4+∂M6

With the performance adjustments (negative deltas due to bugs), it becomes clear that the conclusions that we could possibly have drawn after Iteration #2 cannot be drawn with any level of confidence without looking at the bigger picture.

Normalisation of the random variation

The previous example shows that if we want to be able to measure relative performance accurately, we need to find ways to normalise our input performance data. In the example, we used negative deltas of performance to impact an individual’s performance over subsequent iterations. We could also sample our metrics on features with a given complexity. Evidently, the best possible measurement would be to allocate the same set of features to each developer, but that would be counter-productive and difficult to implement in a real-world project.

Finally, we need to acknowledge that the complexity estimation for a given feature might be substantially wrong and that solving a bug for that feature could uncover a whole new range of complexity, rendering ineffective our negative delta adjustment; we therefore would need to mitigate errors and changes in complexity in our performance evaluation.

There is probably more to it than that, and I happily welcome comments and discussion. How do you measure your team’s performance?

Sphere: Related Content

Agile? Of course we are agile!

May 23rd, 2008

It seems that all companies these days are all doing “agile”. Here’s a pot-pourri collected recently from a few interviews and personnal experiences…

Of course we are agile…

  • we are doing pair programming; that is, as long as each of the programmers work on their own computer… you know we wouldn’t want you to look like you’re doing nothing
  • we are doing stand-up meetings… around the coffee machine… before the real meetings!
  • we write user stories; well, we don’t really have the time to write them, so we’ll just use the title in a big MS Project document and talk to you about what you should be doing
  • we use dynamic languages: Javascript, Flash…
  • look, we must be agile, we are even using open source libraries; as a matter of fact, we use as many open source libraries as possible to show that we are willing to leverage as much existing code as possible - we even have our own architect who tells you which library you will use!
  • Sometimes, I wish they would proudly say “Agile? Nay, we do waterfall here!“, at least it would be honest… :)

    Sphere: Related Content

The case against Inversion of Control

April 3rd, 2008

Inversion of Control is a refactoring that rose to fame with the implementations of the likes of Spring or PicoContainer. This is a quite misunderstood feature of software design (notably its implications), but a rather often used one - mainly through the use of the above cited frameworks. This article is an attempt at debunking the myth and presenting you with the hard, crude reality.

Not a pattern, more a refactoring

Of course, I would love to be able to antagonise refactorings and patterns, but it is not that simple and the separation between the two is easily crossed.

The refactoring we are using with IoC comes along the following scenario: we have a working system, but somehow we would like to reduce dependencies between some of its components; so we decide to refactor it and feed the dependencies to the components instead of having them explicitly request them. Under this light, I have to admit that I prefer the term Dependency Injection to Inversion of Control!

To realise this refactoring, we use the Builder pattern: a component knows how to built another component so that it works correctly without having to know how to build itself.

So, now we are on the same level about IoC, here is what is really bugging me with it.

Replaces compile-time validation with run-time errors

The main issue for me, is that you cannot verify your dependencies when you are actually coding. You need to run your application and deal with a hundred configuration errors before you can even start testing your feature.

Because all the wiring of your classes happen at runtime, you will not know until you are actually using a feature if it is correctly configured or not (with all its dependencies, ad infinitum). But this shouldn’t really scare you because you have tests for everything in your system, don’t you?

Introduces more indirection and delegation

To create an efficient IoC refactoring, you need to break down your dependency into an interface (depend on things less likely to change) and it’s implementation. That’s one level of indirection.

Now when you actually configure your dependency in you XML file, you don’t use actual objects, but names… text… that’s another level of indirection!

In order to manage all this indirection, your IoC container will use intermediary objects, proxies, to actually wire your classes together.

And that’s were it hurts a second time: when debugging, you have to traverse layers upon layers of classes and methods to understand where things go wrong! And don’t even get me started on actually trying to follow the path of a piece of code at development time!!

Hides dependencies but doesn’t reduce them

After all this meddling with your class’ instances, said class remains dependent on other objects but you have now lost aggregation dependency in the midst of the framework; that is, you don’t really know any more which dependencies the class needs to do its job (e.g. a data access objects) and which are just here for delegating (e.g. listeners).

Worse, if you follow the “best practices” enunciated in the documentation of your IoC container, it is very likely that you have now introduced dependencies to the framework in many places of your system: annotations? interceptors? direct use of the IoC’s object factory?

Refactoring, not central piece of architecture (bis repetita)

As a conclusion, I would like to insist that IoC should really be a refactoring for specific parts of your system and that you shouldn’t try to have everything dependency-injected, that’s bad for your system, and that’s bad for your sanity!

These days, you can be dubbed an Architect (notice the capital A, as before) very easily: just move every single instanciation into a IoC container and you get this very enterprisey architecture that’s as easy to maintain as it was before, with the addition of all the indirection… it makes you look good when new developers just stare blankly at your 2Mb XML configuration file.

Nota bene: I really have to thank Jason for motivating me to write up this post that I drafted earlier in January!

Sphere: Related Content

Depending on abstractions: a bad idea?

March 4th, 2008

I have seen my share of code where everything you are dealing with is an interface. And I have seen my share of bugs in this kind of code. While it is proven practice to depend on abstraction, this is also a source of many common mistakes.

nota bene: examples use the Java syntax, but as far as I know, the issue is the same in C#, C++… (name your mainstream language)

The culprit

Consider the following interface declaration:

	public interface Executable {

		public void init(Map config);

		public String execute();


Quite easy to understand, right? Any class implementing this interface can accept initialisation parameters with the init(Map) method and will be executed with the execute() method.

Wrong! This interface doesn’t tell you anything. Essentially, it would mean exactly the same if it was written as such:

	public interface XYZ {

		public void bbb(Map x);

		public String aaaa();


The problem with abstractions

The contract is fuzzy

The interfaces’ contract is really loose:

  • parameters are unspecified: nothing prepevents the user to pass null or strange values to methods; in our example, what is the type of objects in our Map parameter (it can help if we are using Java5+ - e.g. Map - but it would not be a panacea)?
  • return is unspecified: should we expect nulls? is there a domain for the values (X, Y or Z)?
  • call order is unspecified: when you are implementing the interface, what guarantees you that init() will be called before aaaa()?

You can’t test an abstraction

I hate to state the bloody obvious, but, unless you provide an implementation (be it a mock one), the best you can test with an interface is its existence in the system!

Your IDE can’t help you

When following code in your IDE, each time you encounter an interface, you have to guess what is the most likely implementation to be provided at that moment.If anything, you could still run your application in debug mode to find out, but you might not have that luxury… I know that feeling! :)

Dealing with it

The issue is very clear with interfaces, but it is exactly the same with abstract or overridden classes!

Of course, I wouldn’t give away the abstraction capabilities of an OO language just because that language has a poor contract management.

The only way of dealing with it for the moment is to make your code depending on interfaces completely foolproof: expect exceptions to be thrown, expect parameters not to accept nulls, expect return values to be null or completely out of your domain… and try to mitigate the risk that the method execution order might not be respected (when you are implementing an interface).

The way forward would be to implement a contract language for interfaces (OCL anyone?)… as a matter of fact I kind of see a way of doing it in Java. I need to put more thought into it though, any suggestion welcome!

Sphere: Related Content

Oracle Application Server - SNAFU

February 4th, 2008

A long time ago, I had had to deploy a J2EE application on Oracle Application Server/OC4J… I can’t remember which version it was but it doesn’t really matter because the issues I had then still exist today: the bloody thing doesn’t work out of the box, deals with my class loading scheme like it deals with a bad ho’, and the administration tooling provision looks like it could have been built by our friend Neanderthal…

Server configuration

Now I might be a bit overreacting, but what interest is there in releasing an application server onto which you can barely deploy an application? I keep running into “java.lang.OutOfMemoryError: PermGen space”… at least these errors are pretty well known: they happen when the JVM can’t allocate enough space for objects that won’t be garbage collected.

They are also quite easy to circumvent by adding -XX:MaxPermSize=256m to your java command (default is 8m, and you can increase this number as much as you need to and your box - I found that 256m was usually the right figure).

To do that in Oracle Application Server you can go into the EnterpriseManager and in the Administration tab of your OC4J instance, choose “Server Properties” and add a new line with the -XX:MaxPermSize option… done!
If you have OC4J as a standalone instance, you can simply edit the bin/oc4j (or oc4j.cmd on Windows) file and add the -XX:MaxPermSize option directly after the java command.

But you would think that the Oracle guys would have read that kind of documentation (Tuning Garbage Collection with the 5.0 Java[tm] Virtual Machine) and implemented sensible parameters so that the server works as best as possible when you start it for the first time, wouldn’t you?

Class loading issues

I am dreaming that one day, some bloke will build a NNWAS (No-Nonsense Web Application Server), where you would be able to deploy you web application without having to worry about the interactions between the class loading tree of your application and that of the container you deploy it upon.

The issue I am talking about is better described in this excellent article (http://www.qos.ch/logging/classloader.jsp). In a few words, the issue arises when one of your libraries (let’s call it high-level library, because it is often a library of abstractions) is attempting to load classes dynamically from a dependent library (the low-level library, containing the implementations), but the JVM has loaded a different copy of your high-level library in a classloader that is a parent of your application’s classloader.

This is depicted in the following diagram: The Server’s High-level library gets loaded first and when it requests the loading of a class in the Low-level library, the class loading mechanism is looking up in the classloaders tree and can’t find it.

Class loading issues diagram

One solution is to add the low-level library in the shared libraries of the server (in effect, pushing it up the tree of classloaders) or removing the high-level library from the server’s installation (in effect, promoting the application’s equivalent library to the right level of classloading in order to reach the low-level library), with all the possible risks that this strategy might incur.

Bad tooling

Finally, I would like to automatically deploy my application on my OC4J container after each Maven build, after each change in the CVS and overnight…
To that effect, Oracle provides a set of Ant tasks to deploy/undeploy the webapp, publish the shared libraries, create DataSources and connection pools, and even restart the server.

However, I wonder why those tools have been developed as Ant tasks if you can’t use them within a build system: the build will fail if you attempt to undeploy a webapp without it being deployed first (same for shared libraries and connection pools), that means you can’t use the same script to install fresh as you would to repeat install!
That also means that if, for some reason (invalid web.xml, missing dependency…), your build breaks during the deployment phase, you can’t simply re-run you build script once the issue is fixed, you also have to put the server in a stable state again manually.
Nearly a deal breaker for continuous integration, but moreover, really annoying when you are in the ramp-up phase of a project!

Situation Normal: All Fu##ed Up

Sphere: Related Content

Fun with Java generics

January 31st, 2008

Playing around with Spring beans and Hibernate, combined with inheritance of my domain and service classes (not an easy ride, I can tell you), I was trying to implement a generic service to cater for CRUD operations while, at the same time, keeping Spring’s autowiring facilities at bay…

So far, no luck. But in the process of implementing my generic class, I faced a problem that took me a while to circumvent. Consider the following classes:

  class A<T> {

    // reference to an instance of the B class
    public B b;

    public void doOp() {

  class B {

    * this method takes a class as parameter
    public void callOp(Class c) {
      // …

The code that is problematic (as you might have guessed with the use of italics) is the following:


Well… it just doesn’t compile!
Apparently, there is no language feature to simply refer to the parameterized type class.

However, there’s still a solution:

(Class<T>) ((ParameterizedType) getClass().getGenericSuperclass()).getActualTypeArguments()[0];

As documented here: ParameterizedType.getActualTypeArguments()

Thought I would share the goodness…

Sphere: Related Content