I really care… do you?

September 28th, 2007

Do you care about software? Do you care about it being specified correctly, designed accordingly, developed qualitatively, maintained easily and that it gives the intended value to the business? Then take the pledge: icareaboutsoftware.org

Analysts, developers, database administrators, stakeholders… you are all welcome to raise the profile of this campaign!

via icareaboutsoftware.org - Be Part of the First 100

Sphere: Related Content

Observable systems

September 27th, 2007

So, you have been landed that juicy development contract: requirements, development, test, deployment, maintenance… you’ve got it all covered! But have you? Whatever method you will choose for your development, there is an area that is often overlooked when developing software: production. How do you know your system is running well? How do you know what it is doing? Or even why it is doing it? In this post, I would like to introduce a paradigm in software development methodologies: the observable system.

Your client knows nothing about maintenance

The problem with developing maintainable software is two-fold: the quality of code and the fact that you haven’t thought of it in the first place. You can’t be blamed though.

When you develop software, you focus essentially on client requirements. When you are Agile, this focus takes a new dimension: in very short iterations, you pick the client’s brain and implement straight ahead. When you are not Agile, you gather a huge mass of client requirements before trying to design and develop all of them at the same time (and often fail delivering most of the software’s intended value). In any case, the approach is to listen and transcribe.

But the story the client will tell you is one about what he cares about: making their business leaner, simpler, more competitive… (well, in fact, we wish they would all actually care about that, don’t we?)

What the client doesn’t know is that you will have to care for the newborn system. If someone is to maintain the software in production, they will need a wealth of information on the system’s behaviour; and be able to react to this behaviour.

Observe the newborn system

I would like to take this newborn analogy further: when you have your first baby, you take all the care in the world for conception and development. You care for quality of the food for the mother, her resting patterns, multiple measurements (blood samples, morphological measurements, scans, etc.)… but come the preparation for the baby coming and you can’t actually prepare specifically except by buying everything that any baby could ever be in need of; and let me tell you that’s costly! That’s the (untrained) parents take.

baby dashboard
image found via

The midwives approach is to have a set of defined measurements to track baby’s needs in the early stages: weight, eight, hours of sleep, amount of milk taken, length and frequency of feeding time… from these metrics, they decide to switch to formula, decrease the doses, add food complements or send your baby to foster care!

The observable system paradigm

Intent: make a system maintainable by allowing observation of its behaviour while in production
Motivation: systems are usually used in production as if they were black boxes, and their maintenance is operated by investigating production data and comparing to test data and systems; there is a need to gather instant meta-information on the system, its processes and its state.
Applicability: virtually every software system can benefit from this paradigm

observable system structure
click for a larger image

Collaboration: the System exposes Meta-Information and the Maintainer accesses the Meta-Information to use it to asses the state of the System
Known uses: Java’s MXBeans (JMX)

Implement your observable system

In a further post, I will show an example of observable system. In the mean time, I would very much like to hear about your own experience of observable systems: as usual, contact@<my main domain>.com

Sphere: Related Content

BCS - here I come!

September 7th, 2007

I just received fantastic news: I am now a member of the British Computer Society with the Chartered IT Professional (CITP) grade!

I can’t wait to go to their events… :)

Sphere: Related Content

Cartoonesque ProgressBar

August 10th, 2007

I second Romain Guy when he says that Progress Bars are boring: let’s part with this idea that a progress bar has to be a simple long rectangle that is half full or half empty… I am sure that we can find better paradigms, best suited for each case:

  • copy of a bunch of files: have a decreasing stack of files on the left and an increasing one on the right
  • CD burning: have a CD on fire that becomes just a pile of ashes when finished

Here’s my weak attempt at creating a Cartoon-like Progress Bar (you will need Java installed to run that), but it probably took me more time to configure the JNLP than to actually create the animation…


By the way, I greatly recommend Romain’s book Filthy Rich Clients that I have added to my Safari bookshelf and that I think I should finish to read, if you consider my attempt at animating Swing… :)

Sphere: Related Content

Evolutionary vs Extensive: Refactorbot explained

August 1st, 2007

Yesterday, I was discussing the Refactorbot with a friend of mine. Although he has been a developer for longer than me, he couldn’t grasp why the Refactorbot could not possibly test every solution and had to fall back to evolutionary design.

A bit of maths

If you want to test every possible solution for a given design, you have to test every possible number of packages to hold classes and every possible combination of those classes in the packages.

If you have done some probabilities, statistics or enumerative combinatorics, you certainly will see where I am going to with that…

I have never been very good myself at mathematics, but my guess is that the number of possible solutions to our design problem can be represented with the following formula.

This is the sum of all combinations, without repetition, of n classes in p packages for p ranging from 1 (all classes in 1 packages) to n (each class in its own package):

Equations for all solutions


Equation for all solutions, for a given number of packages and classes

(for example, for 30 classes and 5 packages, this gives us 17100720 possibilities for this case alone)

We then quickly understand that it would be impossible for a program to test each and every possible design for a sizeable system: the sheer number of possibilities would make it impossible to test in a lifetime!

Refactorbot v0.0.0.3

If you want to have fun staring at an empty console while your CPU is hitting 100%, I have created an implementation of such a refactoring method.

You can download the Refactorbot v0.0.0.3 which includes this method. Good luck!

The case for small systems

As much as my mathematical demonstration convinced me, myself and I, I had to tell you about my experiments on the design challenge (included in the test cases for the Refactorbot)…

It seems that for a small number of classes, say 8, the method is not only quick but very effective: I found the best ever solution to the design challenge!

ref model: 3 packages/8 classes (D=0.4444444444444445, R=0.8333333333333334)
-> p1 (3 classes): B A C
-> p2 (3 classes): D E F
-> p3 (2 classes): G H

new model: 3 packages/8 classes (D=0.08333333333333333, R=1.0)
-> p0 (4 classes): G C A B
-> p1 (3 classes): E H D
-> p2 (1 classes): F

However, don’t be fooled by this stunning result! It would become increasingly tedious to try and find a solution for bigger and bigger systems.

The deal with evolutionary refactorings

I don’t think that the purpose of the Refactorbot is to come up with the best design - and I will write a piece soon about a definition of the BEST design - possible, but rather with a better design.

If we consider that the design of a software is its DNA, then you can probably try small improvements and keep them or discard them in an evolutionary fashion. Better design would emerge by trial and error; the Refactorbot should do just that.

What is perfect design anyway?

Sphere: Related Content

Refactorbot v0.0.0.2: just use it!

July 27th, 2007

Following the term coined by Jason Gorman, here is a new version of the Refactorbot that you can use to test your models with:

download Zip file
(binaries + libraries + source + Eclipse workspace = 4Mb).

XMI loading capability

In this version, you can feed it an XMI file (I create mines with ArgoUML) and it will attempt to create a better model for it.

The XMI loading, however, is very - very - slow for big models… I coded it using XPath (Xalan-j precisely) and that proved so sluggish that I implemented a cache system: after the first creation of the in-memory model of your XMI file (be patient), it will create a .ser version of your XML file in the same directory and reuse it for next runs.

Because of the nature of the algorithm (using random refactorings), you may want to execute the program many times for the same model, and I can guarantee you that this cache will come quite handy!

New refactoring algorithm

I have implemented a new algorithm that changes only one class at each iteration: it will randomly select a class, randomly decide to create a new package and/or randomly choose the package to put the class in. It will then run the metrics again and keep or discard the new model based on the new metrics.

Don’t worry, the original algorithm that attempted to fathom a complete new model is still here. It is just that I thought it would be interesting to have different algorithms to play with.

Furthermore, I think that this second algorithm is closer to Jason’s initial view that the Refactorbot would do 1 random refactoring and then run all tests to prove that the system has been improved…

Using it

For you lucky windows users with JSE 1.5+ already installed, there’s a batch file in the archive that let’s you just run the application; just run:

refactorbot.bat myxmifile.xmi 1000 p

The others will have to install a version of Java greater or equal to 1.5 and launch the refactorbot.improvemetrics.ImproveMetrics class. The required libraries are provided in the lib folder.

The output is still very crude because it will only tell you the list of packages it has generated and the the classes they contain. I should produce an XMI output very soon, but that’ll wait until I learn a bit more about XMI!

Your impressions

My own results are quite encouraging: I have tried the Refactorbot with a sizeable model (177 classes in 25 packages), and although the first loading of the CSV file is slow (it has 625 associations in a 20Mb file, and that’s what takes most of the time), the improvement of the model is quite fast! Granted, it is quite easy to improve on that model (that I reverse-engineered from a project’s codebase with ArgoUML), but the insight I got was still very invaluable!

However, this is probably the first usable version of the Refactorbot, so I would like to hear from your own experience with the automatic refactoring of your own models! Send me an email at contact@<my domain here>.com, that’ll help improving on the program…

Oh and by the way, I care about software!

Sphere: Related Content

Automated Design Improvement

July 20th, 2007

Jason Gorman, again, inpired me for a new blog post. Some time ago, he offered an OO design challenge in which a design’s maintainability had to be improved upon without any knowledge of the underlying business case.
I quickly gathered that you could solely rely on metrics for determining quality level for a design and did a few trials myself (see below) to improve on the metrics[1].
Jason posted his own solution yesterday, and I suddenly remembered one of his earlier posts that suggested we should be able to automate this process. I detail such a solution further in this article and I give one possible (better?) solution to the design challenge.

Trying to find a methodology

My first attempt at making a better model was to try and see the patterns “through” the model. I moved a few classes around in ArgoUML, generated the Java classes and ran the Metrics plugin for Eclipse… alas, although the normalized distance from main sequence[1] was quite good (D=0.22), the relational cohesion (R=0.55) was worse than the reference model’s (R=0.83)!

First attempt at design challenge

In order to be able to improve models metrics with consistency, I had to devise a methodology.
I ordered the classes by their distance in the dependency graph: the more dependable, the better for maintainability. The dependency arcs are as follows:

B -> A -> C -> G
B -> A -> C
B -> A
B -> D
E -> D
E -> F
H -> D
H -> F

This prompted me to put the classes in four different packages like this:
Second attempt at design challenge
Not very different from my model created without applied methodology, but it has a great flaw: it has a cycle between p2 and p3! (and awful metrics too: D=0.40 and R=0.66)
Moving class A back to package p2 does break the cycle and improve the normalized distance, though only slightly (D=0.375).

Automating the search for a solution

At that point, I went to bed and left it running as a background thought until Jason posted his own solution to the challenge… the way he was proceeding with the refactorings reminded me of one of his earlier posts, though I can’t seem to be able to find it any more, that suggested we might be able to build a robot to perform random refactorings that would be kept if they improved the overall design of a system… if I couldn’t devise a method for solving this problem, I had better leave it to chance completely!

So I built a simple version of such a lucky robot with a very simple design, that would just pick classes from the reference model and, for each of them, decide randomly if it should create a new package or choose, still randomly, a package to put it in…
Once the model is fully built, it runs a few metrics and compare them to the reference model and, if it shows an improvement, keeps the generated model as reference model (otherwise discards it) and does another cycle.
And here is what it produced, after a few thousand cycles:

Third attempt at design challenge

It is definitely much more complex than any other model I could have come up with by hand, but it translates into nice metrics: D=0.1875 and R=1.0!

This leads me to believe that with a bit of work and time we could come up with a more advanced solution that would help designers get the best out of their design without risking to break everything…

You can download the rather crude source code if you wish to have a look at the application.

[1] see http://aopmetrics.tigris.org/metrics.html for a good explanation of a few software metrics metrics

Sphere: Related Content

Bug in the W3C HTML Validator?

July 16th, 2007

It seems that I have found a quirk in the W3C Markup Validation Service

When I validate this page (template of the Who’s Who in France website, without any content), the Validation Service passes (see the validation result).

When I validate this page (same page with home page content) it raises an error about an unclosed </div> (see the validation result)

But I can’t seem to be able to find the error! I have even tried to validate the document in an Eclipse plugin and it does indeed validate; I’m going bonkers here!

Am I that daft, or could it be that I found a bug of some kind in the HTML validation service?

Comments welcome!

Sphere: Related Content

The road to hell is paved with loose typing

June 13th, 2007

I am now (I just took a break to type this note) assessing the quality of a Java codebase using many methods, including actually trying to understand how everything is tied together. Well, it would be an euphemism to say it is convoluted .

The developers chose to have most methods accept as parameters (or return) objects, arrays of objects, 2 dimensions arrays of objects or a whole range of unspecified collections (Hashmaps, Treemaps, ArrayLists and Vectors without any consideration about their actual intended use), and they definitely didn’t use Java 5 generics!

Moreover, they can return collections containing collections of 1 element, arrays of objects where only the value at a specific index is populated or used, arrays of objects containing very different kinds of types (a String at index 0, an Object[] at index 1, null at index 2 and a String or Integer at index 3, depending on the size of the array at index 1…).

Worst of all, the Javadoc associated with the methods is either non-existent, wrong (cut and pasted from an other method), useless (the description is just the name of the parameter split into words) or completely unintelligible.

It is impossible that this code has ever been reviewed by any competent Java developer. I think the assessment I am running is actually turning into a code review.

Now I must find a way to quickly improve code quality and communicate it efficiently to the new developers team… any idea?

Sphere: Related Content

Object Oriented old rope

June 5th, 2007

I found a very good refresher article on Object-Oriented Design at DDJ that I definitely would recommend reading: Software Complexity: Bringing Order to Chaos.

It explains how we need to divide a complex system into much smaller systems (divide and rule) in order to be able to cope with the complexity, and that it is much simpler to extend a system when you have confidence in its sub-components (automated testing anyone?).

Don’t miss the sidebar for a crash-course on different design methods (I suddenly realise why I studied SADT, Merise and OO)

Oh, and by the way, you might also learn that we all have short attention spans (limited to 7 subjects)… so no boasting allowed anymore! :)

Sphere: Related Content