journal of team1

For our colleagues in Geneva, but also for you bloggers, we have put together a report on what happened within team1 during the contest.

1. Journal

First we all quickly read the entire specification (20 pages!); then we joined to discuss about the global vision of the project. We designed a sketch of the URL space, designed a rough UML diagram of the data model, and chose MySQL for persistent storage. Cédric immediately started implementing the schema while Laurent did the ORM layer (description of the tables and their relationships for automatic generation of Perl classes and methods) and Jean-Christophe designed the initial Web pages.

Cédric and Jean-Christophe joined to implement registration and login while Laurent implemented the temperament test  (parsing the questions, displaying the form and analysing the results). Around 17:00 we had those two parts ready and decided to publish an initial release. Cédric deployed the code on the public server, but found out that our particular network configuration (private subnetwork behind the plat-forms proxy) had unexpected side-effects (broken links). We had no expert in sysadmin, so the problem was only solved at 18:25.

Meanwhile Jean-Christophe had prepared an HTML form for searching members and Laurent started working at the query. It seemed a good idea to let it do do all in one go on the database side, but just thinking of the proper logic for the SQL query took some time, and then implementing it through SQL::Abstract (our SQL generation layer) pushed that module to its extreme limits.

Jean-Christophe redesigned the look and feel of the application, introducing nice images and styles for the second release at 22:37. Then he started working at the graphical plot. We knew before the contest that some graphics would come up, and were prepared to generate SVG, but this turned out to be not appropriate, so the first thing to do was to choose a proper package. Well, sometimes there is too much choice on CPAN (the public Perl repository), so reading docs, installing and trying various packages took a bit of time.

Cédric spent the evening trying to do the SOAP interface, but was getting discouraged : this was much more complex than the exercises done at home before the contest. We had not enough good tools to parse the WSDL interface and generate corresponding code. Furthermore it turned out that the specification required a notion of "session", which we hadn’t planned to use. Catalyst has good ready-made plugins for sessions, but with lots of dependencies, so adding them into the architecture at that point was quite a major shift. However that was not enough to solve the SOAP issues, so finally  Cédric dropped that task and helped Jean-Christophe on the plotting task. Together they managed to produce a first graph with Graph::Plot.

We had a short rest from 05:00 to 08:00.

When coming back we announced to the customer that SOAP would not be delivered. We offered to maybe try an alternative (REST Web services or XML-RPC), but this wasn’t useful to the customer, so we dropped the whole thing.

Laurent struggled a little bit to install the new imported modules on his Windows machine (because these Perl modules were relying on C code libraries), and to fix a couple of problems. We published a new release at 10:06, solving previous bugs and answering a couple of remarks that had been posted by testers of our previous releases.

Unfortunately very short after the release, we discovered that it had a new bug due to interaction between some modules in our assembly that had never been put together before. Laurent tried to understand what was going on by running the development server through the Perl debugger, but without success. After having lost a lot of time debugging, we finally took another path which was actually put together in short time and should have been done much earlier.

Meanwhile Jean-Christophe had progressed on displaying member lists and plotting them on a chart, with dynamic change of the plotting axis, and Cédric had produced a nice script to populate the database with fake data, so that we could test the display with realistic numbers of records.

In the last minute, Laurent added some code for sending "requests for contact details" to the displayed members.  Then, without any tests, it was time to wrap up. Cédric very efficiently deployed the final release and went through all steps for delivering the DVD, finishing just two minutes before the deadline.

After it was over we felt quite happy about what had been constructed and about the experience. However we were also frustrated of having delivered something below our initial quality expectations.


2. Things we are happy about

·         The team. We formed an excellent team : 3 people with complementary skills and specialized for complementary tasks, yet with enough common culture to have a good understanding of what everybody is doing. We didn't need any detailed task assignments : in two words it was clear who would do what, and how. Collaboration always remained harmonious, even under stress and tiredness.

·         The process. Let’s call it "develop by intuition". From our previous work together, we had a pretty clear vision about the architecture. So we did very little planning, and just talked from time to time about what needs to be done next, thus progressing in informal iterations of a few hours each.

·         The languages. The expressiveness of Perl did wonders, as usual, especially in its multi-paradigm features : object-oriented structure for the application, functional idioms for some parts of the algorithms, and imperative for the helper scripts. The template language of TT2 was excellent for massaging data into Web pages.

·         The development tools. With Vim or Xemacs as powerful editors, with the Catalyst HTTP Engine running as development server on each machine, and with Subversion for sharing and archiving the code, we had a highly flexible and reactive environment. The balance between individual developments and merges into the common pool went very smoothly.

·         The framework. Catalyst brought a rich structure to start with, and a natural skeleton for structuring our code.

·         The feedback. We got several useful comments and suggestions because of our intermediate releases.


3. Lessons learned

·         priorization. Because we were quite confident of doing the whole thing, we did not care too much about priorizing the goals. Seing it retrospectively, that was a big mistake, because we ended up with an application that implements several low-priority “should” and “may” requirements, while some important “must” requirements are still missing.

·         preparation. Even if we didn’t have too much time to prepare for the context, several things could have been anticipated better :

o        running the production server as a Linux vmware image inside a Windows laptop was a bit risky, and generated some configuration problems.

o        using the same server for public releases and for our internal tools (Subversion, phpMyAdmin) was not very wise.

o        having heterogeneous development machines was risky too, especially knowing that some modules do not install smoothly on Windows.

o        preparing some pre-packaged Web templates (graphics, css) would have gained time

o        we expected that some SOAP and graphics would come up at the contest, and did a few exercises on those topics, but not sufficiently broad and deep.

·         facing problems honestly. In some occasions, when confronted to a problem, it would have been better to ask for help earlier, or to recognize that the problem was hard and try to find another path, rather than insisting on solving it. The ego kicks in and we want to prove that we can master things, but meanwhile the clock is running…

·         over-optimistic estimations. Everybody knows that developers always announce wrong estimations … but even knowing it, we still fell into the trap!