What has been going on? - part 3

Today we will conclude our little series of preliminary results. After the robustness data in part 1 and the activity data in part 2, today we will show you the completeness of our participants' solutions.


For each requirement from the requirements document two randomly assigned judges evaluated a solution's implementation of that requirement and categorized it as missing, partially implemented, fully-implemented-but-flawed (e.g. failing in over 25% of the use cases or being buggy), fully implemented, and particularly-well-fully-implemented. Figure 1 shows all fully implemented requirements, i.e. omits missing and only partially implemented requirements.

Figure 1: Number of fully implemented requirements by requirement priority and team. The rightmost bar for comparison shows the possible maximum.