Results 2007

Results 2007

A short summary of the results of Plat_Forms 2007 and a link to the detailed result report

Below is an excerpt from chapter 16 of the results report of Plat_Forms 2007, summarizing a few of the results. The results report document contains more results besides these and many more details about the results and how we arrived at them.

Here is the excerpt:

  • Java-centric platform differences:
    • Java was the only platform for which all three solutions handle HTML tags in text field input so as to avoid client-side cross-site-scripting (XSS) security issues (see Section 7).
    • The amount of functionality implemented by the Java teams is much less uniform than for the others (see Figures 4.4 and 4.5). Also, the nature of the changes required for adding a user profile data field into the application was more dissimilar among the Java solutions than among the solutions of the other platforms (see Section 13.2.1).
    • Java team members were less often observed to perform activities of type 'talking' than were Perl team members (see Figure 5.6).
    • The design approach chosen for representing the content of the TTT questionnaire in the application tended to be more heavyweight in the Java solutions than in the others (see Section 13.2.2). Also, the directory structure of the Java solutions is more deeply nested overall than the others (see Figure 11.6).
  • Perl-centric platform differences
    • The Perl solutions are smaller than the Java and the PHP solutions (see Figure 10.7).
    • The number of changes required for adding a user profile data field into the application was smallest for Perl (see Section 13.2.1).
    • Only the Perl solutions are consistent in not having any substantial implementation of the webservice requirements (see Figure 4.6). In the postmortem questionnaire, the Perl teams also reported the highest level of frustration with the webservice requirements (due to lack of WSDL support for Perl, see Sections 14.2.1 and 14.2.3).
    • Perl was the only platform for which all three solutions' SQL handling did not resist even a simple form of manipulated HTTP requests (see Figure 7.1).
  • PHP-centric platform differences
    • The amount of functionality implemented by the PHP teams is larger than that of the Perl teams (see Figure 4.4).
    • The amount of functionality implemented by the three different PHP teams is more uniform than it is for the other platforms (see Figures 4.4 and 4.5).
    • A similar statement holds for several other criteria as well: The PHP teams are more similar to one another in those respects than are the teams from the other platforms. See for instance the fraction of implementations that have low quality from the user's point of view (Figure 6.1), the size and composition of the source distribution (visible in Figures 10.3 to 10.6, regarding various subsets and views), the number of lines of code required on average for implementing one of the requirements (Figure 10.8), or the depth of the solutions' directory structure (see Figure 11.6).
    • PHP was the only platform for which all three solutions' SQL handling properly resisted our manipulated HTTP requests (see Figure 7.1).
    • PHP was the only platform for which all three solutions performed sufficient validation of email address input during registration and for which all three solutions could fully handle international characters (see Figure 7.1).