dev_process/eclipse-quality.php

<?php
	include($_SERVER['DOCUMENT_ROOT'] . "/eclipse.org-common/system/include-before-definitions.php");

$pageTitle 		= "Eclipse Quality";
$pageKeywords	= "development process";
$pageAuthor		= "Bjorn Freeman-Benson Nov 20/05";

	include($_SERVER['DOCUMENT_ROOT'] . "/eclipse.org-common/system/include-after-definitions.php");
?>
    <div id="maincontent">
	<div id="midcolumn">
		
<h1>Eclipse Quality</h1>
<p><i>version 017 - August 6, 2005</i></p>
<p><a href="index.php">[This is one page of the larger document.]</a></p>
<h2>Eclipse Quality</h2>

<p><i>Eclipse Quality</i> means extensible frameworks and exemplary tools
developed in an open, inclusive, and predictable process involving the entire
community. From the &quot;consumption perspective,&quot; Eclipse Quality means
good for users (exemplary tools - cool/compelling to use, indicative of what is
possible) and ready for plug-in developers (deliver usable building blocks -
with APIs). From the &quot;creation perspective,&quot; Eclipse Quality means
working with a transparent and open process, open and welcoming to participation
from technical leaders, regardless of affiliation.</p>
<p><i>Eclipse Quality </i>is not a binary statement. Rather, quality is a
spectrum and Eclipse projects are expected to, release-by-release, improve their
quality. Eclipse Quality is an evolution over the life of the project and the
evaluation of a project's quality must be appropriate to the project's maturity
level. For example, a project in the <a href="validation-phase.php">Validation Phase</a> will have more Provisional APIs and fewer Platform APIs than a project
in the <a href="implementation-phase.php">Implementation Phase</a>. Similarly, a 1.0 Release
will have fewer API clients than a 2.0 Release; and a 3.0 Release will have a
better release-to-release migration plan than a 2.0 Release; etc.</p>
<p>Note the Eclipse Quality is about both extensible frameworks and exemplary tools - great tools are important for attracting the users,
who then attract the ecosystem, that then provide members, who then contribute
resources, who then create additional valuable frameworks and tools. Neither
frameworks without users nor tools without frameworks are interesting points
along the software development spectrum.</p>
<p>Eclipse Quality goals:</p>
<ul>
  <li>Platform quality frameworks. (note that &quot;Platform quality&quot; here
    refers to high quality <a href="http://en.wikipedia.org/wiki/API">APIs</a>
    (such as <a href="http://openide.netbeans.org/nonav/tutorial/api-design.html#api">a
    module API in NetBeans</a> or a <a href="http://java.sun.com/j2se/javadoc/writingapispecs/index.html">Java
    platform API</a> or the <a href="http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winprog/winprog/windows_api_start_page.asp">Windows
    API</a>) and <i>not</i> to the Eclipse Platform project).</li>
  <li>Platform quality tools.<sup>1</sup></li>
  <li>All exemplary tools are built on platform APIs.</li>
  <li>Performance and scalability to enterprise class use.</li>
  <li>Automated tests and a quality plan that can be used by any Eclipse
    developer.<br>
    We want any Eclipse user to be able to run all the tests and verify that
    they have a working and correct installation of the frameworks and tools.</li>
  <li>A release-to-release migration plan potentially including automated tools
    for conversions. Release-to-release migration includes not only the APIs,
    but also the artifacts generated by the tools (e.g., configuration files,
    persisted tables, etc).</li>
  <li>API stability.</li>
  <li>Predictable behavior and a predictable rate of change.</li>
  <li>Demonstrated community involvement.</li>
  <li>IP due diligence as per the <a href="/org/documents/Eclipse%20IP%20Policy2003_12_03%20Final.pdf">Eclipse
    IP Policy</a> and <a href="/legal/committerguidelines.php">other</a>
    <a href="/legal/">Eclipse</a> <a href="/org/documents/main.html">processes</a>.
    (This is the only required Quality element, the rest are goals.)</li>
</ul>
<h3>APIs, Provisional or Not</h3>
<p>In an earlier draft document, <i><a href="Eclipse%20Quality%20APIs%20v2.pdf">Eclipse
Quality APIs</a></i>, a platform API was defined to have:</p>
<ul>
  <li><b>A specification.</b> A description of the cover story and the necessary
    details. Specifications should be clear about what is defined and what is
    not defined. Furthermore, specifications should indicate what areas are
    likely to change in the future (e.g., this two-valued parameter will
    probably have N-values in the future). A specification is a difficult
    document to write, and it is more than just a paragraph or two about the
    component.</li>
  <li><b>A test suite.</b></li>
  <li><b>An implementation.</b></li>
  <li><b>One or more clients. </b>Usually, just having “a client” is not
    sufficient – an Eclipse quality API client is ideally developed by a
    separate team. A client (API user) that is written by the same team that
    also creates the API implementation usually suffers from information leakage
    and thus is not a good test of the API. Even better is two or more clients,
    each developed by a separate team, all of whom communicate only through the
    API documents.</li>
  <li><b>A support promise. </b>An implicit or explicit promise about how stable
    the API will be from release to release.</li>
</ul>
<p>Note that the <a href="http://en.wikipedia.org/wiki/API">API</a> (Application
Programming Interface) is the entire public programming surface of the
framework, thus it includes not only Java classes, packages, and interfaces, but
also extension points, file formats, generated meta-data, and even objects that
are passed through to internal packages.</p>
<p>The question arises of what to do about code that is intended to become API
in a future release, but is not up to these standards in the current release. We
can consider the following cases:</p>
<div align="center">
  <center>
  <table border="1" cellspacing="0" cellpadding="2">
    <tr>
      <td>&nbsp;</td>
      <td align="center"><b>Specification</b></td>
      <td align="center"><b>Test Suite</b></td>
      <td align="center"><b>Implementation</b></td>
      <td align="center"><b>Clients</b></td>
      <td align="center"><b>Support Promise</b></td>
      <td align="center"><b>Package</b></td>
    </tr>
    <tr>
      <td><b>Platform API</b></td>
      <td align="center">yes</td>
      <td align="center">yes</td>
      <td align="center">yes</td>
      <td align="center">yes</td>
      <td align="center">yes</td>
      <td align="center">public</td>
    </tr>
    <tr>
      <td><b>Provisional</b></td>
      <td align="center">yes</td>
      <td align="center">yes</td>
      <td align="center">yes</td>
      <td align="center">yes</td>
      <td align="center"><i>not quite</i></td>
      <td align="center">public</td>
    </tr>
    <tr>
      <td><b>Incomplete</b></td>
      <td align="center"><i>incomplete</i></td>
      <td align="center"><i>incomplete</i></td>
      <td align="center">yes</td>
      <td align="center">yes</td>
      <td align="center">yes</td>
      <td align="center">public</td>
    </tr>
    <tr>
      <td><b>Experimental</b></td>
      <td align="center">yes</td>
      <td align="center">yes</td>
      <td align="center">yes</td>
      <td align="center">yes</td>
      <td align="center">?</td>
      <td align="center">internal.provisional</td>
    </tr>
    <tr>
      <td><b>Non-API</b></td>
      <td align="center">-</td>
      <td align="center">-</td>
      <td align="center">yes</td>
      <td align="center">-</td>
      <td align="center">none</td>
      <td align="center">internal</td>
    </tr>
  </table>
  </center>
</div>
<p>Provisional, Incomplete, and Experiment APIs are collectively referred to as <i>Transient
APIs</i>. The interesting cases are <i>Provisional</i>  and <i>Incomplete</i>. Under
this definition, Provisional API is real API in all senses (good specification,
good Javadoc, good tests, etc) but the project does not feel that it has had
sufficient community feedback to completely freeze the APIs yet. The <a href="/tptp"> TPTP</a>
project uses Provisional APIs for all new API introductions: the new API is
released as provisional in release X and then hardened into platform APIs in
release X+1.</p>
<p>Incomplete APIs are those where the project wrote the code before defining
the APIs and thus the documentation and/or test suite is incomplete, but the
project intends to support the API. Incomplete APIs may appear in milestone
releases (although the lack of documentation will make it difficult for the
plug-in developer community to adopt and verify those APIs), but Incomplete APIs
must not appear in releases.</p>

<hr>
<p><sup>1</sup> Admittedly, there is not a good definition for &quot;platform
quality tools&quot; yet. A future version of this document will try to clarify
this.</p>

<?php
	include($_SERVER['DOCUMENT_ROOT'] . "/eclipse.org-common/system/include-end-of-page.php");
?>