platform-core/documents/update.html - www.eclipse.org/eclipse - Git at Google

 <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN2">
 <html>
 <head>
 <title>Update Site Proposal</title>
     <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
     <meta http-equiv="Content-Style-Type" content="text/css">
     <link href="http://dev.eclipse.org/default_style.css" rel="stylesheet" type="text/css">
 </head>
 <body bgcolor="#ffffff">

 <table width="100%">
 <tr><td style="background:#0080C0"><b><span style="color:white">Proposal</span></b></td></tr>
 </table>
 <h1>Eclipse Update Site Proposal</h1>
 <blockquote>
   <p><b>Summary</b> <br>
     The current packaging of the Eclipse SDK as a set of large zips (one per platform)
     is a convenient one stop shop for getting up and running with the SDK. Unfortunately,
     this approach does not scale well. As more and more Eclipse projects (e.g.,
     EMF, GEF, ...) come into common use, there is increasing interest in including
     these in pre-packaged zips. Similarly, as Eclipse is used in more varied scenarios,
     so increases the demand for different packagings (smaller and larger). It
     is not feasible for eclipse.org to create, manage and distribute pre-packaged
     configurations to meet all the demands. This proposal details a structure
     which enables Eclipse component providers to contribute their features and
     plug-ins and for Eclipse consumers to efficiently select, download and install
     their components of interest.</p>
   <p>    Last Modified: November 22, 2004</p>
 </blockquote>

 <h2>Problem Definition</h2>
 <p>The base Eclipse function is currently distributed as roughly the following
    zip files (as of 3.1M3)</p>
 <ul>
   <li>RCP Binary (one per target OS/WS) (8 x ~5MB)</li>
   <li>RCP SDK (one per target OS/WS) (8 x ~18MB)</li>
   <li>Platform Binary (one per target OS/WS) (8 x ~25MB)</li>
   <li>Platform SDK (one per target OS/WS) (8 x ~56MB)</li>
   <li>JDT Binary (OS/WS independent) (~15MB)</li>
   <li>JDT SDK (OS/WS independent) (~27MB)</li>
   <li>PDE Binary (OS/WS independent) (~4MB)</li>
   <li>PDE SDK (OS/WS independent) (~5MB)</li>
   <li>Full Eclipse SDK zips including source and all doc (one per target OS/WS)
     (8 x ~88MB)</li>
   <li>assorted other zips (e.g., examples, webdav, ...)</li>
 </ul>
 <p>Eclipse tools and technology project downloads are available separately and
   come in a variety of flavours, sizes and organizations. People wanting the base
   SDK are quite comfortable with the current situation. They need only download
   the single zip and go. If they need to run on two different platforms they download
   the zip for that platform and ignore the fact that 95% of the second download
   is the same as the first.</p>
 <p>Users looking to combine the base Eclipse with additional plug-ins/features
   are somewhat less happy. They have to search over several web sites, identifying
   downloads (zips) of interest and correlating versions manually. In general,
   the suppliers of these additional zips do not make it easy for users to understand
   their offerings.</p>
 <p>The main issues with the current setup are:</p>
 <ul>
   <li>Does not promote the easy provision, acquisition and use of plug-ins from
      a variety of sources. The basic premise of Eclipse is that plug-in consumers

     can gather a set of interesting plug-ins, put them together in an Eclipse
     platform  configuration and enjoy an integrated seemless environment. Currently
     plug-in
     producers provide their plug-ins in various forms and the lack of any managed
      acquisition mechanism forces potential users to grovel around for plug-ins

     and then hope that they got the right thing.</li>
   <li>Does not scale as the scope of Eclipse increases. The current approach addresses
     the provision/acquisition problem by packaging collections of interesting
     plug-ins together in a single zip. This works for Eclipse since it is at the
     bottom of the plug-in stack. However, others providers are not so lucky and
     are challenged with questions like &quot;how many of my prerequisites should
     I include in my download?&quot;. Further, as more and more plug-ins are added
     to the Eclipse world, the size of the complete zip increases. A zip of a reasonable
     set of SDK drops from projects on eclipse.org could easily reach 150MB (i.e.,
     double the Eclipse SDK size). Following the current model, there would be
     one of these uber-zips per target OS/WS.</li>
   <li>Does not scale as the Eclipse use scenarios increase. As the range of people
      using Eclipse broadens, there will be increasing demand for alternative
     collections
     of plug-ins. These will contain either more or less than the Eclipse SDK.
     While  we can take a stab and produce some common packagings, we are are
     not likely to satisfy all/most people.</li>
   <li>Makes poor use of existing disk and bandwidth resources. As mentioned earlier,
     something like 95% of each OS/WS specific SDK zip is identical content. Further,
     from build to build, not all plug-ins change. The user doc for example tends
     to change slowly. Near the end of a milestone or release cycle we frequently
     rebuild to fix isolated problems. Rather than rebuilding just the plug-ins
     in question and providing these for download, we rebuild the whole stack and
     repackage an entire set of full zips. Subsequently, interested developers
     are forced to redownload the bulk of what they already have. This wastes diskspace
     and bandwidth for both eclipse.org and our users.</li>
   <li>Finally, it is somewhat ironic that the update/install mechanism has been
      designed in part to address exactly these kinds of concerns yet it is one

     of the few components that does not see regular use by the Eclipse team itself.
      As a result, the update/install team misses out on the regular feedback
     enjoyed
     by the other Eclipse teams. It is seemingly hard to justify putting update/install
      forward as a useful technology while not using it ourselves. </li>
 </ul>
 <h2>Scenarios</h2>
 <h3>Scenario 1: minimal install</h3>
 <p>User starts with a minimal Eclipse base install. This contains just the runtime,
   some UI elements and an Update manager UI. Based on this they want to build
   an install containing additional elements from various sources.</p>
 <ol>
   <li>download and unzip the minimal install form eclipse.org or a mirror</li>
   <li>start the new install </li>
   <li>the user is presented with an update manager UI </li>
   <li>user discovers or is presented with update sites </li>
   <li>user selects features to install (the candidates presented are appropriate
     to the current environment e.g., OS/WS/etc as well as already installed
     components)</li>
   <li>the selected features are downloaded</li>
   <li>the features are installed into the current configuration</li>
   <li>the appropriate product/application (one that was just downloaded) is started
     and the update manager GUI goes away</li>
 </ol>
 <h4>Variations</h4>
 <p>After following the above scenario, the user can manage their configuration
   either using the steps outlined in Scenario 2 or by restarting their configuration
   with an indication that the update manager UI should be started. This puts them
   back at step 3 but with all their previously installed plug-ins still installed.</p>
 <h3>Scenario 2: building on a product install</h3>
 <p>In this scenario, the user has an Eclipse product installed (possibly just
   the Eclipse SDK from a downloaded zip file or the configuration from the steps
   in scenario 1) and wants to add more function. This pretty much follows the
   normal update/install scenarios.</p>
 <ol>
   <li>user installs an eclipse based product (e.g., Eclipse SDK)</li>
   <li>at some point, while running the product, the user invokes the update manager
     UI </li>
   <li>user discovers or is presented with update sites </li>
   <li>user selects features to install (the candidates presented are appropriate
     to the current environment e.g., OS/WS/etc as well as already installed
     components)</li>
   <li>the selected features are downloaded</li>
   <li>the features are installed into the current configuration</li>
   <li>the installed function is now available to the user</li>
 </ol>
 <h3>Scenario 3: Web install</h3>
 <p>Note: This is an advanced scenario that may not be supported in the near term.</p>
 <p>This scenario is very much like scenario 1. The main difference is in how the
   initial install is obtained. Rather than manually downloading and unzipping
   the minimal install, JNLP (Webstart) is used to fetch and run the minimal install.
 </p>
 <ol>
   <li>user browses some website and clicks on a JNLP link</li>
   <li>the appropriate jars are downloaded and &quot;installed&quot; using the
     JNLP mechanism</li>
   <li>the minimal Eclipse install is started </li>
   <li>go to step 3 of scenario 1</li>
 </ol>
 <h2>Problem Solution</h2>
 <p>We propose to use the Eclipse update technology to address the described problems
    and implement the listed scenarios. Note that this new structure is a complement

 to the existing large zip packagings of Eclipse. We are not proposing to stop

 creating the familiar SDK etc zips. (we leave it to someone more brave to propose
   that!)</p>
 <h3>Update site</h3>
 <p>eclipse.org (and others) will provide update sites sporting all the relevant
   features and plug-ins. Collections of projects can cooperate and share an update
   site or each supply/support their own update site. Managing your own eliminates
   the need for some sort of shared site update mechanism. However, the user must
   be able to navigate these individual sites seamlessly (see the section on Update
   manager). </p>
 <p>There will be one update site per <em>release family</em>. A release family
   is a set of releases which form a substantial and separable effort. For example,
   Eclipse 2.0, 2.1, 3.0 and 3.1 are all different release families. The life
   of release family continues after its initial release (e.g., 3.0) to include
   maintenance releases (e.g., 3.0.1, 3.0.2, ...). Isolating release families
   to their own update sites helps maintain a separation between development going
   on for the next release family and maintenance work on the previous family.</p>
 <h3>Versioning</h3>
 <p>Update sites are not possible unless we use the Eclipse version numbering
   scheme  effectively. The main challenge is for plug-ins to have version numbers
   that
   accurately relate to their content. The essential element here is to ensure
   that if two plug-ins have the same id and version, their content is identical.
   The
   easiest
   way to do this is to use the version qualifier (i.e., the forth version element).
    For normal (not nightly) builds, the qualifier should be the version tag from
   the map file used for the build
   which generated them (e.g., 3.1.0.v20041122). The map file defines the content
    going into the build. If that does not change, then we can assume that the
   content
   of the output plug-in will not change. In this way, identical plug-ins will
   not  be duplicated on the update site and consumers will not have to download
   them
   if they already have them. For more detail on plugin versioning, see the <a href="http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-core-home/documents/plugin-versioning.html">Plug-in
   Versioning Proposal</a>.</p>
 <p>Features on the other hand are collections of plug-ins. They are also relatively
    small. Since the content of a feature is indirectly defined by the content
   of  the plug-ins included in the feature, it is hard to detect change. The
   safe bet
   here is to use the build stamp as the qualifier for feature version (e.g.,
   3.0.0.200402131200).  This has the benificial effect of clearly identifying
   the features for a particular
   build. Further, since the entire feature set is regenerated, the collection
    of features with the same version number completely defines the entire build

 output.</p>
 <p>Nightly builds do not go into an update site. There is no easy way to correlate
    the content in a nightly build (out of HEAD) with that of an integration build

   (using versioned resources). As a result, all plug-ins would be new and the
   economies  of the proposed solution would not be realized. Further, nightly
   builds are
   primarily there for the individual teams to ensure they have not adversely
   affected  other components etc. That is, they do not figure prominently in
   the scenarios
   driving this proposal.</p>
 <p>Plug-in directories currently reflect the id and version of the plug-in. With
   the addition of the map stamp, the directory names will grow by typically ~10
   chars). It is unlikely but this may push some path length limits. The use of
   the map stamp in a file/dir name may also place additional constraints the set
   of chars that can be used in a map stamp (i.e, CVS tag).</p>
 <p>Sidenote on version qualifiers: The qualifier (fourth segment) is
   not interpreted. Qualifiers are compared using lexographical sorting. As such,
   care should be
   taken to ensure that the qualifier for subsequent builds monotonically increase.
   For example, if a build type identifier is to be included in the qualifier,
   it should be appended and
   the build date used
   as the main
   body (e.g., 20040217-I and 20040213-M7). See the complementary proposal on
   version numbering for more information.</p>
 <h3>Update manager</h3>
 <p>The main challenges for the update manager are in making all of this scale
   and usable. </p>
 <ul>
   <li>the UI must support a structured (hierarchical?) view of the available features.
     This view (and in fact the underlying feature structure) must be able to span
     update sites/servers. </li>
   <li>users must be able to select some set of root features and versions and
     then click &quot;all required features&quot;. Again, this should span update
     sites/servers</li>
   <li>using these capabilities, user can then create their own uber-features/sites
     which simply reference features/sites. This becomes in effect a favourites
     list. They select their favourite root, click &quot;all required&quot; and
     get everything they need.</li>
   <li>the update manager mechanism may cache downloaded elements
      (features and plug-ins) for use in future install operations. That is, if
     you  already have the plug-in/feature, you don't need to download it again.
     The
     cache location should be (optionally) set by the user.</li>
   <li>ideally the update/install scenarios described will be possible when running
     offline providing that all required elements have been previously cached.
     That is,

     if the user has already installed the features for a build, she should be
      able to create a new install without touching a server.</li>
   <li>as an advanced (i.e., longer term) item, update manager needs a way of
     managing its cache/store -- users need a way of purging old elements. Given
     a set
     of 1000
     plugin jars
     of various
     versions,
     determining which to keep and which to delete is not possible without working
      from higher level feature groupings. In the near term, the user can delete
     the whole cache.</li>
 </ul>
 <p>Interesting side note: using the runtime's new ability to run directly out
   of jars, it is possible to have many different eclipse configurations sharing
   the same actual plug-in files from the update manager's cache/store. This will
   break some assumptions about plug-ins being together in a plug-ins directory etc
   (the update reconciler may get upset) and it introduces some challenges for
   cache clean up but it is an interesting possibility.<br>
 </p>
 <h3>PDE Build</h3>
 <p>PDE Build already updates feature.xml files with the version numbers of the
   plug-ins involved in the build. To support the version numbering scheme outlined
   above, PDE Build will be updated to </p>
 <ul>
   <li>modify the plug-in version number in each plugin.xml (i.e., add the map stamp).
     The version number of a plugin.xml is updated only if it contains &lt;_map
     stamp_&gt; as the version qualifier. This allows plug-ins to opt out of this
     update mechanism.</li>
   <li>modify the version number in the feature.xmls (i.e., add the build stamp).
     The version number of a feature.xml is updated only if it contains &lt;_map
     stamp_&gt; as the version qualifier. This allows features to opt out of this
     update mechanism. </li>
   <li>(advanced) modify the version number on &lt;import&gt; statements. Some
     plug-ins and features may specify a perfect match rule in when listing their
     dependencies. While this is not recommended, it is certainly possible. Unfortunately,
     there is a conflict if the intention was that plug-in A depend perfectly on
     the version of plug-in B with which it was built. During the build, B's version
     number will be updated according to its map stamp. Plug-in A's &lt;import&gt;
     statement for B needs to be updated to reflect this new version. While this
     modification is technically possible, it requires more complex analysis of
     the plugin.xml file. Updating the version of the plug-in itself can be done
     using simple string replacement. In this case the builder has to
     <ul>
       <li>look at each &lt;import&gt; using perfect match and wanting build-time
         map stamp substitutoin, </li>
       <li>identify the pluign being imported, </li>
       <li>discover the new version number</li>
       <li>replace the &quot;substitutution desired&quot; flag in the version number
         with the correct version number without loosing any other formatting,
         comments etc.</li>
     </ul>
   </li>
 </ul>
 <p>Note: the plugin.xml updating outlined above must be carried out on any manifest.mf
   files present in the build.</p>
 <p>PDE Build will also be updated to output directly to update site format. This
   is already largely supported but there maybe some modest work required.</p>
 <p>A possible longer term advance is to change PDE build to only build the plug-ins
   which actually changed. Since we know the map stamp of the plug-ins we want in
   the output, we can first look for those in the update site (or some cache).
   If present, there is no need to rebuild them. This will have a huge (positive)
   impact on rebuilds during release cycles.</p>
 <h2>Summary</h2>
 <p>The problem outlined above is making it hard to scale Eclipse. It is hard for
   people to use large numbers of Eclipse components (e.g., EMF, GEF, ...) and
   we are failing to put forward a good model of how others would build/stock component
   systems. The ability to do this easily is key to the overall Eclipse component
   model. The steps proposed here are a good first try at solving the problem.
   The result will not be perfect but will allow us to understand what is wrong
   and fix it. Fortunately, the bulk of the technology required exists in Eclipse
   today. Also, since we are proposing a complementary approach to the current
   download strategy, we can stage the implementation of this new mechanism both
   as we have time and as we learn more.</p>
 <p>The key steps that should be taken for 3.0 are </p>
 <ul>
   <li>create/populate an update site</li>
   <li>update PDE build to automate the stamping of plug-ins and features</li>
   <li>a modest amount of work on update manager to enable reasonable workflows
     (i.e., set a stake in the ground)</li>
 </ul>
 </body>
 </html>
	<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN2">
	<html>
	<head>
	<title>Update Site Proposal</title>
	<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
	<meta http-equiv="Content-Style-Type" content="text/css">
	<link href="http://dev.eclipse.org/default_style.css" rel="stylesheet" type="text/css">
	</head>
	<body bgcolor="#ffffff">

	<table width="100%">
	<tr><td style="background:#0080C0"><b><span style="color:white">Proposal</span></b></td></tr>
	</table>
	<h1>Eclipse Update Site Proposal</h1>
	<blockquote>
	<p><b>Summary</b> <br>
	The current packaging of the Eclipse SDK as a set of large zips (one per platform)
	is a convenient one stop shop for getting up and running with the SDK. Unfortunately,
	this approach does not scale well. As more and more Eclipse projects (e.g.,
	EMF, GEF, ...) come into common use, there is increasing interest in including
	these in pre-packaged zips. Similarly, as Eclipse is used in more varied scenarios,
	so increases the demand for different packagings (smaller and larger). It
	is not feasible for eclipse.org to create, manage and distribute pre-packaged
	configurations to meet all the demands. This proposal details a structure
	which enables Eclipse component providers to contribute their features and
	plug-ins and for Eclipse consumers to efficiently select, download and install
	their components of interest.</p>
	<p> Last Modified: November 22, 2004</p>
	</blockquote>

	<h2>Problem Definition</h2>
	<p>The base Eclipse function is currently distributed as roughly the following
	zip files (as of 3.1M3)</p>
	<ul>
	<li>RCP Binary (one per target OS/WS) (8 x ~5MB)</li>
	<li>RCP SDK (one per target OS/WS) (8 x ~18MB)</li>
	<li>Platform Binary (one per target OS/WS) (8 x ~25MB)</li>
	<li>Platform SDK (one per target OS/WS) (8 x ~56MB)</li>
	<li>JDT Binary (OS/WS independent) (~15MB)</li>
	<li>JDT SDK (OS/WS independent) (~27MB)</li>
	<li>PDE Binary (OS/WS independent) (~4MB)</li>
	<li>PDE SDK (OS/WS independent) (~5MB)</li>
	<li>Full Eclipse SDK zips including source and all doc (one per target OS/WS)
	(8 x ~88MB)</li>
	<li>assorted other zips (e.g., examples, webdav, ...)</li>
	</ul>
	<p>Eclipse tools and technology project downloads are available separately and
	come in a variety of flavours, sizes and organizations. People wanting the base
	SDK are quite comfortable with the current situation. They need only download
	the single zip and go. If they need to run on two different platforms they download
	the zip for that platform and ignore the fact that 95% of the second download
	is the same as the first.</p>
	<p>Users looking to combine the base Eclipse with additional plug-ins/features
	are somewhat less happy. They have to search over several web sites, identifying
	downloads (zips) of interest and correlating versions manually. In general,
	the suppliers of these additional zips do not make it easy for users to understand
	their offerings.</p>
	<p>The main issues with the current setup are:</p>
	<ul>
	<li>Does not promote the easy provision, acquisition and use of plug-ins from
	a variety of sources. The basic premise of Eclipse is that plug-in consumers

	can gather a set of interesting plug-ins, put them together in an Eclipse
	platform configuration and enjoy an integrated seemless environment. Currently
	plug-in
	producers provide their plug-ins in various forms and the lack of any managed
	acquisition mechanism forces potential users to grovel around for plug-ins

	and then hope that they got the right thing.</li>
	<li>Does not scale as the scope of Eclipse increases. The current approach addresses
	the provision/acquisition problem by packaging collections of interesting
	plug-ins together in a single zip. This works for Eclipse since it is at the
	bottom of the plug-in stack. However, others providers are not so lucky and
	are challenged with questions like "how many of my prerequisites should
	I include in my download?". Further, as more and more plug-ins are added
	to the Eclipse world, the size of the complete zip increases. A zip of a reasonable
	set of SDK drops from projects on eclipse.org could easily reach 150MB (i.e.,
	double the Eclipse SDK size). Following the current model, there would be
	one of these uber-zips per target OS/WS.</li>
	<li>Does not scale as the Eclipse use scenarios increase. As the range of people
	using Eclipse broadens, there will be increasing demand for alternative
	collections
	of plug-ins. These will contain either more or less than the Eclipse SDK.
	While we can take a stab and produce some common packagings, we are are
	not likely to satisfy all/most people.</li>
	<li>Makes poor use of existing disk and bandwidth resources. As mentioned earlier,
	something like 95% of each OS/WS specific SDK zip is identical content. Further,
	from build to build, not all plug-ins change. The user doc for example tends
	to change slowly. Near the end of a milestone or release cycle we frequently
	rebuild to fix isolated problems. Rather than rebuilding just the plug-ins
	in question and providing these for download, we rebuild the whole stack and
	repackage an entire set of full zips. Subsequently, interested developers
	are forced to redownload the bulk of what they already have. This wastes diskspace
	and bandwidth for both eclipse.org and our users.</li>
	<li>Finally, it is somewhat ironic that the update/install mechanism has been
	designed in part to address exactly these kinds of concerns yet it is one

	of the few components that does not see regular use by the Eclipse team itself.
	As a result, the update/install team misses out on the regular feedback
	enjoyed
	by the other Eclipse teams. It is seemingly hard to justify putting update/install
	forward as a useful technology while not using it ourselves. </li>
	</ul>
	<h2>Scenarios</h2>
	<h3>Scenario 1: minimal install</h3>
	<p>User starts with a minimal Eclipse base install. This contains just the runtime,
	some UI elements and an Update manager UI. Based on this they want to build
	an install containing additional elements from various sources.</p>
	<ol>
	<li>download and unzip the minimal install form eclipse.org or a mirror</li>
	<li>start the new install </li>
	<li>the user is presented with an update manager UI </li>
	<li>user discovers or is presented with update sites </li>
	<li>user selects features to install (the candidates presented are appropriate
	to the current environment e.g., OS/WS/etc as well as already installed
	components)</li>
	<li>the selected features are downloaded</li>
	<li>the features are installed into the current configuration</li>
	<li>the appropriate product/application (one that was just downloaded) is started
	and the update manager GUI goes away</li>
	</ol>
	<h4>Variations</h4>
	<p>After following the above scenario, the user can manage their configuration
	either using the steps outlined in Scenario 2 or by restarting their configuration
	with an indication that the update manager UI should be started. This puts them
	back at step 3 but with all their previously installed plug-ins still installed.</p>
	<h3>Scenario 2: building on a product install</h3>
	<p>In this scenario, the user has an Eclipse product installed (possibly just
	the Eclipse SDK from a downloaded zip file or the configuration from the steps
	in scenario 1) and wants to add more function. This pretty much follows the
	normal update/install scenarios.</p>
	<ol>
	<li>user installs an eclipse based product (e.g., Eclipse SDK)</li>
	<li>at some point, while running the product, the user invokes the update manager
	UI </li>
	<li>user discovers or is presented with update sites </li>
	<li>user selects features to install (the candidates presented are appropriate
	to the current environment e.g., OS/WS/etc as well as already installed
	components)</li>
	<li>the selected features are downloaded</li>
	<li>the features are installed into the current configuration</li>
	<li>the installed function is now available to the user</li>
	</ol>
	<h3>Scenario 3: Web install</h3>
	<p>Note: This is an advanced scenario that may not be supported in the near term.</p>
	<p>This scenario is very much like scenario 1. The main difference is in how the
	initial install is obtained. Rather than manually downloading and unzipping
	the minimal install, JNLP (Webstart) is used to fetch and run the minimal install.
	</p>
	<ol>
	<li>user browses some website and clicks on a JNLP link</li>
	<li>the appropriate jars are downloaded and "installed" using the
	JNLP mechanism</li>
	<li>the minimal Eclipse install is started </li>
	<li>go to step 3 of scenario 1</li>
	</ol>
	<h2>Problem Solution</h2>
	<p>We propose to use the Eclipse update technology to address the described problems
	and implement the listed scenarios. Note that this new structure is a complement

	to the existing large zip packagings of Eclipse. We are not proposing to stop

	creating the familiar SDK etc zips. (we leave it to someone more brave to propose
	that!)</p>
	<h3>Update site</h3>
	<p>eclipse.org (and others) will provide update sites sporting all the relevant
	features and plug-ins. Collections of projects can cooperate and share an update
	site or each supply/support their own update site. Managing your own eliminates
	the need for some sort of shared site update mechanism. However, the user must
	be able to navigate these individual sites seamlessly (see the section on Update
	manager). </p>
	<p>There will be one update site per <em>release family</em>. A release family
	is a set of releases which form a substantial and separable effort. For example,
	Eclipse 2.0, 2.1, 3.0 and 3.1 are all different release families. The life
	of release family continues after its initial release (e.g., 3.0) to include
	maintenance releases (e.g., 3.0.1, 3.0.2, ...). Isolating release families
	to their own update sites helps maintain a separation between development going
	on for the next release family and maintenance work on the previous family.</p>
	<h3>Versioning</h3>
	<p>Update sites are not possible unless we use the Eclipse version numbering
	scheme effectively. The main challenge is for plug-ins to have version numbers
	that
	accurately relate to their content. The essential element here is to ensure
	that if two plug-ins have the same id and version, their content is identical.
	The
	easiest
	way to do this is to use the version qualifier (i.e., the forth version element).
	For normal (not nightly) builds, the qualifier should be the version tag from
	the map file used for the build
	which generated them (e.g., 3.1.0.v20041122). The map file defines the content
	going into the build. If that does not change, then we can assume that the
	content
	of the output plug-in will not change. In this way, identical plug-ins will
	not be duplicated on the update site and consumers will not have to download
	them
	if they already have them. For more detail on plugin versioning, see the <a href="http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-core-home/documents/plugin-versioning.html">Plug-in
	Versioning Proposal</a>.</p>
	<p>Features on the other hand are collections of plug-ins. They are also relatively
	small. Since the content of a feature is indirectly defined by the content
	of the plug-ins included in the feature, it is hard to detect change. The
	safe bet
	here is to use the build stamp as the qualifier for feature version (e.g.,
	3.0.0.200402131200). This has the benificial effect of clearly identifying
	the features for a particular
	build. Further, since the entire feature set is regenerated, the collection
	of features with the same version number completely defines the entire build

	output.</p>
	<p>Nightly builds do not go into an update site. There is no easy way to correlate
	the content in a nightly build (out of HEAD) with that of an integration build

	(using versioned resources). As a result, all plug-ins would be new and the
	economies of the proposed solution would not be realized. Further, nightly
	builds are
	primarily there for the individual teams to ensure they have not adversely
	affected other components etc. That is, they do not figure prominently in
	the scenarios
	driving this proposal.</p>
	<p>Plug-in directories currently reflect the id and version of the plug-in. With
	the addition of the map stamp, the directory names will grow by typically ~10
	chars). It is unlikely but this may push some path length limits. The use of
	the map stamp in a file/dir name may also place additional constraints the set
	of chars that can be used in a map stamp (i.e, CVS tag).</p>
	<p>Sidenote on version qualifiers: The qualifier (fourth segment) is
	not interpreted. Qualifiers are compared using lexographical sorting. As such,
	care should be
	taken to ensure that the qualifier for subsequent builds monotonically increase.
	For example, if a build type identifier is to be included in the qualifier,
	it should be appended and
	the build date used
	as the main
	body (e.g., 20040217-I and 20040213-M7). See the complementary proposal on
	version numbering for more information.</p>
	<h3>Update manager</h3>
	<p>The main challenges for the update manager are in making all of this scale
	and usable. </p>
	<ul>
	<li>the UI must support a structured (hierarchical?) view of the available features.
	This view (and in fact the underlying feature structure) must be able to span
	update sites/servers. </li>
	<li>users must be able to select some set of root features and versions and
	then click "all required features". Again, this should span update
	sites/servers</li>
	<li>using these capabilities, user can then create their own uber-features/sites
	which simply reference features/sites. This becomes in effect a favourites
	list. They select their favourite root, click "all required" and
	get everything they need.</li>
	<li>the update manager mechanism may cache downloaded elements
	(features and plug-ins) for use in future install operations. That is, if
	you already have the plug-in/feature, you don't need to download it again.
	The
	cache location should be (optionally) set by the user.</li>
	<li>ideally the update/install scenarios described will be possible when running
	offline providing that all required elements have been previously cached.
	That is,

	if the user has already installed the features for a build, she should be
	able to create a new install without touching a server.</li>
	<li>as an advanced (i.e., longer term) item, update manager needs a way of
	managing its cache/store -- users need a way of purging old elements. Given
	a set
	of 1000
	plugin jars
	of various
	versions,
	determining which to keep and which to delete is not possible without working
	from higher level feature groupings. In the near term, the user can delete
	the whole cache.</li>
	</ul>
	<p>Interesting side note: using the runtime's new ability to run directly out
	of jars, it is possible to have many different eclipse configurations sharing
	the same actual plug-in files from the update manager's cache/store. This will
	break some assumptions about plug-ins being together in a plug-ins directory etc
	(the update reconciler may get upset) and it introduces some challenges for
	cache clean up but it is an interesting possibility.<br>
	</p>
	<h3>PDE Build</h3>
	<p>PDE Build already updates feature.xml files with the version numbers of the
	plug-ins involved in the build. To support the version numbering scheme outlined
	above, PDE Build will be updated to </p>
	<ul>
	<li>modify the plug-in version number in each plugin.xml (i.e., add the map stamp).
	The version number of a plugin.xml is updated only if it contains <_map
	stamp_> as the version qualifier. This allows plug-ins to opt out of this
	update mechanism.</li>
	<li>modify the version number in the feature.xmls (i.e., add the build stamp).
	The version number of a feature.xml is updated only if it contains <_map
	stamp_> as the version qualifier. This allows features to opt out of this
	update mechanism. </li>
	<li>(advanced) modify the version number on <import> statements. Some
	plug-ins and features may specify a perfect match rule in when listing their
	dependencies. While this is not recommended, it is certainly possible. Unfortunately,
	there is a conflict if the intention was that plug-in A depend perfectly on
	the version of plug-in B with which it was built. During the build, B's version
	number will be updated according to its map stamp. Plug-in A's <import>
	statement for B needs to be updated to reflect this new version. While this
	modification is technically possible, it requires more complex analysis of
	the plugin.xml file. Updating the version of the plug-in itself can be done
	using simple string replacement. In this case the builder has to
	<ul>
	<li>look at each <import> using perfect match and wanting build-time
	map stamp substitutoin, </li>
	<li>identify the pluign being imported, </li>
	<li>discover the new version number</li>
	<li>replace the "substitutution desired" flag in the version number
	with the correct version number without loosing any other formatting,
	comments etc.</li>
	</ul>
	</li>
	</ul>
	<p>Note: the plugin.xml updating outlined above must be carried out on any manifest.mf
	files present in the build.</p>
	<p>PDE Build will also be updated to output directly to update site format. This
	is already largely supported but there maybe some modest work required.</p>
	<p>A possible longer term advance is to change PDE build to only build the plug-ins
	which actually changed. Since we know the map stamp of the plug-ins we want in
	the output, we can first look for those in the update site (or some cache).
	If present, there is no need to rebuild them. This will have a huge (positive)
	impact on rebuilds during release cycles.</p>
	<h2>Summary</h2>
	<p>The problem outlined above is making it hard to scale Eclipse. It is hard for
	people to use large numbers of Eclipse components (e.g., EMF, GEF, ...) and
	we are failing to put forward a good model of how others would build/stock component
	systems. The ability to do this easily is key to the overall Eclipse component
	model. The steps proposed here are a good first try at solving the problem.
	The result will not be perfect but will allow us to understand what is wrong
	and fix it. Fortunately, the bulk of the technology required exists in Eclipse
	today. Also, since we are proposing a complementary approach to the current
	download strategy, we can stage the implementation of this new mechanism both
	as we have time and as we learn more.</p>
	<p>The key steps that should be taken for 3.0 are </p>
	<ul>
	<li>create/populate an update site</li>
	<li>update PDE build to automate the stamping of plug-ins and features</li>
	<li>a modest amount of work on update manager to enable reasonable workflows
	(i.e., set a stake in the ground)</li>
	</ul>
	</body>
	</html>