<!doctype html public "-//w3c//dtd html 4.0 transitional//en"> | |
<html> | |
<head> | |
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> | |
<meta name="GENERATOR" content="Mozilla/4.75 [en] (Windows NT 5.0; U) [Netscape]"> | |
<title>JDT - Notes on Workspace structure</title> | |
</head> | |
<body> | |
<h2> | |
Notes on Workspace Structure</h2> | |
Last revised 19:40 Wednesday September 12, 2001 | |
<p>In large Java development efforts, it is not uncommon for several inter-related | |
projects to be under development at the same time. For the purposes of | |
this discussion, assume teams, projects, and components align. All teams | |
are working in the same repository, sharing a set of projects via a single | |
stream; each project contains the source code for a single component. | |
<p>Our example scenario has 4 projects: P1, P2, P3, and P4, with components | |
C1 through C4, respectively. C1 is the only component that does not depend | |
on any others; C2 depends on C1; C3 depends on C1 and C2; C4 depends on | |
C2 (N.B., but not on C1 or C3). | |
<p>We assume that there is a single CVS repository that contains all 4 | |
projects in source code form. In addition, we assume that centralized builds | |
are done periodically and posted to a web server where they can be downloaded | |
as a unit. These downloads take the form of a zipped directory which includes | |
a binary jar for each component, along with corresponding source jars to | |
aid debugging. | |
<p>We assume that each developer can download builds from the web server | |
and install them on their local machine. To do their work, they set up | |
Eclipse workspaces on their local machine and load one or more projects | |
from the CVS repository. They also upgrade their workspace from time to | |
time as new builds become available. | |
<p>These assumptions are a plausible abstraction of what goes on in open | |
source projects. (It is an open question as to how closely the Eclipse | |
project with follow this work model.) | |
<h3> | |
Developing, Using, and Patching Components</h3> | |
The following things are the norm for someone actively <b>developing</b> | |
a component: | |
<ul> | |
<li> | |
source code is available for entire component</li> | |
<li> | |
source code is being browsed and changed</li> | |
<li> | |
source code is being compiled regularly</li> | |
<li> | |
source code changes are shared occasionally with other team members</li> | |
</ul> | |
At the other end of the spectrum are components that are passively <b>used</b>: | |
<ul> | |
<li> | |
component is available as pre-compiled binary library</li> | |
<li> | |
source code is available for browsing but not for editing</li> | |
<li> | |
since used component is static, no changes to share with other team members</li> | |
</ul> | |
Components that are being <b>patched</b> are one step from being a used | |
component in the direction of being a component being developed: | |
<ul> | |
<li> | |
component is available as pre-compiled binary library</li> | |
<li> | |
source code is selectively available for editing (patching)</li> | |
<li> | |
patched source code is compiled and used instead of pre-compiled binary</li> | |
<li> | |
patched source code is for local consumption (not shared with other team | |
members)</li> | |
</ul> | |
The general problem can be stated as follows: each developer needs their | |
own workspace so that they can <i>develop</i> their assigned component. | |
To to do, they will need to <i>use</i> the components that their component | |
depends on, and perhaps use some of the other components that depend on | |
theirs. In order to do their work, a developer may need to <i>patch</i> | |
a component that they would ordinarily just <i>use</i>; in some cases, | |
they might even need to actively develop a component that they would ordinarily | |
just <i>use</i>. How can the developers structure their workspaces so that | |
they retain sufficient flexibly to switch between using and patching (or | |
developing) these other components? | |
<p>The various scenarios presented below are all plausible workspace setups, | |
each with certain advantages and disadvantages: | |
<ul> | |
<li> | |
<a href="#Component plus libraries">Component plus libraries</a></li> | |
<li> | |
<a href="#Source for everything">Source for everything</a></li> | |
<li> | |
<a href="#Classpath variables to switch between libraries and sources">Classpath | |
variables to switch between libraries and sources</a></li> | |
<li> | |
<a href="#Unshared Proxy library projects">Unshared proxy library projects</a></li> | |
<li> | |
<a href="#Shared proxy library projects">Shared proxy library projects</a></li> | |
<li> | |
<a href="#Other Scenarios">Assorted Other Scenarios</a></li> | |
</ul> | |
<h3> | |
<a NAME="Component plus libraries"></a>Component plus libraries</h3> | |
<ul> | |
<li> | |
Team member's workspace contains just the project for their component.</li> | |
<li> | |
Each team members downloads latest binary build to c:\temp\build\ or some | |
such and sets classpath variable BUILD = c:\temp\build\</li> | |
<li> | |
Build classpath references individual libraries relative to BUILD classpath | |
variable.</li> | |
<li> | |
Use of classpath variable allows each developer to choose where to install | |
the build.</li> | |
<li> | |
Classpath includes libraries for all prerequisite components.</li> | |
<li> | |
Pro: simple workspace setup (load a single shared project from repository | |
and set single classpath variable).</li> | |
<li> | |
Pro: no restiction on number of jars per project.</li> | |
<li> | |
Pro: easy to upgrade to another binary build (unzip build in place, and | |
close and reopen project in workspace to force a refresh).</li> | |
<li> | |
Con: unable to view code for a component that is not a prerequisite (requires | |
loading an additional project).</li> | |
<li> | |
Con: unable to browse code for prerequiste components in proper content | |
(code completion).</li> | |
<li> | |
Con: switching to developing another component involves loading another | |
project and changing classpaths on dependent components (bad, because these | |
are shared).</li> | |
<li> | |
Con: patching only via switching to active development.</li> | |
<li> | |
Note: uses classpath variables whose value is a library jar (works in R1.0).</li> | |
<li> | |
Note: this is roughly the way PDE 1.0 works.</li> | |
</ul> | |
project P1 (shared) | |
<br> build classpath = source /P1/C1 | |
<br> output folder /P1/bin | |
<br>| | |
<br>project P2 (shared) | |
<br> build classpath = source /P2/C2; BUILD/c1.jar+BUILD/c1src.zip | |
<br> output folder /P2/bin | |
<br>| | |
<br>project P3 (shared) | |
<br> build classpath = source /P3/C3; BUILD/c1.jar+BUILD/c1src.zip; | |
BUILD/c2.jar+BUILD/c2src.zip | |
<br> output folder /P3/bin | |
<br>| | |
<br>project P4 (shared) | |
<br> build classpath = source /P4/C4; BUILD/c2.jar+BUILD/c2src.zip | |
<br> output folder /P4/bin | |
<h3> | |
<a NAME="Source for everything"></a>Source for everything</h3> | |
<ul> | |
<li> | |
Team member's workspace contains all four projects and works entirely from | |
source code.</li> | |
<li> | |
Build classpath references individual dependent projects by name.</li> | |
<li> | |
Classpath includes projects for all prerequisite components.</li> | |
<li> | |
Pro: able to browse code for all components.</li> | |
<li> | |
Pro: able to develop any component.</li> | |
<li> | |
Pro: easy to notice downstream impact of any changes made since all components | |
are present.</li> | |
<li> | |
Pro: easy to keep current (by catching up with stream in repository).</li> | |
<li> | |
Con: Very slow workspace startup and large footprint because entails recompiling | |
source code for all components.</li> | |
<li> | |
Con: Initial workspace setup involving multiple shared projects (automation | |
possible).</li> | |
<li> | |
Con: difficulty handling projects with precompiled library jars that must | |
be exported but do not exist in source (e.g., the Eclipse debugger's JDI | |
jar)</li> | |
<li> | |
Note: uses required projects (works in R1.0).</li> | |
</ul> | |
project P1 (shared) | |
<br> build classpath = source /P1/C1 | |
<br> output folder /P1/bin | |
<br>& | |
<br>project P2 (shared) | |
<br> build classpath = source /P2/C2; project P1 | |
<br> output folder /P2/bin | |
<br>& | |
<br>project P3 (shared) | |
<br> build classpath = source /P3/C3; project P1; project | |
P2 | |
<br> output folder /P3/bin | |
<br>& | |
<br>project P4 (shared) | |
<br> build classpath = source /P4/C4; project P2 | |
<br> output folder /P4/bin | |
<h3> | |
<a NAME="Classpath variables to switch between libraries and sources"></a>Classpath | |
variables to switch between libraries and sources</h3> | |
<ul> | |
<li> | |
Team member's workspace contains just the project for their component.</li> | |
<li> | |
Each team members downloads latest binary build to c:\temp\build\ or some | |
such and sets family of classpath variables, one per component:</li> | |
<ul> | |
<li> | |
P1_LIB = c:\temp\build\c1.jar</li> | |
<li> | |
P2_LIB = c:\temp\build\c2.jar</li> | |
<li> | |
P3_LIB = c:\temp\build\c3.jar</li> | |
<li> | |
P4_LIB = c:\temp\build\c4.jar</li> | |
</ul> | |
<li> | |
Build classpath references an individual component via a classpath variable.</li> | |
<li> | |
Use of classpath variable allows each developer to choose where to install | |
the build.</li> | |
<li> | |
Classpath includes entries for all prerequisite components.</li> | |
<li> | |
Pro: switch to development involves loading an additional project into | |
workspace and rebinding a single classpath variable to refer to that project. | |
For example, load project P2 and rebind P2_LIB = /P2.</li> | |
<li> | |
Pro: easy to upgrade to another binary build (unzip build in place, and | |
close and reopen project in workspace to force a refresh).</li> | |
<li> | |
Con: difficuly attaching debug source to library jar</li> | |
<li> | |
Con: only works if there is exactly one library jar per project.</li> | |
<li> | |
Con: initial workspace setup involving a single shared project and multiple | |
variable bindings (automation required).</li> | |
<li> | |
Con: classpath variable names must be agreed on across components.</li> | |
<li> | |
Con: unable to view code for a component that is not a prerequisite (requires | |
loading an additional project).</li> | |
<li> | |
Con: unable to browse code for prerequiste components in proper content | |
(code completion).</li> | |
<li> | |
Note: uses classpath variables whose value is either a project or a library | |
(in R1.0, JDT core supports these but JDT UI does not expose).</li> | |
</ul> | |
project P1 (shared) | |
<br> build classpath = source /P1/C1 | |
<br> output folder /P1/bin | |
<br>| | |
<br>project P2 (shared) | |
<br> build classpath = source /P2/C2; P1_LIB | |
<br> output folder /P2/bin | |
<br>| | |
<br>project P3 (shared) | |
<br> build classpath = source /P3/C3; P1_LIB; P2_LIB | |
<br> output folder /P3/bin | |
<br>| | |
<br>project P4 (shared) | |
<br> build classpath = source /P4/C4; P2_LIB | |
<br> output folder /P4/bin | |
<h3> | |
<a NAME="Unshared Proxy library projects"></a>Unshared Proxy library projects</h3> | |
<ul> | |
<li> | |
Team member's workspace contains a project for each component.</li> | |
<li> | |
Main project is in source and is shared via the repository.</li> | |
<li> | |
All other projects in workspace are proxy library projects (same name as | |
source project, but not shared via repository).</li> | |
<li> | |
Each team members downloads latest binary build to c:\temp\build\ or some | |
such and sets classpath variable BUILD = c:\temp\build\</li> | |
<li> | |
Build classpath exports libraries relative to BUILD classpath variable.</li> | |
<li> | |
Use of classpath variable allows each developer to choose where to install | |
the build.</li> | |
<li> | |
Build classpath references individual dependent projects by name.</li> | |
<li> | |
Pro: able to browse code for all components in context.</li> | |
<li> | |
Pro: easy to switch to another binary build.</li> | |
<li> | |
Pro: switch to patching involves adding source and output folders to non-shared | |
project, populating with selected source files, and building; no other | |
classpaths need to be changed.</li> | |
<li> | |
Pro: switch to development involves loading an additional shared project | |
into workspace over top of non-shared proxy library project; no classpaths | |
need to be changed.</li> | |
<li> | |
Pro: easy to upgrade to another binary build (unzip build in place, and | |
close and reopen project in workspace to force a refresh).</li> | |
<li> | |
Con: initial workspace setup involving multiple non-shared and shared projects | |
(automation required; e.g., preconstructed base workspace).</li> | |
<li> | |
Con: project holding component under development is lost in sea of proxy | |
library projects.</li> | |
<li> | |
Pro: no restriction on number of jars per project.</li> | |
<li> | |
Note: the exported libraries are external to the project.</li> | |
<li> | |
Note: uses library projects (proposed for R2.0).</li> | |
</ul> | |
The base workspace for developers on all teams looks like: | |
<p>project P1 (not shared) | |
<br> library project | |
<br> build classpath = export BUILD/c1.jar+BUILD/c1src.zip | |
<br>& | |
<br>project P2 (not shared) | |
<br> library project | |
<br> build classpath = export BUILD/c2.jar+BUILD/c2src.zip; | |
project P1 | |
<br>& | |
<br>project P3 (not shared) | |
<br> library project | |
<br> build classpath = export BUILD/c3.jar+BUILD/c3src.zip; | |
project P1; project P2 | |
<br>& | |
<br>project P4 (not shared) | |
<br> library project | |
<br> build classpath = export BUILD/c4.jar+BUILD/c3src.zip; | |
project P2 | |
<p>None of these projects are shared; however, they have the same names | |
as the source projects in repository. This means that any of the library | |
projects can be replaced by loading the corresponding source project from | |
the repository. (None of the other projects in the workspace need to change.) | |
<p>project P1 (shared) | |
<br> build classpath = source /P1/C1 | |
<br> output folder /P1/bin | |
<br>| | |
<br>project P2 (shared) | |
<br> build classpath = source /P2/C2; project P1 | |
<br> output folder /P2/bin | |
<br>| | |
<br>project P3 (shared) | |
<br> build classpath = source /P3/C3; project P1; project | |
P2 | |
<br> output folder /P3/bin | |
<br>| | |
<br>project P4 (shared) | |
<br> build classpath = source /P4/C4; project P2 | |
<br> output folder /P4/bin | |
<p>Regarding automation, the centralized build could create a simple XML | |
document describing the collection of proxy library projects for the workspace. | |
Given this document, a special purpose plug-in could be written that would | |
create (or modify existing) unshared proxy library projects in the workspace. | |
<p><tt><projects></tt> | |
<br><tt> <project name="P1"></tt> | |
<br><tt> <natures</tt> | |
<br><tt> | |
<nature id="org.eclipse.jdt.core.javanature"/></tt> | |
<br><tt> </natures></tt> | |
<br><tt> <libraries></tt> | |
<br><tt> | |
<classpathentry kind="var" path="BUILD/c1.jar" sourcepath="BUILD/c1src.zip" | |
export="true"/></tt> | |
<br><tt> </libraries></tt> | |
<br><tt> </project></tt> | |
<br><tt> <project name="P2"></tt> | |
<br><tt> <natures</tt> | |
<br><tt> | |
<nature id="org.eclipse.jdt.core.javanature"/></tt> | |
<br><tt> </natures></tt> | |
<br><tt> <libraries></tt> | |
<br><tt> | |
<classpathentry kind="var" path="BUILD/c2.jar" sourcepath="BUILD/c2src.zip" | |
export="true"/></tt> | |
<br><tt> | |
<classpathentry kind="project" path="/P1"/></tt> | |
<br><tt> </libraries></tt> | |
<br><tt> </project></tt> | |
<br><tt> <project name="P3"></tt> | |
<br><tt> <natures</tt> | |
<br><tt> | |
<nature id="org.eclipse.jdt.core.javanature"/></tt> | |
<br><tt> </natures></tt> | |
<br><tt> <libraries></tt> | |
<br><tt> | |
<classpathentry kind="var" path="BUILD/c3.jar" sourcepath="BUILD/c3src.zip" | |
export="true"/></tt> | |
<br><tt> | |
<classpathentry kind="project" path="/P1"/></tt> | |
<br><tt> | |
<classpathentry kind="project" path="/P2"/></tt> | |
<br><tt> </libraries></tt> | |
<br><tt> </project></tt> | |
<br><tt> <project name="P4"></tt> | |
<br><tt> <natures</tt> | |
<br><tt> | |
<nature id="org.eclipse.jdt.core.javanature"/></tt> | |
<br><tt> </natures></tt> | |
<br><tt> <libraries></tt> | |
<br><tt> | |
<classpathentry kind="var" path="BUILD/c4.jar" sourcepath="BUILD/c4src.zip" | |
export="true"/></tt> | |
<br><tt> | |
<classpathentry kind="project" path="/P2"/></tt> | |
<br><tt> </libraries></tt> | |
<br><tt> </project></tt> | |
<br><tt></projects></tt> | |
<h3> | |
<a NAME="Shared proxy library projects"></a>Shared proxy library projects</h3> | |
In the previous approach to using proxy library projects, these projects | |
were not under VCM. In this variant of it, the proxy library projects are | |
obtained from a repository as well. We will assume that the proxy library | |
projects are stored in a different repository from the source projects. | |
By assuming they're in a separate binary repository, it is easy to use | |
the same project names for both source and binary forms (doing so in the | |
same repository would required introducing non-standard binary-only and | |
source-only branches in the project version histories). | |
<ul> | |
<li> | |
Team member's workspace contains a project for each component.</li> | |
<li> | |
Main project is in source and is shared via the repository.</li> | |
<li> | |
All other projects in workspace are proxy library projects; same name as | |
source project, but in a different repository.</li> | |
<li> | |
Each proxy library project contains and exports one or more binary jar | |
libraries (plus attached source zips) equivalent to the source.</li> | |
<li> | |
Build classpath references individual dependent projects by name.</li> | |
<li> | |
Pro: able to browse code for all components.</li> | |
<li> | |
Pro: easy to switch to another binary build.</li> | |
<li> | |
Pro: switch to patching involves adding source and output folders to shared | |
proxy library project, populating with selected source files, and building; | |
no other classpaths need to be changed.</li> | |
<li> | |
Pro: switch to development involves loading shared source project into | |
workspace over top of shared proxy library project with the same name; | |
no classpaths need to be changed.</li> | |
<li> | |
Pro: easy to upgrade to another binary build (by catching up with binary | |
stream in repository).</li> | |
<li> | |
Pro: no restriction on number of jars per project.</li> | |
<li> | |
Pro: Proxy library projects are also under VCM.</li> | |
<li> | |
Con: initial workspace setup involving multiple shared projects (automation | |
possible).</li> | |
<li> | |
Con: Additional effort of building and maintained proxy library projects | |
in a separate repository (automation possible, perhaps as part of centralized | |
build process).</li> | |
<li> | |
Con: project holding component under development is lost in sea of proxy | |
library projects.</li> | |
<li> | |
Note: the exported libraries are internal to the project.</li> | |
<li> | |
Note: uses library projects (proposed for R2.0).</li> | |
</ul> | |
The base workspace for developers on all teams looks like: | |
<p>project P1 (shared via binary repository) | |
<br> library project | |
<br> build classpath = export /P1/c1.jar+BUILD/c1src.zip | |
<br>& | |
<br>project P2 (shared via binary repository) | |
<br> library project | |
<br> build classpath = export /P2/c2.jar+BUILD/c2src.zip; | |
project P1 | |
<br>& | |
<br>project P3 (shared via binary repository) | |
<br> library project | |
<br> build classpath = export /P3/c3.jar+BUILD/c3src.zip; | |
project P1; project P2 | |
<br>& | |
<br>project P4 (shared via binary repository) | |
<br> library project | |
<br> build classpath = export /P4/c4.jar+BUILD/c4src.zip; | |
project P2 | |
<p>The proxy library projects have the same names as the source projects, | |
but are stored in a separate repository. | |
<p>project P1 (shared via source repository) | |
<br> build classpath = source /P1/C1 | |
<br> output folder /P1/bin | |
<br>| | |
<br>project P2 (shared via source repository) | |
<br> build classpath = source /P2/C2; project P1 | |
<br> output folder /P2/bin | |
<br>| | |
<br>project P3 (shared via source repository) | |
<br> build classpath = source /P3/C3; project P1; project | |
P2 | |
<br> output folder /P3/bin | |
<br>| | |
<br>project P4 (shared via source repository) | |
<br> build classpath = source /P4/C4; project P2 | |
<br> output folder /P4/bin | |
<h3> | |
<a NAME="Other Scenarios"></a>Assorted Other Scenarios</h3> | |
A couple of other scenarios can be constructed using a combination of the | |
techniques employed above: | |
<ul> | |
<li> | |
Unshared proxy library projects with different project names.</li> | |
<br>Use a classpath variable per component; bind it initially to the name | |
of the proxy library project; rebind to name of the source project. | |
<ul> | |
<li> | |
Pro: Allows side-by-side proxy library project and source project.</li> | |
<li> | |
Con: Uses both classpath variables and proxy library projects.</li> | |
</ul> | |
<li> | |
Shared proxy library projects with different project names.</li> | |
<br>Same as preceding except the proxy library projects can be shared. | |
Use a classpath variable per component; bind it initially to the name of | |
the proxy library project; rebind to name of the source project. Since | |
the proxy library project and source projects have different names, there | |
are no VCM anomalies. | |
<ul> | |
<li> | |
Pro: Allows side-by-side proxy library project and source project.</li> | |
<li> | |
Pro: Proxy library projects are also under VCM.</li> | |
<li> | |
Con: Uses both classpath variables and proxy library projects.</li> | |
</ul> | |
<li> | |
Shared proxy library projects with same project names and same repository.</li> | |
<br>The proxy library projects can use the same name as the source project | |
and be maintained under VCM in the same repository as the source projects. | |
Note that this is a non-standard arrangement for a VCM project. The source | |
and binary versions of a project need to be thought of as two permanently | |
separate branches in the version history: the source versions contains | |
source code but no binaries, whereas the binary versions contains binaries | |
but no source code. The differences are also reflected in the project's | |
.classpath file. In Eclipse, this can be achieved through the use of a | |
binaries-only stream separate from the usual stream in which the source | |
is maintained. In order to avoid generating unwanted outgoing or incoming | |
changes when switching from binary to source, proceed as follows: unshare | |
from binary stream; delete from workspace; load and share from source stream. | |
<ul> | |
<li> | |
Pro: Proxy library projects are also under VCM.</li> | |
<li> | |
Pro: One repository holds everything.</li> | |
<li> | |
Con: Project has non-standard dual version history.</li> | |
<li> | |
Con: Separate binaries-only stream means separate branch in version history.</li> | |
</ul> | |
</ul> | |
<h3> | |
Discussion</h3> | |
Of the various workspace setups discussed, the "Component plus libraries" | |
approach is the most straightforward, but also the weakest. It should work | |
well in cases where developers work on exactly the component they are assigned. | |
But it is not recommended in cases where developers assigned to one component | |
would need to patch or develop another component in the same workspace. | |
<p>The "Source for everything" approach is best for developers who are | |
all involved in the joint active development of all of the components. | |
There is a limit to how large it will scale, since the cost of recompiling | |
everything from source increases method the number and size of components. | |
It is not recommended in situations where many of the components are not | |
under active development; it will be more efficient to use pre-compiled | |
binary libraries for the static components. | |
<p>Approaches involving classpath variables do not have much to recommend | |
them. They share most of the disadvantages (and none of the advantages) | |
of using proxy library projects. | |
<p>The "Proxy Library Projects" workspace setup allows each developer to | |
work on their assigned component while providing ready access to all other | |
components. The arrangement is flexible in allowing easy switching from | |
using a component to patching it (or to actively developing it). The two | |
setups outlined in detail show how the proxy binary projects can be constructed | |
from a downloadable binary build or obtained from a version-managed repository. | |
Each has there pluses and minuses. The main drawback of using unshared | |
proxy library projects is the significant task of setting up a workspace | |
in the first place. This task would have to be automated by some means; | |
it would too tedious and error prone to have each developer create a workspace | |
from scratch. For shared proxy library projects, the tedium is in creating | |
new versions of the projects in their repository. This task could be automated | |
as part of the centralized build process. | |
<h3> | |
Document History</h3> | |
15: 45 Wednesday September 5, 2001 - first version sent for comments | |
<br>13:30 Thursday September 6, 2001 - revised for first round of comments | |
<br>19:40 Wednesday September 12, 2001 - automation for unshared proxy | |
library projects | |
</body> | |
</html> |