<!doctype html public "-//w3c//dtd html 4.0 transitional//en"> | |
<html> | |
<head> | |
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> | |
<meta name="Author" content="Eclipse Project"> | |
<title>Flexible Project Structure</title> | |
<link rel="stylesheet" href="../../default_style.css" type="text/css"> | |
</head> | |
<body bgcolor="#FFFFFF" > | |
<div align="left"> | |
<h1>Flexible Project Structure</h1> | |
</div> | |
<blockquote> | |
<p><b>Summary</b> <br> | |
Eclipse requires that the contents of each project be stored in a single directory on disk. | |
Every file and folder in that directory tree on disk must belong to the project in the workspace. | |
This restriction makes it difficult to use Eclipse in conjunction with tools that have specific layout | |
requirements for their files, or with users who have legacy file base layouts that they need | |
to maintain. We would like to improve the situation for the 2.1 release of Eclipse. This document | |
is the third of a trilogy. The first described the | |
<a href="http://www.eclipse.org/eclipse/development/inflexible-projects-problem.html"> | |
nature of the problem</a>, and the second mapped out the various | |
<a href="http://www.eclipse.org/eclipse/development/towards-more-flexible-projects.html"> | |
options for addressing it</a>. This third document describes the concrete proposed solution | |
in detail.<br><br> | |
Last Modified: 18:30 October 24, 2002</p> | |
</blockquote> | |
<hr/> | |
<h2>Overview</h2> | |
<p> | |
Analysis of the problem identified three characteristics of Eclipse projects that made them | |
difficult to use in conjunction with existing project layouts. Namely, Eclipse projects are | |
currently <b>single-rooted</b>, <b>all-inclusive</b>, and <b>non-overlapping</b>. | |
The proposed solution includes a new mechanism in the Platform core for allowing projects | |
to be multi-rooted, a new JDT facility for interacting with a selective subset of a project, and a relaxation | |
of the restriction that resources cannot overlap in the file system. For readers of the previous | |
document that mapped out the solution space, these are solutions P-2, Q-1, M-2, S-2, V-1 | |
and V-2. | |
</p> | |
<p> | |
Relaxing these restrictions is necessarily a breaking change for many tools built on the Eclipse platform. | |
Some plug-ins may be able to adapt to the new behavior in future releases, but some may never | |
work in the presence of projects that relax these layout restrictions. To minimize the impact, the | |
proposed solution adheres to the following principles: | |
<ul> | |
<li>Only expose the new functionality in the contexts where flexibility problems have been reported. | |
For example, users will be able to change the location of JDT source folders and the output folder, | |
but not be able to change the location of folders at arbitrary depths. Exposure of the new flexibility can | |
be widened to fit new contexts where compelling use cases arise. | |
<li>Cause no breakage for projects that maintain the old constraints. That is, if a user's project | |
continues to be single-rooted, all-inclusive, and non-overlapping, then all plug-ins should continue | |
to function with no code changes needed.</li> | |
<li>Maintain binary compatibility with 2.0- and 1.0-based plug-ins.</li> | |
</ul> | |
</p> | |
<p> | |
These solutions will not fundamentally change the workspace resource model. The workspace | |
still maintains a faithful representation of corresponding files on the local disk. Local refresh will | |
still cause the workspace to be updated to accurately reflect the changed contents on disk. Operations | |
on resources in the workspace will continue to have direct impact on files in the file system -- deletion | |
from the workspace will cause deletion of files from disk, etc. Directories on disk that are associated with | |
resources in the workspace will continue to "belong" to the workspace, and this association | |
will still be recursively all-inclusive. Projects will still have a single, non-overlapping root directory. | |
The only significant change is that a project can now attach contents that are not in its root directory. | |
This allows a single project to bring together contents from several different directories on disk. | |
</p> | |
<hr/> | |
<h2>1. Multi-Rooted Projects</h2> | |
<p> | |
Two categories of problems were identified in relation to the single-rootedness of Eclipse projects. | |
The most common complaint is that it is impossible to locate the Java builder's output folder | |
in another location. The other complaint is that users wanted to draw together sources from several | |
locations into a single project. This second case arises commonly for users of the Eclipse JDT, but also | |
for users of the CDT (C/C++ tools project). In general, this problem boils down to allowing the project to | |
contain folders located elsewhere in the file system. | |
</p> | |
<h3>New Platform Core API</h3> | |
<p> | |
Part of the proposed solution is to add new API in the Platform core to allow <i>linking</i> | |
external resources into projects (solution M-2 from the previous document). This will | |
allow clients of the workspace API to logically connect a resource sub-tree located outside | |
of the project's content area into a project. | |
</p> | |
<p> | |
The <code>link</code> operation is invoked on an IFile or IFolder handle in a fashion similar | |
to the <code>create</code> method. That is, it must be called on a resource that does | |
not yet exist, and the result of the operation is that the resource (and possibly child | |
resources) exists. No files or folders in the file system are moved or modified as | |
a result of creating the link. The link operation essentially just creates resources in the | |
Eclipse workspace tree corresponding to all files at the given file system location. | |
Similarly, deletion of a link removes the corresponding resources from | |
the workspace tree, but leaves the files in the file system untouched. The new terms | |
<i>linked folder</i> and <i>linked file</i> (generally, <i>linked resource</i>) will be used | |
to describe these new entities. This document will also use the term <i>link location</i> | |
to refer to the file system path corresponding to a given linked resource. Children of linked | |
folders are <b>not</b> referred to as linked resources. | |
</p> | |
<p> | |
Support will only be added for linking resources as direct children of a project. Restricting | |
links to only the first level makes the model easier to understand for clients and | |
users. Essentially, the project becomes a logical container that can have arbitrary resources | |
linked in, and the semantics of other files and folders remain the same. None of the other IDEs surveyed | |
allowed linking external resources at arbitrary depths. Finally, supporting links | |
at arbitrary depths would add significant complexity to the implementation, and would affect the | |
performance of most "deep" workspace operations such as deletion, local refresh, move, etc. | |
</p> | |
<h4>Copy, move and deletion of linked resources</h4> | |
<p> | |
Links will have similar semantics to soft symbolic links in Unix. Copy and move semantics are as follows: | |
<ul> | |
<li>If a linked resource is copied or moved into another project, a new linked | |
resource will be created in the destination project that points to the same | |
location.</li> | |
<li>Copying or moving a linked resource into another folder will not be permitted | |
(since links must be direct children of projects). The UI could offer to do | |
a deep copy/move of all resources in this case ("Linked resources can | |
only exist immediately below projects. Do you want to copy/move the contents | |
of the underlying files into /x/y/z?").</li> | |
<li>If a linked resource is renamed, it will continue to be a linked resource | |
pointing to the same location.</li> | |
<li>If a resource contained within a linked folder is copied or moved, it will | |
be copied or moved in the file system in the normal way. No links will be | |
created or deleted.</li> | |
<li>If a project containing linked resources is copied or moved, the new project | |
will contain linked resources to the same locations.</li> | |
</ul> | |
Deletion semantics: | |
<ul> | |
<li>Deletion of a linked resource will never delete the contents on disk. This | |
is symetrical to the behavior of link creation: you cannot create the target | |
of a linked resource from within Eclipse, and you cannot delete the target | |
of a linked resource from within Eclipse. We will add a flag for deleting | |
the contents of the linked resource in the file system, but it will not be | |
the default behaviour of the <code>delete</code> operation.</li> | |
<li>Deletion of a resource contained within a linked folder will cause the underlying | |
files to be deleted from disk in the normal way.</li> | |
<li>Deletion of a project containing a linked resource will also never delete | |
the contents of the linked resource on disk.</li> | |
</ul> | |
<h4>Pre-validation of link locations</h4> | |
<p> | |
A new validation method (<code>IWorkspace.validateLinkLocation</code>) will be introduced | |
to allow clients to check if a proposed file system location is an acceptable location to link into the | |
workspace. This method will return a status with severity <code>IStatus.ERROR</code> if the | |
proposed location is not acceptable (for example, if it contains characters that Eclipse does not | |
allow in filenames, or if it overlaps the workspace metadata area). The method will return a | |
status with severity <code>IStatus.WARNING</code> if the proposed location overlaps | |
another resource location in the workspace. The section on overlapping resources will | |
discuss this in more detail. | |
</p> | |
<h4>Persistence of link data</h4> | |
<p> | |
Link locations will be represented by entries in the .project file. Modifying the .project file | |
can cause links to be created or deleted. A new kind of Core path variables will | |
be introduced, and link locations can be relative to these variables. Resources will be | |
created for all links described in the .project file, even if they cannot be accessed (for | |
example if based on an undefined variable or location that is not accessible). | |
</p> | |
<h4>Facility for disabling links</h4> | |
<p> | |
There may be some plug-ins that, for whatever reason, do not want to tolerate links | |
in projects they operate on. For example, a web site management tool may | |
rely on the file system structure of a project being the same as the structure in the | |
resource tree. A mechanism will be introduced to allow plug-ins to prevent | |
links from being added to a given project. | |
</p> | |
<p> | |
An attribute will be added to project natures that specifies whether the nature | |
allows links. This will default to being true (links allowed), if the | |
nature does not specify it. If any nature installed on a project does not allow | |
links, then attempts to create links will fail. Attempting to install a nature | |
that does not allow links on a project that already has links will fail. | |
</p> | |
<p> | |
Additionally, a hook will be added so that team providers have an opportunity to disallow | |
links. Team providers based on 2.0 will <b>not</b> allow links on projects that they | |
manage. Team providers based on 2.1 or greater must specify that they tolerate links. | |
</p> | |
<p> | |
If either of these mechanisms decides to veto links, users will not be | |
able to override this decision (except by removing the intolerant plug-in). | |
</p> | |
<h3>Implications for team providers</h3> | |
<p> | |
Generally, links will be ignored by team providers. The assumption is that | |
these files are generally taken from some location outside the workspace because they are | |
already managed by an external VCM tool. The java output folder is also usually of | |
no interest to VCM tools. Team providers that have been upgraded to Eclipse 2.1 | |
will be expected to silently tolerate and ignore links. This includes handling of links in the team | |
move/delete hook. If team providers wish to provide more support for linked resources, | |
they may do so. As mentioned earlier, team providers based on Eclipse 2.0 will not | |
tolerate the presence of links on projects they manage. | |
</p> | |
<h3>Platform UI Changes</h3> | |
<p> | |
The Platform UI will add new operations to allow users to create links as top | |
level folders and files. It may be useful to provide a warning in the UI when linking is | |
first attempted to warn users that this may cause problems for other plugins (A warning | |
dialog with the option, "Don't tell me again" would be appropriate). | |
Linked resources should have some visual annotation in the resource navigator to make it | |
easy to distinguish linked resources from normal resources in the UI. | |
</p> | |
<p> | |
The Platform UI will also add a preference page and dialogs for adding, removing, and editing the platform | |
core variables used in link locations. This will be similar to the current UI for java classpath variables. | |
</p> | |
<h3>JDT Changes</h3> | |
<p> | |
JDT core will need to revisit compiler and search infrastructure that currently relies | |
on direct navigation of resources in the file system. Since it is no longer guaranteed that all | |
source folders are children of the project's local directory, they will need to consult the | |
workspace API to find the correct locations. | |
</p> | |
<p>New flexibility will also be added for java output folders, specifically: | |
<ul> | |
<li>Linked output folders can be used for placing build output outside the project location (P-2).</li> | |
<li>Each source folder can specify a separate output folder (Q-1). | |
<li>Multiple projects can use the same output folder.</li> | |
</ul> | |
</p> | |
<p> | |
Allowing multiple projects to share an output folder introduces a new problem. Currently, each java | |
builder assumes that it has complete control of the output folder. On full build, it currently deletes the | |
entire output folder, which will obviously not be acceptable in a shared scenario. If the builder | |
does not delete the output folder before full builds, then any extra .class files in the output | |
folder will influence the classpath of that project. This can have unexpected results at runtime | |
if stale classes in the output folder obscure legitimate classes in other projects. Neither approach | |
(scrub everything or scrub nothing) is entirely appealing. However, a flag will be introduced on | |
output folders to allow either of those two options. If the user turns scrubbing off, they will | |
have to deal with the problem of performing "make clean" before full builds by themselves. | |
</p> | |
<hr/> | |
<h2>2. Selectively Inclusive Projects</h2> | |
<p> | |
In some of the identified problems, users wanted to operate on only a selected subset | |
of the resources in a project. There are two different angles to this problem: | |
<ol> | |
<li>Wanting to completely exclude some resources from the Eclipse workspace</li> | |
<li>Wanting to exclude some resources from consideration by certain tools (such as the Java builder)</li> | |
</ol> | |
</p> | |
<p> | |
The first problem is more difficult to address, and would cause more grief for tools that | |
rely on the all-inclusive nature of the workbench. The difficulty in supporting complete | |
exclusion boils down to tension between plug-ins that have different needs. One plug-in | |
may wish to exclude certain resources, while another plug-in operating on the same resource | |
tree may not want to exclude those same resources. Furthermore, external tools that may | |
operate on resources in the workspace (such as Netscape™ or Ant scripts), will not be aware | |
of any resource exclusion mechanism introduced in the platform, and would thus be fundamentally | |
at odds with such a mechanism. | |
</p> | |
<p> For these reasons, support for exclusion will only be introduced at the JDT | |
level. JDT Core will provide a new facility for adding exclusion rules to source | |
classpath entries. These exclusion rules will support matching of path prefixes, | |
e.g., exclude com/xyz/package1/*. Excluded resources will be omitted completely | |
from the Java model, and will not be considered for compilation. These exclusion | |
rules will be stored in the .classpath file, and thus automatically shared with | |
other team members. This approach is equivalent to solution S-2 from the previous | |
document. </p> | |
<hr/> | |
<h2>3. Overlapping Projects</h2> | |
<p> | |
The proposed solution is to have almost no restrictions on overlapping link locations. | |
Existing restrictions on non-overlapping project locations will remain. Linked resources | |
will only have a small number of restrictions to prevent situations impossible to support. | |
This breaks down as: | |
<ul> | |
<li>Project locations cannot overlap other project locations.</li> | |
<li>Link locations can overlap other link locations, and other project locations.</li> | |
<li>Link locations cannot be the same as or a parent of the root directory of the | |
project the link is contained in (this would result in an infinitely recursive project tree).</li> | |
<li>Link locations cannot overlap the workspace metadata area (can also result in | |
infinite recursion of folder structures).</li> | |
<li>The .project file cannot be a linked file (chicken and egg problem | |
because link information is stored in the .project file).</li> | |
</ul> | |
</p> | |
<p> | |
The platform core change will introduce some new behavior and wrinkles | |
in how resources behave. If two resources correspond to the same underlying | |
file, then changing one resource will incur a resource delta on both. Also, if a | |
<code>refreshLocal</code> is invoked on an overlapping portion of the workspace, | |
then several different resources may be refreshed as a result. Because of the potential | |
dangers of having overlapping resources (generally that changing one will affect all duplicates), | |
a warning will be presented when a user tries to create a linked resource whose location | |
overlaps another resource in the workspace. Calculation of this warning will be done | |
by the new <code>validateLinkLocation</code> method. | |
</p> | |
<p> | |
This change has ramifications for the following Platform Core methods: | |
<br><br><code> | |
IFile IWorkspaceRoot.getFileForLocation(IPath location);<br> | |
IContainer IWorkspaceRoot.getContainerForLocation(IPath location); | |
</code><br><br> | |
The specification of these methods will be clarified to indicate that they will | |
ignore linked resources. They will continue to only look in each project's root | |
location for the file system path in question. If a given file exists under both a | |
project's root directory and under a link location, it will always return the resource | |
under the project's root directory. Since project root directories cannot overlap, | |
there is no ambiguity to this solution. New link-aware API methods will be introduced for finding | |
the set of resources corresponding to a given file system location. | |
</p> | |
<h3>JDT Changes</h3> | |
<p> | |
JDT Core will add the ability to introduce nested source folders, subject to the condition | |
that the nested portion is excluded from the parent folder. Users will not be allowed to | |
link the same folder twice to create two different source folders at the same location in a given project. | |
</p> | |
<hr/> | |
<h2>Examples Redux</h2> | |
<p> | |
To show that the proposed solution is sufficient to solve many of the problems described | |
in the problem definition, we will revisit the four example workspaces from the problem | |
statement and show how they can be supported by the proposals in this document. | |
</p> | |
<p>Example 1:</p> | |
<p> | |
|- AllProducts<br> | |
|- Product1<br> | |
|- JavaSourceFiles<br> | |
|- Product2<br> | |
|- JavaSourceFiles<br> | |
</p> | |
<p> | |
This configuration can be supported using only the new linked resources. | |
Projects would be left in the default workspace content area, and the source folders | |
would be linked into the monolithic directory tree. Eclipse artifacts such as .project, | |
along with build output, would remain in the default area, thus causing no pollution | |
of the source tree with transient or Eclipse-specific files. This solution assumes that | |
the resources are not being managed by an eclipse team provider: | |
<br><br> | |
/Product1 at default location<br> | |
/src - java source folder linked at file://AllProducts/Product1/JavaSourceFiles<br> | |
/Product2 at default location<br> | |
/src - java source folder linked at file://AllProducts/Product2/JavaSourceFiles<br> | |
</p> | |
<p> | |
If such a monolithic resource tree needs to use an Eclipse team provider, then an | |
extra project can be used whose location is the "AllProducts" folder. This | |
extra project would be a simple project without a java builder, and would be used for | |
source control purposes only. All other projects would not have a team provider. | |
</p> | |
<p> | |
/MasterProject at file://AllProducts (shared with team provider)<br> | |
/Product1 at default location (no team provider)<br> | |
/src - java source folder linked at file://AllProducts/Product1/JavaSourceFiles<br> | |
/Product2 at default location (no team provider)<br> | |
/src - java source folder linked at file://AllProducts/Product2/JavaSourceFiles<br> | |
</p> | |
<p>Example 2:</p> | |
<p> | |
|- AllJavaSourceFiles<br> | |
|- com<br> | |
|- xyz<br> | |
|- product1<br> | |
| |
|- P1Main.java<br> | |
|- product2<br> | |
| |
|- P2Main.java<br> | |
</p> | |
<p> | |
This common configuration requires support from all three proposal areas. Again projects | |
would remain in the default content area, one project per product. All projects would link | |
the same external folder as a source folder, <tt>AllJavaSourceFiles</tt>. Each project's source | |
folder would then exclude all other product packages from its build classpath. | |
</p> | |
<p> | |
/Product1 at default location (no team provider)<br> | |
/src - java source folder linked at file://AllJavaSourceFiles, excluding com/xyz/product2.<br> | |
/Product2 at default location (no team provider)<br> | |
/src - java source folder linked at file://AllJavaSourceFiles, excluding com/xyz/product1.<br> | |
</p> | |
<p> | |
If such a monolithic resource tree needs to make use of an Eclipse team provider, it | |
can be arranged in a similar fashion to example 1. An extra project would correspond | |
to AllJavaSourceFiles, and would have an Eclipse team provider installed. | |
</p> | |
<p> | |
/MasterProject at file://AllJavaSourceFiles (shared with team provider)<br> | |
/Product1 at default location (no team provider)<br> | |
/src - java source folder linked at file://AllJavaSourceFiles, excluding com/xyz/product2.<br> | |
/Product2 at default location (no team provider)<br> | |
/src - java source folder linked at file://AllJavaSourceFiles, excluding com/xyz/product1.<br> | |
</p> | |
<p>Example 3:</p> | |
<p> | |
|- Product1<br> | |
|- JavaSourcesFiles<br> | |
|- com<br> | |
|- xyz<br> | |
| |
|- product1<br> | |
| |
| |
|- P1Main.java<br> | |
|- tests<br> | |
|- com<br> | |
| |
|- xyz<br> | |
| |
| |
|- product1<br> | |
| |
| |
|- tests<br> | |
| |
|- | |
P1Test.java<br> | |
</p> | |
<p> | |
This directory structure could be supported in several ways, but the most likely solution is as follows. | |
A single project would be created. The path Product1/JavaSourceFiles/ would be specified as a source folder. | |
This folder would have an exclusion rule to omit the "tests" sub-directory. The path | |
Product1/JavaSourceFiles/tests would be specified as a second source folder, with no exclusion rules. If desired, | |
the two source folders could be placed in separate Eclipse projects to provide greater separation between | |
application code and tests. | |
</p> | |
<p> | |
/Product1 at default location <br> | |
/JavaSourceFiles - java source folder excluding the "tests" sub-directory.<br> | |
/tests - java source folder<br> | |
</p> | |
<p>Example 4: </p> | |
<p>|- CommonFramework<br> | |
|- JavaSourceFiles<br> | |
|- Product1<br> | |
|- JavaSourceFiles<br> | |
|- Product2<br> | |
|- JavaSourceFiles<br> | |
</p> | |
<p> | |
Again this structure can be supported in several ways. The most likely is to create three projects, | |
mapping to CommonFramework, Product1 and Product2. Product1 and Product2 would create a | |
linked source folder CommonFramework/JavaSourceFiles. In this case CommonFramework | |
would not need to be a java project, as it mainly just acts as the VCM container for that set of files. | |
<p> | |
/CommonFramework at file://CommonFramework<br> | |
/Product1 at file://Product1<br> | |
/JavaSourceFiles - java source folder<br> | |
/src-common - java source folder linked at file://CommonFramework/JavaSourceFiles<br> | |
/Product2 at file://Product2<br> | |
/JavaSourceFiles - java source folder<br> | |
/src-common - java source folder linked at file://CommonFramework/JavaSourceFiles<br> | |
</p> | |
</body> | |
</html> |