<!doctype html public "-//w3c//dtd html 4.0 transitional//en">
<html>
<head>
   <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
   <meta name="Author" content="Eclipse Project">
   <title>Flexible Project Structure</title>
   <link rel="stylesheet" href="../../default_style.css" type="text/css">
</head>
<body bgcolor="#FFFFFF" >

<div align="left">
  <h1>Flexible Project Structure</h1>
</div>
<blockquote> 
  <p><b>Summary</b> <br>
  Eclipse requires that the contents of each project be stored in a single directory on disk.
  Every file and folder in that directory tree on disk must belong to the project in the workspace.
  This restriction makes it difficult to use Eclipse in conjunction with tools that have specific layout
  requirements for their files, or with users who have legacy file base layouts that they need
  to maintain. We would like to improve the situation for the 2.1 release of Eclipse. This document 
  is the third of a trilogy.  The first described the 
  <a href="http://www.eclipse.org/eclipse/development/inflexible-projects-problem.html">
   nature of the problem</a>, and the second mapped out the various 
   <a href="http://www.eclipse.org/eclipse/development/towards-more-flexible-projects.html">
   options for addressing it</a>.  This third document describes the concrete proposed solution
   in detail.<br><br>
    Last Modified: 18:30 October 24, 2002</p>
</blockquote>
<hr/>
<h2>Overview</h2>
<p>
Analysis of the problem identified three characteristics of Eclipse projects that made them 
difficult to use in conjunction with existing project layouts.  Namely, Eclipse projects are
currently <b>single-rooted</b>, <b>all-inclusive</b>, and <b>non-overlapping</b>.
The proposed solution includes a new mechanism in the Platform core for allowing projects
to be multi-rooted, a new JDT facility for interacting with a selective subset of a project, and a relaxation
of the restriction that resources cannot overlap in the file system.  For readers of the previous
document that mapped out the solution space, these are solutions P-2, Q-1, M-2, S-2, V-1 
and V-2.
</p>
<p>
Relaxing these restrictions is necessarily a breaking change for many tools built on the Eclipse platform.
Some plug-ins may be able to adapt to the new behavior in future releases, but some may never 
work in the presence of projects that relax these layout restrictions.  To minimize the impact, the 
proposed solution adheres to the following principles:
<ul>
<li>Only expose the new functionality in the contexts where flexibility problems have been reported.
For example, users will be able to change the location of JDT source folders and the output folder,
but not be able to change the location of folders at arbitrary depths.  Exposure of the new flexibility can
be widened to fit new contexts where compelling use cases arise.
<li>Cause no breakage for projects that maintain the old constraints.  That is, if a user's project
continues to be single-rooted, all-inclusive, and non-overlapping, then all plug-ins should continue
to function with no code changes needed.</li>
<li>Maintain binary compatibility with 2.0- and 1.0-based plug-ins.</li>
</ul>
</p>
<p>
These solutions will not fundamentally change the workspace resource model.  The workspace
still maintains a faithful representation of corresponding files on the local disk.  Local refresh will 
still cause the workspace to be updated to accurately reflect the changed contents on disk.  Operations
on resources in the workspace will continue to have direct impact on files in the file system -- deletion 
from the workspace will cause deletion of files from disk, etc.  Directories on disk that are associated with
resources in the workspace will continue to &quot;belong&quot; to the workspace, and this association
will still be recursively all-inclusive.  Projects will still have a single, non-overlapping root directory.
The only significant change is that a project can now attach contents that are not in its root directory.
This allows a single project to bring together contents from several different directories on disk.
</p>
<hr/>
<h2>1. Multi-Rooted Projects</h2>
<p>
Two categories of problems were identified in relation to the single-rootedness of Eclipse projects.
The most common complaint is that it is impossible to locate the Java builder's output folder
in another location.  The other complaint is that users wanted to draw together sources from several
locations into a single project.  This second case arises commonly for users of the Eclipse JDT, but also
for users of the CDT (C/C++ tools project).  In general, this problem boils down to allowing the project to
contain folders located elsewhere in the file system.
</p>
<h3>New Platform Core API</h3>
<p>
Part of the proposed solution is to add new API in the Platform core to allow <i>linking</i> 
external resources into projects (solution M-2 from the previous document).  This will
allow clients of the workspace API to logically connect a resource sub-tree located outside 
of the project's content area into a project.  
</p>
<p>
The <code>link</code> operation is invoked on an IFile or IFolder handle in a fashion similar
to the <code>create</code> method.  That is, it must be called on a resource that does
not yet exist, and the result of the operation is that the resource (and possibly child
resources) exists. No files or folders in the file system are moved or modified as 
a result of creating the link.  The link operation essentially just creates resources in the 
Eclipse workspace tree corresponding to all files at the given file system location.  
Similarly, deletion of a link removes the corresponding resources from 
the workspace tree, but leaves the files in the file system untouched.  The new terms
<i>linked folder</i> and <i>linked file</i> (generally, <i>linked resource</i>) will be used 
to describe these new entities.  This document will also use the term <i>link location</i> 
to refer to the file system path corresponding to a given linked resource. Children of linked 
folders are <b>not</b> referred to as linked resources.
</p>
<p>
Support will only be added for linking resources as direct children of a project.  Restricting 
links to only the first level makes the model easier to understand for clients and 
users.  Essentially, the project becomes a logical container that can have arbitrary resources 
linked in, and the semantics of other files and folders remain the same.  None of the other IDEs surveyed 
allowed linking external resources at arbitrary depths.  Finally, supporting links
at arbitrary depths would add significant complexity to the implementation, and would affect the
performance of most &quot;deep&quot; workspace operations such as deletion, local refresh, move, etc.
</p>
<h4>Copy, move and deletion of linked resources</h4>
<p>
Links will have similar semantics to soft symbolic links in Unix.  Copy and move semantics are as follows:
<ul>
  <li>If a linked resource is copied or moved into another project, a new linked 
    resource will be created in the destination project that points to the same 
    location.</li>
  <li>Copying or moving a linked resource into another folder will not be permitted 
    (since links must be direct children of projects). The UI could offer to do 
    a deep copy/move of all resources in this case (&quot;Linked resources can 
    only exist immediately below projects. Do you want to copy/move the contents 
    of the underlying files into /x/y/z?&quot;).</li>
  <li>If a linked resource is renamed, it will continue to be a linked resource 
    pointing to the same location.</li>
  <li>If a resource contained within a linked folder is copied or moved, it will 
    be copied or moved in the file system in the normal way. No links will be 
    created or deleted.</li>
  <li>If a project containing linked resources is copied or moved, the new project 
    will contain linked resources to the same locations.</li>
</ul>
Deletion semantics: 
<ul>
  <li>Deletion of a linked resource will never delete the contents on disk. This 
    is symetrical to the behavior of link creation: you cannot create the target 
    of a linked resource from within Eclipse, and you cannot delete the target 
    of a linked resource from within Eclipse. We will add a flag for deleting 
    the contents of the linked resource in the file system, but it will not be 
    the default behaviour of the <code>delete</code> operation.</li>
  <li>Deletion of a resource contained within a linked folder will cause the underlying 
    files to be deleted from disk in the normal way.</li>
  <li>Deletion of a project containing a linked resource will also never delete 
    the contents of the linked resource on disk.</li>
</ul>

<h4>Pre-validation of link locations</h4>
<p>
A new validation method (<code>IWorkspace.validateLinkLocation</code>) will be introduced 
to allow clients to check if a proposed file system location is an acceptable location to link into the 
workspace.  This method will return a status with severity <code>IStatus.ERROR</code> if the 
proposed location is not acceptable (for example, if it contains characters that Eclipse does not 
allow in filenames, or if it overlaps the workspace metadata area).  The method will return a
status with severity <code>IStatus.WARNING</code> if the proposed location overlaps
another resource location in the workspace.  The section on overlapping resources will
discuss this in more detail.
</p>
<h4>Persistence of link data</h4>
<p>
Link locations will be represented by entries in the .project file.  Modifying the .project file 
can cause links to be created or deleted.  A new kind of Core path variables will 
be introduced, and link locations can be relative to these variables.  Resources will be 
created for all links described in the .project file, even if they cannot be accessed (for 
example if based on an undefined variable or location that is not accessible).
</p>
<h4>Facility for disabling links</h4>
<p>
There may be some plug-ins that, for whatever reason, do not want to tolerate links
in projects they operate on.  For example, a web site management tool may 
rely on the file system structure of a project being the same as the structure in the 
resource tree.  A mechanism will be introduced to allow plug-ins to prevent 
links from being added to a given project.
</p>
<p>
An attribute will be added to project natures that specifies whether the nature
allows links.  This will default to being true (links allowed), if the
nature does not specify it.  If any nature installed on a project does not allow
links, then attempts to create links will fail.  Attempting to install a nature
that does not allow links on a project that already has links will fail.
</p>
<p>
Additionally, a hook will be added so that team providers have an opportunity to disallow
links.  Team providers based on 2.0 will <b>not</b> allow links on projects that they 
manage.  Team providers based on 2.1 or greater must specify that they tolerate links.
</p>
<p>
If either of these mechanisms decides to veto links, users will not be 
able to override this decision (except by removing the intolerant plug-in).
</p>

<h3>Implications for team providers</h3>
<p>
Generally, links will be ignored by team providers.  The assumption is that
these files are generally taken from some location outside the workspace because they are 
already managed by an external VCM tool. The java output folder is also usually of
no interest to VCM tools.  Team providers that have been upgraded to Eclipse 2.1
will be expected to silently tolerate and ignore links.  This includes handling of links in the team 
move/delete hook.  If team providers wish to provide more support for linked resources,
they may do so.  As mentioned earlier, team providers based on Eclipse 2.0 will not 
tolerate the presence of links on projects they manage.
</p>

<h3>Platform UI Changes</h3>
<p>
The Platform UI will add new operations to allow users to create links as top 
level folders and files.   It may be useful to provide a warning in the UI when linking is 
first attempted to warn users that this may cause problems for other plugins (A warning 
dialog with the option, "Don't tell me again" would be appropriate).  
Linked resources should have some visual annotation in the resource navigator to make it 
easy to distinguish linked resources from normal resources in the UI.
</p>
<p>
The Platform UI will also add a preference page and dialogs for adding, removing, and editing the platform
core variables used in link locations.  This will be similar to the current UI for java classpath variables.
</p>
<h3>JDT Changes</h3>
<p>
JDT core will need to revisit compiler and search infrastructure that currently relies 
on direct navigation of resources in the file system.  Since it is no longer guaranteed that all
source folders are children of the project's local directory, they will need to consult the
workspace API to find the correct locations.
</p>
<p>New flexibility will also be added for java output folders, specifically:
<ul>
<li>Linked output folders can be used for placing build output outside the project location (P-2).</li>
<li>Each source folder can specify a separate output folder (Q-1).
<li>Multiple projects can use the same output folder.</li>
</ul>
</p>
<p>
Allowing multiple projects to share an output folder introduces a new problem.  Currently, each java
builder assumes that it has complete control of the output folder.  On full build, it currently deletes the
entire output folder, which will obviously not be acceptable in a shared scenario.  If the builder
does not delete the output folder before full builds, then any extra .class files in the output 
folder will influence the classpath of that project.  This can have unexpected results at runtime
if stale classes in the output folder obscure legitimate classes in other projects.  Neither approach
(scrub everything or scrub nothing) is entirely appealing.  However, a flag will be introduced on
output folders to allow either of those two options.  If the user turns scrubbing off, they will
have to deal with the problem of performing &quot;make clean&quot; before full builds by themselves.
</p>
<hr/>
<h2>2. Selectively Inclusive Projects</h2>
<p>
In some of the identified problems, users wanted to operate on only a selected subset
of the resources in a project.  There are two different angles to this problem:
<ol>
<li>Wanting to completely exclude some resources from the Eclipse workspace</li>
<li>Wanting to exclude some resources from consideration by certain tools (such as the Java builder)</li>
</ol>
</p>
<p>
The first problem is more difficult to address, and would cause more grief for tools that 
rely on the all-inclusive nature of the workbench.  The difficulty in supporting complete
exclusion boils down to tension between plug-ins that have different needs.  One plug-in
may wish to exclude certain resources, while another plug-in operating on the same resource
tree may not want to exclude those same resources.  Furthermore, external tools that may 
operate on resources in the workspace (such as Netscape&#153; or Ant scripts), will not be aware
of any resource exclusion mechanism introduced in the platform, and would thus be fundamentally
at odds with such a mechanism.
</p>
<p> For these reasons, support for exclusion will only be introduced at the JDT 
  level. JDT Core will provide a new facility for adding exclusion rules to source 
  classpath entries. These exclusion rules will support matching of path prefixes, 
  e.g., exclude com/xyz/package1/*. Excluded resources will be omitted completely 
  from the Java model, and will not be considered for compilation. These exclusion 
  rules will be stored in the .classpath file, and thus automatically shared with 
  other team members. This approach is equivalent to solution S-2 from the previous 
  document. </p>
<hr/>
<h2>3. Overlapping Projects</h2>
<p>
The proposed solution is to have almost no restrictions on overlapping link locations.  
Existing restrictions on non-overlapping project locations will remain. Linked resources
will only have a small number of restrictions to prevent situations impossible to support.
This breaks down as:
<ul>
<li>Project locations cannot overlap other project locations.</li>
<li>Link locations can overlap other link locations, and other project locations.</li>
<li>Link locations cannot be the same as or a parent of the root directory of the 
project the link is contained in (this would result in an infinitely recursive project tree).</li>
<li>Link locations cannot overlap the workspace metadata area (can also result in
infinite recursion of folder structures).</li>
<li>The .project file cannot be a linked file (chicken and egg problem
because link information is stored in the .project file).</li>
</ul>
</p>
<p>
The platform core change will introduce some new behavior and wrinkles
in how resources behave.  If two resources correspond to the same underlying
file, then changing one resource will incur a resource delta on both.  Also, if a 
<code>refreshLocal</code> is invoked on an overlapping portion of the workspace,
then several different resources may be refreshed as a result.  Because of the potential
dangers of having overlapping resources (generally that changing one will affect all duplicates), 
a warning will be presented when a user tries to create a linked resource whose location 
overlaps another resource in the workspace.  Calculation of this warning will be done
by the new <code>validateLinkLocation</code> method.
</p>
<p>
This change has ramifications for the following Platform Core methods:
<br><br><code>
IFile IWorkspaceRoot.getFileForLocation(IPath location);<br>
IContainer IWorkspaceRoot.getContainerForLocation(IPath location);
</code><br><br>
The specification of these methods will be clarified to indicate that they will
ignore linked resources.  They will continue to only look in each project's root 
location for the file system path in question. If a given file exists under both a 
project's root directory and under a link location, it will always return the resource 
under the project's root directory.  Since project root directories cannot overlap, 
there is no ambiguity to this solution. New link-aware API methods will be introduced for finding 
the set of resources corresponding to a given file system location.
</p>
<h3>JDT Changes</h3>
<p>
JDT Core will add the ability to introduce nested source folders, subject to the condition
that the nested portion is excluded from the parent folder.  Users will not be allowed to
link the same folder twice to create two different source folders at the same location in a given project.
</p>
<hr/>
<h2>Examples Redux</h2>
<p>
To show that the proposed solution is sufficient to solve many of the problems described
in the problem definition, we will revisit the four example workspaces from the problem
statement and show how they can be supported by the proposals in this document.
</p>
<p>Example 1:</p>
<p>
|- AllProducts<br>
&nbsp;&nbsp;&nbsp;&nbsp;|- Product1<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |- JavaSourceFiles<br>
&nbsp;&nbsp;&nbsp;&nbsp;|- Product2<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |- JavaSourceFiles<br>
</p>
<p>
This configuration can be supported using only the new linked resources.
Projects would be left in the default workspace content area, and the source folders
would be linked into the monolithic directory tree.  Eclipse artifacts such as .project,
along with build output, would remain in the default area, thus causing no pollution
of the source tree with transient or Eclipse-specific files.  This solution assumes that
the resources are not being managed by an eclipse team provider:
<br><br>
/Product1 at default location<br>
&nbsp;&nbsp;&nbsp;/src - java source folder linked at file://AllProducts/Product1/JavaSourceFiles<br>
/Product2 at default location<br>
&nbsp;&nbsp;&nbsp;/src - java source folder linked at file://AllProducts/Product2/JavaSourceFiles<br>
</p>
<p>
If such a monolithic resource tree needs to use an Eclipse team provider, then an
extra project can be used whose location is the &quot;AllProducts&quot; folder.  This
extra project would be a simple project without a java builder, and would be used for
source control purposes only.  All other projects would not have a team provider.
</p>
<p>
/MasterProject at file://AllProducts (shared with team provider)<br>
/Product1 at default location (no team provider)<br>
&nbsp;&nbsp;&nbsp;/src - java source folder linked at file://AllProducts/Product1/JavaSourceFiles<br>
/Product2 at default location (no team provider)<br>
&nbsp;&nbsp;&nbsp;/src - java source folder linked at file://AllProducts/Product2/JavaSourceFiles<br>
</p>

<p>Example 2:</p>
<p>
|- AllJavaSourceFiles<br>
&nbsp;&nbsp;&nbsp;&nbsp;|- com<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |- xyz<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |- product1<br>
&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
|- P1Main.java<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |- product2<br>
&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
|- P2Main.java<br>
</p>
<p>
This common configuration requires support from all three proposal areas.  Again projects
would remain in the default content area, one project per product.  All projects would link
the same external folder as a source folder, <tt>AllJavaSourceFiles</tt>.  Each project's source
folder would then exclude all other product packages from its build classpath.
</p>
<p>
/Product1 at default location (no team provider)<br>
&nbsp;&nbsp;&nbsp;/src - java source folder linked at file://AllJavaSourceFiles, excluding com/xyz/product2.<br>
/Product2 at default location (no team provider)<br>
&nbsp;&nbsp;&nbsp;/src - java source folder linked at file://AllJavaSourceFiles, excluding com/xyz/product1.<br>
</p>
<p>
If such a monolithic resource tree needs to make use of an Eclipse team provider, it
can be arranged in a similar fashion to example 1.  An extra project would correspond 
to AllJavaSourceFiles, and would have an Eclipse team provider installed.
</p>
<p>
/MasterProject at file://AllJavaSourceFiles (shared with team provider)<br>
/Product1 at default location (no team provider)<br>
&nbsp;&nbsp;&nbsp;/src - java source folder linked at file://AllJavaSourceFiles, excluding com/xyz/product2.<br>
/Product2 at default location (no team provider)<br>
&nbsp;&nbsp;&nbsp;/src - java source folder linked at file://AllJavaSourceFiles, excluding com/xyz/product1.<br>
</p>
<p>Example 3:</p>
<p>
|- Product1<br>
&nbsp;&nbsp;&nbsp;&nbsp;|- JavaSourcesFiles<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |- com<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |- xyz<br>
&nbsp;&nbsp;&nbsp;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |- product1<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
|- P1Main.java<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |- tests<br>
&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |- com<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
|- xyz<br>
&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
|- product1<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
|- tests<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |-
P1Test.java<br>
</p>
<p>
This directory structure could be supported in several ways, but the most likely solution is as follows.
A single project would be created.  The path Product1/JavaSourceFiles/ would be specified as a source folder.  
This folder would have an exclusion rule to omit the &quot;tests&quot; sub-directory.  The path 
Product1/JavaSourceFiles/tests would be specified as a second source folder, with no exclusion rules.  If desired,
the two source folders could be placed in separate Eclipse projects to provide greater separation between 
application code and tests.
</p>
<p>
/Product1 at default location <br>
&nbsp;&nbsp;&nbsp;/JavaSourceFiles - java source folder excluding the &quot;tests&quot; sub-directory.<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;/tests - java source folder<br>

</p>
<p>Example 4:&nbsp;</p>
<p>|- CommonFramework<br>
&nbsp;&nbsp;&nbsp; |- JavaSourceFiles<br>
|- Product1<br>
&nbsp;&nbsp;&nbsp; |- JavaSourceFiles<br>
|- Product2<br>
&nbsp;&nbsp;&nbsp; |- JavaSourceFiles<br>
</p>
<p>
Again this structure can be supported in several ways.  The most likely is to create three projects, 
mapping to CommonFramework, Product1 and Product2.  Product1 and Product2 would create a
linked source folder CommonFramework/JavaSourceFiles.   In this case CommonFramework 
would not need to be a java project, as it mainly just acts as the VCM container for that set of files.
<p>
/CommonFramework at file://CommonFramework<br>
/Product1 at file://Product1<br>
&nbsp;&nbsp;&nbsp;/JavaSourceFiles - java source folder<br>
&nbsp;&nbsp;&nbsp;/src-common - java source folder linked at file://CommonFramework/JavaSourceFiles<br>
/Product2 at file://Product2<br>
&nbsp;&nbsp;&nbsp;/JavaSourceFiles - java source folder<br>
&nbsp;&nbsp;&nbsp;/src-common - java source folder linked at file://CommonFramework/JavaSourceFiles<br>
</p>
</body>
</html>