| <!doctype html public "-//w3c//dtd html 4.0 transitional//en"> |
| <html> |
| <head> |
| <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> |
| <meta name="Author" content="Eclipse Project"> |
| <title>Flexible Project Structure</title> |
| <link rel="stylesheet" href="../../default_style.css" type="text/css"> |
| </head> |
| <body bgcolor="#FFFFFF" > |
| |
| <div align="left"> |
| <h1>Flexible Project Structure</h1> |
| </div> |
| <blockquote> |
| <p><b>Summary</b> <br> |
| Eclipse requires that the contents of each project be stored in a single directory on disk. |
| Every file and folder in that directory tree on disk must belong to the project in the workspace. |
| This restriction makes it difficult to use Eclipse in conjunction with tools that have specific layout |
| requirements for their files, or with users who have legacy file base layouts that they need |
| to maintain. We would like to improve the situation for the 2.1 release of Eclipse. This document |
| is the third of a trilogy. The first described the |
| <a href="http://www.eclipse.org/eclipse/development/inflexible-projects-problem.html"> |
| nature of the problem</a>, and the second mapped out the various |
| <a href="http://www.eclipse.org/eclipse/development/towards-more-flexible-projects.html"> |
| options for addressing it</a>. This third document describes the concrete proposed solution |
| in detail.<br><br> |
| Last Modified: 18:30 October 24, 2002</p> |
| </blockquote> |
| <hr/> |
| <h2>Overview</h2> |
| <p> |
| Analysis of the problem identified three characteristics of Eclipse projects that made them |
| difficult to use in conjunction with existing project layouts. Namely, Eclipse projects are |
| currently <b>single-rooted</b>, <b>all-inclusive</b>, and <b>non-overlapping</b>. |
| The proposed solution includes a new mechanism in the Platform core for allowing projects |
| to be multi-rooted, a new JDT facility for interacting with a selective subset of a project, and a relaxation |
| of the restriction that resources cannot overlap in the file system. For readers of the previous |
| document that mapped out the solution space, these are solutions P-2, Q-1, M-2, S-2, V-1 |
| and V-2. |
| </p> |
| <p> |
| Relaxing these restrictions is necessarily a breaking change for many tools built on the Eclipse platform. |
| Some plug-ins may be able to adapt to the new behavior in future releases, but some may never |
| work in the presence of projects that relax these layout restrictions. To minimize the impact, the |
| proposed solution adheres to the following principles: |
| <ul> |
| <li>Only expose the new functionality in the contexts where flexibility problems have been reported. |
| For example, users will be able to change the location of JDT source folders and the output folder, |
| but not be able to change the location of folders at arbitrary depths. Exposure of the new flexibility can |
| be widened to fit new contexts where compelling use cases arise. |
| <li>Cause no breakage for projects that maintain the old constraints. That is, if a user's project |
| continues to be single-rooted, all-inclusive, and non-overlapping, then all plug-ins should continue |
| to function with no code changes needed.</li> |
| <li>Maintain binary compatibility with 2.0- and 1.0-based plug-ins.</li> |
| </ul> |
| </p> |
| <p> |
| These solutions will not fundamentally change the workspace resource model. The workspace |
| still maintains a faithful representation of corresponding files on the local disk. Local refresh will |
| still cause the workspace to be updated to accurately reflect the changed contents on disk. Operations |
| on resources in the workspace will continue to have direct impact on files in the file system -- deletion |
| from the workspace will cause deletion of files from disk, etc. Directories on disk that are associated with |
| resources in the workspace will continue to "belong" to the workspace, and this association |
| will still be recursively all-inclusive. Projects will still have a single, non-overlapping root directory. |
| The only significant change is that a project can now attach contents that are not in its root directory. |
| This allows a single project to bring together contents from several different directories on disk. |
| </p> |
| <hr/> |
| <h2>1. Multi-Rooted Projects</h2> |
| <p> |
| Two categories of problems were identified in relation to the single-rootedness of Eclipse projects. |
| The most common complaint is that it is impossible to locate the Java builder's output folder |
| in another location. The other complaint is that users wanted to draw together sources from several |
| locations into a single project. This second case arises commonly for users of the Eclipse JDT, but also |
| for users of the CDT (C/C++ tools project). In general, this problem boils down to allowing the project to |
| contain folders located elsewhere in the file system. |
| </p> |
| <h3>New Platform Core API</h3> |
| <p> |
| Part of the proposed solution is to add new API in the Platform core to allow <i>linking</i> |
| external resources into projects (solution M-2 from the previous document). This will |
| allow clients of the workspace API to logically connect a resource sub-tree located outside |
| of the project's content area into a project. |
| </p> |
| <p> |
| The <code>link</code> operation is invoked on an IFile or IFolder handle in a fashion similar |
| to the <code>create</code> method. That is, it must be called on a resource that does |
| not yet exist, and the result of the operation is that the resource (and possibly child |
| resources) exists. No files or folders in the file system are moved or modified as |
| a result of creating the link. The link operation essentially just creates resources in the |
| Eclipse workspace tree corresponding to all files at the given file system location. |
| Similarly, deletion of a link removes the corresponding resources from |
| the workspace tree, but leaves the files in the file system untouched. The new terms |
| <i>linked folder</i> and <i>linked file</i> (generally, <i>linked resource</i>) will be used |
| to describe these new entities. This document will also use the term <i>link location</i> |
| to refer to the file system path corresponding to a given linked resource. Children of linked |
| folders are <b>not</b> referred to as linked resources. |
| </p> |
| <p> |
| Support will only be added for linking resources as direct children of a project. Restricting |
| links to only the first level makes the model easier to understand for clients and |
| users. Essentially, the project becomes a logical container that can have arbitrary resources |
| linked in, and the semantics of other files and folders remain the same. None of the other IDEs surveyed |
| allowed linking external resources at arbitrary depths. Finally, supporting links |
| at arbitrary depths would add significant complexity to the implementation, and would affect the |
| performance of most "deep" workspace operations such as deletion, local refresh, move, etc. |
| </p> |
| <h4>Copy, move and deletion of linked resources</h4> |
| <p> |
| Links will have similar semantics to soft symbolic links in Unix. Copy and move semantics are as follows: |
| <ul> |
| <li>If a linked resource is copied or moved into another project, a new linked |
| resource will be created in the destination project that points to the same |
| location.</li> |
| <li>Copying or moving a linked resource into another folder will not be permitted |
| (since links must be direct children of projects). The UI could offer to do |
| a deep copy/move of all resources in this case ("Linked resources can |
| only exist immediately below projects. Do you want to copy/move the contents |
| of the underlying files into /x/y/z?").</li> |
| <li>If a linked resource is renamed, it will continue to be a linked resource |
| pointing to the same location.</li> |
| <li>If a resource contained within a linked folder is copied or moved, it will |
| be copied or moved in the file system in the normal way. No links will be |
| created or deleted.</li> |
| <li>If a project containing linked resources is copied or moved, the new project |
| will contain linked resources to the same locations.</li> |
| </ul> |
| Deletion semantics: |
| <ul> |
| <li>Deletion of a linked resource will never delete the contents on disk. This |
| is symetrical to the behavior of link creation: you cannot create the target |
| of a linked resource from within Eclipse, and you cannot delete the target |
| of a linked resource from within Eclipse. We will add a flag for deleting |
| the contents of the linked resource in the file system, but it will not be |
| the default behaviour of the <code>delete</code> operation.</li> |
| <li>Deletion of a resource contained within a linked folder will cause the underlying |
| files to be deleted from disk in the normal way.</li> |
| <li>Deletion of a project containing a linked resource will also never delete |
| the contents of the linked resource on disk.</li> |
| </ul> |
| |
| <h4>Pre-validation of link locations</h4> |
| <p> |
| A new validation method (<code>IWorkspace.validateLinkLocation</code>) will be introduced |
| to allow clients to check if a proposed file system location is an acceptable location to link into the |
| workspace. This method will return a status with severity <code>IStatus.ERROR</code> if the |
| proposed location is not acceptable (for example, if it contains characters that Eclipse does not |
| allow in filenames, or if it overlaps the workspace metadata area). The method will return a |
| status with severity <code>IStatus.WARNING</code> if the proposed location overlaps |
| another resource location in the workspace. The section on overlapping resources will |
| discuss this in more detail. |
| </p> |
| <h4>Persistence of link data</h4> |
| <p> |
| Link locations will be represented by entries in the .project file. Modifying the .project file |
| can cause links to be created or deleted. A new kind of Core path variables will |
| be introduced, and link locations can be relative to these variables. Resources will be |
| created for all links described in the .project file, even if they cannot be accessed (for |
| example if based on an undefined variable or location that is not accessible). |
| </p> |
| <h4>Facility for disabling links</h4> |
| <p> |
| There may be some plug-ins that, for whatever reason, do not want to tolerate links |
| in projects they operate on. For example, a web site management tool may |
| rely on the file system structure of a project being the same as the structure in the |
| resource tree. A mechanism will be introduced to allow plug-ins to prevent |
| links from being added to a given project. |
| </p> |
| <p> |
| An attribute will be added to project natures that specifies whether the nature |
| allows links. This will default to being true (links allowed), if the |
| nature does not specify it. If any nature installed on a project does not allow |
| links, then attempts to create links will fail. Attempting to install a nature |
| that does not allow links on a project that already has links will fail. |
| </p> |
| <p> |
| Additionally, a hook will be added so that team providers have an opportunity to disallow |
| links. Team providers based on 2.0 will <b>not</b> allow links on projects that they |
| manage. Team providers based on 2.1 or greater must specify that they tolerate links. |
| </p> |
| <p> |
| If either of these mechanisms decides to veto links, users will not be |
| able to override this decision (except by removing the intolerant plug-in). |
| </p> |
| |
| <h3>Implications for team providers</h3> |
| <p> |
| Generally, links will be ignored by team providers. The assumption is that |
| these files are generally taken from some location outside the workspace because they are |
| already managed by an external VCM tool. The java output folder is also usually of |
| no interest to VCM tools. Team providers that have been upgraded to Eclipse 2.1 |
| will be expected to silently tolerate and ignore links. This includes handling of links in the team |
| move/delete hook. If team providers wish to provide more support for linked resources, |
| they may do so. As mentioned earlier, team providers based on Eclipse 2.0 will not |
| tolerate the presence of links on projects they manage. |
| </p> |
| |
| <h3>Platform UI Changes</h3> |
| <p> |
| The Platform UI will add new operations to allow users to create links as top |
| level folders and files. It may be useful to provide a warning in the UI when linking is |
| first attempted to warn users that this may cause problems for other plugins (A warning |
| dialog with the option, "Don't tell me again" would be appropriate). |
| Linked resources should have some visual annotation in the resource navigator to make it |
| easy to distinguish linked resources from normal resources in the UI. |
| </p> |
| <p> |
| The Platform UI will also add a preference page and dialogs for adding, removing, and editing the platform |
| core variables used in link locations. This will be similar to the current UI for java classpath variables. |
| </p> |
| <h3>JDT Changes</h3> |
| <p> |
| JDT core will need to revisit compiler and search infrastructure that currently relies |
| on direct navigation of resources in the file system. Since it is no longer guaranteed that all |
| source folders are children of the project's local directory, they will need to consult the |
| workspace API to find the correct locations. |
| </p> |
| <p>New flexibility will also be added for java output folders, specifically: |
| <ul> |
| <li>Linked output folders can be used for placing build output outside the project location (P-2).</li> |
| <li>Each source folder can specify a separate output folder (Q-1). |
| <li>Multiple projects can use the same output folder.</li> |
| </ul> |
| </p> |
| <p> |
| Allowing multiple projects to share an output folder introduces a new problem. Currently, each java |
| builder assumes that it has complete control of the output folder. On full build, it currently deletes the |
| entire output folder, which will obviously not be acceptable in a shared scenario. If the builder |
| does not delete the output folder before full builds, then any extra .class files in the output |
| folder will influence the classpath of that project. This can have unexpected results at runtime |
| if stale classes in the output folder obscure legitimate classes in other projects. Neither approach |
| (scrub everything or scrub nothing) is entirely appealing. However, a flag will be introduced on |
| output folders to allow either of those two options. If the user turns scrubbing off, they will |
| have to deal with the problem of performing "make clean" before full builds by themselves. |
| </p> |
| <hr/> |
| <h2>2. Selectively Inclusive Projects</h2> |
| <p> |
| In some of the identified problems, users wanted to operate on only a selected subset |
| of the resources in a project. There are two different angles to this problem: |
| <ol> |
| <li>Wanting to completely exclude some resources from the Eclipse workspace</li> |
| <li>Wanting to exclude some resources from consideration by certain tools (such as the Java builder)</li> |
| </ol> |
| </p> |
| <p> |
| The first problem is more difficult to address, and would cause more grief for tools that |
| rely on the all-inclusive nature of the workbench. The difficulty in supporting complete |
| exclusion boils down to tension between plug-ins that have different needs. One plug-in |
| may wish to exclude certain resources, while another plug-in operating on the same resource |
| tree may not want to exclude those same resources. Furthermore, external tools that may |
| operate on resources in the workspace (such as Netscape™ or Ant scripts), will not be aware |
| of any resource exclusion mechanism introduced in the platform, and would thus be fundamentally |
| at odds with such a mechanism. |
| </p> |
| <p> For these reasons, support for exclusion will only be introduced at the JDT |
| level. JDT Core will provide a new facility for adding exclusion rules to source |
| classpath entries. These exclusion rules will support matching of path prefixes, |
| e.g., exclude com/xyz/package1/*. Excluded resources will be omitted completely |
| from the Java model, and will not be considered for compilation. These exclusion |
| rules will be stored in the .classpath file, and thus automatically shared with |
| other team members. This approach is equivalent to solution S-2 from the previous |
| document. </p> |
| <hr/> |
| <h2>3. Overlapping Projects</h2> |
| <p> |
| The proposed solution is to have almost no restrictions on overlapping link locations. |
| Existing restrictions on non-overlapping project locations will remain. Linked resources |
| will only have a small number of restrictions to prevent situations impossible to support. |
| This breaks down as: |
| <ul> |
| <li>Project locations cannot overlap other project locations.</li> |
| <li>Link locations can overlap other link locations, and other project locations.</li> |
| <li>Link locations cannot be the same as or a parent of the root directory of the |
| project the link is contained in (this would result in an infinitely recursive project tree).</li> |
| <li>Link locations cannot overlap the workspace metadata area (can also result in |
| infinite recursion of folder structures).</li> |
| <li>The .project file cannot be a linked file (chicken and egg problem |
| because link information is stored in the .project file).</li> |
| </ul> |
| </p> |
| <p> |
| The platform core change will introduce some new behavior and wrinkles |
| in how resources behave. If two resources correspond to the same underlying |
| file, then changing one resource will incur a resource delta on both. Also, if a |
| <code>refreshLocal</code> is invoked on an overlapping portion of the workspace, |
| then several different resources may be refreshed as a result. Because of the potential |
| dangers of having overlapping resources (generally that changing one will affect all duplicates), |
| a warning will be presented when a user tries to create a linked resource whose location |
| overlaps another resource in the workspace. Calculation of this warning will be done |
| by the new <code>validateLinkLocation</code> method. |
| </p> |
| <p> |
| This change has ramifications for the following Platform Core methods: |
| <br><br><code> |
| IFile IWorkspaceRoot.getFileForLocation(IPath location);<br> |
| IContainer IWorkspaceRoot.getContainerForLocation(IPath location); |
| </code><br><br> |
| The specification of these methods will be clarified to indicate that they will |
| ignore linked resources. They will continue to only look in each project's root |
| location for the file system path in question. If a given file exists under both a |
| project's root directory and under a link location, it will always return the resource |
| under the project's root directory. Since project root directories cannot overlap, |
| there is no ambiguity to this solution. New link-aware API methods will be introduced for finding |
| the set of resources corresponding to a given file system location. |
| </p> |
| <h3>JDT Changes</h3> |
| <p> |
| JDT Core will add the ability to introduce nested source folders, subject to the condition |
| that the nested portion is excluded from the parent folder. Users will not be allowed to |
| link the same folder twice to create two different source folders at the same location in a given project. |
| </p> |
| <hr/> |
| <h2>Examples Redux</h2> |
| <p> |
| To show that the proposed solution is sufficient to solve many of the problems described |
| in the problem definition, we will revisit the four example workspaces from the problem |
| statement and show how they can be supported by the proposals in this document. |
| </p> |
| <p>Example 1:</p> |
| <p> |
| |- AllProducts<br> |
| |- Product1<br> |
| |- JavaSourceFiles<br> |
| |- Product2<br> |
| |- JavaSourceFiles<br> |
| </p> |
| <p> |
| This configuration can be supported using only the new linked resources. |
| Projects would be left in the default workspace content area, and the source folders |
| would be linked into the monolithic directory tree. Eclipse artifacts such as .project, |
| along with build output, would remain in the default area, thus causing no pollution |
| of the source tree with transient or Eclipse-specific files. This solution assumes that |
| the resources are not being managed by an eclipse team provider: |
| <br><br> |
| /Product1 at default location<br> |
| /src - java source folder linked at file://AllProducts/Product1/JavaSourceFiles<br> |
| /Product2 at default location<br> |
| /src - java source folder linked at file://AllProducts/Product2/JavaSourceFiles<br> |
| </p> |
| <p> |
| If such a monolithic resource tree needs to use an Eclipse team provider, then an |
| extra project can be used whose location is the "AllProducts" folder. This |
| extra project would be a simple project without a java builder, and would be used for |
| source control purposes only. All other projects would not have a team provider. |
| </p> |
| <p> |
| /MasterProject at file://AllProducts (shared with team provider)<br> |
| /Product1 at default location (no team provider)<br> |
| /src - java source folder linked at file://AllProducts/Product1/JavaSourceFiles<br> |
| /Product2 at default location (no team provider)<br> |
| /src - java source folder linked at file://AllProducts/Product2/JavaSourceFiles<br> |
| </p> |
| |
| <p>Example 2:</p> |
| <p> |
| |- AllJavaSourceFiles<br> |
| |- com<br> |
| |- xyz<br> |
| |- product1<br> |
| |
| |- P1Main.java<br> |
| |- product2<br> |
| |
| |- P2Main.java<br> |
| </p> |
| <p> |
| This common configuration requires support from all three proposal areas. Again projects |
| would remain in the default content area, one project per product. All projects would link |
| the same external folder as a source folder, <tt>AllJavaSourceFiles</tt>. Each project's source |
| folder would then exclude all other product packages from its build classpath. |
| </p> |
| <p> |
| /Product1 at default location (no team provider)<br> |
| /src - java source folder linked at file://AllJavaSourceFiles, excluding com/xyz/product2.<br> |
| /Product2 at default location (no team provider)<br> |
| /src - java source folder linked at file://AllJavaSourceFiles, excluding com/xyz/product1.<br> |
| </p> |
| <p> |
| If such a monolithic resource tree needs to make use of an Eclipse team provider, it |
| can be arranged in a similar fashion to example 1. An extra project would correspond |
| to AllJavaSourceFiles, and would have an Eclipse team provider installed. |
| </p> |
| <p> |
| /MasterProject at file://AllJavaSourceFiles (shared with team provider)<br> |
| /Product1 at default location (no team provider)<br> |
| /src - java source folder linked at file://AllJavaSourceFiles, excluding com/xyz/product2.<br> |
| /Product2 at default location (no team provider)<br> |
| /src - java source folder linked at file://AllJavaSourceFiles, excluding com/xyz/product1.<br> |
| </p> |
| <p>Example 3:</p> |
| <p> |
| |- Product1<br> |
| |- JavaSourcesFiles<br> |
| |- com<br> |
| |- xyz<br> |
| |
| |- product1<br> |
| |
| |
| |- P1Main.java<br> |
| |- tests<br> |
| |- com<br> |
| |
| |- xyz<br> |
| |
| |
| |- product1<br> |
| |
| |
| |- tests<br> |
| |
| |- |
| P1Test.java<br> |
| </p> |
| <p> |
| This directory structure could be supported in several ways, but the most likely solution is as follows. |
| A single project would be created. The path Product1/JavaSourceFiles/ would be specified as a source folder. |
| This folder would have an exclusion rule to omit the "tests" sub-directory. The path |
| Product1/JavaSourceFiles/tests would be specified as a second source folder, with no exclusion rules. If desired, |
| the two source folders could be placed in separate Eclipse projects to provide greater separation between |
| application code and tests. |
| </p> |
| <p> |
| /Product1 at default location <br> |
| /JavaSourceFiles - java source folder excluding the "tests" sub-directory.<br> |
| /tests - java source folder<br> |
| |
| </p> |
| <p>Example 4: </p> |
| <p>|- CommonFramework<br> |
| |- JavaSourceFiles<br> |
| |- Product1<br> |
| |- JavaSourceFiles<br> |
| |- Product2<br> |
| |- JavaSourceFiles<br> |
| </p> |
| <p> |
| Again this structure can be supported in several ways. The most likely is to create three projects, |
| mapping to CommonFramework, Product1 and Product2. Product1 and Product2 would create a |
| linked source folder CommonFramework/JavaSourceFiles. In this case CommonFramework |
| would not need to be a java project, as it mainly just acts as the VCM container for that set of files. |
| <p> |
| /CommonFramework at file://CommonFramework<br> |
| /Product1 at file://Product1<br> |
| /JavaSourceFiles - java source folder<br> |
| /src-common - java source folder linked at file://CommonFramework/JavaSourceFiles<br> |
| /Product2 at file://Product2<br> |
| /JavaSourceFiles - java source folder<br> |
| /src-common - java source folder linked at file://CommonFramework/JavaSourceFiles<br> |
| </p> |
| </body> |
| </html> |
