| <!doctype html public "-//w3c//dtd html 4.0 transitional//en"> | |
| <html> | |
| <head> | |
| <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> | |
| <meta name="Author" content="Eclipse Project"> | |
| <title>Flexible Project Structure</title> | |
| <link rel="stylesheet" href="../../default_style.css" type="text/css"> | |
| </head> | |
| <body bgcolor="#FFFFFF" > | |
| <div align="left"> | |
| <h1>Flexible Project Structure</h1> | |
| </div> | |
| <blockquote> | |
| <p><b>Summary</b> <br> | |
| Eclipse requires that the contents of each project be stored in a single directory on disk. | |
| Every file and folder in that directory tree on disk must belong to the project in the workspace. | |
| This restriction makes it difficult to use Eclipse in conjunction with tools that have specific layout | |
| requirements for their files, or with users who have legacy file base layouts that they need | |
| to maintain. We would like to improve the situation for the 2.1 release of Eclipse. This document | |
| is the third of a trilogy. The first described the | |
| <a href="http://www.eclipse.org/eclipse/development/inflexible-projects-problem.html"> | |
| nature of the problem</a>, and the second mapped out the various | |
| <a href="http://www.eclipse.org/eclipse/development/towards-more-flexible-projects.html"> | |
| options for addressing it</a>. This third document describes the concrete proposed solution | |
| in detail.<br><br> | |
| Last Modified: 18:30 October 24, 2002</p> | |
| </blockquote> | |
| <hr/> | |
| <h2>Overview</h2> | |
| <p> | |
| Analysis of the problem identified three characteristics of Eclipse projects that made them | |
| difficult to use in conjunction with existing project layouts. Namely, Eclipse projects are | |
| currently <b>single-rooted</b>, <b>all-inclusive</b>, and <b>non-overlapping</b>. | |
| The proposed solution includes a new mechanism in the Platform core for allowing projects | |
| to be multi-rooted, a new JDT facility for interacting with a selective subset of a project, and a relaxation | |
| of the restriction that resources cannot overlap in the file system. For readers of the previous | |
| document that mapped out the solution space, these are solutions P-2, Q-1, M-2, S-2, V-1 | |
| and V-2. | |
| </p> | |
| <p> | |
| Relaxing these restrictions is necessarily a breaking change for many tools built on the Eclipse platform. | |
| Some plug-ins may be able to adapt to the new behavior in future releases, but some may never | |
| work in the presence of projects that relax these layout restrictions. To minimize the impact, the | |
| proposed solution adheres to the following principles: | |
| <ul> | |
| <li>Only expose the new functionality in the contexts where flexibility problems have been reported. | |
| For example, users will be able to change the location of JDT source folders and the output folder, | |
| but not be able to change the location of folders at arbitrary depths. Exposure of the new flexibility can | |
| be widened to fit new contexts where compelling use cases arise. | |
| <li>Cause no breakage for projects that maintain the old constraints. That is, if a user's project | |
| continues to be single-rooted, all-inclusive, and non-overlapping, then all plug-ins should continue | |
| to function with no code changes needed.</li> | |
| <li>Maintain binary compatibility with 2.0- and 1.0-based plug-ins.</li> | |
| </ul> | |
| </p> | |
| <p> | |
| These solutions will not fundamentally change the workspace resource model. The workspace | |
| still maintains a faithful representation of corresponding files on the local disk. Local refresh will | |
| still cause the workspace to be updated to accurately reflect the changed contents on disk. Operations | |
| on resources in the workspace will continue to have direct impact on files in the file system -- deletion | |
| from the workspace will cause deletion of files from disk, etc. Directories on disk that are associated with | |
| resources in the workspace will continue to "belong" to the workspace, and this association | |
| will still be recursively all-inclusive. Projects will still have a single, non-overlapping root directory. | |
| The only significant change is that a project can now attach contents that are not in its root directory. | |
| This allows a single project to bring together contents from several different directories on disk. | |
| </p> | |
| <hr/> | |
| <h2>1. Multi-Rooted Projects</h2> | |
| <p> | |
| Two categories of problems were identified in relation to the single-rootedness of Eclipse projects. | |
| The most common complaint is that it is impossible to locate the Java builder's output folder | |
| in another location. The other complaint is that users wanted to draw together sources from several | |
| locations into a single project. This second case arises commonly for users of the Eclipse JDT, but also | |
| for users of the CDT (C/C++ tools project). In general, this problem boils down to allowing the project to | |
| contain folders located elsewhere in the file system. | |
| </p> | |
| <h3>New Platform Core API</h3> | |
| <p> | |
| Part of the proposed solution is to add new API in the Platform core to allow <i>linking</i> | |
| external resources into projects (solution M-2 from the previous document). This will | |
| allow clients of the workspace API to logically connect a resource sub-tree located outside | |
| of the project's content area into a project. | |
| </p> | |
| <p> | |
| The <code>link</code> operation is invoked on an IFile or IFolder handle in a fashion similar | |
| to the <code>create</code> method. That is, it must be called on a resource that does | |
| not yet exist, and the result of the operation is that the resource (and possibly child | |
| resources) exists. No files or folders in the file system are moved or modified as | |
| a result of creating the link. The link operation essentially just creates resources in the | |
| Eclipse workspace tree corresponding to all files at the given file system location. | |
| Similarly, deletion of a link removes the corresponding resources from | |
| the workspace tree, but leaves the files in the file system untouched. The new terms | |
| <i>linked folder</i> and <i>linked file</i> (generally, <i>linked resource</i>) will be used | |
| to describe these new entities. This document will also use the term <i>link location</i> | |
| to refer to the file system path corresponding to a given linked resource. Children of linked | |
| folders are <b>not</b> referred to as linked resources. | |
| </p> | |
| <p> | |
| Support will only be added for linking resources as direct children of a project. Restricting | |
| links to only the first level makes the model easier to understand for clients and | |
| users. Essentially, the project becomes a logical container that can have arbitrary resources | |
| linked in, and the semantics of other files and folders remain the same. None of the other IDEs surveyed | |
| allowed linking external resources at arbitrary depths. Finally, supporting links | |
| at arbitrary depths would add significant complexity to the implementation, and would affect the | |
| performance of most "deep" workspace operations such as deletion, local refresh, move, etc. | |
| </p> | |
| <h4>Copy, move and deletion of linked resources</h4> | |
| <p> | |
| Links will have similar semantics to soft symbolic links in Unix. Copy and move semantics are as follows: | |
| <ul> | |
| <li>If a linked resource is copied or moved into another project, a new linked | |
| resource will be created in the destination project that points to the same | |
| location.</li> | |
| <li>Copying or moving a linked resource into another folder will not be permitted | |
| (since links must be direct children of projects). The UI could offer to do | |
| a deep copy/move of all resources in this case ("Linked resources can | |
| only exist immediately below projects. Do you want to copy/move the contents | |
| of the underlying files into /x/y/z?").</li> | |
| <li>If a linked resource is renamed, it will continue to be a linked resource | |
| pointing to the same location.</li> | |
| <li>If a resource contained within a linked folder is copied or moved, it will | |
| be copied or moved in the file system in the normal way. No links will be | |
| created or deleted.</li> | |
| <li>If a project containing linked resources is copied or moved, the new project | |
| will contain linked resources to the same locations.</li> | |
| </ul> | |
| Deletion semantics: | |
| <ul> | |
| <li>Deletion of a linked resource will never delete the contents on disk. This | |
| is symetrical to the behavior of link creation: you cannot create the target | |
| of a linked resource from within Eclipse, and you cannot delete the target | |
| of a linked resource from within Eclipse. We will add a flag for deleting | |
| the contents of the linked resource in the file system, but it will not be | |
| the default behaviour of the <code>delete</code> operation.</li> | |
| <li>Deletion of a resource contained within a linked folder will cause the underlying | |
| files to be deleted from disk in the normal way.</li> | |
| <li>Deletion of a project containing a linked resource will also never delete | |
| the contents of the linked resource on disk.</li> | |
| </ul> | |
| <h4>Pre-validation of link locations</h4> | |
| <p> | |
| A new validation method (<code>IWorkspace.validateLinkLocation</code>) will be introduced | |
| to allow clients to check if a proposed file system location is an acceptable location to link into the | |
| workspace. This method will return a status with severity <code>IStatus.ERROR</code> if the | |
| proposed location is not acceptable (for example, if it contains characters that Eclipse does not | |
| allow in filenames, or if it overlaps the workspace metadata area). The method will return a | |
| status with severity <code>IStatus.WARNING</code> if the proposed location overlaps | |
| another resource location in the workspace. The section on overlapping resources will | |
| discuss this in more detail. | |
| </p> | |
| <h4>Persistence of link data</h4> | |
| <p> | |
| Link locations will be represented by entries in the .project file. Modifying the .project file | |
| can cause links to be created or deleted. A new kind of Core path variables will | |
| be introduced, and link locations can be relative to these variables. Resources will be | |
| created for all links described in the .project file, even if they cannot be accessed (for | |
| example if based on an undefined variable or location that is not accessible). | |
| </p> | |
| <h4>Facility for disabling links</h4> | |
| <p> | |
| There may be some plug-ins that, for whatever reason, do not want to tolerate links | |
| in projects they operate on. For example, a web site management tool may | |
| rely on the file system structure of a project being the same as the structure in the | |
| resource tree. A mechanism will be introduced to allow plug-ins to prevent | |
| links from being added to a given project. | |
| </p> | |
| <p> | |
| An attribute will be added to project natures that specifies whether the nature | |
| allows links. This will default to being true (links allowed), if the | |
| nature does not specify it. If any nature installed on a project does not allow | |
| links, then attempts to create links will fail. Attempting to install a nature | |
| that does not allow links on a project that already has links will fail. | |
| </p> | |
| <p> | |
| Additionally, a hook will be added so that team providers have an opportunity to disallow | |
| links. Team providers based on 2.0 will <b>not</b> allow links on projects that they | |
| manage. Team providers based on 2.1 or greater must specify that they tolerate links. | |
| </p> | |
| <p> | |
| If either of these mechanisms decides to veto links, users will not be | |
| able to override this decision (except by removing the intolerant plug-in). | |
| </p> | |
| <h3>Implications for team providers</h3> | |
| <p> | |
| Generally, links will be ignored by team providers. The assumption is that | |
| these files are generally taken from some location outside the workspace because they are | |
| already managed by an external VCM tool. The java output folder is also usually of | |
| no interest to VCM tools. Team providers that have been upgraded to Eclipse 2.1 | |
| will be expected to silently tolerate and ignore links. This includes handling of links in the team | |
| move/delete hook. If team providers wish to provide more support for linked resources, | |
| they may do so. As mentioned earlier, team providers based on Eclipse 2.0 will not | |
| tolerate the presence of links on projects they manage. | |
| </p> | |
| <h3>Platform UI Changes</h3> | |
| <p> | |
| The Platform UI will add new operations to allow users to create links as top | |
| level folders and files. It may be useful to provide a warning in the UI when linking is | |
| first attempted to warn users that this may cause problems for other plugins (A warning | |
| dialog with the option, "Don't tell me again" would be appropriate). | |
| Linked resources should have some visual annotation in the resource navigator to make it | |
| easy to distinguish linked resources from normal resources in the UI. | |
| </p> | |
| <p> | |
| The Platform UI will also add a preference page and dialogs for adding, removing, and editing the platform | |
| core variables used in link locations. This will be similar to the current UI for java classpath variables. | |
| </p> | |
| <h3>JDT Changes</h3> | |
| <p> | |
| JDT core will need to revisit compiler and search infrastructure that currently relies | |
| on direct navigation of resources in the file system. Since it is no longer guaranteed that all | |
| source folders are children of the project's local directory, they will need to consult the | |
| workspace API to find the correct locations. | |
| </p> | |
| <p>New flexibility will also be added for java output folders, specifically: | |
| <ul> | |
| <li>Linked output folders can be used for placing build output outside the project location (P-2).</li> | |
| <li>Each source folder can specify a separate output folder (Q-1). | |
| <li>Multiple projects can use the same output folder.</li> | |
| </ul> | |
| </p> | |
| <p> | |
| Allowing multiple projects to share an output folder introduces a new problem. Currently, each java | |
| builder assumes that it has complete control of the output folder. On full build, it currently deletes the | |
| entire output folder, which will obviously not be acceptable in a shared scenario. If the builder | |
| does not delete the output folder before full builds, then any extra .class files in the output | |
| folder will influence the classpath of that project. This can have unexpected results at runtime | |
| if stale classes in the output folder obscure legitimate classes in other projects. Neither approach | |
| (scrub everything or scrub nothing) is entirely appealing. However, a flag will be introduced on | |
| output folders to allow either of those two options. If the user turns scrubbing off, they will | |
| have to deal with the problem of performing "make clean" before full builds by themselves. | |
| </p> | |
| <hr/> | |
| <h2>2. Selectively Inclusive Projects</h2> | |
| <p> | |
| In some of the identified problems, users wanted to operate on only a selected subset | |
| of the resources in a project. There are two different angles to this problem: | |
| <ol> | |
| <li>Wanting to completely exclude some resources from the Eclipse workspace</li> | |
| <li>Wanting to exclude some resources from consideration by certain tools (such as the Java builder)</li> | |
| </ol> | |
| </p> | |
| <p> | |
| The first problem is more difficult to address, and would cause more grief for tools that | |
| rely on the all-inclusive nature of the workbench. The difficulty in supporting complete | |
| exclusion boils down to tension between plug-ins that have different needs. One plug-in | |
| may wish to exclude certain resources, while another plug-in operating on the same resource | |
| tree may not want to exclude those same resources. Furthermore, external tools that may | |
| operate on resources in the workspace (such as Netscape™ or Ant scripts), will not be aware | |
| of any resource exclusion mechanism introduced in the platform, and would thus be fundamentally | |
| at odds with such a mechanism. | |
| </p> | |
| <p> For these reasons, support for exclusion will only be introduced at the JDT | |
| level. JDT Core will provide a new facility for adding exclusion rules to source | |
| classpath entries. These exclusion rules will support matching of path prefixes, | |
| e.g., exclude com/xyz/package1/*. Excluded resources will be omitted completely | |
| from the Java model, and will not be considered for compilation. These exclusion | |
| rules will be stored in the .classpath file, and thus automatically shared with | |
| other team members. This approach is equivalent to solution S-2 from the previous | |
| document. </p> | |
| <hr/> | |
| <h2>3. Overlapping Projects</h2> | |
| <p> | |
| The proposed solution is to have almost no restrictions on overlapping link locations. | |
| Existing restrictions on non-overlapping project locations will remain. Linked resources | |
| will only have a small number of restrictions to prevent situations impossible to support. | |
| This breaks down as: | |
| <ul> | |
| <li>Project locations cannot overlap other project locations.</li> | |
| <li>Link locations can overlap other link locations, and other project locations.</li> | |
| <li>Link locations cannot be the same as or a parent of the root directory of the | |
| project the link is contained in (this would result in an infinitely recursive project tree).</li> | |
| <li>Link locations cannot overlap the workspace metadata area (can also result in | |
| infinite recursion of folder structures).</li> | |
| <li>The .project file cannot be a linked file (chicken and egg problem | |
| because link information is stored in the .project file).</li> | |
| </ul> | |
| </p> | |
| <p> | |
| The platform core change will introduce some new behavior and wrinkles | |
| in how resources behave. If two resources correspond to the same underlying | |
| file, then changing one resource will incur a resource delta on both. Also, if a | |
| <code>refreshLocal</code> is invoked on an overlapping portion of the workspace, | |
| then several different resources may be refreshed as a result. Because of the potential | |
| dangers of having overlapping resources (generally that changing one will affect all duplicates), | |
| a warning will be presented when a user tries to create a linked resource whose location | |
| overlaps another resource in the workspace. Calculation of this warning will be done | |
| by the new <code>validateLinkLocation</code> method. | |
| </p> | |
| <p> | |
| This change has ramifications for the following Platform Core methods: | |
| <br><br><code> | |
| IFile IWorkspaceRoot.getFileForLocation(IPath location);<br> | |
| IContainer IWorkspaceRoot.getContainerForLocation(IPath location); | |
| </code><br><br> | |
| The specification of these methods will be clarified to indicate that they will | |
| ignore linked resources. They will continue to only look in each project's root | |
| location for the file system path in question. If a given file exists under both a | |
| project's root directory and under a link location, it will always return the resource | |
| under the project's root directory. Since project root directories cannot overlap, | |
| there is no ambiguity to this solution. New link-aware API methods will be introduced for finding | |
| the set of resources corresponding to a given file system location. | |
| </p> | |
| <h3>JDT Changes</h3> | |
| <p> | |
| JDT Core will add the ability to introduce nested source folders, subject to the condition | |
| that the nested portion is excluded from the parent folder. Users will not be allowed to | |
| link the same folder twice to create two different source folders at the same location in a given project. | |
| </p> | |
| <hr/> | |
| <h2>Examples Redux</h2> | |
| <p> | |
| To show that the proposed solution is sufficient to solve many of the problems described | |
| in the problem definition, we will revisit the four example workspaces from the problem | |
| statement and show how they can be supported by the proposals in this document. | |
| </p> | |
| <p>Example 1:</p> | |
| <p> | |
| |- AllProducts<br> | |
| |- Product1<br> | |
| |- JavaSourceFiles<br> | |
| |- Product2<br> | |
| |- JavaSourceFiles<br> | |
| </p> | |
| <p> | |
| This configuration can be supported using only the new linked resources. | |
| Projects would be left in the default workspace content area, and the source folders | |
| would be linked into the monolithic directory tree. Eclipse artifacts such as .project, | |
| along with build output, would remain in the default area, thus causing no pollution | |
| of the source tree with transient or Eclipse-specific files. This solution assumes that | |
| the resources are not being managed by an eclipse team provider: | |
| <br><br> | |
| /Product1 at default location<br> | |
| /src - java source folder linked at file://AllProducts/Product1/JavaSourceFiles<br> | |
| /Product2 at default location<br> | |
| /src - java source folder linked at file://AllProducts/Product2/JavaSourceFiles<br> | |
| </p> | |
| <p> | |
| If such a monolithic resource tree needs to use an Eclipse team provider, then an | |
| extra project can be used whose location is the "AllProducts" folder. This | |
| extra project would be a simple project without a java builder, and would be used for | |
| source control purposes only. All other projects would not have a team provider. | |
| </p> | |
| <p> | |
| /MasterProject at file://AllProducts (shared with team provider)<br> | |
| /Product1 at default location (no team provider)<br> | |
| /src - java source folder linked at file://AllProducts/Product1/JavaSourceFiles<br> | |
| /Product2 at default location (no team provider)<br> | |
| /src - java source folder linked at file://AllProducts/Product2/JavaSourceFiles<br> | |
| </p> | |
| <p>Example 2:</p> | |
| <p> | |
| |- AllJavaSourceFiles<br> | |
| |- com<br> | |
| |- xyz<br> | |
| |- product1<br> | |
| | |
| |- P1Main.java<br> | |
| |- product2<br> | |
| | |
| |- P2Main.java<br> | |
| </p> | |
| <p> | |
| This common configuration requires support from all three proposal areas. Again projects | |
| would remain in the default content area, one project per product. All projects would link | |
| the same external folder as a source folder, <tt>AllJavaSourceFiles</tt>. Each project's source | |
| folder would then exclude all other product packages from its build classpath. | |
| </p> | |
| <p> | |
| /Product1 at default location (no team provider)<br> | |
| /src - java source folder linked at file://AllJavaSourceFiles, excluding com/xyz/product2.<br> | |
| /Product2 at default location (no team provider)<br> | |
| /src - java source folder linked at file://AllJavaSourceFiles, excluding com/xyz/product1.<br> | |
| </p> | |
| <p> | |
| If such a monolithic resource tree needs to make use of an Eclipse team provider, it | |
| can be arranged in a similar fashion to example 1. An extra project would correspond | |
| to AllJavaSourceFiles, and would have an Eclipse team provider installed. | |
| </p> | |
| <p> | |
| /MasterProject at file://AllJavaSourceFiles (shared with team provider)<br> | |
| /Product1 at default location (no team provider)<br> | |
| /src - java source folder linked at file://AllJavaSourceFiles, excluding com/xyz/product2.<br> | |
| /Product2 at default location (no team provider)<br> | |
| /src - java source folder linked at file://AllJavaSourceFiles, excluding com/xyz/product1.<br> | |
| </p> | |
| <p>Example 3:</p> | |
| <p> | |
| |- Product1<br> | |
| |- JavaSourcesFiles<br> | |
| |- com<br> | |
| |- xyz<br> | |
| | |
| |- product1<br> | |
| | |
| | |
| |- P1Main.java<br> | |
| |- tests<br> | |
| |- com<br> | |
| | |
| |- xyz<br> | |
| | |
| | |
| |- product1<br> | |
| | |
| | |
| |- tests<br> | |
| | |
| |- | |
| P1Test.java<br> | |
| </p> | |
| <p> | |
| This directory structure could be supported in several ways, but the most likely solution is as follows. | |
| A single project would be created. The path Product1/JavaSourceFiles/ would be specified as a source folder. | |
| This folder would have an exclusion rule to omit the "tests" sub-directory. The path | |
| Product1/JavaSourceFiles/tests would be specified as a second source folder, with no exclusion rules. If desired, | |
| the two source folders could be placed in separate Eclipse projects to provide greater separation between | |
| application code and tests. | |
| </p> | |
| <p> | |
| /Product1 at default location <br> | |
| /JavaSourceFiles - java source folder excluding the "tests" sub-directory.<br> | |
| /tests - java source folder<br> | |
| </p> | |
| <p>Example 4: </p> | |
| <p>|- CommonFramework<br> | |
| |- JavaSourceFiles<br> | |
| |- Product1<br> | |
| |- JavaSourceFiles<br> | |
| |- Product2<br> | |
| |- JavaSourceFiles<br> | |
| </p> | |
| <p> | |
| Again this structure can be supported in several ways. The most likely is to create three projects, | |
| mapping to CommonFramework, Product1 and Product2. Product1 and Product2 would create a | |
| linked source folder CommonFramework/JavaSourceFiles. In this case CommonFramework | |
| would not need to be a java project, as it mainly just acts as the VCM container for that set of files. | |
| <p> | |
| /CommonFramework at file://CommonFramework<br> | |
| /Product1 at file://Product1<br> | |
| /JavaSourceFiles - java source folder<br> | |
| /src-common - java source folder linked at file://CommonFramework/JavaSourceFiles<br> | |
| /Product2 at file://Product2<br> | |
| /JavaSourceFiles - java source folder<br> | |
| /src-common - java source folder linked at file://CommonFramework/JavaSourceFiles<br> | |
| </p> | |
| </body> | |
| </html> |