| <!doctype html public "-//w3c//dtd html 4.0 transitional//en"> |
| <html> |
| <head> |
| <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> |
| <meta name="Author" content="Build"> |
| <meta name="GENERATOR" content="Microsoft FrontPage 4.0"> |
| <title>Eclipse Platform Core RFC 0009 - Improved Build Story</title> |
| </head> |
| <body> |
| |
| <h1> |
| Improved Build Story</h1> |
| <i><font size=-1>Last revised 17:15 Thursday February 14, 2002</font></i> |
| <p>This proposal covers a series of additions and changes to Eclipse to improve the build story |
| and provider better Ant integration. |
| <h2> |
| Ant</h2> |
| |
| <h3> |
| Running existing Ant scripts inside Eclipse</h3> |
| Part of the perceived value of Eclipse's Ant support is that the Ant scripts |
| written outside Eclipse can be run without modification within Eclipse. |
| This is crucial if Eclipse is to attract open source developers working |
| with existing code bases and Ant scripts. |
| <p>This is not well supported in Eclipse 1.0. The problem is that Eclipse |
| Ant assumes tight integration with Eclipse, with all that entails. Some |
| of things that don't work well: |
| <ul> |
| <li> |
| Ant scripts that use optional predefined Ant tasks, like javac. (The JDK |
| tools.jar needs to be on Ant's runtime classpath.)</li> |
| |
| <li> |
| Ant scripts that use specialized Ant tasks. (There needs to be a way to |
| get other jar libraries on Ant's runtime classpath.)</li> |
| |
| <li> |
| Ant scripts that call System.exit. (These cause Eclipse to exit.)</li> |
| <li> |
| <p ALIGN="LEFT">Ant scripts that call javac don't cooperate with incremental |
| builders. Since auto building is on by default, users can easily end up in a |
| situation where both autobuild and Ant script generates class files, but in |
| different locations. For running the program, the auto built class files are |
| used.</li> |
| <li> |
| <p ALIGN="LEFT">Ant scripts that call javac generate compiler error messages |
| that just up in the Ant console. There is no way to navigate from the error |
| message to the source file.</li> |
| </ul> |
| <p> |
| As a general goal, we need to find ways to make Eclipse Ant support these |
| developers. Here are some general properties that the solution should have:</p> |
| <ul> |
| <li> |
| Running Ant as a separate OS process should have the same semantics as |
| running Ant as part of the running Eclipse, except when Eclipse-specific |
| tasks are invoked. </li> |
| |
| <li> |
| Communication from Eclipse proper to Ant should be via straightforward |
| Ant command line parameters that users could plausibly furnish themselves. |
| This makes it easy to call existing existing Ant scripts in the way they |
| are already familiar with.</li> |
| |
| <li> |
| Eclipse Ant should work with Ant buildfiles wherever they are, whether |
| outside or inside the workspace.</li> |
| |
| <li> |
| The user should control the choice of working directory that Ant runs in.</li> |
| </ul> |
| If it is easy to run inside Eclipse Ant scripts written for a non-Eclipse |
| environment, the converse follows automatically: developers working in |
| Eclipse should be able to write and run scripts that they intend to also |
| use outside Eclipse. Obviously, these scripts should avoid Eclipse-specific |
| Ant tasks, which are not available outside Eclipse. While Eclipse Ant should |
| provide added value for Eclipse users, we should avoid doing things that |
| gratuitously lock users into Eclipse. |
| <p>Using Ant's "if" or "unless" clause makes it quite easy to conditionalize |
| scripts so that they work both inside and outside Eclipse. All that is required |
| is a single well known variable that is given some value when Ant is run inside |
| Eclipse.</p> |
| <p>The proposed changes to Eclipse Ant are as follows: |
| <ul> |
| <li>Run Ant using the same VM as Eclipse, but use a special class loader for |
| running Ant, so that the classpath available to Ant can be controlled.</li> |
| <li>Since running an Ant script is a long-running operation, it will happen in |
| a separate ModalContext thread. <font FACE="Times New Roman">This allows |
| that the UI to update and the user to press a Cancel button. However, it |
| implies that </font> Eclipse-specific Ant tasks will have to be written in |
| such a way that they do not deadlock the system.</li> |
| <li>If feasible, install a security manager to prevent an Ant task from using System.exit |
| to kill Eclipse.</li> |
| <li>Use code from the Ant distribution as is; remove existing hacks.</li> |
| <li>Provide an execution environment consistent with Ant run outside Eclipse.</li> |
| <li>Use the Ant logger class feature to collect output for the Ant Console..</li> |
| <li>Investigate how to provide progress and cancellation while Ant is running |
| a script.</li> |
| </ul> |
| <h3> |
| The "refresh" problem</h3> |
| The following problem becomes painfully apparent when an Ant script written |
| to run outside Eclipse is run from inside Eclipse. The norm for an Ant |
| script is to process some files and create, overwrite, or delete others. |
| For an Eclipse user, there is a good chance that the files in question |
| are somewhere in the workspace. The problem is that the Eclipse workspace |
| has no way of knowing where in the workspace the affected files might be. |
| Without further information, a refresh local on the entire workspace would |
| be needed to ensure that the in-memory resource tree accurately reflects |
| the local file system. This is a very bad situation. Refreshing the entire |
| workspace is time-consuming (at least 0.2 ms per file). But if it is not |
| done automatically, the user would have to worry about it and handle it |
| manually. The user may have more knowledge that would allow them to refresh |
| more selectively --- but it would still be their responsiblity to do it. |
| <p>By its nature, the refresh operation modifies the resource tree. Modifying |
| the resource tree triggers resource change event notifications. And if |
| auto-build is turned on, resource tree modifications also triggers incremental |
| project builders. Thus even a simple thing like refresh can have far-reaching |
| consequences. |
| <p>Here are three options for dealing with this: |
| <ul> |
| <li> |
| <b>Include calls to RefreshLocal in Ant scripts.</b></li> |
| |
| The RefreshLocal task is Eclipse-specific and would only be available |
| when the script in run from Eclipse. This is the most precise option since |
| script authors are in the best position to pin point files in need of refreshing. |
| <li> |
| <b>Selectively refresh workspace after running Ant scripts.</b></li> |
| |
| For each Ant command, allow the user to specify which part of the workspace |
| needs to be refreshed: none, entire workspace, or specific project (more |
| generally, a resource working set, if there were such a thing). This makes |
| it easy for the user to ensure that the refresh happens automatically and |
| to control how much of the workspace is scanned. |
| <li> |
| <b>Autonomous "smart" refresh.</b></li> |
| |
| A background task scans the local files system for files that are out |
| of sync with the workspace and compiles a list. Through the UI, the user |
| controls when the files on this list are refreshed in the workspace. This |
| is a very general option, with no particular connection with Ant scripts. |
| Its drawbacks are that it requires a somewhat pervasive visibility in the |
| UI (like a global toolbar action), and that it is not a programmatic solution. |
| </ul> |
| <p>We plan to support the first and second options, and pursue the third option as a separate work |
| item.</p> |
| |
| <h3> |
| Running Ant Scripts from the Workbench</h3> |
| In Eclipse 1.0, Ant has a very simple UI: |
| <ul> |
| <li> |
| Ant preference page to set verbosity options and change colors or Ant Console</li> |
| |
| <li> |
| Toolbar button for running Ant with a history of previously launched scripts.</li> |
| |
| <li> |
| Context menu item "Run Ant..." appears on *.xml resources.</li> |
| |
| <li> |
| Optional global action set with pulldown action for running an Ant script.</li> |
| |
| <li> |
| Ant Console view, which shows the output in a structured way, grouped by task.</li> |
| <li> |
| <p ALIGN="LEFT">Errors in Ant scripts are shown with problem markers.</li> |
| </ul> |
| The main user requirements are: |
| <ul> |
| <li> |
| It should be easy for a user to configure and run an Ant script. This includes |
| selecting the Ant buildfile, selecting one of the Ant targets, and supplying any command line arguments that |
| might be required. Some of the users will be unfamiliar with Ant and would |
| not able to read an Ant script.</li> |
| |
| <li> |
| <p ALIGN="LEFT">It should be convenient for the user to select which Ant |
| target to run from among the choices in an Ant script.</li> |
| |
| <li> |
| It should be easy for a user to re-run an Ant script that they have run |
| recently.</li> |
| </ul> |
| The "Run..." action of the Windows Start menu is a simple UI that covers |
| both use cases, and serves as a good model. |
| <p>The proposal is to add a new item to the "Workbench" menu, called "Tool |
| Script", which would bring up a Tool Scripts dialog. The Tool Scripts |
| dialog would allow the user to specify the path of the Ant buildfile and |
| command line arguments; the last N such commands would be remembered in |
| a pulldown list. |
| <p>[There are a couple of places in the Eclipse UI where similar kinds of things |
| are done (e.g., Debug and Run pulldowns, search). The UI guidelines should |
| recommend a style of presentation for these kinds of things; this would help |
| fight a proliferation of different presentations, or similar presentations that |
| are not similar enough.] |
| <p>Additional properties: |
| <ul> |
| <li> |
| The dialog should show the full command line so that users can use cut/copy/paste |
| to transfer commands between themselves. All information which Eclipse |
| communicates with the Ant script should be disclosed.</li> |
| |
| <li> |
| Allow the user to indicate the portion of the workspace to refresh after running |
| the script: none, or specified container (working set?).</li> |
| |
| <li> |
| Allow anything that is runnable on that OS in addition to Ant buildfiles; |
| e.g., on Windows, let the user select any *.exe or *.bat file.</li> |
| |
| <li> |
| <font color="#000000">Provide general support for external commands, and |
| augmented support for Ant (prompt for target, structured output, verbosity |
| levels, etc.).</font></li> |
| |
| <li> |
| Allow Ant buildfiles (or other runnables) to be outside of workspace as |
| well as inside a project.</li> |
| |
| <li> |
| Paths for buildfiles outside the workspace should ordinarily be specified |
| with absolute paths in the local file system.</li> |
| |
| <li> |
| Browse button to navigate to Ant buildfile (or other runnable).</li> |
| |
| <li> |
| Eclipse provides small set of documented parameters that can be used in |
| the command line. These parameters get macro expanded each time the command |
| is run. Parameters include: ${project-<i>projectName</i>} - the absolute local |
| file system path of the named project's root directory; other suggestions |
| are the path of the currently selected resource or file, current date or |
| time, location of "workspace", name of currently selected project.</li> |
| |
| <li> |
| It should be possible for plug-ins to extend the set of available parameters. For example, JDT |
| might provide parameters like ${project-output-folder-<i>projectName</i>} so |
| that the user can readily direct a script to generate class files into a Java |
| project's output folder.</li> |
| |
| <li> |
| Paths for buildfiles inside project should ordinarily be specified in project-relative |
| terms; e.g., as ${project-myproject}/scripts/ant1.xml where "myproject" is the |
| name of a workspace project.</li> |
| |
| <li> |
| Consider allowing tool scripts to show up in a submenu of the Tool Scripts menu |
| item so that we can map an accelerator to the actions.</li> |
| |
| <li> |
| The Tool Script action is in a global action set that could be added to |
| a perspective's tool bar (either programmatically, or by the user via Perspective |
| -> Customize).</li> |
| |
| </ul> |
| <p> |
| Implementation details:</p> |
| <ul> |
| <li> |
| Command lines in the Tool Scripts dialog are saved as UI state.</li> |
| |
| <li> |
| By default, command lines would not be considered to be shared metadata. |
| If there were a mechanism for sharing metadata between users (another work |
| item), there might be merit in letting the user specify that certain commands |
| were to be shared.</li> |
| </ul> |
| Other issues to be resolved: |
| <ul> |
| |
| <li> |
| Ant Console view presents the output from running an Ant script to the user.</li> |
| |
| <ul> |
| <li> |
| Could the Ant Console be made to show output from running other runnables?</li> |
| |
| <li> |
| How does the Ant Console view come to the user's attention? Would this view be |
| pre-configured into the standard perspectives? Or would the user have to ask for |
| it?</li> |
| |
| <li> |
| To be consistent with the Debug console, the Ant Console should be made visible when output is written |
| to it.</li> |
| </ul> |
| |
| <li> |
| Support back navigation from the Ant Console to other views. Without the ability to do this (at least in some limited cases), we don't have an integration story. |
| We should use a pluggable approach similar to that used for the Open Selection |
| action in the JDT Console. |
| <ul> |
| <li>N.B. The CDT commander View lets you specify patterns for the line and define how to interpret a line. This is used for error navigation.</li> |
| </ul> |
| </li> |
| </ul> |
| |
| <h2> |
| Build Story</h2> |
| |
| <p> |
| By the Eclipse build story we mean the mechanisms underlying the |
| "Build" and "Rebuild all" actions that appear in the generic |
| workbench UI.</p> |
| |
| <h3> |
| Running Ant Scripts as Builders</h3> |
| A ordinary user would be unlikely to care which incremental builders a |
| project has, or what order they run in. The user just hits "Build". These |
| details are for the plug-in developer who provides the incremental project |
| builders and the project natures which configure them. |
| <p>However, we would like a way to give advanced user the ability to extend |
| what happens when the user hits "Build". |
| <p>The proposal is to allow the user to add additional Ant commands to |
| the project's list of builders. |
| <ul> |
| <li> |
| Build commands in a project build spec would be extended to include things |
| other than incremental project builders.</li> |
| |
| <li> |
| Would allow arbitrary commands as per Tools Script dialog.</li> |
| |
| <li> |
| Could be couched as running a special incremental project builder with |
| all info stuffed in arguments. (There is a bug in current implementation |
| that precludes multiple build commands from using the same builder id.)</li> |
| |
| <li> |
| Project properties page for viewing and extended a project's incremental |
| project builders.</li> |
| |
| <li> |
| User can add or remove extra commands before or after any step in build |
| sequence.</li> |
| |
| <li> |
| User cannot add, remove, change, rearrange, or disable builders. This is a responsibility of |
| the plug-in/project nature.</li> |
| |
| <li> |
| Additional parameter variable available for command line: ${thisProject} |
| - the absolute local file system path of this project's root directory; |
| ${buildType} |
| - "full", "incremental", or "auto".</li> |
| |
| <li> |
| Multiple commands per step so that it is possible to configure one for incremental |
| build and one for |
| full build.</li> |
| |
| <li> |
| Check box to indicate whether autobuilds should be included with other |
| incremental builds.</li> |
| |
| <li> |
| A way to handle scripts which are failing (a "stop on error" checkbox to fail |
| the Build operation when this happens?).</li> |
| |
| <li> |
| Allow the user to indicate the portion of the workspace to refresh after running |
| the script: none, or specified container (working set?).</li> |
| |
| <li> |
| Project build spec is persistent project metadata, shared via the project's |
| description.</li> |
| |
| <li> |
| Extra build steps are therefore automatically shared.</li> |
| |
| <li> |
| Individual builders provide a pre-project preference setting to allow the user to |
| disable it if they can forsee users wanting to (and being permitted to) replace |
| it. For example, it should be possible to disable the Java builder for a project |
| so that |
| the user can run an Ant script that calls a javac instead.</li> |
| |
| <li> |
| Core should treat missing scripts (and other problems) like it does missing/failing |
| builders.</li> |
| |
| <li> |
| Core must catch reentrant case of IncrementalBuild task being invoked from an |
| Ant script being run as a builder.</li> |
| |
| </ul> |
| |
| <h3> |
| Ordering projects</h3> |
| When several projects are involved in a build, the individual projects get |
| built, one at a time, in a certain order. Unless explicitly overridden by the |
| user, the order of project is the default ordering of projects in the workspace |
| as computed by Core using a topological sort based on project references (if |
| project X refers to project Y then build Y before building X). |
| <p>At the very least, we can improve the user experience by taking the following |
| steps :</p> |
| <ul> |
| <li>Users should be discouraged from overriding the default project order |
| computed by Core. This is because only the default project order gets |
| updated automatically when project references change or additional projects |
| are introduced into the workspace.</li> |
| <li>Cycles in project references are detected and quietly broken when Core |
| computes the default project order, often leading to miserable orders which |
| do not work and only frustrate and confuse the user. When Core is unable to |
| compute a surefire default project order, the user must be informed. The |
| user should be kept aware of the problem as long as it persists; otherwise, |
| the user is liable to forget about the problem and release to a team |
| repository a project with screwed up project references, thereby propagating |
| the problem to other developer workspaces.</li> |
| </ul> |
| |
| <h3> |
| Ordering builders within a project</h3> |
| Particular incremental project builders are typically inserted in the project's |
| build specification when it is configured to have a project nature (the |
| code for the project nature does this). However, each project can be configured |
| to have a number of project natures, in no particular order. At present, |
| this mechanism is quite weak. How can a plug-in add a builder I after some |
| other builder J when the project does not have a J? It is reminiscent of |
| each program adding a line to the beginning (or end) of autoexec.bat when |
| it gets installed, with no guarantee that things will work properly if |
| you don't install the programs in the expected order. |
| <p>Note that this is not the kind of question that we want to involve the |
| user in. This is an internal matter for the plug-ins or project natures |
| that are configuring a project. A reasonable solution therefore requires |
| some scheme or mechanism so that plug-in can definitely resolve builder |
| ordering issues. |
| <p>In a separate work item, we are proposing to impose constraints on the |
| order project natures get added to (or removed from) a project. For instance, |
| if project nature B was known to have nature A as a prerequiste, then Core |
| could arrange that a project to be configured with both A and B would get |
| A first, then B. This would mean that the code for project nature B would |
| be able to assume that A had already added its builders to the project. |
| Consequently, A could determine where to add its builders to the project |
| taking into account the positions of B's builders (assuming that A knows |
| the ids of the builders that B adds). |
| <br> |
| </body> |
| </html> |