| <!doctype html public "-//w3c//dtd html 4.0 transitional//en"> |
| <html> |
| <head> |
| <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> |
| <title>The Language Toolkit: An API for Automated Refactorings in |
| Eclipse-based IDEs</title> |
| <link rel="stylesheet" href="../../default_style.css"> |
| </head> |
| <body> |
| <div align="right"><font size="-1">Copyright © 2005 Leif Frenzel</font></div> |
| <table border=0 cellspacing=0 cellpadding=2 width="100%"> |
| <tr> |
| <td align=LEFT valign=TOP colspan="2" bgcolor="#0080C0"><b><font |
| face="Arial,Helvetica" color="#FFFFFF"> Eclipse Corner Article</font></b></td> |
| </tr> |
| </table> |
| |
| <center> |
| <h1>The Language Toolkit: An API for Automated Refactorings in |
| Eclipse-based IDEs</h1> |
| </center> |
| <blockquote><b>Summary</b> <br> |
| Anyone who supports a programming language in an Eclipse-based IDE will |
| be asked sooner or later to offer automated refactorings - similar to |
| what is provided by the Java Development Tools (JDT). Since the release |
| of Eclipse 3.1, at least part of this task?which is by no means |
| simple?is supported by a language neutral API: the Language Toolkit |
| (LTK). But how is this API used? <br> |
| <p><b>Leif Frenzel</b> <br> |
| <font size="-1">April 12, 2006 (originally appeared in <a |
| href="http://www.eclipse-magazin.de/" target="_blank">Eclipse Magazin</a> |
| Vol. 5, January 2006)</font></p> |
| </blockquote> |
| <p>"Refactoring" is a programming technique for improving the |
| quality of the source text (e.g. with regard to maintainability or |
| extendibility) while preserving existing behaviour. This is done by the |
| gradual reformulation of the code in which the semantics of the program |
| part being edited is left unchanged. In order to make sure that no |
| inadvertent errors are introduced, each refactoring is checked using a |
| suite of unit tests.</p> |
| <p>Larger refactorings can involve a considerable amount of program code |
| and work. As long as they are implemented in small steps, however, they |
| can often be regarded simply as a sequence of atomic changes. These |
| changes, such as the renaming of variables or the extraction of a |
| program block into a new method, are also described as refactoring. An |
| extensive catalogue with a precise description of the steps required - |
| as well as the potential risks - is provided by <a href="#endnote1">[1]</a>.</p> |
| <p>Reliability is the greatest when the atomic steps of the refactoring |
| can be automated - not only because a tool guarantees the correctness of |
| the automated steps but also because the developers that carry out the |
| refactoring are able to pay attention to the larger context instead of |
| having to carry out the numerous small changes to the actual source |
| code. The Java IDE of Eclipse (JDT) now offers a variety of such |
| automated refactorings (the most common of these are described in detail |
| in <a href="#endnote2">[2]</a>).</p> |
| The JDT team has extracted the recurring elements in the implementation |
| of these refactorings into a language neutral layer that is now |
| available in the Eclipse platform as a standalone API and can thus be |
| used independently of the Java support. This layer is called Language |
| Toolkit (LTK) and can be found in the |
| <code>org.eclipse.ltk.core.refactoring</code> |
| and |
| <code>org.eclipse.ltk.ui.refactoring</code> |
| plug-ins. It also includes an infrastructure (the so-called refactoring |
| participants), which enables other plug-ins to take part in refactorings |
| selectively via extensions (please see the box " |
| <a href="#refactoring_participants">Refactoring Participants</a> |
| "). |
| <p> |
| <blockquote> |
| <table border="1" cellpadding="5" |
| style="border-collapse: collapse; border: solid;" width="100%"> |
| <tbody> |
| <tr> |
| <td> |
| <h2><a name="refactoring_participants">Refactoring Participants (as |
| of Eclipse 3.2M2)</a></h2> |
| <p>PDE (the Plug-in Development Environment of Eclipse) provides an |
| excellent example of the use of Refactoring Participants. It |
| manipulates the manifest files for the development of plug-ins |
| (primarily <code>Manifest.mf</code> and <code>plugin.xml</code>), in |
| which Java classes are often named. If the name of a class changes |
| during a refactoring of the Java source code, then the declaration of |
| this class must also change in the manifest file. PDE participates in |
| Java refactorings in order to meet this requirement. The renaming of |
| classes that are referenced in manifest files is recognised by the |
| PDE and the appropriate adjustments are made.</p> |
| |
| <h3>Refactorings outside the Java-IDE</h3> |
| <p>Eclipse is probably still best known as a Java IDE. The potential |
| of Eclipse as a platform for the integration of tools that come from |
| different programming languages environments is also becoming |
| increasingly evident, however. This can be shown particularly well |
| using the refactoring participants: A project that is developing an |
| IDE for C/C++ based on Eclipse has been in progress for a number of |
| years at Eclipse.org: The C/C++ Development Tools (CDT) project. |
| Since the appearance of its version 3.0 <a href="#endnote3">[3]</a>, |
| CDT offers automated refactorings that are implemented based on LTK.</p> |
| <p>Since these refactorings can now be used by CDT users, it is a |
| small step to use refactoring participants in an area that combines |
| both programming languages: Java Native Interface (JNI) bindings. The |
| JNI makes it possible to bind functions from a library implemented in |
| C to methods declared as native in a Java class so that they can be |
| invoked from Java code. To do this, however, the C function must |
| comply with certain naming conventions (i.e. the name of the Java |
| method and class that it belongs to is encoded in its name). The |
| native implementation of the following method:</p> |
| <pre> |
| package com.xyz; |
| public class NativeImpl { |
| private static native void someMethod(); |
| }</pre> |
| <p>is bound to in the C-code as follows:</p> |
| <pre>JNIEXPORT void JNICALL Java_com_xyz_NativeImpl_someMethod(JNIEnv* env, jclass cl) </pre> |
| |
| <p>In a workspace in which Java and C development are combined, it |
| would be desirable to have refactorings on the Java side (like the |
| renaming or moving of native methods or classes which contain such |
| methods), as well as those on the C side (the renaming of JNI-bound |
| functions) be recognised on the corresponding side, and if the |
| appropriate changes could be performed automatically. The refactoring |
| participants would have to be implemented in a standalone plug-in in |
| order for JDT and CDT to remain mutually independent of each other. |
| Such a plug-in would ideally be an extension of both development |
| environments.</p> |
| </td> |
| </tr> |
| </tbody> |
| </table> |
| </blockquote> |
| <h2>Refactoring of Properties</h2> |
| <p>The following implementation <a href="LTK-article-example.zip">example</a> |
| should enable the renaming of keys in properties files. Several such |
| files (which may be in a single project, but could also be distributed |
| throughout the entire workspace) are often combined into a bundle for |
| the localisation of user interface texts. A file containing default |
| values might be named <code>texts.properties</code> (no language |
| information is included in the file name).</p> |
| <p>Related files are identified with the help of a naming convention. |
| The file <code>texts_de.properties</code> contains translations into |
| German, <code>texts_fr.properties</code> contains French texts, etc. The |
| renaming of a key in such a file usually entails changes in all other |
| files of the bundle. Such a process is ideal for automation - and the |
| example implementation makes this possible.</p> |
| <p>When the user opens a properties file in a text editor and selects a |
| property key, he can start a refactoring by selecting an entry in the |
| context menu that renames the key in the file, optionally in all files |
| of the associated bundle or even in all properties files in the |
| workspace. This refactoring leverages the same familiar user interface |
| as those provided by the Java IDE; including the the same preview |
| option. This is implemented with the help of LTK.</p> |
| <p align="center"><a name="figure1"><img src="images/image001.png" /><br /> |
| Figure 1: Refactoring of property keys</a></p> |
| <h2>Preconditions</h2> |
| <p>Refactorings are transformations of the program; in most cases, they |
| are a manipulation of source code. In order to carry out such |
| transformations (and the analyses preceding them), a programmatically |
| manipulable representation of the developed system is required.</p> |
| <p>JDT for example has an object model of all language elements |
| (classes, fields, methods - both in the source text and in libraries or |
| compiled class files) for the entire Eclipse-workspace (all projects) - |
| as well as a (generated on demand) representation of individual source |
| files as an abstract syntax graph (Abstract Syntax Tree - AST). This |
| enables a specific search for occurrences and language elements. Even |
| simple refactorings contain partial processes such as "Find all |
| places where the variable x is used". A plug-in will hardly be able |
| to provide automated refactorings without tools for such analyses.</p> |
| <p>The theoretical background is just as essential (almost even more |
| important): A very good understanding of the places in which the source |
| code must change is required (and what the code should look like after |
| the change) so that the program semantic is preserved. A formal |
| description derived from the language specification is ideally |
| available. With such a description, the code manipulations themselves as |
| well as the corresponding test cases can be developed.</p> |
| <h2>The life cycle of a refactoring</h2> |
| <p>Refactorings in Eclipse follow an exact, predefined procedure:</p> |
| <ol> |
| <li>The refactoring is started by the user.</li> |
| <li>An initial quick check is performed to determine whether the |
| refactoring is applicable at all in the context desired by the user (<code>checkInitialConditions()</code>).</li> |
| <li>The user is asked for additional information if necessary.</li> |
| <li>After all necessary information is available for executing the |
| refactoring, a careful check is triggered (<code>checkFinalConditions()</code>) |
| and the individual changes in the source text are calculated (<code>createChange()</code>).</li> |
| <li>The preview dialogue displays the changes; the user confirms them |
| and the LTK applies them to the workspace.</li> |
| </ol> |
| <p>The API classes in both the LTK core and UI plug-ins incorporate this |
| sequence. The individual steps will be explained in detail below based |
| on the code of our example plug-in.</p> |
| <h2>Core and UI</h2> |
| <p>A subclass of <code>org.eclipse.ltk.core.refactoring.Refactoring</code> |
| must always be created. This class is the core of a refactoring; all |
| validations of the necessary operations are performed within it; even |
| the actual changes to the source text are calculated here. The methods |
| mentioned in the process under 2. and 4. are part of the protocol of |
| this class.</p> |
| <p>If refactoring participants should be supported, the class <code>ProcessorBasedRefactoring</code> |
| must be extended instead; the actual functionality is then delegated to |
| an implementation of a <code>RefactoringProcessor</code>. In addition to |
| the lifecycle methods of the refactorings, this also defines an entry |
| point for loading the participants. (Since the example for this article |
| does not provide for any participants, the implementation remains |
| empty.)</p> |
| <p>A subclass of <code>org.eclipse.ltk.ui.refactoring.RefactoringWizard</code> |
| is required on the UI side. It is primarily responsible for managing the |
| individual pages of the Wizard, which the user scrolls through by means |
| of 'Next' and 'Back', and for invoking the final operation when |
| finishing the process. All these tasks are already taken over by the |
| super class (<code>RefactoringWizard</code>). It is necessary to provide |
| the wizard with a reference to a <code>Refactoring</code> subclass and |
| to insert one?s own additional wizard pages (derived from <code>UserInputWizardPage</code>) |
| if required. These pages are displayed during the course of the |
| refactoring in the wizard. |
| <h2>Action!</h2> |
| <p>Step 1. The typical trigger for a refactoring is an action of the |
| user when working in an editor or a click on a menu item in the Outline |
| View. In both cases, it is an implementation of an Eclipse action that |
| provides the entry point for the refactoring. In the <code>run()</code> |
| method, instances of <code>RenamePropertyRefactoring</code> and <code>RenamePropertyWizard</code> |
| (the respective subclasses of <code>Refactoring</code> or <code>RefactoringWizard</code> |
| in the example implementation) are created and the refactoring lifecycle |
| of the LTK is started by means of a <code>RefactoringWizardOpenOperation</code> |
| (<a href="#listing1">Listing 1</a>).</p> |
| <p>First, the available information about what the user selected when he |
| triggered the refactoring is collected. With the current selection, it |
| is possible to determine which language element should be renamed, moved |
| or manipulated in another way. The example plug-in has a text selection; |
| the selected text and its position in the text document are saved in an |
| Info object, which is evaluated later in the <code>RenamePropertyRefactoring</code>. |
| </p> |
| <p><a name="listing1">Listing 1 Class RenameProperty (ui.actions)</a></p> |
| <pre> |
| RefactoringProcessor processor = new RenamePropertyProcessor( info ); |
| RenamePropertyRefactoring ref = new RenamePropertyRefactoring( processor ); |
| RenamePropertyWizard wizard = new RenamePropertyWizard( ref, info ); |
| RefactoringWizardOpenOperation op = new RefactoringWizardOpenOperation(wizard ); |
| try { |
| String titleForFailedChecks = ""; //$NON-NLS-1$ |
| op.run(getShell(), titleForFailedChecks ); |
| } catch( InterruptedException irex ) { |
| // operation was cancelled |
| }</pre> |
| <p>Step 2. The LTK assumes control of the subsequent process. It first |
| invokes the method <code>checkInitialConditions()</code> on the |
| refactoring object. This method should perform a quick check to |
| determine whether the basic conditions have been met for the |
| refactoring. The example implementation merely verifies that the user |
| has actually selected an existing key in a properties file. If it |
| encounters a fatal situation (e.g. a write-protected source file), it |
| generates an object of the type RefactoringStatus, which contains more |
| information about the problem (<a href="#listing2">Listing 2</a>). In |
| such a case, the LTK does not continue with the refactoring but instead |
| informs the user about the abort by means of a dialogue box.</p> |
| <p><a name="listing2">Listing 2 Class RenamePropertyDelegate (core)</a></p> |
| <pre> |
| RefactoringStatus checkInitialConditions() { |
| RefactoringStatus result = new RefactoringStatus(); |
| IFile sourceFile = info.getSourceFile(); |
| if( sourceFile == null || !sourceFile.exists() ) { |
| result.addFatalError( CoreTexts.renamePropertyDelegate_noSourceFile ); |
| } else if( info.getSourceFile().isReadOnly() ) { |
| result.addFatalError( CoreTexts.renamePropertyDelegate_roFile ); |
| } else if( isEmpty( info.getOldName() ) || !isPropertyKey( info.getSourceFile(), info.getOldName() ) ) { |
| result.addFatalError( CoreTexts.renamePropertyDelegate_noPropertyKey ); |
| } |
| return result; |
| }</pre> |
| <h2>More Interaction</h2> |
| <p>Step 3. After determining that there are no fundamental obstacles |
| preventing the renaming of the key, the wizard is displayed and asks the |
| user for additional information: For example, the new name that he wants |
| to assign for the language element; as well as information about the |
| scope where the refactoring should be applied (only in the current |
| source file, in the project, or in the entire workspace). The UI |
| required for this purpose is implemented as subclasses of <code>UserInputWizardPage</code>. |
| The data entered here is made available to the <code>RenamePropertyRefactoring</code> |
| with the help of the info object (which plays the role of a presentation |
| model).</p> |
| <p> |
| <h2>Detail work</h2> |
| <p>Step 4. Before the wizard shows the last page, which offers a |
| detailed preview on the changes to be performed, two things happen: The |
| method <code>checkFinalConditions()</code> is invoked on the refactoring |
| object and the actual changes are calculated on all of the files |
| concerned. The latter must be implemented in the method <code>createChange()</code>.</p> |
| <p>The example implementation demonstrates these two steps by first |
| searching all properties files concerned in the workspace and checking |
| to see whether they can be edited; at the same time, occurrences of the |
| property key are found. A <code>TextFileChange</code> instance that |
| describes the replacement of a range of text at a given position in the |
| file is generated for each of these occurrences (<a href="#listing3">Listing |
| 3</a>). These Change objects are arranged into a tree-like structure |
| whose root is returned by the method <code>createChange()</code>. (It is |
| out of the scope of this article to discuss the manipulation of text |
| files across the entire workplace. A good introduction to this topic is |
| offered by <a href="#endnote4">[4]</a>).</p> |
| <p><a name="listing3">Listing 3 Class RenamePropertyDelegate (core) </a></p> |
| <pre> |
| private Change createRenameChange() { |
| // create a change object for the file that contains the property |
| // which the user has selected to rename |
| IFile file = info.getSourceFile(); |
| TextFileChange result = new TextFileChange( file.getName(), file ); |
| // a file change contains a tree of edits, first add the root of them |
| MultiTextEdit fileChangeRootEdit = new MultiTextEdit(); |
| result.setEdit( fileChangeRootEdit ); |
| |
| // edit object for the text replacement in the file, this is the only child |
| ReplaceEdit edit = new ReplaceEdit( info.getOffset(), |
| info.getOldName().length(), |
| info.getNewName() ); |
| fileChangeRootEdit.addChild( edit ); |
| return result; |
| }</pre> |
| <p>Step 5. If the user does not skip this by pressing the |
| "Finish" button immediately, a preview is displayed as the |
| last page of the wizard; here the user can view all the pending changes |
| in detail (and deselect some if necessary). No files have actually been |
| changed in the workspace up to this point. All changes to the files are |
| applied only after they have been confirmed by the user. (If change |
| objects from the LTK are used exclusively, then undo-functionality is |
| provided as a small present.)</p> |
| <p align="center"><a name="figure2"><img src="images/image003.png" /><br /> |
| Figure 2: The preview page gives an overview of all changes in the |
| course of the refactoring.</a></p> |
| <h2>Conclusion</h2> |
| <p>The Language Toolkit facilitates the implementation of automated |
| refactorings. It defines the process of a refactoring and provides |
| abstract classes in the core (<code>Refactoring</code>, <code>Change</code>) |
| as well as UI components (<code>RefactoringWizard</code>) that largely |
| cover the language-neutral part of an implementation. The language |
| specific calculation of the changes in the source text can be inserted |
| in this frame so that users can implement the refactoring in the |
| familiar form of the Java refactoring support known from JDT.</p> |
| |
| <blockquote><i><b>Leif Frenzel</b> is the Senior Architect at |
| Innoopract. His main field of work is the Eclipse-Distribution Yoxos as |
| well as development environments based on the Eclipse platform. In |
| addition to this, he is the author of Open-Source-Plug-ins for the |
| Support of Functional Programming Languages in Eclipse.</i></blockquote> |
| <h2>Endnotes</h2> |
| <ul> |
| <li><a name="endnote1">[1]</a> Martin Fowler: Refactoring. Improving |
| the Design of Existing Code, Addison-Wesley, 2000</li> |
| <li><a name="endnote2">[2]</a> Martin Lippert, Andreas Havenstein: |
| Elementares Handwerkszeug. Refactorings mit Eclipse schrittweise, |
| effizient und sicher durchführen (Basic skills. Refactoring with |
| Eclipse step by step, efficiently and easily), in Eclipse Magazin Vol. |
| 4</li> |
| <li><a name="endnote3">[3]</a> Markus Knauer, Elias Volanakis: Back to |
| the Future. C- und C++-Entwicklung mit Eclipse und CDT 3.0 (C and C++ |
| Development with Eclipse and CDT 3.0), in Eclipse Magazin Vol. 4</li> |
| <li><a name="endnote4">[4]</a> Kai-Uwe Mätzel: Safely Manipulating the |
| Contents of Files - How to Get it Right, Presentation at the EclipseCon |
| 2005: <a |
| href="http://www.eclipsecon.org/2005/presentations/EclipseCon2005_5.1Maetzel.pdf">http://www.eclipsecon.org/2005/presentations/EclipseCon2005_5.1Maetzel.pdf</a></li> |
| </body> |
| </html> |