presentation/eclipsecon2012/hits/index.html

<!DOCTYPE html>

<!--
  Google HTML5 slide template

  Authors: Luke Mahé (code)
           Marcin Wichary (code and design)
           
           Dominic Mazzoni (browser compatibility)
           Charles Chen (ChromeVox support)
           John Arthorne (Eclipse template)

  URL: http://code.google.com/p/html5slides/
-->

<html>
  <head>
    <title>Eclipse SDK's Greatest Hits: The First Ten Years</title>

    <meta charset='utf-8'>
    <script src='http://eclipse.org/eclipse/presentation/html/slides.js'></script>
  <style>
    /* Your individual styles here, or just use inline styles if that's
       what you want. */
    
    
  </style>
  </head>
  

  <body style='display: none;'>

    <section class='slides layout-regular template-eclipse'>
      
      <!-- Your slides (<article>s) go here. Delete or comment out the
           slides below. -->
        
      
      <article id='splash' class='biglogo'>
      </article>

      <article id='title'>
        <h1>
          Eclipse SDK's Greatest Hits:
          <br>
          The First Ten Years
        </h1>
        <p>
          John Arthorne
          <br>
          March, 2012
        </p>
      </article>

      <article id="disclaimer">
        <h3>
          Disclaimer
        </h3>
        <ul>
          <li>
            This is a semi-random, subjective list
          </li>
          <li>
            Just some deep Eclipse implementation details that I think are cool, from my perspective
          </li>
          <li>
            This presentation contains code
          </li>
        </ul>
        <p>
          <img style='height: 300px;' src='images/warning.png'>
        </p>
        <div class='source'>
          Source: http://www.flickr.com/photos/cara_vsangel/
        </div>
      </article>

      <article id="overview">
        <h3>
          What we're going to look at
        </h3>
        <ul>
          <li>
            Workspace tree representation
          </li>
          <li>
            Incremental Java build
          </li>
          <li>
            RCP refactoring
          </li>
          <li>
            Command framework
          </li>
          <li>
            p2 resolver
          </li>
        </ul>
      </article>

      <!-- ************************************************************************ -->

      <article id="tree-intro">
        <h3>
          Workspace tree representation
        </h3>
        <q>
          It's ElementTree my dear Watson, ElementTree
        </q>
        <div class='author'>
          ElementTree.java, line 75
        </div>
      </article>

      <article id="tree-motivation">
        <h3>
          ElementTree motivation
        </h3>
        <ul>
          <li>
            Typical edit/compile/deploy cycle for a developer focuses on a small segment of a potentially very large code base
          </li>
          <li>
            Want to aggressively optimize for this common cycle: make performance cost proportional to the change, rather than to the size of the workspace
          </li>
          <li>
            Create and manipulate &quot;units of change&quot; that are passed around to interested parties
          </li>
        </ul>
      </article>
      
      <article id="tree-diag1">
        <h3>
          Data flow
        </h3>
        <p>
          <img style='height: 520px;' src='images/tree.png'>
        </p>
      </article>

      <article id="tree-observations">
        <h3>
          Additional observations
        </h3>
        <ul>
          <li>Clients want a delta of the workspace state between two moments in time</li>
          <li>Different clients may want deltas with different start and/or end points</li>
          <li>May need to keep track of hundreds of tree states at once</li>
          <li>How to efficiently represent all these different deltas in memory?</li>
          <li>How to compute these deltas without traversing entire workspaces?</li>
        </ul>
      </article>

      <article id="tree-diag2">
        <h3>
          Representing two tree states
        </h3>
        <p>
          <img style='height: 520px;' src='images/tree2.png'>
        </p>
      </article>

      <article id="tree-diag3">
        <h3>
          Representing tree states as deltas
        </h3>
        <p>
          <img style='height: 520px;' src='images/tree3.png'>
        </p>
      </article>

      <article id="tree-details">
        <h3>
          More details
        </h3>
        <ul>
          <li>Have only one complete tree, and represent all others as deltas</li>
          <li>Only one delta is mutable - representing current operation</li>
          <li>Immutability allows trees to share common parent and nodes to share children</li>
          <li>Each tree seems complete when traversing using API, but internal structure quite different</li>
          <li>Computing deltas is nearly free for most common operations</li>
        </ul>
      </article>
      <article>
        <h3>
          Lessons learned
        </h3>
        <ul>
          <li>Not all optimization is a speed-space tradeoff. Here we greatly reduced space required to represent states, and made delta calculation very fast</li>
          <li>Textbook data structures work very well in most programming situations, but sometimes 
          a very specialized structure is called for</li>
          <li>Internal representation of a data structure can be radically different from external appearance</li>
        </ul>
      </article>

      <!-- ************************************************************************ -->

      <article id="builder-intro" class='smaller'>
        <h3>
          Incremental Java build
        </h3>
        <q>
          Conducts a trial on a single compilation unit. Returns true if the unit
          is guilty. The sentence is compilation without possibility of parole.
        </q>
        <div class='author'>
          IncrementalImageBuilder.java v_146, line 1570
        </div>
      </article>

      <article id="builder-assumptions">
        <h3>
          Starting assumptions
        </h3>
        <ul>
          <li>Compilation is expensive</li>
          <li>Compiling a single file can create a ripple effect where many more files need compiling</li>
        </ul>
        <p>
          <img class='centered' style='height: 350px;' src='images/ripple.jpg'>
        </p>
        <div class='source'>
          Source: Bill Gracey
        </div>
      </article>

      <article id="builder-1-0">
        <h3>
          Eclipse 1.0 Java builder
        </h3>
        <ul>
          <li>Used a trial metaphor</li>
          <li>When a file was compiled it produced evidence (e.g., class B is a subclass of A)</li>
          <li>When A is changed, all files that refer to it are put on trial (e.g., method foo() removed from A)</li>
          <li>All the corner cases are considered: does B refer to foo(), does it also declare foo() and invoke super?</li>
          <li>Compile guilty files; repeat</li>
        </ul>
      </article>

      <article id="builder-revisit">
        <h3>
          Revisiting assumptions
        </h3>
        <ul>
          <li>Meanwhile JDT compiler was rewritten to be blazingly fast</li>
          <li>You only need to recompile dependents when there are structural changes</li>
          <li>Structural changes have little or no ripple effect</li>
          <li>Keeping all the evidence around is expensive and slows down compilation </li>
        </ul>
      </article>
  
      <article id="builder-2-0">
        <h3>
          Builder 2.0
        </h3>
        <ul>
          <li>When a file is compiled, check for structural changes by comparing signatures with previous .class file</li>
          <li>If no structural changes, we are done</li>
          <li>If there are structural changes, recompile all dependents</li>
          <li>Faster on average and vastly more memory efficient than Eclipse 1.0 builder</li>
        </ul>
      </article>

      <article id="builder-lessons">
        <h3>
          Lessons learned
        </h3>
        <ul>
          <li>Sometimes you need to revisit your starting assumptions</li>
          <li>Sometimes the optimal solution is not the best!</li>
          <li>A simpler solution is often easier to maintain (unnecessary compilation no longer a bug)</li>
        </ul>
      </article>

      <!-- ************************************************************************ -->

      <article id="rcp-intro">
        <h3>
          RCP refactoring
        </h3>
        <q>
          Performs arbitrary actions when the event loop crashes.
        </q>
        <div class='author'>
          WorkbenchAdvisor.java, line 297
        </div>
      </article>

      <article id="rcp-background">
        <h3>
          RCP history
        </h3>
        <ul>
          <li>As an <b>integration</b> platform, Eclipse focus was on seamless integration</li>
          <li>Plugins by design had little or no control over general appearance</li>
          <li>Plugins could only add; not remove</li>
          <li>But some people wanted to be able to build applications that looked very different</li>
          <li>Wanted to remove some of the built in functionality</li>
        </ul>
      </article>

      <article id="rcp-refactoring">
        <h3>
          RCP refactoring
        </h3>
        <ul>
          <li>Mechanical part was scrubbing "tool" artifacts from API</li>
          <li>Harder part: decoupling workbench from the application so other applications could use it</li>
          <li>Required inversion of control - separating policy from mechanism</li>
          <li>IDE-specific behavior and functionality separated from the workbench</li>
          <li>Can't just create preferences - must be under application control</li>
        </ul>
      </article>

      <article id="rcp-advisor">
        <h3>Policy vs mechanism</h3>
        <table>
          <tr>
            <th>
              Mechanism
            </th>
            <th>
              Policy
            </th>
          </tr>
          <tr>
            <td>
              Error handling
            </td>
            <td>
              Show error dialog, log, exit
            </td>
          </tr>
          <tr>
            <td>
              Menus
            </td>
            <td>
              What menus appear, what do they contain
            </td>
          </tr>
          <tr>
            <td>
              Toolbars
            </td>
            <td>
              Decide what commands appear in toolbar
            </td>
          </tr>
          <tr>
            <td>
              Default perspective
            </td>
            <td>
              Select default perspective
            </td>
          </tr>
          <tr>
            <td>
              Startup
            </td>
            <td>
              Restore settings, welcome screen, login
            </td>
          </tr>
          <tr>
            <td>
              Window trim
            </td>
            <td>
              Progress, status line, title
            </td>
          </tr>
        </table>
      </article>

      <article id="rcp-solution">
        <h3>Workbench advisor</h3>
        <ul>
          <li>Application in charge of creating a workbench if desired: <code>PlatformUI.createAndRunWorkbench(Display, WorkbenchAdvisor)</code></li>
          <li><code>WorkbenchAdvisor</code> is consulted on aspects of window appearance, and has
            "hooks" that are run on startup, window open, shutdown, etc.</li>
          <li>Advisor has complete control over default set of commands in menu and toolbars</li>
          <li>Plugins can still contribute their own extra commands if the menu/toolbar exists</li>
        </ul>
      </article>

      <article id="rcp-extensibility">
        <h3>Many ways to add policy</h3>
        <table>
          <tr>
            <th>
              Mechanism
            </th>
            <th>
              Arity
            </th>
            <th>
              User Control
            </th>
            <th>
              App Control
            </th>
            <th>
              Plugin Control
            </th>
          </tr>
          <tr>
            <td>
              Extension points
            </td>
            <td>
              Many
            </td>
            <td>
              No
            </td>
            <td>
              Transforms
            </td>
            <td>
              Yes
            </td>
          </tr>
          <tr>
            <td>
              Services
            </td>
            <td>
              Many
            </td>
            <td>
              No
            </td>
            <td>
              No
            </td>
            <td>
              Yes
            </td>
          </tr>
          <tr>
            <td>
              Preferences
            </td>
            <td>
              Many
            </td>
            <td>
              Maybe
            </td>
            <td>
              Yes
            </td>
            <td>
              Yes
            </td>
          </tr>
          <tr>
            <td>
              System prop
            </td>
            <td>
              One
            </td>
            <td>
              Yes
            </td>
            <td>
              Yes
            </td>
            <td>
              Yes
            </td>
          </tr>
          <tr>
            <td>
              Advisor
            </td>
            <td>
              One
            </td>
            <td>
              No
            </td>
            <td>
              Yes
            </td>
            <td>
              No
            </td>
          </tr>
        </table>
        
      </article>
      
      <article id="rcp-lessons">
        <h3>Lessons learned</h3>
        <ul>
          <li>The "policy" vs "mechanism" distinction burned into committer minds</li>
          <li>When designing functionality, always have to ask if another app might want to do it differently</li>
          <li>Is it something any plugin should be able to control? Or only application? Only user?
          <li>There are many ways to offer extensibility with various tradeoffs</li>
        </ul>
      </article>

      <!-- ************************************************************************ -->

      <article id="command-intro">
        <h3>
          Command framework
        </h3>
        <q>
          Okay. Have a seat. Relax a while. This is going to be a bumpy ride.
        </q>
        <div class='author'>
          WidgetMethodHandler.java, line 81
        </div>
      </article>

      <article id="commands-history">
        <h3>
          Key bindings history
        </h3>
        <ul>
          <li>Eclipse 1.0: hard-coded, unmodifiable</li>
          <li>Eclipse 2.0: plugins can contribute bindings, only editor bindings can be customized</li>
          <li>Eclipse 2.1: user contributed key bindings, concept of scopes but limited (global vs editor)</li>
          <li>Eclipse 3.0: key bindings in dialogs, expanded concept of scopes</li>
          <li>Eclipse 3.1: Introduced command framework.. views can override global commands</li>
        </ul>
      </article>

      <article id="commands-challenges">
        <h3>
          Challenges
        </h3>
        <ul>
          <li>Decisions on every key stroke</li>
          <li>Bindings change with active part/shell</li>
          <li>Ctrl+B could be Build or Bold</li>
          <li>F2 in Package Explorer could be refactor or file rename</li>
          <li>Incrementally bolting on functionality to original design starting to become untenable</li>
          <li>Committers leaving to open breweries, become apple farmers, or to live in an Ashram</li>
        </ul>
      </article>

      <article id="commands-framework">
        <h3>
          Command framework
        </h3>
        <ul>
          <li>Introduced separation between command and handler</li>
          <li>Key bindings are mapped to commands, command get bound to handlers</li>
          <li>Mapping from binding to command is stable -&gt; Cache!</li>
          <li>Relationship between command and handler still needs updating on each part activation</li>
          <li>Only need to process commands that have a handler specific to that part</li>
        </ul>
      </article>

      <article id="commands-lessons">
        <h3>
          Lessons learned
        </h3>
        <ul>
          <li>Incrementally adding functionality over time can reach a breaking point where 
            you need a fundamental rethink</li>
          <li>Creating a pure "model" layer between user inputs and resulting behaviour was very powerful</li>
          <li>Different parts of the problem could be isolated and optimized</li>
        </ul>
      </article>

      <!-- ************************************************************************ -->

      <article id="p2-intro">
        <h3>p2 resolver</h3>
        <q>
          The fix is to avoid the workaround.
        </q>
        <div class='author'>
          Control.java (Motif), 4076
        </div>
      </article>
      
      <article id="p2-background">
        <h3>p2 Background</h3>
        <ul>
          <li>Update manager only resolved feature dependencies</li>
          <li>Features: once installed, always installed</li>
          <li>A feature you don't really care about can block install of something you really want</li>
          <li>Resolving dependencies could sometimes be slow but was generally usable</li>
          <li>Redundant expression of dependencies at feature and plugin level was a source of bugs</li>
        </ul>
      </article>

      <article id="p2-observations">
        <h3>Observations</h3>
        <ul>
          <li>Some installed things are "roots" that the user chose to install, 
other things are installed only to satisfy dependencies	</li>
          <li>In p2 we wanted to ensure all dependencies were satisfied - feature, bundle, and package level</li>
          <li>This is an NP hard constraint satisfaction problem, and our hand-crafted resolver wasn't good enough</li>
        </ul>
      </article>

      <article>
        <h3>Boolean satisfaction problem</h3>
        <ul>
          <li>A series of boolean expressions (AND, OR, and NOT)</li>
          <li>Is there a set of variable assignments that will satisfy all expressions</li>
          <li>Or, what set of values satisfies the most expressions (MAX-SAT)</li>
        </ul>
        <pre>
1: x &amp; !y
2: x | y

Solution: x=true, y=false
</pre>
      </article>

      <article>
        <h3>Mapping install to SAT (Projector)</h3>
        <pre>
a := Plugin A 1.0 - requires B [1.0,2.0)
b := Plugin B 1.0
c := Plugin B 1.1

1: (a &amp; b) | (a &amp; c)
2: (b &amp; !c) | (!b &amp; c)

Solution: a=true, b=false, c=true
</pre>
        <ul>
          <li>Also assign weights to expressions to cause higher plugin version to be favoured</li>
          <li>Throw the problem at a SAT solver</li>
        </ul>
      </article>

      <article id="p2-lessons-learned">
        <h3>Lessons learned</h3>
        <ul>
          <li>Need to recognize when you have an NP hard problem</li>
          <li>There are probably others who have solved this problem more efficiently</li>
          <li>Those things you learned in school about algorithm complexity and isomorphism sometimes come in handy!</li>
        </ul>
      </article>

      <article id="the-end" class='nobackground'>
        <h3>
          Thank you
        </h3>
        <iframe src='http://www.eclipsecon.org/2012/program/session-schedule'></iframe>
      </article>

    </section>

  </body>
</html>