<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
  <title>Remaining Eclipse 3.0 responsiveness issues</title>
  <link rel="stylesheet" href="http://dev.eclipse.org/default_style.css" type="text/css">
</head>
<body text="#000000" bgcolor="#FFFFFF">
 
<h1>Remaining Eclipse 3.0 responsiveness issues</h1>
<font size="-1">Last modified: November 6, 2003</font> 
<p>
This document outlines the remaining UI responsiveness issues at the start of the
3.0 M5 cycle.  A dynamic team has been set up with representatives from Platform 
Core, Platform UI, JDT Core, JDT UI, and Team to address these remaining issues during
the M5 development cycle. 
</p>
<h3>Consistent progress API</h3>
<p>
There are currently at least three different APIs for reporting progress on a blocking
(modal) operation: the SWT BusyIndicator, ProgressMonitorDialog, and IStatusLineManager.
There are many more ad-hoc mechanisms used by different components, as well
as various dialogs (such as wizard dialog) that have their own progress mechanisms.
This results in inconsistent progress presentation for blocking operations.  Furthermore,
BusyIndicator operates in the UI thread, and the other APIs provide an option to run
in the current thread (fork=false).
</p>
<p>
Proposed solution:<br>
<ol>
<li>Introduce new progress API  that doesn't commit to a particular presentation.
Prototype API is included in current 3.0 stream I-builds.  See
<tt>IWorkbench.getProgressManager</tt> and <tt>IProgressManager.busyCursorWhile</tt>.
This should probably be renamed to <tt>IProgressService</tt> to be more consistent
with other workbench APIs, and avoid committing to the busy cursor in the method name
(e.g., <tt>IProgressService.busyRun(IRunnableWithProgress)</tt>).</li>
<li>Decide what to do with remaining APIs.  Most likely deprecate. It may be possible
to port some old APIs to forward to new APIs, but not BusyIndicator, and not ModalContext 
in fork=false case.</li>
<li>Port all existing uses of old progress mechanisms to use new progress service (TBD).</li>
</ol>
<h3>Presentation of blocking operation progress</h3>
<p>
How should long running progress be presented by this new progress API?  The two
likely candidates are progress dialog or status line progress indicator.  Tendency
has been to move away from a modal dialog to the status line because a modal
dialog has usability problems: breaks user's task concentration, hides information the 
user may currently be reading.  Requirements for a progress presentation are:
<ul>
<li>Graphical presentation of progress</li>
<li>A text message describing the current task</li>
<li>Cancelation</li>
<li>Indication when the foreground operation is blocked by a background operation</li>
<li>A mechanism to allow the user to resolve the blockage (such as presentation of
progress view to allow cancelation of background jobs).</li>
</ul>
<p>
The current prototype uses a busy cursor, followed by a progress monitor dialog
after a sufficient delay.  This avoids the distraction of a popping dialog for most cases,
but gives enough room to describe the progress for longer running cases and blocked
operations.
</p>
<p>
<img src="../images/new.gif" width="31" height="14"> We also need a story for how dialogs with embedded
progress indicators will work (e.g.: wizards, launch configuration dialog, 
preference dialog).  Having a nested progress dialog within another
dialog does not look nice.  Need to revisit all implementors of IRunnableContext
to see if new API is needed.  Dirk suggested keeping IRunnableContext for consistency,
possibly just changing the javadoc to indicate that the &quot;cancel&quot; parameter 
may be ignored (cancelation should always be possible).
</p>
<h3>Presentation of non-modal progress</h3>
<p>
We are learning about a number of situations where background operations
are causing presentation problems in views. The following are descriptions of
usability problems introduced by background activty. We are looking for input
from other teams on other scenarios not covered here.
</p>
<h4>1. Incomplete/busy views</h4>
<p>
Some background jobs cause information to be incrementally added to a view.
This results in a state where the view is &quote;busy&quote; for the duration of the job.
During that period, the user needs to know that the information they are seeing is
possibly incomplete.  They also need to be aware that the view is in a state where
it may change suddenly, causing unexpected flashing and scrolling.  There also needs
to be visible feedback when the view is no longer in this state.
</p>
<p>
Examples:
<ul>
<li>JUnit view when tests are running</li>
<li>CVS repository view when an element is being expanded</li>
<li>Search view during a background search</li>
</ul>
</p>
<p>
<img src="../images/new.gif" width="31" height="14"> This new functionality should have a generic component
that provides a uniform presentation of a busy view.  Possible solutions include
an icon overlay, changing title bar gradient, changing appearance of part pane border,
and changing the cursor.
Could also use a *brief* flashing title bar to indicate completion of long running task.
The mechanism should have a hook method on WorkbenchPart that allows subclasses
to add additional busyness indicators within the view body.
</p>
<h4>2. Stale views</h4>
<p>
Many views contain information that is *possibly* incorrect during a background
job.  The important distinction from the busy view scenario is that the information
in the view during the job may actually be valid before, during, and after the job
has been run.  It is generally too much work to decide in advance what information
is invalid so it can be hidden or removed.  Generally, the job is currently doing the 
work to decide whether the information needs to be updated.  This leads to errors
when the user tries to perform actions on this stale information (for example jumping
to a missing marker, or open a missing file).
</p>
<p>
Examples:
<ul>
<li>Task and Problem views when a build is occurring</li>
<li>Editor marker bar when document is dirty and while build is ocurring</li>
<li>Team Synchronize view while background refresh is running</li>
<li>Navigator view during import or checkout</li>
</ul>
</p>

<h4>3. Inadequate indication of background activity</h4>
<p>
The current indication of background activity is weak.  It is important that the user
is aware when background operations are going on.  If they know that <b>something</b>
is happening, they are more likely to tolerate stale information, busy views, pauses,
etc.  Most applications use an animated 32x32 icon in the upper right hand corner of
the window.  Eclipse currently uses a 16x16 icon in the lower left corner, due to layout
problems on certain platforms (i.e. Mac) with icons in the upper right.  Even people working 
on the responsive UI have been in a state where they did not notice that background 
activity was happening, and were frustrated by stale information as a result.
</p>
<p>
Suggestion from discussions this week: the &quot;3.0 look&quot; group should look into this.
An indicator that &quot;breaks out&quot; of the workbench window using the new
SWT non-rectangular shell support might give enough feedback (in addition to looking
slick).
</p>

<h3>Long running activity in UI thread</h3>
<p>
Currently many &quot;short&quot; (1-10 seconds) operations run directly in the
UI thread.  This causes the entire view to become unresponsive (no painting). 
Further, this leaves absolutely no way for the UI to inform the user that they are
blocked by background activity, and no way for the user to cancel.  For these reasons,
any operation that may be blocked by a very long running activity must be run 
outside the UI thread. This includes all operations that modify resources and markers.
</p>
<p>
As of I20031029, marker changes no longer require any scheduling rule.
This means there is nothing that will block marker changes for extended periods
due to conflicts with ongoing background activity. The remaining problem is that almost
all marker changes are wrapped within an <tt>IWorkspaceRunnable</tt>, which for backwards
compatibility reasons must lock the entire workspace.  A new <tt>IWorkspace.run</tt> method
is now available that takes a resource scheduling rule.  This method is used for finer-grained
locking of resources during the invocation of an <tt>IWorkspaceRunnable</tt>.  
A 3.0 porting guide section will be needed to describe why clients should update their
calls to <tt>IWorkspace.run</tt> to use the method that supports finer-grained locking.
UI clients will need to do a sweep to begin using this method where applicable.
</p>
<p>
Since marker changes are now less likely to block, it is ok for them to run in the
UI thread.  However, it is still the case that all resource (file, folder) changes should
be moved into a background thread using the new UI progress service.  This will
allow users to cancel where the change is not compatible with background activity.
</p>
<p>
A suggestion was made to resolve this issue by shifting the burden 
to the implementors of background operations.  If the UI thread tries to make a 
change that conflicts with the background operation, the background operation could 
be suspended to allow the UI thread to proceed.  This would require that we provide
the background operation with a resource delta describing what changes occurred
while it was suspended.  It could then decide whether it should proceed, abort, or
rollback depending on the nature of the conflicting change.  However, even a resource
delta would not always provide sufficient information for it to make a decision.  If
the operation was in the middle of making a series of changes to a file when interrupted,
it would not be able to rollback correctly without a binary delta describing how the file
changed while it was interrupted.  These transaction semantics would be extremely 
difficult to implement.  Another approach is to require the background operation
to break up long running work into small transactions, and require the operation
to support cancelation after each transaction.  This is the approach taken by the 
automatic cancelation support in background autobuild.  To be effective, each transaction
would need to be of sufficiently small duration that locking the UI for that duration
would be acceptable.  It is not clear in the general case that all background operations
could be implemented in this way.
</p>

<p>
We also discussed the possibility of hacking in support for cancelation even when
operations are run in the UI thread. This can be sold as a &quot;backwards compatibility&quot;
story for UI components that do not move their long running operations out of the
UI thread.  This can be achieved by adding a method to IProgressProvider to specify
a default progress monitor when none is provided by the operation.  This monitor
can then spin the event loop (ala EventLoopProgressMonitor), and open a dialog
when the foreground operation is blocked.
</p>
<p>
<img src="../images/new.gif" width="31" height="14"> A suggestion was raised that this support could work by
opening a second display with its own event loop, just for the purpose of giving
some form of progress feedback when people writing long running code in the 
UI thread.  Mcq believes this will not work on all platforms.
</p>
<h3>Slow computer preference</h3>
<p>
In 3.0 M4 a slider preference was introduced to allow the user to indicate that they
have a slow computer.  The intent is that highly CPU intensive features should consult
this preference before deciding whether they should run.  This can also be used to
throttle the amount of background activity, and to increase the sleep times used by non-critical
background activities such as decoration, editor reconciling, and indexing. We need to
identify what features should use this preference, and begin making use of it.
</p>
<h3>Periodic notifications</h3>
<p>
One of the items in the <a href="plan_concurrency_listeners.html">builders and listeners</a> proposal
was to introduce resource change notifications during long running operations.  There
are several motivations for this:
<ul>
<li>Provides additional feedback that the system is busy, since views will be updating more often</li>
<li>Allows the user to begin working with the output of an operation before the operation
completes. For example, the user may want to begin editing files in a project that is 
currently being checked out from the repository.</li>
<li>If short resource changing operations happen in one thread while a long running
operation is happening in another, the results of the short operation need to be
displayed in a timely manner.  For example, if a user adds a breakpoint during a CVS
checkout, the breakpoint icon must appear quickly.  This requires a resource change
notification.</li>
</ul>
Drawbacks of more frequent notifications include:
<ul>
<li>Breaking 2.1 API contracts that guarantee batching during invocation of an 
IWorkspaceRunnable.</li>
<li>Wasted listener update work when a resource changes several times during an operation.</li>
<li>Hindered performance of background operation due to delta calculations and broadcast,
and increased activity in UI thread.</li>
<li>Flashy user interface</li>
</ul>
</p>
<p>
The new <tt>IWorkspace.run</tt> method now also takes a
&quot;style bit&quot; that can be used to specify that periodic notifications are not
wanted.  This allows clients to provide a hint to core that it doesn't make sense
to perform resource change notfications during that operation.  The old <tt>run</tt>
method forwards to the new method and allows specifies the <tt>AVOID_UPDATE</tt>
flag.  So, clients calling the old API will get old style batching for free.  This flag is phrased
as a hint because notifications are sometimes required when multiple operations run
concurrently.  I.e, if two threads are modifying the workspace, one using <tt>AVOID_UPDATE</tt>
and the other not, then periodic notifications will occur for the duration of the operation
that did not specify <tt>AVOID_UPDATE</tt>.
</p>
<p>
Another suggestion was to provide partial workspace updates.  
If a notifying operation runs concurrently with a non-notifying operation, the 
workspace could broadcast a partial delta reflecting only the changes made by the 
notifying operation. See the 
<a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=43718#c13">bug report discussion</a>
on this proposal.  In short, this would be quite complex to implement and would open
up new breaking changes for listeners.
</p>
<h3>Porting guide</h3>
<p>
Enumerate API breakages and deprecations, and major non-API behaviour changes
that may impact 2.1-based clients.  Begin drafting of 3.0 porting guide chapters
for the responsive UI work.
</p>
<h3>Miscellaneous</h3>
<h4>Posting behind current operation</h4>
<p>
<img src="../images/new.gif" width="31" height="14"> Dirk to provide more detail here since the phone connection 
was breaking up.  The issue is when you want to &quot;post&quot; a chunk of
code to run after the current operation is completed.  This is currently accomplished
using asyncExec, but if the operation is run in the UI thread, then the asyncExec
may be processed before the operation completes due to hacks like
EventLoopProgressMonitor.  This is possible with operations built on the 3.0 jobs
API, but not for foreground (non-job) operations.  For jobs, clients can call IJobManager.currentJob
to find the currently running job, then hook a listener on the job to fire another 
job/operation when the first job completes.  There were no recommendations for
how to handle this for clients not using the jobs API.
</p>
<h4>Concurrency test harness</h4>
<p>
<img src="../images/new.gif" width="31" height="14"> The core concurrency test harness may have some pieces that can be generalized
and pushed down into the test.harness project, to help other components in
writing concurrency tests.
</p>
<hr>
<p>
Go <a href="../planning/3.0/plan_concurrency.html">back to concurrency home</a>.
</p>
(Plan item bug reference: <a
 href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=36957">36957</a>)<br>
</body>
</html>