platform-core/documents/3.1/large_scale_issues.html - www.eclipse.org/eclipse - Git at Google

 <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
 <html>
 <head>
   <title>Large-scale development issues</title>
   <link rel="stylesheet" href="http://dev.eclipse.org/default_style.css" type="text/css">
 </head>
 <body text="#000000" bgcolor="#FFFFFF">

 <table width="100%">
 <tr><td style="background:#0080C0"><b><span style="color:white">Large-scale development issues</span></b></td></tr>
 </table>
 <font size="-1">Last modified: February 24, 2005</font>
 <p>
 One of the major development themes for Eclipse 3.1 is to improve support for
 "Large-scale development" in Eclipse. This includes improving collaboration for
 large, distributed teams, but it also encompasses support for large workspaces.
 This document captures requirements submitted in bug reports, mailing lists,
 and other discussions from people using Eclipse for large-scale project development.
 Not all of these issues are committed to be solved in Eclipse 3.1, but this list presents,
 in no particular order, a problem scope from which work items can be chosen. Some
 of these items are already present on the Eclipse 3.1 plan, but are included here
 for completeness.
 </p>
 <h3>1. Memory footprint</h3>
 <p>
 Eclipse imposes a significant RAM footprint when working with a large workspace.
 Identify principal areas of memory consumption and explore opportunities to
 reduce current footprint.
 <ul>
 <li><img src="../../images/ok.gif"  height="10" width="10" border="0">
 	Extension registry footprint.  Eclipse maintains the registry of plug-ins, extensions,
 	and extension points in memory.  As the number of plug-ins and extensions
 	grows, so does this footprint.  Convert the registry to a cache structure that
 	stores infrequently referenced portions on disk and brings extensions into memory
 	in a lazy and transient manner.
 </li>
 <li><img src="../../images/ok.gif"  height="10" width="10" border="0">
 	Workspace tree footprint.  The workspace is represented in memory as a tree
 	containing various data such as resource names, attributes, markers, etc.
 	Explore reducing the amount of data stored in memory for each resource, and
 	other optimizations such as uniquification of strings.
 <li><img src="../../images/ok.gif"  height="10" width="10" border="0">
 	Team/CVS metadata footprint.  The CVS plug-ins store significant information
 	in memory about the synchronization and "dirty" state of each resource. Explore
 	reducing the footprint of this data or using lazy caching to only bring this information
 	into memory when needed.
 <li><img src="../../images/ok.gif"  height="10" width="10" border="0">
 	Message bundles (bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=37712">37712</a>).
 	Most plug-ins store translated strings in <tt>ResourceBundle</tt>
 	objects.  These bundles are not space-efficient, and often use lengthy string-based
 	keys for message lookup.  Explore a more efficient representation, integer-based keys,
 	or a disk-based bundle for infrequently used messages.
 </li>
 </ul>
 <h3>2. Performance of I/O-bound operations</h3>
 <p>
 Large teams often store development artifacts (code, diagrams, documentation)
 on a network file system in order to increase reliability, facilitate backup and restoring of data,
 and to simplify integration and building. I/O-bound operations in Eclipse are typically
 much slower in such environments.  Explore optimization of I/O-bound operations,
 and moving lengthy operations into a background thread.
 <ul>
 <li><img src="../../images/ok.gif"  height="10" width="10" border="0">
 	Project creation. Creating a project at a file system location that contains
 	a large number of existing files and folders requires significant I/O to discover
 	all the files and to gather local information such as time stamps. This project
 	discovery can be moved into a background thread.
 </li>
 <li><img src="../../images/ok.gif"  height="10" width="10" border="0">
 	Resource copy, move, and delete.  Most operations that act on trees of resources
 	still cannot be run in the background.  When these operations take a long time,
 	the user is forced to wait until they complete. These should be converted to "user" jobs
 	that can optionally be run in the background.
 </li>
 <li><font class="indexsub" color="#8080ff">[&gt;3.1]</font>
 Recursive deletion 	(bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=10628">10628</a>).
 Java provides no API for recursively deleting a directory
 containing files and other directories. This means deleting a large resource tree
 requires two native I/O calls per directory (one to list the children, one to delete), and
 one native I/O call per file.  This particularly impacts compilation, which often needs
 to delete large trees of resources in the output (bin) folder. Consider adding a native
 method to improve recursive deletion performance.
 </li>
 </ul>
 </p>
 <h3>3. Project interchange</h3>
 <p>
 Eclipse has always emphasized first-class support for integration of repository tools,
 and has treated repositories as the primary vehicle for code sharing among team members.
 This leaves behind groups that either don't use a repository, or don't use a repository
 that has Eclipse integration plug-ins. The Import/Export wizards are typically used
 by such groups to share code.  Some improvements to the import and export tools
 would them more powerful as a project interchange (sharing) mechanism.
 <ul>
 <li><img src="../../images/ok.gif"  height="10" width="10" border="0">
 	Import multiple projects at once
 	(bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=22698">22698</a>).
 	If you have unzipped, untarred, or checked out a large group of projects from a repository,
 	there is no way to load them all into a workspace at once.  The current "existing project"
 	import wizard only allows importing one project at a time.
 </li>
 <li><img src="../../images/ok.gif"  height="10" width="10" border="0">
 	Import project inside zip file
 	(bugs <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=66798">66798</a>,
 		<a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=67808">67808</a>).
 	The Export wizard allows you to export an entire project into a ZIP or JAR file.  The corresponding
 	Import wizards don't allow you to import that ZIP or JAR file back into a workspace
 	as a top-level project.  The user has to unzip the file and import as existing project,
 	or create a new project with the same name and import the contents.  This should
 	be made easier.  Similarly, it should be possible to import a ZIP containing multiple
 	projects.
 </li>
 <li><font class="indexsub" color="#8080ff">[&gt;3.1]</font>
 	Rename project on import
 	(bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=40493">40493</a>).
 	The "import existing project" wizard doesn't allow you to import a project but
 	pick a different name for the project in the workspace.  This is often needed
 	by users who check out projects from a repository into the file system, and then
 	want to call them something different in the workspace (one common example
 	is when working on multiple streams of the same project in a single workspace).
 </li>
 </ul>
 </p>
 <h3>4. Support for non-incremental builders</h3>
 <p><img src="../../images/ok.gif"  height="10" width="10" border="0">
 	The workspace builder infrastructure is designed primarily with efficient
 	incremental compilers in mind.  Auto-build is turned on by default, and this
 	is only realistic for fast builders.  The workspace should have support
 	for inherently non-incremental or slower builders (such as C compilers and Ant-based builders).
 	In particular, we need to support users working in a heterogeneous environment with
 	some fast incremental builders and some slow non-incremental builders, sometimes
 	with both on the same project
 	(bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=60803">60803</a>).
 	Read the <a href="large_scale_builders.html">proposal</a>.
 </p>
 <h3>5. Improved working sets</h3>
 <p>
 With very large workspaces, working sets are often used to filter the amount of
 information showed in various views, and for scoping long-running tasks such
 as builds and searches. The current working set support has some problems:
 <ul>
 <li>Shared notion of a working set
 	(bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=22328">22328</a>).
 	Each view has to be explicitly and manually
 	scoped to a given working set.  Each long running search or build also needs to have
 	the working set manually chosen.  One particularly bad example is the Java browsing
 	perspective, which has four views that each needs to have its working set specified manually.
 	Consider adding a global notion of a current working set, or a current working set
 	per window.
 </li>
 <li>
 	Dynamic working sets.  A working set is defined as a static list of elements.
 	Consider adding mechanisms to make working sets more flexible, such as
 	wildcards (bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=62646">62646</a>),
 	exclusion (bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=22362">22362</a>),
 	and tracking project creations (bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=15941">15941</a>)
 	and moves (bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=15938">15938</a>).
 	Aggregating or nesting working sets would also be useful for very large scale workspaces.
 </li>
 </ul>
 </p>

 <!-- LEGEND -->
 <table border="0" cellspacing="5" cellpadding="2" width="100%">
 	<tbody>
 		<tr>
 			<td align="left" valign="top" colspan="4" bgcolor="#0080c0"
 				width="100%"><b><font face="Arial,Helvetica"><font color="#ffffff">Legend
 			</font></font></b></td>
 		</tr>
 		<tr>
 			<td valign="top"><img src="../../images/progress.gif" height="5" width="14" border="0"></td>
 			<td width="50%">item is under development.</td>
 			<td valign="top"><img src="../../images/glass.gif"  height="10" width="10" border="0"></td>
 			<td width="50%">item is under investigation.</td>
 		</tr>
 		<tr>
 			<td valign="top"><img src="../../images/ok.gif"  height="10" width="10" border="0"></td>
 			<td width="50%">item is finished.</td>
 			<td valign="top">( )</td>
 			<td width="50%">item is time permitted.</td>
 		</tr>
 		<tr>
 			<td valign="top"><font class="indexsub" color="#8080ff">[&gt;3.1]</font></td>
 			<td width="50%">item is deferred.</td>
 			<td valign="top"><font class="indexsub"><img border="0" src="../../images/new.gif"></font></td>
 			<td width="50%">new</td>
 		</tr>
 	</tbody>
 </table>
 </body>
 </html>
	<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
	<html>
	<head>
	<title>Large-scale development issues</title>
	<link rel="stylesheet" href="http://dev.eclipse.org/default_style.css" type="text/css">
	</head>
	<body text="#000000" bgcolor="#FFFFFF">

	<table width="100%">
	<tr><td style="background:#0080C0"><b><span style="color:white">Large-scale development issues</span></b></td></tr>
	</table>
	<font size="-1">Last modified: February 24, 2005</font>
	<p>
	One of the major development themes for Eclipse 3.1 is to improve support for
	"Large-scale development" in Eclipse. This includes improving collaboration for
	large, distributed teams, but it also encompasses support for large workspaces.
	This document captures requirements submitted in bug reports, mailing lists,
	and other discussions from people using Eclipse for large-scale project development.
	Not all of these issues are committed to be solved in Eclipse 3.1, but this list presents,
	in no particular order, a problem scope from which work items can be chosen. Some
	of these items are already present on the Eclipse 3.1 plan, but are included here
	for completeness.
	</p>
	<h3>1. Memory footprint</h3>
	<p>
	Eclipse imposes a significant RAM footprint when working with a large workspace.
	Identify principal areas of memory consumption and explore opportunities to
	reduce current footprint.
	<ul>
	<li><img src="../../images/ok.gif" height="10" width="10" border="0">
	Extension registry footprint. Eclipse maintains the registry of plug-ins, extensions,
	and extension points in memory. As the number of plug-ins and extensions
	grows, so does this footprint. Convert the registry to a cache structure that
	stores infrequently referenced portions on disk and brings extensions into memory
	in a lazy and transient manner.
	</li>
	<li><img src="../../images/ok.gif" height="10" width="10" border="0">
	Workspace tree footprint. The workspace is represented in memory as a tree
	containing various data such as resource names, attributes, markers, etc.
	Explore reducing the amount of data stored in memory for each resource, and
	other optimizations such as uniquification of strings.
	<li><img src="../../images/ok.gif" height="10" width="10" border="0">
	Team/CVS metadata footprint. The CVS plug-ins store significant information
	in memory about the synchronization and "dirty" state of each resource. Explore
	reducing the footprint of this data or using lazy caching to only bring this information
	into memory when needed.
	<li><img src="../../images/ok.gif" height="10" width="10" border="0">
	Message bundles (bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=37712">37712</a>).
	Most plug-ins store translated strings in <tt>ResourceBundle</tt>
	objects. These bundles are not space-efficient, and often use lengthy string-based
	keys for message lookup. Explore a more efficient representation, integer-based keys,
	or a disk-based bundle for infrequently used messages.
	</li>
	</ul>
	<h3>2. Performance of I/O-bound operations</h3>
	<p>
	Large teams often store development artifacts (code, diagrams, documentation)
	on a network file system in order to increase reliability, facilitate backup and restoring of data,
	and to simplify integration and building. I/O-bound operations in Eclipse are typically
	much slower in such environments. Explore optimization of I/O-bound operations,
	and moving lengthy operations into a background thread.
	<ul>
	<li><img src="../../images/ok.gif" height="10" width="10" border="0">
	Project creation. Creating a project at a file system location that contains
	a large number of existing files and folders requires significant I/O to discover
	all the files and to gather local information such as time stamps. This project
	discovery can be moved into a background thread.
	</li>
	<li><img src="../../images/ok.gif" height="10" width="10" border="0">
	Resource copy, move, and delete. Most operations that act on trees of resources
	still cannot be run in the background. When these operations take a long time,
	the user is forced to wait until they complete. These should be converted to "user" jobs
	that can optionally be run in the background.
	</li>
	<li><font class="indexsub" color="#8080ff">[>3.1]</font>
	Recursive deletion (bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=10628">10628</a>).
	Java provides no API for recursively deleting a directory
	containing files and other directories. This means deleting a large resource tree
	requires two native I/O calls per directory (one to list the children, one to delete), and
	one native I/O call per file. This particularly impacts compilation, which often needs
	to delete large trees of resources in the output (bin) folder. Consider adding a native
	method to improve recursive deletion performance.
	</li>
	</ul>
	</p>
	<h3>3. Project interchange</h3>
	<p>
	Eclipse has always emphasized first-class support for integration of repository tools,
	and has treated repositories as the primary vehicle for code sharing among team members.
	This leaves behind groups that either don't use a repository, or don't use a repository
	that has Eclipse integration plug-ins. The Import/Export wizards are typically used
	by such groups to share code. Some improvements to the import and export tools
	would them more powerful as a project interchange (sharing) mechanism.
	<ul>
	<li><img src="../../images/ok.gif" height="10" width="10" border="0">
	Import multiple projects at once
	(bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=22698">22698</a>).
	If you have unzipped, untarred, or checked out a large group of projects from a repository,
	there is no way to load them all into a workspace at once. The current "existing project"
	import wizard only allows importing one project at a time.
	</li>
	<li><img src="../../images/ok.gif" height="10" width="10" border="0">
	Import project inside zip file
	(bugs <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=66798">66798</a>,
	<a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=67808">67808</a>).
	The Export wizard allows you to export an entire project into a ZIP or JAR file. The corresponding
	Import wizards don't allow you to import that ZIP or JAR file back into a workspace
	as a top-level project. The user has to unzip the file and import as existing project,
	or create a new project with the same name and import the contents. This should
	be made easier. Similarly, it should be possible to import a ZIP containing multiple
	projects.
	</li>
	<li><font class="indexsub" color="#8080ff">[>3.1]</font>
	Rename project on import
	(bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=40493">40493</a>).
	The "import existing project" wizard doesn't allow you to import a project but
	pick a different name for the project in the workspace. This is often needed
	by users who check out projects from a repository into the file system, and then
	want to call them something different in the workspace (one common example
	is when working on multiple streams of the same project in a single workspace).
	</li>
	</ul>
	</p>
	<h3>4. Support for non-incremental builders</h3>
	<p><img src="../../images/ok.gif" height="10" width="10" border="0">
	The workspace builder infrastructure is designed primarily with efficient
	incremental compilers in mind. Auto-build is turned on by default, and this
	is only realistic for fast builders. The workspace should have support
	for inherently non-incremental or slower builders (such as C compilers and Ant-based builders).
	In particular, we need to support users working in a heterogeneous environment with
	some fast incremental builders and some slow non-incremental builders, sometimes
	with both on the same project
	(bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=60803">60803</a>).
	Read the <a href="large_scale_builders.html">proposal</a>.
	</p>
	<h3>5. Improved working sets</h3>
	<p>
	With very large workspaces, working sets are often used to filter the amount of
	information showed in various views, and for scoping long-running tasks such
	as builds and searches. The current working set support has some problems:
	<ul>
	<li>Shared notion of a working set
	(bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=22328">22328</a>).
	Each view has to be explicitly and manually
	scoped to a given working set. Each long running search or build also needs to have
	the working set manually chosen. One particularly bad example is the Java browsing
	perspective, which has four views that each needs to have its working set specified manually.
	Consider adding a global notion of a current working set, or a current working set
	per window.
	</li>
	<li>
	Dynamic working sets. A working set is defined as a static list of elements.
	Consider adding mechanisms to make working sets more flexible, such as
	wildcards (bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=62646">62646</a>),
	exclusion (bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=22362">22362</a>),
	and tracking project creations (bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=15941">15941</a>)
	and moves (bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=15938">15938</a>).
	Aggregating or nesting working sets would also be useful for very large scale workspaces.
	</li>
	</ul>
	</p>

	<!-- LEGEND -->
	<table border="0" cellspacing="5" cellpadding="2" width="100%">
	<tbody>
	<tr>
	<td align="left" valign="top" colspan="4" bgcolor="#0080c0"
	width="100%"><b><font face="Arial,Helvetica"><font color="#ffffff">Legend
	</font></font></b></td>
	</tr>
	<tr>
	<td valign="top"><img src="../../images/progress.gif" height="5" width="14" border="0"></td>
	<td width="50%">item is under development.</td>
	<td valign="top"><img src="../../images/glass.gif" height="10" width="10" border="0"></td>
	<td width="50%">item is under investigation.</td>
	</tr>
	<tr>
	<td valign="top"><img src="../../images/ok.gif" height="10" width="10" border="0"></td>
	<td width="50%">item is finished.</td>
	<td valign="top">( )</td>
	<td width="50%">item is time permitted.</td>
	</tr>
	<tr>
	<td valign="top"><font class="indexsub" color="#8080ff">[>3.1]</font></td>
	<td width="50%">item is deferred.</td>
	<td valign="top"><font class="indexsub"><img border="0" src="../../images/new.gif"></font></td>
	<td width="50%">new</td>
	</tr>
	</tbody>
	</table>
	</body>
	</html>