sketch/proposal.html - www.eclipse.org/proposals - Git at Google

 <!--
 	This document is provided as a template along with some guidance for creating
 	your project proposal. This is just a template. Feel free to change it as
 	you see fit (add sections, remove section). We feel, however, that the
 	suggestions represented in this document represent the reasonable minimum
 	amount of information to move forward.

 	Please keep the formatting in this document simple. Please do not edit
 	this document in Microsoft Word as it adds huge piles of markup that make
 	it difficult to restyle.

 	More information is available here:

 	http://wiki.eclipse.org/Development_Resources/HOWTO/Pre-Proposal_Phase

 	Direct any questions about this template to emo@eclipse.org
  -->

 <html>
 <head>

 <!--
 	Include the title here. We will parse it out of here and include it on the
 	rendered webpage. Do not duplicate the title within the text of your page.
  -->

 <title>Sketch</title>
 <link rel="stylesheet" href="eclipse-proposal.css" type="text/css" media="screen" />
 <style type="text/css" media="screen">
 div.floatright {
 border-color:black;
 border-style:dotted;
 border-width:0 0 0 0;
 clear:right;
 float:right;
 margin-left:10px;
 padding-left:6px;
 position:relative;
 }
 div.floatright p {
 font-size:11px;
 }
 div.floatright a {
 color:#FFFFFF;
 }
 div.floatleft {
 border-color:white;
 border-style:solid;
 border-width:0.5em 1.4em 0.8em 0;
 float:left;
 margin:0.3em 0.5em 0.5em 0;
 position:relative;
 }
 div.floatleft p {
 font-size:10px;
 }
 </style>
 </head>

 <body>
 <p>The Sketch project is a proposed open source project under the <a
 	href="http://www.eclipse.org/projects/project_summary.php?projectid=technology">Technology
 Container Project</a>.</p>

 <p>This proposal is in the Project Proposal Phase (as defined in the
 Eclipse Development Process) and is written to declare its intent and
 scope. We solicit additional participation and input from the Eclipse
 community. Please send all feedback to
 the <a href="http://www.eclipse.org/forums/eclipse.sketch">Sketch forum</a>.</p>

 <h2>Background</h2>

 <!--
 	Optionally provide the background that has lead you to creating this project.
  -->

 <!-- SIDE IMAGES -->
 <div class="floatright">
 <p>An example of the current development state of the API:</p>
 <object width="450" height="361"><param name="allowfullscreen" value="true" /><param name="allowscriptaccess" value="always" /><param name="movie" value="http://vimeo.com/moogaloop.swf?clip_id=2252782&amp;server=vimeo.com&amp;show_title=0&amp;show_byline=0&amp;show_portrait=0&amp;color=ffffff&amp;fullscreen=1" /><embed src="http://vimeo.com/moogaloop.swf?clip_id=2252782&amp;server=vimeo.com&amp;show_title=0&amp;show_byline=0&amp;show_portrait=0&amp;color=ffffff&amp;fullscreen=1" type="application/x-shockwave-flash" allowfullscreen="true" allowscriptaccess="always" width="450" height="361"></embed></object>
 </div>

 <div class="floatright">
 <p>
 Overview of the API structure, the line divides its integration with GEF:
 </p>
 <a href="Sketch_Class_Diagram.png"><img width='450' src="Sketch_Class_Diagram.thumb.png" alt="Overview of the API structure" /></a>
 </div>


 <p>
 Nowadays, with the increasing popularity of touch-enabled devices, it is getting more common to enable users to insert information directly on the screen, using their fingers or a pen, instead of using a mouse. This project aims to enrich user's interaction with GEF/GMF editors using the capabilities provided by touch/tablet devices.
 </p>
 <p>
 Sketch provides an API to allow users to insert elements (EditParts) on GEF/GMF editors by capturing its gestures, typically using a pen on a tablet. The idea is to allow user might freely draw the representation of an element on the editor's area. The API would then transform this rough user's representation on something processable, interpreting what element the user meant to insert.
 </p>
 <p>
 The API might learn from user's input, recognizing each sketch in the way it is drawn by different users. In cases when it is impossible to determine what element is meant by a sketch, the API 'asks' the user, through a small dialog, what he/she meant by that sketch. Thus the API is able to learn for the next times.
 </p>

 <h2>Scope</h2>

 <!--
 	All projects must have a well-defined scope. Describe, concisely, what
 	is in-scope and (optionally) what is out-of-scope. An Eclipse project
 	cannot have an open-ended scope.
  -->

 The project will provide an API and an basic exemplary editor for shapes such as Squares, Triangles and Circles. The API is targeted to developers who wish to add sketching capabilities to GEF/GMF editors, doing so through a small set of configurations, in which is possible to specify, among other things, the elements that should be recognized. The exemplary editor will show how to use the API and expose the provided features to an end user.

 <h2>Description</h2>

 <!--
 	Describe the project here. Be concise, but provide enough information that
 	somebody who doesn't already know very much about your project idea or domain
 	has at least a fighting chance of understanding its purpose.
  -->
 <div>
 <p>
 Traditionally, GEF editors uses one tool for each element of the domain, thus the user must select a tool from the palette and click at the editor, adding a new element to the diagram. The goal of Sketch is to enable users to draw and connect the elements inside the editor's area, using their own gestures.
 </p>
 <p>
 The Sketch API is targeted primarily at editors with a large set of elements, in which the user have to search through the whole GEF palette in order to add them to the editor.
 </p>


 </div>
 <div>
 <h2>Initial Contribution</h2>

 <!--
 	Describe any existing code that will be contributed to the project.
  -->

 <p>
 This API is currently composed of three parts: a sketch <b>Tool</b>, a <b>Recognizer</b> and a sketch <b>Bank</b> (or database).  The figure below shows how the API integrates with GEF, through AbstractTool and GraphicalEditor. The upper part of the figure is an example of integration -- it shows a GEF editor (ShapesDiagramEditor) and three tools (for Square, Triangle and a connection). The integration is made through the SketchTool -- a tool that observes the user's clicks and drags and provides a visual feedback of what is being drawn on the editor. Once the sketch is finished, the tool also passes the points to the SketchManager, which in turn processes those points and updates the editor.
 </p>


 <p>

 Basically, the recognition is made using a chain of algorithms to take part in a decision. The SketchManager gives a list of points captured by the SketchTool to SketchElementRecognizer, and then periodically probes the recognizer for a result. The SketchElementRecognizer creates the chain using a set of algorithms. If the first algorithm of the chain is not 'sure' about what the sketch means, it can pass the decision to the next one on the chain, returning an element or null, if no element could be determined.
 </p>
 <div class="floatleft">
 <a><img  width='350' src="Levenshtein.png" alt="" /></a>
 </div>
 <p>
 The first algorithm on the current chain was made using an approach based on Levenshtein's algorithm for string distance. It receives the user's sketch transformed in a set of points and places them on a grid, then each point is transformed to a number according to the next point's position. This approach is described in detail <a href='http://www.springerlink.com/content/u0r7261w01248q1t/'>here</a>.
 </p>

 <p>
 For each element there is a set of stored '''words''' that describes its many forms, like drawn by the user. The recognizer compares each new sketch with the words of each element, calculating the distance between them. If the distance is smaller for a square than for a triangle, for example, then the element is probably a square.
 </p>
 <p>
 The Chain of Responsibility pattern is applied to allow other algorithms to take part on the decision regarding what element is meant by a given sketch. Currently if a sketch is not recognized by the LevenshteinHandler, it is passed to ConnectionHandler, which analyses if the user is trying to connect two elements.
 </p>
 </div>


 <h2>Committers</h2>

 <!--
 	List any initial committers that should be provisioned along with the
 	new project. Include affiliation, but do not include email addresses at
 	this point.
  -->

 <p>The following individuals are proposed as initial committers to the project:</p>

 <ul>
 	<li>Ugo Sangiorgi (Project Lead)</li>
 	<li>Mariot Chauvin, Obeo  </li>
 </ul>


 <h2>Mentors</h2>

 <!--
 	New Eclipse projects require a minimum of two mentors from the Architecture
 	Council. You need to identify two mentors before the project is created. The
 	proposal can be posted before this section is filled in (it's a little easier
 	to find a mentor when the proposal itself is public).
  -->

 <p>The following Architecture Council members will mentor this
 project:</p>

 <ul>
 	<li>Chris Aniszczyk</li>
 	<li>Ed Merks</li>
 </ul>

 <h2>Interested Parties</h2>

 <!--
 	Provide a list of individuals, organisations, companies, and other Eclipse
 	projects that are interested in this project. This list will provide some
 	insight into who your project's community will ultimately include. Where
 	possible, include affiliations. Do not include email addresses.
  -->

 <p>The following individuals, organisations, companies and projects have
 expressed interest in this project:</p>

 <ul>
 	<li>Chris Aniszczyk</li>
 	<li>Ed Merks</li>
 	<li>Ugo Sangiorgi </li>
 	<li>Obeo (http://www.obeo.fr)</li>
 	<li>Aurélien Pupier (BonitaSoft)</li>
 	<li>Mickael Istria (BonitaSoft)</li>
 	<li>Oisín Hurley</li>
 	<li>David Sciamma (Sierra Wireless)</li>
 	<li>Jens von Pilgrim (GEF3D)</li>
 	<li>Kristian Duske (GEF3D) </li>
 </ul>

 <h2>Project Scheduling</h2>

 <!--
 	Describe, in rough terms, what the basic scheduling of the project will
 	be. You might, for example, include an indication of when an initial contribution
 	should be expected, when your first build will be ready, etc. Exact
 	dates are not required.
  -->
 An initial build should be ready within a month passed the project approval, together with a working example application.

 <h2>Changes to this Document</h2>

 <!--
 	List any changes that have occurred in the document here.
 	You only need to document changes that have occurred after the document
 	has been posted live for the community to view and comment.
  -->

 <table>
 	<tr>
 		<th>Date</th>
 		<th>Change</th>
 	</tr>
 	<tr>
 		<td>02-02-2010</td>
 		<td>Document created</td>
 	</tr>
 </table>
 </body>
 </html>
	<!--
	This document is provided as a template along with some guidance for creating
	your project proposal. This is just a template. Feel free to change it as
	you see fit (add sections, remove section). We feel, however, that the
	suggestions represented in this document represent the reasonable minimum
	amount of information to move forward.

	Please keep the formatting in this document simple. Please do not edit
	this document in Microsoft Word as it adds huge piles of markup that make
	it difficult to restyle.

	More information is available here:

	http://wiki.eclipse.org/Development_Resources/HOWTO/Pre-Proposal_Phase

	Direct any questions about this template to emo@eclipse.org
	-->

	<html>
	<head>

	<!--
	Include the title here. We will parse it out of here and include it on the
	rendered webpage. Do not duplicate the title within the text of your page.
	-->

	<title>Sketch</title>
	<link rel="stylesheet" href="eclipse-proposal.css" type="text/css" media="screen" />
	<style type="text/css" media="screen">
	div.floatright {
	border-color:black;
	border-style:dotted;
	border-width:0 0 0 0;
	clear:right;
	float:right;
	margin-left:10px;
	padding-left:6px;
	position:relative;
	}
	div.floatright p {
	font-size:11px;
	}
	div.floatright a {
	color:#FFFFFF;
	}
	div.floatleft {
	border-color:white;
	border-style:solid;
	border-width:0.5em 1.4em 0.8em 0;
	float:left;
	margin:0.3em 0.5em 0.5em 0;
	position:relative;
	}
	div.floatleft p {
	font-size:10px;
	}
	</style>
	</head>

	<body>
	<p>The Sketch project is a proposed open source project under the <a
	href="http://www.eclipse.org/projects/project_summary.php?projectid=technology">Technology
	Container Project</a>.</p>

	<p>This proposal is in the Project Proposal Phase (as defined in the
	Eclipse Development Process) and is written to declare its intent and
	scope. We solicit additional participation and input from the Eclipse
	community. Please send all feedback to
	the <a href="http://www.eclipse.org/forums/eclipse.sketch">Sketch forum</a>.</p>

	<h2>Background</h2>

	<!--
	Optionally provide the background that has lead you to creating this project.
	-->

	<!-- SIDE IMAGES -->
	<div class="floatright">
	<p>An example of the current development state of the API:</p>
	<object width="450" height="361"><param name="allowfullscreen" value="true" /><param name="allowscriptaccess" value="always" /><param name="movie" value="http://vimeo.com/moogaloop.swf?clip_id=2252782&server=vimeo.com&show_title=0&show_byline=0&show_portrait=0&color=ffffff&fullscreen=1" /><embed src="http://vimeo.com/moogaloop.swf?clip_id=2252782&server=vimeo.com&show_title=0&show_byline=0&show_portrait=0&color=ffffff&fullscreen=1" type="application/x-shockwave-flash" allowfullscreen="true" allowscriptaccess="always" width="450" height="361"></embed></object>
	</div>

	<div class="floatright">
	<p>
	Overview of the API structure, the line divides its integration with GEF:
	</p>
	<a href="Sketch_Class_Diagram.png"><img width='450' src="Sketch_Class_Diagram.thumb.png" alt="Overview of the API structure" /></a>
	</div>


	<p>
	Nowadays, with the increasing popularity of touch-enabled devices, it is getting more common to enable users to insert information directly on the screen, using their fingers or a pen, instead of using a mouse. This project aims to enrich user's interaction with GEF/GMF editors using the capabilities provided by touch/tablet devices.
	</p>
	<p>
	Sketch provides an API to allow users to insert elements (EditParts) on GEF/GMF editors by capturing its gestures, typically using a pen on a tablet. The idea is to allow user might freely draw the representation of an element on the editor's area. The API would then transform this rough user's representation on something processable, interpreting what element the user meant to insert.
	</p>
	<p>
	The API might learn from user's input, recognizing each sketch in the way it is drawn by different users. In cases when it is impossible to determine what element is meant by a sketch, the API 'asks' the user, through a small dialog, what he/she meant by that sketch. Thus the API is able to learn for the next times.
	</p>

	<h2>Scope</h2>

	<!--
	All projects must have a well-defined scope. Describe, concisely, what
	is in-scope and (optionally) what is out-of-scope. An Eclipse project
	cannot have an open-ended scope.
	-->

	The project will provide an API and an basic exemplary editor for shapes such as Squares, Triangles and Circles. The API is targeted to developers who wish to add sketching capabilities to GEF/GMF editors, doing so through a small set of configurations, in which is possible to specify, among other things, the elements that should be recognized. The exemplary editor will show how to use the API and expose the provided features to an end user.

	<h2>Description</h2>

	<!--
	Describe the project here. Be concise, but provide enough information that
	somebody who doesn't already know very much about your project idea or domain
	has at least a fighting chance of understanding its purpose.
	-->
	<div>
	<p>
	Traditionally, GEF editors uses one tool for each element of the domain, thus the user must select a tool from the palette and click at the editor, adding a new element to the diagram. The goal of Sketch is to enable users to draw and connect the elements inside the editor's area, using their own gestures.
	</p>
	<p>
	The Sketch API is targeted primarily at editors with a large set of elements, in which the user have to search through the whole GEF palette in order to add them to the editor.
	</p>


	</div>
	<div>
	<h2>Initial Contribution</h2>

	<!--
	Describe any existing code that will be contributed to the project.
	-->

	<p>
	This API is currently composed of three parts: a sketch <b>Tool</b>, a <b>Recognizer</b> and a sketch <b>Bank</b> (or database). The figure below shows how the API integrates with GEF, through AbstractTool and GraphicalEditor. The upper part of the figure is an example of integration -- it shows a GEF editor (ShapesDiagramEditor) and three tools (for Square, Triangle and a connection). The integration is made through the SketchTool -- a tool that observes the user's clicks and drags and provides a visual feedback of what is being drawn on the editor. Once the sketch is finished, the tool also passes the points to the SketchManager, which in turn processes those points and updates the editor.
	</p>



	<p>

	Basically, the recognition is made using a chain of algorithms to take part in a decision. The SketchManager gives a list of points captured by the SketchTool to SketchElementRecognizer, and then periodically probes the recognizer for a result. The SketchElementRecognizer creates the chain using a set of algorithms. If the first algorithm of the chain is not 'sure' about what the sketch means, it can pass the decision to the next one on the chain, returning an element or null, if no element could be determined.
	</p>
	<div class="floatleft">
	<a><img width='350' src="Levenshtein.png" alt="" /></a>
	</div>
	<p>
	The first algorithm on the current chain was made using an approach based on Levenshtein's algorithm for string distance. It receives the user's sketch transformed in a set of points and places them on a grid, then each point is transformed to a number according to the next point's position. This approach is described in detail <a href='http://www.springerlink.com/content/u0r7261w01248q1t/'>here</a>.
	</p>

	<p>
	For each element there is a set of stored '''words''' that describes its many forms, like drawn by the user. The recognizer compares each new sketch with the words of each element, calculating the distance between them. If the distance is smaller for a square than for a triangle, for example, then the element is probably a square.
	</p>
	<p>
	The Chain of Responsibility pattern is applied to allow other algorithms to take part on the decision regarding what element is meant by a given sketch. Currently if a sketch is not recognized by the LevenshteinHandler, it is passed to ConnectionHandler, which analyses if the user is trying to connect two elements.
	</p>
	</div>



	<h2>Committers</h2>

	<!--
	List any initial committers that should be provisioned along with the
	new project. Include affiliation, but do not include email addresses at
	this point.
	-->

	<p>The following individuals are proposed as initial committers to the project:</p>

	<ul>
	<li>Ugo Sangiorgi (Project Lead)</li>
	<li>Mariot Chauvin, Obeo </li>
	</ul>


	<h2>Mentors</h2>

	<!--
	New Eclipse projects require a minimum of two mentors from the Architecture
	Council. You need to identify two mentors before the project is created. The
	proposal can be posted before this section is filled in (it's a little easier
	to find a mentor when the proposal itself is public).
	-->

	<p>The following Architecture Council members will mentor this
	project:</p>

	<ul>
	<li>Chris Aniszczyk</li>
	<li>Ed Merks</li>
	</ul>

	<h2>Interested Parties</h2>

	<!--
	Provide a list of individuals, organisations, companies, and other Eclipse
	projects that are interested in this project. This list will provide some
	insight into who your project's community will ultimately include. Where
	possible, include affiliations. Do not include email addresses.
	-->

	<p>The following individuals, organisations, companies and projects have
	expressed interest in this project:</p>

	<ul>
	<li>Chris Aniszczyk</li>
	<li>Ed Merks</li>
	<li>Ugo Sangiorgi </li>
	<li>Obeo (http://www.obeo.fr)</li>
	<li>Aurélien Pupier (BonitaSoft)</li>
	<li>Mickael Istria (BonitaSoft)</li>
	<li>Oisín Hurley</li>
	<li>David Sciamma (Sierra Wireless)</li>
	<li>Jens von Pilgrim (GEF3D)</li>
	<li>Kristian Duske (GEF3D) </li>
	</ul>

	<h2>Project Scheduling</h2>

	<!--
	Describe, in rough terms, what the basic scheduling of the project will
	be. You might, for example, include an indication of when an initial contribution
	should be expected, when your first build will be ready, etc. Exact
	dates are not required.
	-->
	An initial build should be ready within a month passed the project approval, together with a working example application.

	<h2>Changes to this Document</h2>

	<!--
	List any changes that have occurred in the document here.
	You only need to document changes that have occurred after the document
	has been posted live for the community to view and comment.
	-->

	<table>
	<tr>
	<th>Date</th>
	<th>Change</th>
	</tr>
	<tr>
	<td>02-02-2010</td>
	<td>Document created</td>
	</tr>
	</table>
	</body>
	</html>