blob: 4f82f6d286fce610d56f1883d249bd7105029f63 [file] [log] [blame]
<?php require_once($_SERVER['DOCUMENT_ROOT'] . "/eclipse.org-common/system/app.class.php"); require_once($_SERVER['DOCUMENT_ROOT'] . "/eclipse.org-common/system/nav.class.php"); require_once($_SERVER['DOCUMENT_ROOT'] . "/eclipse.org-common/system/menu.class.php"); $App = new App(); $Nav = new Nav(); $Menu = new Menu(); include($App->getProjectCommon()); # All on the same line to unclutter the user's desktop'
$pageTitle = "";
$pageKeywords = "";
$pageAuthor = "";
ob_start();
?>
<div id="maincontent">
<div id="midcolumn"><P STYLE="margin-bottom: 0in">&nbsp;
</P>
<?php
include_once($_SERVER['DOCUMENT_ROOT'] . "/projects/fragments/proposal-page-header.php");
generate_header("Data Tools Top-Level");
?>
<h1>Data
Tools Project</h1>
<P><FONT FACE="Arial, sans-serif">This project proposal is in the
</FONT><A HREF="/org/documents/Eclipse Development Process 2003_11_09 FINAL.pdf"><FONT FACE="Arial, sans-serif">Proposal
Phase</FONT></A> <FONT FACE="Arial, sans-serif">and is posted here to
solicit community feedback, additional project participation, and
ways the project can be leveraged from the Eclipse
membership-at-large. You are invited to comment on and/or join the
project. Please send all feedback to the
</FONT><A HREF="http://www.eclipse.org/newsportal/thread.php?group=eclipse.dtp"><FONT FACE="Arial, sans-serif">http://www.eclipse.org/newsportal/thread.php?group=eclipse.dtp</FONT></A>
<FONT FACE="Arial, sans-serif">newsgroup.</FONT></P>
<h2>Project
Organization</h2>
<P>The Data Tools Project (DTP) is
a proposed open-source Top Level Project of eclipse.org. <a href="charter.html"> The
Charter</a> describes the scope, the organization of the project,
the roles and
responsibilities of the participants, and the top-level development
process for the project.</P>
<h2>Overview</h2>
<P>&ldquo;Data Tools&rdquo;
is a vast domain, yet there are a fairly small number of
foundational requirements when developing with or managing
data-centric systems. A developer is interested in an environment
that is easy to configure, one in which the challenges of
application development are due to the problem domain, not the
complexity of the tools employed. Data management, whether by a
developer working on an application, or an administrator
maintaining or monitoring a production system, should also provide
a consistent, highly usable environment that works well with
associated technologies.</P>
<P>Such an environment
starts with key frameworks designed both for use and
extensibility. Examples include location and management of data
source drivers, and configurations for access to particular data
source instances. Once a connection is successfully made, the next
task often is to explore the data source, making changes as
required. Some of these operations might be carried out by GUI
actions, others directly through commands. For example, users &ndash;
both developers and administrators &ndash; typically will create,
edit, and test SQL for these commands. Assistance in editing SQL
through code completion, formatting, and dialect specialization,
greatly enhances productivity. Further, the ability to execute or
debug commands, both SQL and stored procedures, rounds out the
rapid development process that Eclipse supports so well. Finally,
bridging chasms, whether between relational, object, or other
structures, presents challenges that data management tooling
should address.
</P>
<h2>Project
Principles</h2>
<P CLASS="text-body-indent">Founded
in the spirit of open-source, community-driven principles guiding
eclipse.org itself, this project will concentrate on several key
ideals:</P>
<UL>
<LI><P><I>Vendor neutrality: </I>We
intend to provide data management frameworks and tools not biased
toward any vendor. Our intention is that DTP be leveraged to
provide the Eclipse community with the widest range of choices
possible. To this end, we seek community involvement in
formulating key framework interfaces, so that the largest
possible constituency is represented.</P>
</UL>
<UL>
<LI><P><I>Extensibility:</I> We
recognize both the common need for data tooling infrastructure
and the desire to extend the offerings in new and innovative
ways. To support these efforts, our components will be designed
for, and make good use of, extensibility mechanisms supported by
Eclipse.
</P>
</UL>
<UL>
<LI><P><I>Community Involvement:
</I>Success for DTP, as with other eclipse.org projects, is as
much a factor of community involvement as the technical merit of
its components. We strongly believe that DTP will achieve its
full potential only as the result of deep and broad cooperation
with the Eclipse membership-at-large. Thus, we will make every
effort to accommodate collaboration, reach acceptable
compromises, and provide a project management infrastructure that
includes all contributors, regardless of their affiliation,
location, interests, or level of involvement. Regular meetings
covering all aspects of DTP, open communication channels, and
equal access to process will be key areas in driving successful
community involvement.</P>
</UL>
<UL>
<LI><P><I>Transparency: </I>As
with all projects under the eclipse.org banner, key information
and discussions at every level &ndash; such as requirements,
design, implementation, and testing &ndash; will be easily
accessible to the Eclipse membership-at-large.
</P>
</UL>
<UL>
<LI><P><I>Agile development </I>We
will strive to incorporate into our planning process innovations
that arise once a project is underway, and the feedback from our
user community on our achievements to date. We think an agile
planning and development process, in which progress is
incremental, near-term deliverables are focused, and long-term
planning is flexible, will be the best way to achieve this.</P>
</UL>
<h2>Project
Scope </h2>
<P>Data-centric
applications are those having a connection to a data source, and a
mapping from a data source to an in-memory model. The
distinguishing characteristic of such applications is that their
domain of capabilities is no more specific than <I>data-centric</I>.
For instance, while a Java source file and its in-memory
representation could be considered data, the domain of Java
development is clearly present, and is more specific than just
&ldquo;data.&rdquo; Thus, <I>data-centric</I> delineates an
abstract, foundational domain, which is superseded by more
specific domains when the application manipulates something more
than just &ldquo;data.&rdquo; An application with a more specific
domain is <I>data-dependent</I> rather than <I>data-centric</I>.
<I>Data-dependent</I> applications are not within the scope of
this project.</P>
<UL>
<LI><P>Using a
model-driven approach, the Data Tools Project (DTP) consists of
extensible frameworks and exemplary tools for data-centric
applications. These include:</P>
<LI><P>In-memory representation:
Models providing a domain-based interaction with data, such as
database definitions, query models, result sets, and objects.
These models provide the basis upon which all other DTP
components are constructed.</P>
<LI><P>Connectivity: Specification
and configuration of data source drivers.
</P>
<LI><P>Management: Administration
of data sources including both generic and vendor-specific
configuration options. Examples include adding and removing
tables from a database, setting type information for contained
data, and setting performance parameters.</P>
<LI><P>Data-centric model
transformation: Changing data from one format to another is a
common task in data-centric application development. For example,
there are several popular dialects of the SQL standard. A query
using vendor-specific-dialect extensions is useless when used
with another vendor&rsquo;s data source. Hence, there is a need
to translate between the two dialects.</P>
</UL>
<P>In addition to
these areas, we expect that future versions of DTP will include:</P>
<UL>
<LI><P>Extract-Transform-Load:
Obtaining data from and supplying data to data sources, typically
using large-scale batch movements and involving data checking and
validation. Often includes some operations on data obtained
before loading into target data source.</P>
<LI><P>Data Mapping: Mapping
between data source and in-memory representation, used for
bridging between domains such as object, relational,
hierarchical, and multi-dimensional data structures.
</P>
</UL>
<P>Although the scope
of DTP includes exemplary connectors for popular open source and
commercial data sources, these are not necessarily intended to be
the definitive connectors. Instead, they are intended to serve two
purposes. First, they are intended to enable users to immediately
use these data sources, although possibly with not exploiting all
their features. Second, they are intended to serve as examples to
both commercial and open source developers who want to integrate
data sources into Eclipse. It is consistent with the goals of this
project that the exemplary connectors become superseded by more
complete implementations provided by third parties, both
commercial and open source.</P>
<P><BR>
</P>
<h2>Projects
</h2>
<P>Initially DTP will
contain the following three projects, and emphasize relational
data sources and structures. Since the core frameworks and tools
are data source agnostic, this initial emphasis on relational data
does not preclude contributions in later releases of DTP or from
the community of support for other data source types.</P>
<H3 CLASS="western">Model
Base</h4>
<P>The Model Base
project provides the foundation for DTP. Using industry best
practices such as model-driven development with UML, and taking
advantage of the Eclipse Modeling Framework (EMF), initially
included are models for:</P>
<UL>
<LI><P>Driver
definition</P>
</UL>
<UL>
<LI><P>Database
definition</P>
</UL>
<UL>
<LI><P>SQL</P>
</UL>
<UL>
<LI><P>SQL Query
</P>
</UL>
<P>Key features
supported and benefits provided include:
</P>
<UL>
<LI><P>Support for
change management: Models can be version controlled through
Eclipse team support.</P>
</UL>
<UL>
<LI><P>Broad editing
support: Visual and other means of model editing, integrated with
EMF, enabling seamless editing and EMF model generation.
</P>
</UL>
<UL>
<LI><P>Published
models with documentation (Java Doc) to support DTP consumers.
</P>
</UL>
<UL>
<LI><P>Extensible and
database-agnostic models.</P>
</UL>
<UL>
<LI><P>Complied with
the latest standards, such as SQL.
</P>
</UL>
<UL>
<LI><P>Supporting
JDBC and other connectivity standards</P>
</UL>
<UL>
<LI><P>Sample
code/plug-ins: To demonstrate model usage.</P>
</UL>
<H3 CLASS="western">Connectivity</h4>
<P>The Connectivity
project includes components for defining, connecting to, and
working with data sources. These include:</P>
<UL>
<LI><P>Driver
Management Framework<BR><BR>Access to the appropriate drivers is
a prerequisite for programmatic interaction with data sources.
The Driver Management Framework (DMF) supplies an Eclipse
preference page enabling users to create driver definitions based
on supplied templates. A number of templates are provided in the
base installation, and additional templates can be added by
component developers contributing to DMF extension points.</P>
</UL>
<UL>
<LI><P>Connection
Management Framework<BR><BR>The Connection Management Framework
(CMF) is the foundation upon which specific connection types are
created. The connection types, called <I>Connection Profiles</I>
(CP), are contributed to the CMF through extension points. Users
then connect to data source instances by creating and configuring
a CP for that data source type. Data source-standard
configuration parameters, such as the connection URL, user name,
and password, are provided on CP instance creation and stored as
secure meta-data for the CP. CP allow for host connectivity
checks (&ldquo;ping&rdquo;), connection, auto-connect on CP
startup, and disconnect. Further, CP Extensions enable additional
functionality and content to be added to a CP. For reuse of CP
instance configuration, base export/export functionality is
provided by CPF and surfaced in tools such as the DTP Explorer
(see below). Data source CP then become the connection providers
through which other DTP tooling accesses data source instances.</P>
</UL>
<UL>
<LI><P>JDBC
connection support<BR><BR>DTP will include a JDBC driver template
and CP, as a means of enabling database connectivity, and serving
as an example for further CP development. Database-specific
capabilities can then be surfaced as CP extensions, allowing for
specialization and presentation of differentiating database
functionality directly in that database's CP.</P>
</UL>
<UL>
<LI><P>Data Source
Explorer<BR><BR>The Data Source Explorer (DSE) is an Eclipse view
housing CP instances. From this view, CP capabilities are
surfaced, and data source content is presented. The type and
level of detail for any one instance is constrained only by the
CP itself. DSE also is a provider of CP instance data to clients,
such as drag and drop and API calls. This allows data tooling
requiring connection management to interact with the DSE as a
mediator to CP instances.</P>
</UL>
<UL>
<LI><P>Open Data
Access<BR><BR>The Open Data Access (ODA) component is an open and
flexible data access framework that allows applications to access
data from both standard and custom data sources. It enables data
connectivity between data consumers and data source providers
through published run-time and design-time interfaces. In
addition, the framework also includes an ODA driver management
package that helps an ODA consumer application to manage diverse
behavior of individual ODA data drivers. <BR><BR>A data driver is
created simply by implementing the run-time interfaces defined by
the framework. The run-time interfaces include support for
establishing a connection, accessing meta-data, and executing
queries to retrieve data.&nbsp; A driver can define internal data
source connection profiles and/or work with the CMF's Connection
Profiles extensions. Once developed, the driver can be registered
through an extension point with individual ODA consumer
components to enable data connectivity. The framework also
provides design-time interfaces to integrate custom query
builders within an application designer tool.</P>
</UL>
<H3 CLASS="western">SQL
Development Tools</h4>
<P>The SQL Development Tools
project provides frameworks and tools for deep and broad SQL
support. The frameworks include:</P>
<UL>
<LI><P>Routines Editor Framework<BR><BR>An
extensible framework for editing database routines and SQL
statements. Vender-specific extensions enable specialized support
for particular databases.</P>
</UL>
<UL>
<LI><P>Routines Debugger Framework<BR><BR>While
routine debug support varies widely in existing database
offerings, the Routines Debugger Framework will provide an
extensible base enabling debug support for specific cases in a
manner consistent with existing Eclipse debug infrastructure.</P>
</UL>
<UL>
<LI><P>SQL Query Parser<BR><BR>Although SQL is
defined by a standard, several major dialects exist. Thus, while
the standard must be supported, practicality also demands
flexibility in adjustment to dialects. The SQL Query Parser meets
these needs by providing an extensible framework, enabling
dialect-aware SQL components and tools.</P>
</UL>
<UL>
<LI><P>SQL Execution Plan Framework<BR><BR>The
ability to understand how a SQL evaluation engine will execute a
given query is vital in tuning queries to optimize performance.
The SQL Execution Plan Framework will provide a means for
capturing and presenting execution plans in a generic fashion,
enabling extends to customize support for specific SQL execution
engines.</P>
</UL>
<P>The tools include:</P>
<UL>
<LI><P>SQL Editor<BR><BR>The SQL Editor will
provide an exemplary tool for standard text-based editing of SQL
statements. Providing content assist tied to the SQL Model,
syntax colorization, and multiple statement support, this editor
will provide an essential tool for data-centric development.</P>
</UL>
<UL>
<LI><P>Visual SQL Builder<BR><BR>The Visual
SQL Builder allows for graphical editing of SQL, raising the
level of abstraction, increasing developer productivity, and
making query construction possible for a wider user base.</P>
</UL>
<UL>
<LI><P>Results View<BR><BR>The Results View
displays the output of routine or SQL statement execution in a
tabular form typical of SQL result sets. These results can be
exported to persistent storage in a variety of formats and
reloaded at a later time into the results view.</P>
</UL>
<UL>
<LI><P>Script History<BR><BR>Typically a
number of scripts will be executed repeatedly during the course
data-centric development. The ability to retain a history of
these queries and thereby quickly repeat execution of them and
view results increases productivity. The Script History is a view
meeting these needs, based on a development session.</P>
</UL>
<P><BR>
</P>
<P><BR><BR>
</P>
</div>
</div>
<?php
# Paste your HTML content between the EOHTML markers!
$html = ob_get_contents();
ob_end_clean();
# Generate the web page
$App->generatePage($theme, $Menu, $Nav, $pageAuthor, $pageKeywords, $pageTitle, $html);
?>