eclipse_newsletter/2015/january/content/en_article2.php - gerrit/www.eclipse.org/community - Git at Google

 <?php
 /*******************************************************************************
  * Copyright (c) 2015 Eclipse Foundation and others.
  * All rights reserved. This program and the accompanying materials
  * are made available under the terms of the Eclipse Public License v1.0
  * which accompanies this distribution, and is available at
  * http://eclipse.org/legal/epl-v10.html
  *
  * Contributors:
  *    Eric Poirier (Eclipse Foundation) - Initial implementation
  *******************************************************************************/
 ?>

 <h1 class="article-title"><?php echo $pageTitle; ?></h1>
     <p>Eclipse has always been a platform that unites different
       technology and makes it possible for developers to use the tools
       of their trade such as language editors, debuggers, profilers,
       version control, all in one environment. Until relatively recently
       scientists and researchers were at the mercy of fragmented suites
       of tools or closed-source workbenches. Actually many still are.
       But now with the increased adoption of Eclipse in the science
       sector, they too can look forward to the huge benefits of tool
       integration and interoperability as well as the resulting speed-up
       in productivity.</p>
     <p>
       When it comes to tools-of-the-trade for scientists, Python is high
       on the list. This is in large part thanks to its fast, powerful
       libraries such as <a href="http://scipy-lectures.github.io/"
         target="_blank">numpy and scipy</a>. Also the dynamic and
       easy-to-use nature of the language lends itself well to the
       exploring necessary for experimental work. The <a
         href="http://software-carpentry.org/v5/novice/python/index.html"
         target="_blank">accessibility of learning resources</a> also
       contributes to Python being the language of choice for the
       scientist embracing programming.
     </p>
     <p>
       So how do you go about providing a tight integration of Python in
       Eclipse; one that allows a user to seamlessly move data around and
       use it both interactively and within views? <span id="more-148"></span>This
       was the challenge faced by <a
         href="http://www.diamond.ac.uk/Home.html" target="_blank">Diamond
         Light Source</a> as they developed their science workbench <a
         href="http://www.dawnsci.org/" target="_blank">DAWN</a>. Diamond
       is a synchrotron in the UK countryside which regularly hosts
       scientists and researchers from diverse fields doing
       groundbreaking research on all manner of things: viruses, retinas,
       dinosaur bones and chocolate to name a few. The users collect huge
       amounts of data for their samples and then perform extensive
       analysis to arrive at their conclusions. Analysis of the data from
       the tomography, crystallography or other experiments is done in
       DAWN, the Eclipse-based data analysis workbench developed by
       Diamond.
     </p>
     <p>The video below shows some of the power of the integrated
       environment. Users can plot arrays in views from the interactive
       console and access regions-of-interest that were created
       graphically from the command line. Knowledge of the types being
       used further enhances the experience, for example dragging a file
       node type to the console automatically pastes the code used to
       load the node from file. To achieve this level of integration of
       Python with DAWN required two key pieces of enabling technology:
       PyDev and AnalysisRPC.</p>
     <br />
     <iframe width="588" height="441"
       src="https://www.youtube.com/embed/BFi0vqGEvYU"
       allowfullscreen></iframe>
     <br />
     <h3>PyDev</h3>
     <p>
       Using Eclipse gives access to the huge Eclipse ecosystem, which
       includes <a href="http://pydev.org/" target="_blank">PyDev</a>.
       PyDev is a feature-rich, popular Python IDE. It is made up of a
       set of mature Eclipse plug-ins which were straightforward to
       customise and integrate into DAWN. This immediately gave DAWN
       features such as context-sensitive code completion, an IPython
       interactive console and rich debugging. That last item is
       particularly powerful &#8211; for instance a user can debug their
       script and see their array values in the variable view. However,
       it is hard to make sense of arrays when seeing them as a list of
       values, but in DAWN you can right click on the value and plot the
       array so you can examine it visually to see whether it looks right
       or not.
     </p>
     <div id="attachment_156" style="width: 650px"
       class="wp-caption aligncenter">
       <a
         href="/community/eclipse_newsletter/2015/january/images/pydev.png"><img
         class="wp-image-156 size-large"
         src="/community/eclipse_newsletter/2015/january/images/pydev.png"
         alt="Pydev" width="640" height="360" /></a>
       <p class="wp-caption-text">Visual debugging using PyDev and DAWN</p>
     </div>
     <p>
       By using PyDev, this instantly provided an easy way to use Jython
       which gives easy access to the Java objects and for scripting.
       However, with <a
         href="http://www.scipy.org/scipylib/faq.html#does-numpy-scipy-work-with-jython"
         target="_blank">Jython you cannot access C-based libraries like
         numpy</a>. There was still the need to be able to perform
       plotting operations in the workbench views from the interactive
       console while making use of numpy functionality. So this was the
       next problem that needed solving and gave rise to an in-house
       solution known as AnalysisRPC.
     </p>
     <h3>AnalysisRPC</h3>
     <p>
       AnalysisRPC is a Python-to-Java bridge that provides a generic way
       to call Python functions from Java as well as Java functions from
       Python. Its real value-add comes from its deep understanding of
       complex types such as data sets (<a
         href="http://docs.scipy.org/doc/numpy/reference/arrays.ndarray.html">ndarrays</a>)
       or regions of interest. It provides a consistent API to call
       functions regardless of whether they are local or remote calls by
       using a mechanism referred to as &#8216;flattening&#8217;. It was
       also important to provide a robust way of handling exceptions so
       information was not lost when something went wrong between the two
       layers of Java and Python.
     </p>
     <div id="attachment_154" style="width: 650px"
       class="wp-caption aligncenter">
       <a
         href="/community/eclipse_newsletter/2015/january/images/python2java.png"><img
         class="wp-image-154 size-large"
         src="/community/eclipse_newsletter/2015/january/images/python2java.png"
         alt="python2java" width="640" height="480" /></a>
       <p class="wp-caption-text">Illustration of a plot operation
         calling Java from Python using AnalysisRPC</p>
     </div>
     <p>The initial requirements, such as scripting plotting operations,
       only required CPython to call Java code. However the
       implementation, which uses XML-RPC as the transport mechanism,
       lent itself well to calling Python from Java. It was not long
       before many useful features of Java calling Python came up such as
       when using workflows. Now it is fairly seamless to enhance a
       workflow with a custom algorithm written in Python.</p>
     <div id="attachment_159" style="width: 650px"
       class="wp-caption aligncenter">
       <a
         href="/community/eclipse_newsletter/2015/january/images/java2python.png"><img
         class="wp-image-159 size-large"
         src="/community/eclipse_newsletter/2015/january/images/java2python.png"
         alt="java2python" width="640" height="480" /></a>
       <p class="wp-caption-text">Illustration of calling algorithm in
         Python from Java using AnalysisRPC</p>
     </div>
     <p>
       By combining the existing technology of PyDev with the newly
       developed AnalysisRPC, DAWN provides an ideal environment for
       facing <a href="https://www.youtube.com/watch?v=bFUQVb38scc"
         target="_blank">the challenges of high throughput science</a>.
       Since its initial implementation, the technology continues to
       evolve and use other pieces of technology such as <a
         href="http://py4j.sourceforge.net/" target="_blank">Py4J</a>
       where they fit best. <a
         href="https://github.com/DawnScience/scisoft-core/tree/master/uk.ac.diamond.scisoft.analysis.xmlrpc"
         target="_blank">AnalysisRPC</a> itself is evolving and is
       finding applications in other non-science areas where data needs
       to be exchanged between Java and Python. To this end work is being
       planned to improve the framework for reuse with custom data types
       in custom environments so other areas can have the same benefits
       of integration. This is the true spirit of the Eclipse eco-system.
     </p>
     <p>
       To find out more or keep abreast of the latest developments, sign
       up to the <a
         href="https://dev.eclipse.org/mailman/listinfo/science-iwg"
         target="_blank">Eclipse Science mailing list</a>.
     </p>

 <div class="bottomitem">
   <h3>About the Authors</h3>

   <div class="row">
     <div class="col-sm-12">
       <div class="row">
         <div class="col-sm-8">
           <img class="author-picture"
         src="/community/eclipse_newsletter/2015/january/images/tracy.jpg"
         alt="tracy miranda" height="90" />
         </div>
         <div class="col-sm-16">
           <p class="author-name">
         Tracy Miranda<br />
         <a target="_blank" href="http://kichwacoders.com/">Kichwa Coders</a>
       </p>
       <ul class="author-link">
         <li><a target="_blank" href="https://twitter.com/tracymiranda">Twitter</a>
         </li>
         <li><a target="_blank" href="https://twitter.com/kichwacoders">Twitter
             Kichwa Coders</a></li>
         <!--<li><a target="_blank" href="https://www.linkedin.com/in/marcelbruch">Linkedin</a></li>-->
         <?php echo $og; ?>
       </ul>
         </div>
       </div>
     </div>
   </div>
 </div>
	<?php
	/*******************************************************************************
	* Copyright (c) 2015 Eclipse Foundation and others.
	* All rights reserved. This program and the accompanying materials
	* are made available under the terms of the Eclipse Public License v1.0
	* which accompanies this distribution, and is available at
	* http://eclipse.org/legal/epl-v10.html
	*
	* Contributors:
	* Eric Poirier (Eclipse Foundation) - Initial implementation
	*******************************************************************************/
	?>

	<h1 class="article-title"><?php echo $pageTitle; ?></h1>
	<p>Eclipse has always been a platform that unites different
	technology and makes it possible for developers to use the tools
	of their trade such as language editors, debuggers, profilers,
	version control, all in one environment. Until relatively recently
	scientists and researchers were at the mercy of fragmented suites
	of tools or closed-source workbenches. Actually many still are.
	But now with the increased adoption of Eclipse in the science
	sector, they too can look forward to the huge benefits of tool
	integration and interoperability as well as the resulting speed-up
	in productivity.</p>
	<p>
	When it comes to tools-of-the-trade for scientists, Python is high
	on the list. This is in large part thanks to its fast, powerful
	libraries such as <a href="http://scipy-lectures.github.io/"
	target="_blank">numpy and scipy</a>. Also the dynamic and
	easy-to-use nature of the language lends itself well to the
	exploring necessary for experimental work. The <a
	href="http://software-carpentry.org/v5/novice/python/index.html"
	target="_blank">accessibility of learning resources</a> also
	contributes to Python being the language of choice for the
	scientist embracing programming.
	</p>
	<p>
	So how do you go about providing a tight integration of Python in
	Eclipse; one that allows a user to seamlessly move data around and
	use it both interactively and within views? <span id="more-148"></span>This
	was the challenge faced by <a
	href="http://www.diamond.ac.uk/Home.html" target="_blank">Diamond
	Light Source</a> as they developed their science workbench <a
	href="http://www.dawnsci.org/" target="_blank">DAWN</a>. Diamond
	is a synchrotron in the UK countryside which regularly hosts
	scientists and researchers from diverse fields doing
	groundbreaking research on all manner of things: viruses, retinas,
	dinosaur bones and chocolate to name a few. The users collect huge
	amounts of data for their samples and then perform extensive
	analysis to arrive at their conclusions. Analysis of the data from
	the tomography, crystallography or other experiments is done in
	DAWN, the Eclipse-based data analysis workbench developed by
	Diamond.
	</p>
	<p>The video below shows some of the power of the integrated
	environment. Users can plot arrays in views from the interactive
	console and access regions-of-interest that were created
	graphically from the command line. Knowledge of the types being
	used further enhances the experience, for example dragging a file
	node type to the console automatically pastes the code used to
	load the node from file. To achieve this level of integration of
	Python with DAWN required two key pieces of enabling technology:
	PyDev and AnalysisRPC.</p>
	<br />
	<iframe width="588" height="441"
	src="https://www.youtube.com/embed/BFi0vqGEvYU"
	allowfullscreen></iframe>
	<br />
	<h3>PyDev</h3>
	<p>
	Using Eclipse gives access to the huge Eclipse ecosystem, which
	includes <a href="http://pydev.org/" target="_blank">PyDev</a>.
	PyDev is a feature-rich, popular Python IDE. It is made up of a
	set of mature Eclipse plug-ins which were straightforward to
	customise and integrate into DAWN. This immediately gave DAWN
	features such as context-sensitive code completion, an IPython
	interactive console and rich debugging. That last item is
	particularly powerful – for instance a user can debug their
	script and see their array values in the variable view. However,
	it is hard to make sense of arrays when seeing them as a list of
	values, but in DAWN you can right click on the value and plot the
	array so you can examine it visually to see whether it looks right
	or not.
	</p>
	<div id="attachment_156" style="width: 650px"
	class="wp-caption aligncenter">
	<a
	href="/community/eclipse_newsletter/2015/january/images/pydev.png"><img
	class="wp-image-156 size-large"
	src="/community/eclipse_newsletter/2015/january/images/pydev.png"
	alt="Pydev" width="640" height="360" /></a>
	<p class="wp-caption-text">Visual debugging using PyDev and DAWN</p>
	</div>
	<p>
	By using PyDev, this instantly provided an easy way to use Jython
	which gives easy access to the Java objects and for scripting.
	However, with <a
	href="http://www.scipy.org/scipylib/faq.html#does-numpy-scipy-work-with-jython"
	target="_blank">Jython you cannot access C-based libraries like
	numpy</a>. There was still the need to be able to perform
	plotting operations in the workbench views from the interactive
	console while making use of numpy functionality. So this was the
	next problem that needed solving and gave rise to an in-house
	solution known as AnalysisRPC.
	</p>
	<h3>AnalysisRPC</h3>
	<p>
	AnalysisRPC is a Python-to-Java bridge that provides a generic way
	to call Python functions from Java as well as Java functions from
	Python. Its real value-add comes from its deep understanding of
	complex types such as data sets (<a
	href="http://docs.scipy.org/doc/numpy/reference/arrays.ndarray.html">ndarrays</a>)
	or regions of interest. It provides a consistent API to call
	functions regardless of whether they are local or remote calls by
	using a mechanism referred to as ‘flattening’. It was
	also important to provide a robust way of handling exceptions so
	information was not lost when something went wrong between the two
	layers of Java and Python.
	</p>
	<div id="attachment_154" style="width: 650px"
	class="wp-caption aligncenter">
	<a
	href="/community/eclipse_newsletter/2015/january/images/python2java.png"><img
	class="wp-image-154 size-large"
	src="/community/eclipse_newsletter/2015/january/images/python2java.png"
	alt="python2java" width="640" height="480" /></a>
	<p class="wp-caption-text">Illustration of a plot operation
	calling Java from Python using AnalysisRPC</p>
	</div>
	<p>The initial requirements, such as scripting plotting operations,
	only required CPython to call Java code. However the
	implementation, which uses XML-RPC as the transport mechanism,
	lent itself well to calling Python from Java. It was not long
	before many useful features of Java calling Python came up such as
	when using workflows. Now it is fairly seamless to enhance a
	workflow with a custom algorithm written in Python.</p>
	<div id="attachment_159" style="width: 650px"
	class="wp-caption aligncenter">
	<a
	href="/community/eclipse_newsletter/2015/january/images/java2python.png"><img
	class="wp-image-159 size-large"
	src="/community/eclipse_newsletter/2015/january/images/java2python.png"
	alt="java2python" width="640" height="480" /></a>
	<p class="wp-caption-text">Illustration of calling algorithm in
	Python from Java using AnalysisRPC</p>
	</div>
	<p>
	By combining the existing technology of PyDev with the newly
	developed AnalysisRPC, DAWN provides an ideal environment for
	facing <a href="https://www.youtube.com/watch?v=bFUQVb38scc"
	target="_blank">the challenges of high throughput science</a>.
	Since its initial implementation, the technology continues to
	evolve and use other pieces of technology such as <a
	href="http://py4j.sourceforge.net/" target="_blank">Py4J</a>
	where they fit best. <a
	href="https://github.com/DawnScience/scisoft-core/tree/master/uk.ac.diamond.scisoft.analysis.xmlrpc"
	target="_blank">AnalysisRPC</a> itself is evolving and is
	finding applications in other non-science areas where data needs
	to be exchanged between Java and Python. To this end work is being
	planned to improve the framework for reuse with custom data types
	in custom environments so other areas can have the same benefits
	of integration. This is the true spirit of the Eclipse eco-system.
	</p>
	<p>
	To find out more or keep abreast of the latest developments, sign
	up to the <a
	href="https://dev.eclipse.org/mailman/listinfo/science-iwg"
	target="_blank">Eclipse Science mailing list</a>.
	</p>

	<div class="bottomitem">
	<h3>About the Authors</h3>

	<div class="row">
	<div class="col-sm-12">
	<div class="row">
	<div class="col-sm-8">
	<img class="author-picture"
	src="/community/eclipse_newsletter/2015/january/images/tracy.jpg"
	alt="tracy miranda" height="90" />
	</div>
	<div class="col-sm-16">
	<p class="author-name">
	Tracy Miranda<br />
	<a target="_blank" href="http://kichwacoders.com/">Kichwa Coders</a>
	</p>
	<ul class="author-link">
	<li><a target="_blank" href="https://twitter.com/tracymiranda">Twitter</a>
	</li>
	<li><a target="_blank" href="https://twitter.com/kichwacoders">Twitter
	Kichwa Coders</a></li>
	<!--<li><a target="_blank" href="https://www.linkedin.com/in/marcelbruch">Linkedin</a></li>-->
	<?php echo $og; ?>
	</ul>
	</div>
	</div>
	</div>
	</div>
	</div>