eclipse_newsletter/2019/december/content/en_2.php

<?php
/**
 * *****************************************************************************
 * Copyright (c) 2015, 2016 Eclipse Foundation and others.
 * All rights reserved. This program and the accompanying materials
 * are made available under the terms of the Eclipse Public License v1.0
 * which accompanies this distribution, and is available at
 * http://eclipse.org/legal/epl-v10.html
 *
 * Contributors:
 * Eric Poirier (Eclipse Foundation) - Initial implementation
 * Christopher Guindon (Eclipse Foundation)
 * *****************************************************************************
 */
// This file must be included
if (basename(__FILE__) == basename($_SERVER['PHP_SELF'])) {
  exit();
}
?>
<h1 class="article-title"><?php echo $pageTitle; ?></h1>
<h2>Incorporating Artificial Intelligence Into Real-World Applications</h2>
<p>
  The goal of the<a href="https://projects.eclipse.org/projects/technology.deeplearning4j"> Eclipse
    Deeplearning4j</a> project is simple: To enable artificial intelligence (AI) applications to run
  everywhere, for everyone. We&rsquo;re working to take deep learning and AI applications out of the
  theoretical, academic world and into the real world where they can be applied in useful and
  meaningful ways across industries.
</p>
<p>With the suite of tools we&rsquo;ve made available in the Deeplearning4j project,
  enterprise developers can apply deep learning technology in a way that addresses the technical and
  business challenges in their industry.</p>
<h2>Research Versus Reality</h2>
<p>One of the major challenges in the deep learning field today is that when researchers
  speak publicly about deep learning frameworks, 90 percent of the time, they&rsquo;re talking to
  other researchers about what they know and what they think is most interesting. They&rsquo;re not
  talking to enterprise developers about how the technology can be applied to solve their real-world
  problems.</p>
<p>The media picks up on researchers&rsquo; ideas and the industry gets left behind. Most
  of the information people learn about AI through the media is exaggerated. And, it doesn&rsquo;t
  fit with what industry actually needs.</p>
<p>The reality is that most developers have basic use cases and they&rsquo;re using deep
  learning models that are two to five years old. And, only a very few leading-edge companies take a
  deep learning research paper and deploy the technology and capabilities it describes.</p>
<h2>Targeting Enterprise Developer Challenges</h2>
<p>The Deeplearning4j toolset is for traditional enterprise developers who just want to
  deploy a simple deep learning model in industries such as banking, retail, manufacturing,
  ecommerce, and others. To meet enterprise developers&rsquo; real-world requirements, we&rsquo;ve
  created a decentralized, open, and vendor-neutral ecosystem that targets their need to:</p>
<ul>
  <li>Run AI applications on architectures that most people typically wouldn&rsquo;t have access to,
    such as alternative accelerators</li>
  <li>Take advantage of Java-based microservices architectures such as<a
    href="https://microprofile.io/"
  > Eclipse MicroProfile</a></li>
  <li>Enable AI application deployments at the<a
    href="https://blogs.eclipse.org/post/mike-milinkovich/close-edge"
  > network edge</a></li>
  <li>Run AI applications on application servers such as WildFly and other products in the<a
    href="https://jakarta.ee/compatibility/"
  > Jakarta EE ecosystem</a></li>
  <li>Support the deployment needs of global markets with a wide range of use cases</li>
  <li>Run AI applications on a mix of power-efficient chip types that are outside the mainstream
    NVIDIA and Intel options with no restrictions</li>
  <li>Interoperate with existing and new deep learning frameworks</li>
  <li>Write code in a variety of languages &mdash; Java, Scala, Python, Swift, and others</li>
  <li>Integrate with legacy file formats and protocols</li>
</ul>
<h2>Key Project Components</h2>
<p>Here&rsquo;s a brief summary of some of the key components in the Eclipse
  Deeplearning4j toolset:</p>
<ul>
  <li>SameDiff is our version of TensorFlow or PyTorch. It&rsquo;s a neutral runtime that allows
    developers to build and train deep learning models no matter which research framework
    they&rsquo;re using. It can import TensorFlow models and run most existing TensorFlow models. It
    also interoperates with PyTorch. As a result, it makes a great starting point for building
    models.</li>
  <li>DataVec is our data extract, transform, load (ETL) library. DataVec vectorizes and
    &ldquo;tensorizes&rdquo; data in spreadsheets, images, and other data sources that can&rsquo;t
    be included in deep learning applications in their original format. Once transformed, the data
    is output in n-dimensional arrays.</li>
  <li>Arbiter is our hyperparameter optimization component. Arbiter uses grid searches, random
    searches, and Bayesian methods to automatically tune the neural network, allowing developers to
    automatically build the optimal deep learning model for the data provided.</li>
  <li>RL4J is our reinforcement learning library. It runs on top of Deeplearning4j and gives
    developers the ability to run reinforcement learning applications.</li>
</ul>
<p>We also have:</p>
<ul>
  <li>A set of lower-level C++ libraries that are optimized and pre-compiled for different
    architectures to enable multi-language support</li>
  <li>A component that allows developers to distribute training and batch inferences on Apache Spark</li>
  <li>Two Scala libraries</li>
</ul>
<h2>Opening up the Deep Learning Ecosystem</h2>
<p>
  When you read our component descriptions and visit <a href="https://deeplearning4j.org/">the
    Deeplearning4j website</a>, you&rsquo;ll notice that interoperability among frameworks,
  technologies, and languages is a common theme. Today, there&rsquo;s a huge misconception that all
  of the different framework developers and ecosystem players in the deep learning field are more
  interested in competing than cooperating.
</p>
<p>But, it&rsquo;s not a binary reality. There are different use cases and different ways
  to deploy deep learning models. Competition is good because without it, you simply have an echo
  chamber with only very narrow use cases being served and no alternative ideas. However,
  vendor-specific implementations typically do not interoperate well with one another.</p>
<p>That&rsquo;s why it&rsquo;s so important that the Deeplearning4j SameDiff component can
  import, read, write, and train models in the same way as TensorFlow and PyTorch. SameDiff
  complements TensorFlow and PyTorch, but also provides access to a broader community. </p>
<p>When new standards come out, we plan to support them so we can become the Switzerland
  of deep learning and offer a viable alternative for deep learning that isn&rsquo;t marketed by one
  company or another.</p>
<h2>The Eclipse Foundation Is the Ideal Place to Achieve Our Goals</h2>
<p>Because we believe so strongly in the value of community development, openness, and
  interoperability, the Eclipse Foundation is the best place to achieve the deep learning advances
  listed above. </p>
<p>At the Eclipse Foundation, different people with different perspectives  can
  contribute to the overall progress of projects, advancing technology and adoption faster. Everyone
  has a voice, governance is better and it&rsquo;s vendor-neutral. There&rsquo;s much greater
  opportunity to develop a diverse set of deep learning use cases and a more diverse ecosystem that
  will strengthen the long-term health of the technology. While the Deeplearning4j project is not
  currently associated with an Eclipse Foundation working group, we hope it will be in the future.</p>
<h2>Code Is Ready for Use</h2>
<p>Most of the Deeplearning4j code has been in production for several years at a number of
  different companies. The SameDiff component will be added to the project in January 2020, but
  developers are already using it and examples are available. </p>
<p>Going forward, SameDiff will be the main interface for building models and the
  Deeplearning4j interface that most people know will be supported as a higher-level API.
  We&rsquo;ll also be focusing on further stabilizing the project components and on multi-language
  support end-to-end. And, we&rsquo;re hoping to rework our ETL library to better integrate with
  lower-level math libraries and optimize the math end-to-end. You will also see more examples
  become available.</p>
<h2>Get Started Today</h2>
<p>If you have a use case, consider giving SameDiff a try. Or, you can download a
  pre-trained model from the PyTorch or TensorFlow ecosystem and use that as the starting point to
  build models with SameDiff. If you want to do ETL, use DataVec with either SameDiff or
  Deeplearning4j.</p>
<p>
  For examples, tutorials, documentation, and details about the community support that&rsquo;s
  available,<a href="https://deeplearning4j.org/"> visit our website</a>.
</p>
<p>
  To receive the latest updates about Deeplearning4j and get in touch with our members, visit our<a
    href="https://accounts.eclipse.org/mailing-list/deeplearning4j-dev"
  > mailing list page</a>.
</p>
<p>
  Finally, please consider taking<a href="https://nexent.typeform.com/to/PjbGd4"> our survey</a> to
  help us better understand how deep learning is used in the Java ecosystem.
</p>
<div class="bottomitem">
  <h3>About the Author</h3>
  <div class="row">
    <div class="col-sm-12">
      <div class="row">
        <div class="col-sm-8">
          <img class="img-responsive"
            src="/community/eclipse_newsletter/2019/december/images/adam.png" alt="Adam Gibson"
          />
        </div>
        <div class="col-sm-16">
          <p class="author-name">Adam Gibson</p>
          <p>CTO at Konduit</p>
        </div>
      </div>
    </div>
  </div>
</div>