blob: 31adce2846cdebaa19a566a7ed0dc38ee5a9ef5b [file] [log] [blame]
/*******************************************************************************
* Copyright (c) 2008, 2012 Attensity Europe GmbH and brox IT Solutions GmbH. All rights reserved. This program and the
* accompanying materials are made available under the terms of the Eclipse Public License v1.0 which accompanies this
* distribution, and is available at http://www.eclipse.org/legal/epl-v10.html
*
* Contributors: Andreas Schank, Juergen Schumacher (Attensity Europe GmbH)
*******************************************************************************/
/**
*
This package provides an easy-to-use helper library to access the SMILA REST API. The library hides all the HTTP
* communication and JSON conversion stuff so that you can use the REST API elements using the SMILA data model classes.
* <p>
* See also:
* <ul>
* <li><a href="http://wiki.eclipse.org/SMILA/Documentation/Using_The_ReST_API" target="_blank">Using the REST API</a>
* for general information about the SMILA REST API.
* <li><a href="http://wiki.eclipse.org/SMILA/REST_API_Reference" target="_blank">REST API Reference</a> for an overview
* of available API resources.
* <li><a href="http://wiki.eclipse.org/SMILA/Documentation/HowTo/How_to_access_the_REST_API_with_the_RestClient"
* target="_blank">How to access the REST API with the RestClient</a> for some more details on this library.
* </ul>
*
* <h1>Basics</h1>
*
* <p>
* The following examples and code snippets all apply when you are running SMILA out-of-the box on localhost.
* </p>
* <p>
* If you are running SMILA on a different host or with a different port (or an altered root context), please see <a
* href="#non-default_configuration">non-default configuration</a> on how to use the Rest Client in these cases.
* </p>
*
* <h2>Interfaces and default implementations</h2>
*
* The {@link org.eclipse.smila.http.client.RestClient} interface encapsulates the REST access to SMILA. It provides
* methods for GET, POST, PUT and DELETE calls to the REST API and represents data using SMILA's
* {@link org.eclipse.smila.datamodel.Any} interface and attachments using the
* {@link org.eclipse.smila.http.client.Attachments} interface. The latter allow working with binary data in SMILA.
* <p>
* The package <tt>org.eclipse.smila.http.client.impl</tt> provides a default implementation for the
* {@link org.eclipse.smila.http.client.RestClient} named {@link org.eclipse.smila.http.client.impl.DefaultRestClient}.
* <p>
* There are two helper classes providing the resources as described in REST API Reference:
* <ul>
* <li> {@link org.eclipse.smila.http.client.ResourceHelper} for all resources beginning with <tt>/smila</tt>,
* except for those that are marked as deprecated in the REST API Reference.
* <li> {@link org.eclipse.smila.http.client.TaskManagerClientHelper} to provide workers that are not directly driven by
* the WorkerManager with resources for task handling (internal TaskManager REST API, i.e. the resources beginning with
* <tt>/taskmanager</tt>).
* </ul>
*
* <h2>Accessing SMILA</h2>
*
* To access SMILA via its REST interface, instantiate the {@link org.eclipse.smila.http.client.impl.DefaultRestClient},
* like:
*
* <pre>
* RestClient restClient = new DefaultRestClient();
* </pre>
*
* The following code snippet creates a job definition, sends it to the JobManager and starts it if posting was
* successful:
*
* <pre>
* final RestClient restClient = new DefaultRestClient();
* final ResourceHelper resourceHelper = new ResourceHelper();
* final String jobName = &quot;crawlCData&quot;;
*
* // create job description as an AnyMap
* final AnyMap jobDescription = DataFactory.DEFAULT.createAnyMap();
* jobDescription.put(&quot;name&quot;, jobName);
* jobDescription.put(&quot;workflow&quot;, &quot;fileCrawling&quot;);
* final AnyMap parameters = DataFactory.DEFAULT.createAnyMap();
* parameters.put(&quot;tempStore&quot;, &quot;temp&quot;);
* parameters.put(&quot;jobToPushTo&quot;, &quot;importJob&quot;);
* parameters.put(&quot;dataSource&quot;, &quot;file_data&quot;);
* parameters.put(&quot;rootFolder&quot;, &quot;c:/data&quot;);
* jobDescription.put(&quot;parameters&quot;, parameters);
*
* // the resourcehelper provides us with the resource to the jobs API
* // we send the (AnyMap) job description in the POST body
* restClient.post(resourceHelper.getJobsResource(), jobDescription);
*
* // POST (here without a body) to start the Job,
* // the ResourceHelper provides the resource to the named job
* restClient.post(resourceHelper.getJobResource(jobName));
* </pre>
*
* The following snippet checks if the job with the given name is already running, if not, it is started, and a record
* with an attachment is sent to it.
*
* <pre>
* final RestClient restClient = new DefaultRestClient();
* final ResourceHelper resourceHelper = new ResourceHelper();
* final String jobName = &quot;indexUpdate&quot;;
*
* // check for a current run of this job
* final AnyMap currentJobRun =
* restClient.get(resourceHelper.getJobResource(jobName)).getMap(&quot;runs&quot;).getMap(&quot;current&quot;);
* if (currentJobRun != null &amp;&amp; !currentJobRun.isEmpty()) {
* // a current run exists, so we don't need to start one but it may not be running.
* if (!&quot;RUNNING&quot;.equalsIgnoreCase(currentJobRun.getStringValue(&quot;state&quot;))) {
* // well it's just an example...
* throw new IllegalStateException(&quot;Job '&quot; + jobName + &quot;' is not running but has status '&quot;
* + currentJobRun.getStringValue(&quot;state&quot;) + &quot;'.&quot;);
* }
* } else {
* // no current job run, start another one.
* restClient.post(resourceHelper.getJobResource(jobName));
* }
*
* // create attachment with a file's content
* final File file = new File(&quot;c:/data/notice.html&quot;);
* final Attachments attachments = new AttachmentWrapper(&quot;file&quot;, file);
* // put some sample metadata
* final AnyMap metadata = DataFactory.DEFAULT.createAnyMap();
* metadata.put(&quot;_recordid&quot;, &quot;1&quot;);
* metadata.put(&quot;fileName&quot;, file.getCanonicalPath());
* // now post metadata with an attachment from a file.
* // if we had a Record with attachments, we could POST that one...
* // note: we could add more than one attachment using the AttachmentWrapper.
* restClient.post(resourceHelper.getPushRecordToJobResource(jobName), metadata, attachments);
* </pre>
*
* <h1>Using Attachments with the RestClient</h1>
*
* As seen above, this bundle provides an {@link org.eclipse.smila.http.client.Attachments} interface allowing
* attachments to be POSTed. An attachment consists of a string key and binary data that will be POSTed as
* <tt>application/octet-stream</tt> in a multi-part message.
*
* <h2>Handling attachments manually</h2>
*
* You can use the {@link org.eclipse.smila.http.client.attachments.AttachmentWrapper} in order to add attachments from
* the following sources if you want to handle attachments manually:
* <ul>
* <li>a byte[]
* <li>a {@link java.lang.String}
* <li>a {@link java.io.File}
* <li>an {@link java.io.InputStream}
* </ul>
*
* There are convenience constructors to provide an attachment when constructing an
* {@link org.eclipse.smila.http.client.attachments.AttachmentWrapper} but you can add more than one attachment and mix
* the types.
* <p>
* Example:
*
* <pre>
* final RestClient restClient = new DefaultRestClient();
* byte[] byteAttachment = new byte[1000];
* String stringAttachment = &quot;string attachment&quot;;
* File fileAttachment = new File(&quot;c:/data/notice.html&quot;);
* InputStream inputStreamAttachment = new FileInputStream(fileAttachment);
*
* AttachmentWrapper attachments = new AttachmentWrapper(&quot;byte-data&quot;, byteAttachment);
* attachments.add(&quot;string-data&quot;, stringAttachment);
* attachments.add(&quot;file-data&quot;, fileAttachment);
* attachments.add(&quot;stream-data&quot;, inputStreamAttachment);
*
* restClient.post(resource, parameters, attachments);
* </pre>
*
* <h2>Handling attachments with records</h2>
*
* SMILA Records can also include attachments, and since SMILA's target data units are
* {@link org.eclipse.smila.datamodel.Record}s, it is natural, that the {@link org.eclipse.smila.http.client.RestClient}
* also supports {@link org.eclipse.smila.datamodel.Record}s (with attachments) directly.
* <p>
* That means that the record's metadata will be sent with the {@link org.eclipse.smila.datamodel.Record}s' attachments
* as parts of a multipart message.
* <p>
* Example:
*
* <pre>
* final byte[] data1 = ...;
* final byte[] data2 = ...;
* record.setAttachment("data1", data1);
* record.setAttachment("data2", data2);
*
* // POST the record with the attachments
* restClient.post(resourceHelper.getPushRecordToJobResource(jobName), record);
* </pre>
*
* <h1>Using the RestClient without the complete development environment</h1>
*
* <h2>Setting up the classpath</h2>
*
* This section describes the steps to follow when using the RestClient from a Java application outside SMILA's JRE.
* <ul>
* <li>Build or download the SMILA distribution.
* <li>Set up a new workspace.
* <li>Create a Java project of your gusto.
* <li>Add the following JARs from your downloaded/built SMILA application to the Java Build Path of your new project
* (exact version numbers are omitted in this list and replaced with <tt>$version</tt>, just use the latest version
* you'll find in your SMILA application):
* <li>from the <tt>plugins</tt> directory:
* <ul>
* <li>org.apache.commons.collections_$version.jar
* <li>org.apache.commons.io_$version.jar
* <li>org.apache.commons.lang_$version.jar
* <li>org.apache.httpcomponents.httpclient_$version.jar (>=4.1)
* <li>org.apache.httpcomponents.httpcore_$version.jar (>=4.1)
* <li>org.apache.log4j_$version.jar
* <li>org.codehaus.jackson.core_$version.jar
* <li>org.eclipse.smila.datamodel_$version.jar
* <li>org.eclipse.smila.http.client_$version.jar
* <li>org.eclipse.smila.ipc_$version.jar
* <li>org.eclipse.smila.utils_$version.jar
* </ul>
* <li>from the <tt>plugins/org.apache.commons.logging_$version/lib</tt> directory
* <li>commons-logging-$version.jar
* </ul>
* Now you have all means to access SMILA's REST API from another Java application.
* <p>
* E.g. you could now write a simple program that creates and starts up a crawl job and the indexUpdate-job:
*
* <pre>
* public class CrawlMyData {
*
* public static void main(String[] args) {
* final RestClient restClient = new DefaultRestClient();
* final ResourceHelper resourceHelper = new ResourceHelper();
* final String jobName = &quot;crawlCData&quot;;
*
* // create job description as an AnyMap
* final AnyMap jobDescription = DataFactory.DEFAULT.createAnyMap();
* jobDescription.put(&quot;name&quot;, jobName);
* jobDescription.put(&quot;workflow&quot;, &quot;fileCrawling&quot;);
* final AnyMap parameters = DataFactory.DEFAULT.createAnyMap();
* parameters.put(&quot;tempStore&quot;, &quot;temp&quot;);
* parameters.put(&quot;jobToPushTo&quot;, &quot;indexUpdate&quot;);
* parameters.put(&quot;dataSource&quot;, &quot;file_data&quot;);
* parameters.put(&quot;rootFolder&quot;, &quot;c:/data&quot;);
* jobDescription.put(&quot;parameters&quot;, parameters);
*
* try {
* // start the referred job &quot;indexUpdate&quot; that indexes our sent data.
* // We should check if it is still be running, etc..
* restClient.post(resourceHelper.getJobResource(&quot;indexUpdate&quot;));
* } catch (RestException e) {
* e.printStackTrace();
* } catch (IOException e) {
* e.printStackTrace();
* }
*
* try {
* // create (or update) the job, we could check if it exists or is running, etc...
* restClient.post(resourceHelper.getJobsResource(), jobDescription);
*
* // POST with no body to start the Job in default mode
* restClient.post(resourceHelper.getJobResource(jobName));
* } catch (RestException e) {
* e.printStackTrace();
* } catch (IOException e) {
* e.printStackTrace();
* }
*
* }
* }
* </pre>
*
* <h2>Putting it together</h2>
*
* Now start your SMILA application and when it's up, run the application from above and watch the jobs using your
* preferred REST client (e.g. browser plugin, see Interactive REST tools) at <a
* href="http://localhost:8080/smila/jobmanager/jobs/" target="_blank">http://localhost:8080/smila/jobmanager/jobs/</a>
* <p>
* You should see:
* <ul>
* <li>the newly created job "crawlCData",
* <li>the job "indexUpdate" is <tt>RUNNING</tt>,
* <li>the job "crawlCData" is <tt>FINISHING</tt> (or has already finished, depending on the amount of data in
* your crawled directory).
* </ul>
*
* Wait a bit and you can search your crawled data at <a href="http://localhost:8080/SMILA/search"
* target="_blank">http://localhost:8080/SMILA/search</a>.
*
* <h1><a name="non-default_configuration">Using non-default configuration</a></h1>
*
* <p>
* {@link org.eclipse.smila.http.client.RestClient} and {@link org.eclipse.smila.http.client.ResourceHelper}
* have default constructors using the standard values for the SMILA application. These are:
* <ul>
* <li>Host: localhost</li>
* <li>Port: 8080</li>
* <li>Root context: /smila</li>
* </ul>
* </p>
*
* <p>
* If your bundle uses a different root context path, you have to create your ResourceHelper using the actual context
* path. Also, if your application runs on a different server and/or using a different port, you will have to
* supply this information to the constructor of the {@link org.eclipse.smila.http.client.impl.DefaultRestClient}
* (you can omit the leading http://).
* </p>
*
* <p>
* E.g. the following code snippet:
* <pre>
* final RestClient restClient = new DefaultRestClient(&quot;host.domain.org:80&quot;);
* final ResourceHelper resourceHelper = new ResourceHelper(&quot;/context&quot;);
* </pre>
* creates a client and a resource helper connection to a SMILA instance running on http://host.domain.org:80/context.
* </p>
*
* <p>
* You can also use your own connection manager or limit the number of total connections and max connections per host by
* using the respective constructors of {@link org.eclipse.smila.http.client.impl.DefaultRestClient}.
* </p>
*
* @since 1.1.0
*/
package org.eclipse.smila.http.client;