| <?xml version="1.0" encoding="UTF-8"?> |
| <rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"> |
| <channel> |
| <title>Eclipse Ditto Blog</title> |
| <description>Announcements, tutorials and examples around Eclipse Ditto and Digital Twins</description> |
| <link>https://www.eclipse.org/ditto/</link> |
| <atom:link href="https://www.eclipse.org/ditto/feed.xml" rel="self" type="application/rss+xml"/> |
| <pubDate>Fri, 24 Apr 2020 19:29:56 +0000</pubDate> |
| <lastBuildDate>Fri, 24 Apr 2020 19:29:56 +0000</lastBuildDate> |
| <generator>Jekyll v3.6.2</generator> |
| |
| <item> |
| <title>Digital twins of devices connected via LoRaWAN to TTN</title> |
| <description><figure><img class="docimage" src="images/2020-04-16-Logo_TTVC_color.png" alt="TTVC logo" style="max-width:300px;padding-left:1em;float:right" /></figure> |
| |
| <p><br /></p> |
| |
| <p>A workshop of the <a href="https://www.thethingsnetwork.org/article/the-things-virtual-conference">2020 The Things Virtual Conference</a> |
| on April 16th 2020 is/was about how to connect Eclipse Ditto to “The Things Network” via TTN’s MQTT broker in order to |
| automatically update digital twins of devices connected via LoRaWAN to the TTN backend.</p> |
| |
| <p>You can find the slides <a href="slides/2020_04_16-ttn-virtual-conference/index.html">here</a>.</p> |
| |
| <p>This blogpost helps setting up this kind of connection and shall also be used as a step-by-step tutorial during |
| the workshop.</p> |
| |
| <h2 id="requirements">Requirements</h2> |
| |
| <p>You’ll need:</p> |
| <ul> |
| <li>an operating system capable of running Docker (best use a Linux distribution)</li> |
| <li>4 CPU cores and 4GB of RAM are advised (less can work, but the Ditto cluster startup is more fragile then)</li> |
| <li>to have installed: <code class="highlighter-rouge">curl</code> and <code class="highlighter-rouge">git</code></li> |
| </ul> |
| |
| <p>Also, you’ll need a TTN account and an existing application with at least one device if you want to follow the hands-on |
| part and want to create digital twins of your devices connected to TTN.</p> |
| |
| <h2 id="preparation">Preparation</h2> |
| |
| <p>Please follow these initial preparation steps (if you don’t already have Docker and Docker Compose installed).</p> |
| |
| <p>When you have access to a Kubernetes cluster and already have worked with <a href="https://helm.sh">Helm</a> (the package manager |
| for Kubernetes), you can alternatively install Ditto via its official |
| <a href="https://hub.helm.sh/charts/eclipse-iot/ditto">Helm chart</a>.</p> |
| |
| <h3 id="install-docker">Install Docker</h3> |
| |
| <p>Assumption: You’re running a Debian or Ubuntu based Linux distribution containing the <code class="highlighter-rouge">apt</code> package manager.</p> |
| |
| <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>apt install docker.io |
| <span class="nb">sudo </span>service docker start |
| <span class="nb">sudo </span>usermod <span class="nt">-a</span> <span class="nt">-G</span> docker &lt;your-username&gt; |
| </code></pre></div></div> |
| |
| <p>Logout and login again so that your user gets the “docker” group.</p> |
| |
| <h3 id="install-docker-compose">Install Docker Compose</h3> |
| |
| <p><a href="https://docs.docker.com/compose/install/">Follow the installation guide here</a>, in short:</p> |
| |
| <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>curl <span class="nt">-L</span> <span class="s2">"https://github.com/docker/compose/releases/download/1.25.4/docker-compose-</span><span class="k">$(</span>uname <span class="nt">-s</span><span class="k">)</span><span class="s2">-</span><span class="k">$(</span>uname <span class="nt">-m</span><span class="k">)</span><span class="s2">"</span> <span class="nt">-o</span> /usr/local/bin/docker-compose |
| <span class="nb">sudo </span>chmod +x /usr/local/bin/docker-compose |
| </code></pre></div></div> |
| |
| <h3 id="clone-ditto-codebase">Clone Ditto codebase</h3> |
| |
| <p>That is required to get the <code class="highlighter-rouge">docker-compose.yaml</code> file and other resources required to run Ditto with Docker Compose.</p> |
| |
| <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>git clone <span class="nt">--depth</span> 1 https://github.com/eclipse/ditto.git |
| </code></pre></div></div> |
| |
| <h2 id="startup-ditto-cluster">Startup Ditto cluster</h2> |
| |
| <p>Change directory into the just cloned git repository - optionally adjust the <code class="highlighter-rouge">DITTO_EXTERNAL_PORT</code> variable to where |
| Ditto is reachable after the start:</p> |
| |
| <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">cd </span>ditto/deployment/docker/ |
| <span class="nb">export </span><span class="nv">DITTO_EXTERNAL_PORT</span><span class="o">=</span>80 |
| docker-compose up <span class="nt">-d</span> |
| </code></pre></div></div> |
| |
| <p>Verify that Ditto is running:</p> |
| |
| <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker-compose ps |
| </code></pre></div></div> |
| |
| <p>The output should look similar like this:</p> |
| <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code> Name Command State Ports |
| ---------------------------------------------------------------------------------------- |
| docker_concierge_1 /sbin/tini -- java -jar st ... Up 8080/tcp |
| docker_connectivity_1 /sbin/tini -- java -jar st ... Up 8080/tcp |
| docker_gateway_1 /sbin/tini -- java -Dditto ... Up 0.0.0.0:8081-&gt;8080/tcp |
| docker_mongodb_1 docker-entrypoint.sh mongo ... Up 27017/tcp |
| docker_nginx_1 nginx -g daemon off; Up 0.0.0.0:80-&gt;80/tcp |
| docker_policies_1 /sbin/tini -- java -jar st ... Up 8080/tcp |
| docker_swagger-ui_1 nginx -g daemon off; Up 80/tcp, 8080/tcp |
| docker_things-search_1 /sbin/tini -- java -jar st ... Up 8080/tcp |
| docker_things_1 /sbin/tini -- java -jar st ... Up 8080/tcp |
| </code></pre></div></div> |
| |
| <p>Verify that your Ditto cluster is healthy. Please give it ~1 minute in order to properly start up.</p> |
| |
| <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>curl <span class="nt">-u</span> devops:foobar http://localhost:<span class="k">${</span><span class="nv">DITTO_EXTERNAL_PORT</span><span class="k">}</span>/status/health |
| </code></pre></div></div> |
| |
| <p>The returned output should start with:</p> |
| <div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="s2">"label"</span><span class="p">:</span><span class="s2">"roles"</span><span class="p">,</span><span class="s2">"status"</span><span class="p">:</span><span class="s2">"UP"</span><span class="p">,</span><span class="w"> </span><span class="err">...</span><span class="w"> </span><span class="p">}</span><span class="w"> |
| </span></code></pre></div></div> |
| |
| <p>If your Ditto cluster has trouble starting up (e.g. because you only have less CPU cores than advised), try the following |
| startup command instead:</p> |
| |
| <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker-compose start mongodb<span class="p">;</span> sleep 30<span class="p">;</span> docker-compose start policies things<span class="p">;</span> sleep 60<span class="p">;</span> docker-compose start concierge<span class="p">;</span> sleep 60<span class="p">;</span> docker-compose start things-search<span class="p">;</span> sleep 60<span class="p">;</span> docker-compose start connectivity<span class="p">;</span> sleep 60<span class="p">;</span> docker-compose up <span class="nt">-d</span> |
| </code></pre></div></div> |
| |
| <h2 id="configure-connection-to-ttn-mqtt-broker">Configure connection to TTN MQTT broker</h2> |
| |
| <p>The Things Network provides a built in MQTT broker which you can connect to using your TTN application credentials. |
| For a more detailed description on that topic, please refer to the |
| <a href="https://www.thethingsnetwork.org/docs/applications/mqtt/quick-start.html">TTN MQTT Quick Start</a>.</p> |
| |
| <p>Eclipse Ditto can establish connections to MQTT brokers. This is a schematic picture of what we now will do:</p> |
| |
| <figure><img class="docimage" src="images/../slides/images/ttn-ditto-via-mqtt.png" alt="TTN to Ditto via MQTT" style="max-width:600px;padding-left:1em" /></figure> |
| |
| <p>In order to connect to your own TTN application, perform the following steps.</p> |
| |
| <p>You can find the <code class="highlighter-rouge">&lt;AppId&gt;</code> (application ID) and <code class="highlighter-rouge">&lt;AppKey&gt;</code> (access key) in your TTN console of your application. |
| For <code class="highlighter-rouge">&lt;Region&gt;</code>, e.g. choose <code class="highlighter-rouge">'eu'</code> when your application is in handled by the Handler ‘ttn-handler-eu’.</p> |
| |
| <p>Please export your application’s credentials locally to environment variables:</p> |
| |
| <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">export </span><span class="nv">TTN_REGION</span><span class="o">=</span><span class="s1">'&lt;Region&gt;'</span> |
| <span class="nb">export </span><span class="nv">TTN_APP_ID</span><span class="o">=</span><span class="s1">'&lt;AppID&gt;'</span> |
| <span class="nb">export </span><span class="nv">TTN_APP_KEY</span><span class="o">=</span><span class="s1">'&lt;AppKey&gt;'</span> |
| </code></pre></div></div> |
| |
| <p>After having done that, you can already create the connection of Ditto to the TTN MQTT broker:</p> |
| |
| <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>curl <span class="nt">-X</span> POST <span class="nt">-u</span> devops:foobar <span class="nt">-H</span> <span class="s1">'Content-Type: application/json'</span> <span class="nt">-d</span> <span class="s1">'{ |
| "targetActorSelection": "/system/sharding/connection", |
| "headers": { |
| "aggregate": false |
| }, |
| "piggybackCommand": { |
| "type": "connectivity.commands:createConnection", |
| "connection": { |
| "id": "ttn-connection-via-mqtt", |
| "name": "TTN-MQTT", |
| "connectionType": "mqtt", |
| "connectionStatus": "open", |
| "uri": "tcp://'</span><span class="s2">"</span><span class="k">${</span><span class="nv">TTN_APP_ID</span><span class="k">}</span><span class="s2">"</span><span class="s1">':'</span><span class="s2">"</span><span class="k">${</span><span class="nv">TTN_APP_KEY</span><span class="k">}</span><span class="s2">"</span><span class="s1">'@'</span><span class="s2">"</span><span class="k">${</span><span class="nv">TTN_REGION</span><span class="k">}</span><span class="s2">"</span><span class="s1">'.thethings.network:1883", |
| "failoverEnabled": true, |
| "clientCount": 1, |
| "validateCertificates": false, |
| "sources": [{ |
| "addresses": [ |
| "'</span><span class="s2">"</span><span class="k">${</span><span class="nv">TTN_APP_ID</span><span class="k">}</span><span class="s2">"</span><span class="s1">'/devices/+/up" |
| ], |
| "consumerCount": 1, |
| "qos": 0, |
| "authorizationContext": [ |
| "pre-authenticated:ttn-connection" |
| ], |
| "enforcement": { |
| "input": "{{ source:address }}", |
| "filters": [ |
| "'</span><span class="s2">"</span><span class="k">${</span><span class="nv">TTN_APP_ID</span><span class="k">}</span><span class="s2">"</span><span class="s1">'/devices/{{ thing:name }}/up" |
| ] |
| }, |
| "replyTarget": { |
| "enabled": false |
| }, |
| "payloadMapping": [ |
| "ttn-demo-mapping" |
| ] |
| }], |
| "mappingDefinitions": { |
| "ttn-demo-mapping": { |
| "mappingEngine": "JavaScript", |
| "options": { |
| "incomingScript": "function mapToDittoProtocolMsg(\n headers,\n textPayload,\n bytePayload,\n contentType\n) {\n\n let ttnJson = JSON.parse(textPayload);\n let deviceId = ttnJson['</span><span class="s2">"'"</span><span class="s1">'dev_id'</span><span class="s2">"'"</span><span class="s1">'];\n let payloadFields = ttnJson['</span><span class="s2">"'"</span><span class="s1">'payload_fields'</span><span class="s2">"'"</span><span class="s1">'];\n \n let attributesObj = {\n hardwareSerial: ttnJson['</span><span class="s2">"'"</span><span class="s1">'hardware_serial'</span><span class="s2">"'"</span><span class="s1">'],\n ttnCounter: ttnJson['</span><span class="s2">"'"</span><span class="s1">'counter'</span><span class="s2">"'"</span><span class="s1">']\n };\n \n let featuresObj = {\n temperature: {\n properties: {\n value: payloadFields['</span><span class="s2">"'"</span><span class="s1">'temperature_7'</span><span class="s2">"'"</span><span class="s1">']\n }\n },\n pressure: {\n properties: {\n value: payloadFields['</span><span class="s2">"'"</span><span class="s1">'barometric_pressure_10'</span><span class="s2">"'"</span><span class="s1">']\n }\n },\n humidity: {\n properties: {\n value: payloadFields['</span><span class="s2">"'"</span><span class="s1">'relative_humidity_8'</span><span class="s2">"'"</span><span class="s1">']\n }\n }\n };\n \n let thing = {\n attributes: attributesObj,\n features: featuresObj\n };\n \n let dittoHeaders = {\n '</span><span class="s2">"'"</span><span class="s1">'response-required'</span><span class="s2">"'"</span><span class="s1">': false,\n '</span><span class="s2">"'"</span><span class="s1">'If-Match'</span><span class="s2">"'"</span><span class="s1">': '</span><span class="s2">"'"</span><span class="s1">'*'</span><span class="s2">"'"</span><span class="s1">'\n };\n\n return Ditto.buildDittoProtocolMsg(\n '</span><span class="s2">"'"</span><span class="s1">'org.eclipse.ditto.ttn.demo'</span><span class="s2">"'"</span><span class="s1">',\n deviceId,\n '</span><span class="s2">"'"</span><span class="s1">'things'</span><span class="s2">"'"</span><span class="s1">',\n '</span><span class="s2">"'"</span><span class="s1">'twin'</span><span class="s2">"'"</span><span class="s1">',\n '</span><span class="s2">"'"</span><span class="s1">'commands'</span><span class="s2">"'"</span><span class="s1">',\n '</span><span class="s2">"'"</span><span class="s1">'modify'</span><span class="s2">"'"</span><span class="s1">',\n '</span><span class="s2">"'"</span><span class="s1">'/'</span><span class="s2">"'"</span><span class="s1">',\n dittoHeaders,\n thing\n );\n}", |
| "outgoingScript": "function mapFromDittoProtocolMsg() { return null; }", |
| "loadBytebufferJS": "false", |
| "loadLongJS": "false" |
| } |
| } |
| } |
| } |
| } |
| }'</span> http://localhost:<span class="k">${</span><span class="nv">DITTO_EXTERNAL_PORT</span><span class="k">}</span>/devops/piggyback/connectivity?timeout<span class="o">=</span>8000 |
| </code></pre></div></div> |
| |
| <p>Explanation - what is done here:</p> |
| <ul> |
| <li>using curl with the <code class="highlighter-rouge">devops</code> (admin) user and its initial password <code class="highlighter-rouge">foobar</code> we create a connection of type <code class="highlighter-rouge">mqtt</code> |
| (you can find further information on that in Ditto’s <a href="connectivity-protocol-bindings-mqtt.html">MQTT docs</a>)</li> |
| <li>we use the TTN application credentials in the configured <code class="highlighter-rouge">"uri"</code>, connect via plain TCP |
| (SSL is also possible but in this case a little more complicated as the server certificate of the TTN MQTT broker |
| would have to be imported)</li> |
| <li>we add an entry in <code class="highlighter-rouge">"sources"</code>: |
| <ul> |
| <li>defining the MQTT topic (<code class="highlighter-rouge">"addresses"</code>) to subscribe to</li> |
| <li>specifying in which <code class="highlighter-rouge">"authorizationContext"</code> messages from this connection shall be executed</li> |
| <li>defining in the <code class="highlighter-rouge">"enforcement"</code> that, based on the MQTT topic, a device may only update the Ditto twin having the same name</li> |
| <li>declaring that a custom payload mapping shall be applied for each incoming message</li> |
| </ul> |
| </li> |
| <li>in the <code class="highlighter-rouge">"mappingDefinitions"</code> we define the previously used “ttn-demo-mapping” as JavaScript based mapping: |
| <ul> |
| <li>only an “incoming” script is defined as we don’t handle downstream messages to TTN in this example</li> |
| <li>when you want to understand the script in more depth, please take a look at the <a href="#javascript-payload-mapping-script-in-detail">details about it</a></li> |
| </ul> |
| </li> |
| </ul> |
| |
| <div class="alert alert-success" role="alert"><i class="fa fa-check-square-o"></i> <b>Tip:</b> As you have other custom <code class="highlighter-rouge">payload_fields</code> for your TTN devices, please adjust the script |
| if you want to see the device’s custom payload fields in your Ditto twins.</div> |
| |
| <h2 id="create-a-common-policy-for-the-twins-to-be-created">Create a common policy for the twins to be created</h2> |
| |
| <p>Eclipse Ditto secures each API access to the managed twins by applying authorization of the authenticated user.<br /> |
| Those “rules” which authenticated user may access which twins are defined in <a href="basic-policy.html">Policies</a>.</p> |
| |
| <p>In order to proceed with our scenario, we create a single Policy which shall be used for all twins we create in a later |
| step:</p> |
| |
| <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>curl <span class="nt">-X</span> PUT <span class="nt">-u</span> ditto:ditto <span class="nt">-H</span> <span class="s1">'Content-Type: application/json'</span> <span class="nt">-d</span> <span class="s1">'{ |
| "policyId": "org.eclipse.ditto.ttn.demo:twin-policy", |
| "entries": { |
| "USER": { |
| "subjects": { |
| "nginx:ditto": { |
| "type": "basic auth user authenticated via nginx" |
| } |
| }, |
| "resources": { |
| "thing:/": { |
| "grant": ["READ", "WRITE"], |
| "revoke": [] |
| }, |
| "policy:/": { |
| "grant": ["READ", "WRITE"], |
| "revoke": [] |
| }, |
| "message:/": { |
| "grant": ["READ", "WRITE"], |
| "revoke": [] |
| } |
| } |
| }, |
| "TTN": { |
| "subjects": { |
| "pre-authenticated:ttn-connection": { |
| "type": "used in the connections authorizationContext to the TTN MQTT" |
| } |
| }, |
| "resources": { |
| "thing:/": { |
| "grant": ["WRITE"], |
| "revoke": [] |
| } |
| } |
| } |
| } |
| }'</span> http://localhost:<span class="k">${</span><span class="nv">DITTO_EXTERNAL_PORT</span><span class="k">}</span>/api/2/policies/org.eclipse.ditto.ttn.demo:twin-policy |
| </code></pre></div></div> |
| |
| <p>Explanation - what is done here:</p> |
| <ul> |
| <li>we create a new Policy with the ID <code class="highlighter-rouge">"org.eclipse.ditto.ttn.demo:twin-policy"</code></li> |
| <li>it contains 2 entries: |
| <ul> |
| <li><code class="highlighter-rouge">"USER"</code>: this Policy entry contains the authorization information of the user of the twin APIs (authenticated via the contained “nginx” acting as reverse proxy). |
| This user may READ+WRITE the things (twins), this created policy and may also send and receive messages.</li> |
| <li><code class="highlighter-rouge">"TTN"</code>: this Policy entry contains the authorization information of the connection to the TTN MQTT broker (the subject was configured as <code class="highlighter-rouge">"authorizationContext"</code> when we created the connection. |
| This connection may only WRITE (update) the things (twins).</li> |
| </ul> |
| </li> |
| </ul> |
| |
| <h2 id="create-digital-twins">Create digital twins</h2> |
| |
| <p>Now we have everything in place in order to create digital twins for our devices connected to TTN.</p> |
| |
| <p>Please export all device ids you want to create digital twins for as comma separated environment variable:</p> |
| <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">export </span><span class="nv">TTN_DEVICE_IDS</span><span class="o">=</span><span class="s1">'&lt;comma-separated-list-of-your-device-ids&gt;'</span> |
| </code></pre></div></div> |
| |
| <p>After having done that, we can already create the twins in Ditto as the <code class="highlighter-rouge">ditto</code> user:</p> |
| |
| <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">for </span>dev_id <span class="k">in</span> <span class="k">${</span><span class="nv">TTN_DEVICE_IDS</span><span class="p">//,/ </span><span class="k">}</span> |
| <span class="k">do</span> |
| <span class="c"># call your procedure/other scripts here below</span> |
| <span class="nb">echo</span> <span class="s2">"Creating digital twin with Thing ID: org.eclipse.ditto.ttn.demo:</span><span class="nv">$dev_id</span><span class="s2">"</span> |
| curl <span class="nt">-X</span> PUT <span class="nt">-u</span> ditto:ditto <span class="nt">-H</span> <span class="s1">'Content-Type: application/json'</span> <span class="nt">-d</span> <span class="s1">'{ |
| "policyId": "org.eclipse.ditto.ttn.demo:twin-policy" |
| }'</span> http://localhost:<span class="k">${</span><span class="nv">DITTO_EXTERNAL_PORT</span><span class="k">}</span>/api/2/things/org.eclipse.ditto.ttn.demo:<span class="nv">$dev_id</span> |
| <span class="k">done</span> |
| </code></pre></div></div> |
| |
| <p>Explanation - what is done here:</p> |
| <ul> |
| <li>we split the passed in <code class="highlighter-rouge">TTN_DEVICE_IDS</code> environment variable by <code class="highlighter-rouge">,</code> and iterate over all contained device ids</li> |
| <li>for each device ID we create a new Thing (twin) referencing the already previously created Policy</li> |
| </ul> |
| |
| <h2 id="access-your-digital-twins-via-api">Access your digital twins via API</h2> |
| |
| <p>Congratulations, if you have done it so far your TTN devices do now have digital twin representations in Eclipse Ditto.</p> |
| |
| <div class="alert alert-success" role="alert"><i class="fa fa-check-square-o"></i> <b>Tip:</b> Install the command line tool <code class="highlighter-rouge">jq</code> and pipe the output of the below <code class="highlighter-rouge">curl</code> commands to it in order to get |
| prettified and colored JSON</div> |
| |
| <div class="alert alert-info" role="alert"><i class="fa fa-info-circle"></i> <b>Note:</b> Alternatively to <code class="highlighter-rouge">curl</code>, you may also use the locally deployed swagger-ui at http://localhost:${DITTO_EXTERNAL_PORT}/apidoc/ |
| in order to try out Ditto’s HTTP API - <em>make sure to select</em> <code class="highlighter-rouge">/api/2 - local Ditto</code> in the ‘Servers’ |
| section - when asked for credentials, use username ‘ditto’ and password ‘ditto’</div> |
| |
| <p>You can now, for example, use Ditto’s HTTP APIs in order</p> |
| <ul> |
| <li>to retrieve the latest reported values: <code class="highlighter-rouge">curl -u ditto:ditto http://localhost:${DITTO_EXTERNAL_PORT}/api/2/things/org.eclipse.ditto.ttn.demo:&lt;dev_id&gt;</code></li> |
| <li>to get a live stream of updates to the twins using SSE (Server Sent Events): <code class="highlighter-rouge">curl --http2 -u ditto:ditto -H 'Accept:text/event-stream' -N http://localhost:${DITTO_EXTERNAL_PORT}/api/2/things</code></li> |
| <li>to list all available twins via the search API: <code class="highlighter-rouge">curl -u ditto:ditto http://localhost:${DITTO_EXTERNAL_PORT}/api/2/search/things</code> |
| <ul> |
| <li>alternatively, use your browser and open http://localhost:${DITTO_EXTERNAL_PORT}/api/2/search/things</li> |
| <li>when asked for credentials, use username “ditto” and password “ditto”</li> |
| </ul> |
| </li> |
| <li>formulate a search query, e.g. only searching for twins with a temperature above 24°, sorted by the last modification, the most recent first |
| to get the most active twin as first result: |
| <ul> |
| <li><code class="highlighter-rouge">curl -u ditto:ditto "http://localhost:${DITTO_EXTERNAL_PORT}/api/2/search/things?filter=gt(features/temperature/properties/value,24.0)&amp;option=sort(-_modified),size(5)&amp;fields=thingId,policyId,attributes,features,_modified,_revision"</code></li> |
| </ul> |
| </li> |
| </ul> |
| |
| <h2 id="which-other-possibilities-do-we-now-have">Which other possibilities do we now have?</h2> |
| |
| <p>Now you have all the possibilities Eclipse Ditto as digital twin framework provides, e.g.:</p> |
| <ul> |
| <li>directly use your device’s data in a web application consuming Ditto’s HTTP API</li> |
| <li>directly use your device’s data in a mobile app using Ditto’s bidirectional <a href="httpapi-protocol-bindings-websocket.html">WebSocket</a></li> |
| <li>make use of the <a href="https://github.com/eclpise/ditto-clients/">Eclipse Ditto Java or JavaScript clients</a> which also use the WebSocket to integrate your device’s data</li> |
| <li>create another connection (optionally also applying JavaScript based payload mapping) |
| <ul> |
| <li>to e.g. <a href="connectivity-protocol-bindings-kafka2.html">Apache Kafka</a> and forward all the modifications made to your devices to there</li> |
| <li>or using <a href="connectivity-protocol-bindings-http.html">HTTP push</a> in order to call another HTTP API (e.g. insert time series data into an InfluxDB via its HTTP API)</li> |
| </ul> |
| </li> |
| </ul> |
| |
| <p><br /> |
| <br /></p> |
| |
| <p>For time reasons we do not go deeper into additional topics, they are possible however, please consult the Ditto |
| documentation:</p> |
| <ul> |
| <li>the WebSocket channel and subscribing for change notifications</li> |
| <li>sending downward messages to devices</li> |
| <li>live commands (not retrieving persisted data of devices, but live data)</li> |
| <li>a more detailed introduction into authentication mechanisms (<a href="installation-operating.html#openid-connect">OpenID Connect with OAuth2.0 is possible</a>)</li> |
| <li>possibilities to configure your Policies on every resource level, e.g. allowing individuals to only access certain values of a twin</li> |
| <li>and many other things..</li> |
| </ul> |
| |
| <h2 id="additional-resources">Additional resources</h2> |
| |
| <h3 id="cleanup-after-the-workshop">Cleanup after the workshop</h3> |
| |
| <p>Simply perform in the <code class="highlighter-rouge">ditto/deployment/docker</code> folder:</p> |
| <div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker-compose down |
| </code></pre></div></div> |
| |
| <p>And unistall docker + docker-compose (for docker-compose, just remove the downloaded file) again, if you don’t need it.</p> |
| |
| <h3 id="javascript-payload-mapping-script-in-detail">JavaScript payload mapping script in detail</h3> |
| |
| <p>Similar to the TTN console’s decoding/converting capabilities of “Payload Formats” of an TTN application, Ditto is able |
| to apply a custom JavaScript function for each consumed message.<br /> |
| That is necessary in order to convert the received data into a <a href="protocol-overview.html">Ditto Protocol</a> message |
| including the JSON hierarchy of a so called <a href="basic-thing.html">Thing</a> being the representation of a digital twin.</p> |
| |
| <p>As the above injected JavaScript payload mapping script is formatted in a single line, this is the script we used pretty |
| formatted, including the jsdoc of the provided function and some other inline comments.</p> |
| |
| <p>If you need to adjust the script in order to use your own <code class="highlighter-rouge">payload_fields</code>, please replace all newlines with <code class="highlighter-rouge">\n</code> and |
| escape the single quotes <code class="highlighter-rouge">'</code> in the script with the following replacement: <code class="highlighter-rouge">'"'"'</code>. Otherwise the single quotes won’t get |
| correctly escaped in the bash. You can remove the comments before making a single line of the script.</p> |
| |
| <div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cm">/** |
| * Maps the passed parameters to a Ditto Protocol message. |
| * @param {Object.&lt;string, string&gt;} headers - The headers Object containing all received header values |
| * @param {string} [textPayload] - The String to be mapped |
| * @param {ArrayBuffer} [bytePayload] - The bytes to be mapped as ArrayBuffer |
| * @param {string} [contentType] - The received Content-Type, e.g. "application/json" |
| * @returns {(DittoProtocolMessage|Array&lt;DittoProtocolMessage&gt;)} dittoProtocolMessage(s) - |
| * The mapped Ditto Protocol message, |
| * an array of Ditto Protocol messages or |
| * &lt;code&gt;null&lt;/code&gt; if the message could/should not be mapped |
| */</span> |
| <span class="kd">function</span> <span class="nx">mapToDittoProtocolMsg</span><span class="p">(</span> |
| <span class="nx">headers</span><span class="p">,</span> |
| <span class="nx">textPayload</span><span class="p">,</span> |
| <span class="nx">bytePayload</span><span class="p">,</span> |
| <span class="nx">contentType</span> |
| <span class="p">)</span> <span class="p">{</span> |
| |
| <span class="kd">let</span> <span class="nx">ttnJson</span> <span class="o">=</span> <span class="nx">JSON</span><span class="p">.</span><span class="nx">parse</span><span class="p">(</span><span class="nx">textPayload</span><span class="p">);</span> <span class="c1">// we simply parse the incoming TTN message as JSON</span> |
| <span class="kd">let</span> <span class="nx">deviceId</span> <span class="o">=</span> <span class="nx">ttnJson</span><span class="p">[</span><span class="s1">'dev_id'</span><span class="p">];</span> <span class="c1">// and extract some fields we require</span> |
| <span class="kd">let</span> <span class="nx">payloadFields</span> <span class="o">=</span> <span class="nx">ttnJson</span><span class="p">[</span><span class="s1">'payload_fields'</span><span class="p">];</span> <span class="c1">// the 'payload_fields' content is - obviously - different for your application</span> |
| |
| <span class="kd">let</span> <span class="nx">attributesObj</span> <span class="o">=</span> <span class="p">{</span> <span class="c1">// the attributes of a Thing are meant for unstructured data </span> |
| <span class="na">hardwareSerial</span><span class="p">:</span> <span class="nx">ttnJson</span><span class="p">[</span><span class="s1">'hardware_serial'</span><span class="p">],</span> |
| <span class="na">ttnCounter</span><span class="p">:</span> <span class="nx">ttnJson</span><span class="p">[</span><span class="s1">'counter'</span><span class="p">]</span> |
| <span class="p">};</span> |
| |
| <span class="kd">let</span> <span class="nx">featuresObj</span> <span class="o">=</span> <span class="p">{</span> <span class="c1">// the features of a Thing e.g. contain sensor data of devices</span> |
| <span class="na">temperature</span><span class="p">:</span> <span class="p">{</span> |
| <span class="na">properties</span><span class="p">:</span> <span class="p">{</span> |
| <span class="na">value</span><span class="p">:</span> <span class="nx">payloadFields</span><span class="p">[</span><span class="s1">'temperature_7'</span><span class="p">]</span> |
| <span class="p">}</span> |
| <span class="p">},</span> |
| <span class="na">pressure</span><span class="p">:</span> <span class="p">{</span> |
| <span class="na">properties</span><span class="p">:</span> <span class="p">{</span> |
| <span class="na">value</span><span class="p">:</span> <span class="nx">payloadFields</span><span class="p">[</span><span class="s1">'barometric_pressure_10'</span><span class="p">]</span> |
| <span class="p">}</span> |
| <span class="p">},</span> |
| <span class="na">humidity</span><span class="p">:</span> <span class="p">{</span> |
| <span class="na">properties</span><span class="p">:</span> <span class="p">{</span> |
| <span class="na">value</span><span class="p">:</span> <span class="nx">payloadFields</span><span class="p">[</span><span class="s1">'relative_humidity_8'</span><span class="p">]</span> |
| <span class="p">}</span> |
| <span class="p">}</span> |
| <span class="p">};</span> |
| |
| <span class="kd">let</span> <span class="nx">thing</span> <span class="o">=</span> <span class="p">{</span> <span class="c1">// a Thing can contain both attributes and features</span> |
| <span class="na">attributes</span><span class="p">:</span> <span class="nx">attributesObj</span><span class="p">,</span> |
| <span class="na">features</span><span class="p">:</span> <span class="nx">featuresObj</span> |
| <span class="p">};</span> |
| |
| <span class="kd">let</span> <span class="nx">dittoHeaders</span> <span class="o">=</span> <span class="p">{</span> |
| <span class="s1">'response-required'</span><span class="p">:</span> <span class="kc">false</span><span class="p">,</span> <span class="c1">// we don't expect a response sent back to TTN</span> |
| <span class="s1">'If-Match'</span><span class="p">:</span> <span class="s1">'*'</span> <span class="c1">// we only want to update the thing if it already exists</span> |
| <span class="p">};</span> |
| |
| <span class="k">return</span> <span class="nx">Ditto</span><span class="p">.</span><span class="nx">buildDittoProtocolMsg</span><span class="p">(</span> |
| <span class="s1">'org.eclipse.ditto.ttn.demo'</span><span class="p">,</span> <span class="c1">// this is the namespace used as prefix for Ditto Thing IDs</span> |
| <span class="nx">deviceId</span><span class="p">,</span> <span class="c1">// the TTN device ID is used as "name" part of the Ditto Thing ID </span> |
| <span class="s1">'things'</span><span class="p">,</span> |
| <span class="s1">'twin'</span><span class="p">,</span> |
| <span class="s1">'commands'</span><span class="p">,</span> |
| <span class="s1">'modify'</span><span class="p">,</span> |
| <span class="s1">'/'</span><span class="p">,</span> |
| <span class="nx">dittoHeaders</span><span class="p">,</span> |
| <span class="nx">thing</span> |
| <span class="p">);</span> |
| <span class="p">}</span> |
| </code></pre></div></div> |
| |
| <p>An example message received from the TTN MQTT broker:</p> |
| <div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w"> |
| </span><span class="s2">"app_id"</span><span class="p">:</span><span class="w"> </span><span class="s2">"iot-campus-be12"</span><span class="p">,</span><span class="w"> |
| </span><span class="s2">"dev_id"</span><span class="p">:</span><span class="w"> </span><span class="s2">"node0"</span><span class="p">,</span><span class="w"> |
| </span><span class="s2">"hardware_serial"</span><span class="p">:</span><span class="w"> </span><span class="s2">"70B3D5499A2D3954"</span><span class="p">,</span><span class="w"> |
| </span><span class="s2">"port"</span><span class="p">:</span><span class="w"> </span><span class="mi">2</span><span class="p">,</span><span class="w"> |
| </span><span class="s2">"counter"</span><span class="p">:</span><span class="w"> </span><span class="mi">9449</span><span class="p">,</span><span class="w"> |
| </span><span class="s2">"payload_raw"</span><span class="p">:</span><span class="w"> </span><span class="s2">"B2cA6AhoKwpzJ8oEAwH4"</span><span class="p">,</span><span class="w"> |
| </span><span class="s2">"payload_fields"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w"> |
| </span><span class="s2">"analog_out_4"</span><span class="p">:</span><span class="w"> </span><span class="mf">5.04</span><span class="p">,</span><span class="w"> |
| </span><span class="s2">"barometric_pressure_10"</span><span class="p">:</span><span class="w"> </span><span class="mf">1018.6</span><span class="p">,</span><span class="w"> |
| </span><span class="s2">"relative_humidity_8"</span><span class="p">:</span><span class="w"> </span><span class="mf">21.5</span><span class="p">,</span><span class="w"> |
| </span><span class="s2">"temperature_7"</span><span class="p">:</span><span class="w"> </span><span class="mf">23.2</span><span class="w"> |
| </span><span class="p">},</span><span class="w"> |
| </span><span class="s2">"metadata"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w"> |
| </span><span class="err">...</span><span class="w"> |
| </span><span class="p">}</span><span class="w"> |
| </span><span class="p">}</span><span class="w"> |
| </span></code></pre></div></div> |
| |
| <p>would be transformed to the following Ditto Protocol message:</p> |
| <div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w"> |
| </span><span class="s2">"topic"</span><span class="p">:</span><span class="w"> </span><span class="s2">"org.eclipse.ditto/node0/things/twin/commands/modify"</span><span class="p">,</span><span class="w"> |
| </span><span class="s2">"path"</span><span class="p">:</span><span class="w"> </span><span class="s2">"/"</span><span class="p">,</span><span class="w"> |
| </span><span class="s2">"value"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w"> |
| </span><span class="s2">"attributes"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w"> |
| </span><span class="s2">"hardwareSerial"</span><span class="p">:</span><span class="w"> </span><span class="s2">"70B3D5499A2D3954"</span><span class="p">,</span><span class="w"> |
| </span><span class="s2">"ttnCounter"</span><span class="p">:</span><span class="w"> </span><span class="mi">9449</span><span class="w"> |
| </span><span class="p">},</span><span class="w"> |
| </span><span class="s2">"features"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w"> |
| </span><span class="s2">"temperature"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w"> |
| </span><span class="s2">"properties"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w"> |
| </span><span class="s2">"value"</span><span class="p">:</span><span class="w"> </span><span class="mf">23.2</span><span class="w"> |
| </span><span class="p">}</span><span class="w"> |
| </span><span class="p">},</span><span class="w"> |
| </span><span class="s2">"pressure"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w"> |
| </span><span class="s2">"properties"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w"> |
| </span><span class="s2">"value"</span><span class="p">:</span><span class="w"> </span><span class="mf">1018.6</span><span class="w"> |
| </span><span class="p">}</span><span class="w"> |
| </span><span class="p">},</span><span class="w"> |
| </span><span class="s2">"humidity"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w"> |
| </span><span class="s2">"properties"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w"> |
| </span><span class="s2">"value"</span><span class="p">:</span><span class="w"> </span><span class="mf">21.5</span><span class="w"> |
| </span><span class="p">}</span><span class="w"> |
| </span><span class="p">}</span><span class="w"> |
| </span><span class="p">}</span><span class="w"> |
| </span><span class="p">}</span><span class="w"> |
| </span><span class="p">}</span><span class="w"> |
| </span></code></pre></div></div> |
| |
| <p><br /> |
| <br /></p> |
| <figure><img class="docimage" src="images/ditto.svg" alt="Ditto" style="max-width: 500px" /></figure> |
| |
| <p>–<br /> |
| The Eclipse Ditto team</p> |
| </description> |
| <pubDate>Thu, 16 Apr 2020 00:00:00 +0000</pubDate> |
| <link>https://www.eclipse.org/ditto/2020-04-16-connecting-to-ttn-via-mqtt.html</link> |
| <guid isPermaLink="true">https://www.eclipse.org/ditto/2020-04-16-connecting-to-ttn-via-mqtt.html</guid> |
| |
| <category>blog</category> |
| |
| <category>connectivity</category> |
| |
| |
| </item> |
| |
| <item> |
| <title>Announcing Eclipse Ditto Release 1.0.0</title> |
| <description><p>Today the Eclipse Ditto is thrilled to announce the availability of Eclipse Ditto’s first major release |
| <a href="https://projects.eclipse.org/projects/iot.ditto/releases/1.0.0">1.0.0</a>.</p> |
| |
| <h2 id="maturity">Maturity</h2> |
| |
| <p>The initial code contribution was done in October 2017, 2 years later and 2 releases |
| (<a href="2018-11-28-release-announcement-080.html">0.8.0</a> and <a href="2019-07-10-release-announcement-090.html">0.9.0</a>) later, we |
| think its time to graduate from the Eclipse “incubation” phase and officially declare the project as mature.</p> |
| |
| <p>Recent adoptions and contributions from our community show us that Eclipse Ditto solves problems which also other |
| companies have. Adopters add Eclipse Ditto as a central part of their own IoT platforms.</p> |
| |
| <h3 id="api-stability">API stability</h3> |
| |
| <p>Having reached 1.0.0, some additional promises towards “API stability” do apply:</p> |
| |
| <h4 id="http-api-stability">HTTP API stability</h4> |
| <p>Ditto uses schema versioning (currently schema version 1 and 2) at the HTTP API level in order to being able to |
| evolve APIs. |
| It is backward compatible to the prior versions 0.8.0 and 0.9.0.</p> |
| |
| <h4 id="json-api-stability">JSON API stability</h4> |
| <p>Ditto kept its main JSON APIs (regarding things, policies and search) backwards compatible to 0.8.0 and 0.9.0 releases. |
| The JSON format of “connections” was changed since 0.9.0 and will from 1.0.0 on be kept backwards compatible as well.</p> |
| |
| <h4 id="java-api-stability">Java API stability</h4> |
| <p>The Java APIs will for the 1.x release be kept backwards compatible, so only non-breaking additions to the APIs will be done. This is enforced by a Maven tooling.</p> |
| |
| <p>The following Java modules are treated as API for which compatibility is enforced:</p> |
| |
| <ul> |
| <li>ditto-json</li> |
| <li>ditto-model-*</li> |
| <li>ditto-signals-*</li> |
| <li>ditto-protocol-adapter</li> |
| <li>ditto-utils</li> |
| <li>ditto-client</li> |
| </ul> |
| |
| <h3 id="scalability">Scalability</h3> |
| |
| <p>The focus on the 0.9.0 and 1.0.0 releases regarding non-functionals were laid on horizontal scalability.</p> |
| |
| <p>With Eclipse Ditto 1.0.0 we are confident to face production grade scalability requirements being capable of handling |
| millions of managed things.</p> |
| |
| <h2 id="changelog">Changelog</h2> |
| |
| <p>The main changes compared to the last release, <a href="release_notes_090.html">0.9.0</a>, are:</p> |
| |
| <ul> |
| <li>addition of a Java and a JavaScript client SDK in separate <a href="https://github.com/eclipse/ditto-clients">GitHub repo</a></li> |
| <li>configurable OpenID Connect authorization servers</li> |
| <li>support for OpenID Connect / OAuth2.0 based authentication in Ditto Java Client</li> |
| <li>invoking custom foreign HTTP endpoints as a result of events/messages</li> |
| <li>ability to reflect Eclipse Hono’s device connection state in Ditto’s things</li> |
| <li>configurable throttling of max. consumed WebSocket commands / time interval</li> |
| <li>Addition of “definition” field in thing at model level containing the model ID a thing may follow</li> |
| <li>Improved connection response handling/mapping</li> |
| </ul> |
| |
| <p>Please have a look at the <a href="release_notes_100.html">1.0.0 release notes</a> for a more detailed information on the release.</p> |
| |
| <h2 id="artifacts">Artifacts</h2> |
| |
| <p>The new Java artifacts have been published at the <a href="https://repo.eclipse.org/content/repositories/ditto/">Eclipse Maven repository</a> |
| as well as <a href="https://repo1.maven.org/maven2/org/eclipse/ditto/">Maven central</a>.</p> |
| |
| <p>The Docker images have been pushed to Docker Hub:</p> |
| <ul> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-policies/">eclipse/ditto-policies</a></li> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-things/">eclipse/ditto-things</a></li> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-things-search/">eclipse/ditto-things-search</a></li> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-gateway/">eclipse/ditto-gateway</a></li> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-connectivity/">eclipse/ditto-connectivity</a></li> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-concierge/">eclipse/ditto-concierge</a></li> |
| </ul> |
| |
| <p><br /> |
| <br /></p> |
| <figure><img class="docimage" src="images/ditto.svg" alt="Ditto" style="max-width: 500px" /></figure> |
| |
| <p>–<br /> |
| The Eclipse Ditto team</p> |
| </description> |
| <pubDate>Thu, 12 Dec 2019 00:00:00 +0000</pubDate> |
| <link>https://www.eclipse.org/ditto/2019-12-12-release-announcement-100.html</link> |
| <guid isPermaLink="true">https://www.eclipse.org/ditto/2019-12-12-release-announcement-100.html</guid> |
| |
| <category>blog</category> |
| |
| |
| </item> |
| |
| <item> |
| <title>Announcing Ditto Milestone 1.0.0-M2</title> |
| <description><p>The second and last milestone of the upcoming release |
| <a href="https://projects.eclipse.org/projects/iot.ditto/releases/1.0.0">1.0.0</a> was released today.</p> |
| |
| <p>Have a look at the Milestone <a href="release_notes_100-M2.html">1.0.0-M2 release notes</a> for what changed in detail.</p> |
| |
| <p>The main changes and new features since the last release <a href="release_notes_100-M1a.html">1.0.0-M1a release notes</a> are</p> |
| |
| <ul> |
| <li>invoking custom foreign HTTP endpoints as a result of events/messages</li> |
| <li>ability to reflect Eclipse Hono’s device connection state in Ditto’s things</li> |
| <li>support for OpenID Connect / OAuth2.0 based authentication in Ditto Java Client</li> |
| <li>configurbale throttling of max. consumed WebSocket commands / time interval</li> |
| </ul> |
| |
| <h2 id="artifacts">Artifacts</h2> |
| |
| <p>The new Java artifacts have been published at the <a href="https://repo.eclipse.org/content/repositories/ditto/">Eclipse Maven repository</a> |
| as well as <a href="https://repo1.maven.org/maven2/org/eclipse/ditto/">Maven central</a>.</p> |
| |
| <p>The Docker images have been pushed to Docker Hub:</p> |
| <ul> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-policies/">eclipse/ditto-policies</a></li> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-things/">eclipse/ditto-things</a></li> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-things-search/">eclipse/ditto-things-search</a></li> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-gateway/">eclipse/ditto-gateway</a></li> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-connectivity/">eclipse/ditto-connectivity</a></li> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-concierge/">eclipse/ditto-concierge</a></li> |
| </ul> |
| |
| <p><br /> |
| <br /></p> |
| <figure><img class="docimage" src="images/ditto.svg" alt="Ditto" style="max-width: 500px" /></figure> |
| |
| <p>–<br /> |
| The Eclipse Ditto team</p> |
| </description> |
| <pubDate>Mon, 04 Nov 2019 00:00:00 +0000</pubDate> |
| <link>https://www.eclipse.org/ditto/2019-11-04-milestone-announcement-100-M2.html</link> |
| <guid isPermaLink="true">https://www.eclipse.org/ditto/2019-11-04-milestone-announcement-100-M2.html</guid> |
| |
| <category>blog</category> |
| |
| |
| </item> |
| |
| <item> |
| <title>Integration of HTTP endpoints/webhooks</title> |
| <description><p>By adding another connectivity type - <a href="connectivity-protocol-bindings-http.html">HTTP</a> - to Ditto’s |
| connectivity, it is now (to be released in the next Ditto milestone 1.0.0-M2) possible to to |
| publish <em>twin events</em>, <em>messages</em>, <em>live commands and events</em> to existing HTTP servers/endpoints.</p> |
| |
| <p>That is especially useful for invoking existing APIs (which are most of the time HTTP based) whenever e.g. |
| a digital twin was modified.</p> |
| |
| <p>One example on how to benefit from this new feature is to invoke a custom |
| <a href="https://ifttt.com">IFTTT</a> (if-this-than-that) <a href="https://ifttt.com/maker_webhooks">webhook</a> via a HTTP <code class="highlighter-rouge">POST</code> request |
| which then may trigger other IFTTT follow-up-actions (e.g. send a chat message to a <a href="https://ifttt.com/slack">Slack</a> |
| room).</p> |
| |
| <p>For IFTTT “webhooks” the address would be <code class="highlighter-rouge">POST https://maker.ifttt.com/trigger/&lt;your-event-name&gt;/with/key/&lt;your-key&gt;</code> |
| and the expected JSON body:</p> |
| |
| <div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w"> |
| </span><span class="s2">"value1"</span><span class="p">:</span><span class="w"> </span><span class="s2">"..."</span><span class="p">,</span><span class="w"> |
| </span><span class="s2">"value2"</span><span class="p">:</span><span class="w"> </span><span class="s2">"..."</span><span class="p">,</span><span class="w"> |
| </span><span class="s2">"value3"</span><span class="p">:</span><span class="w"> </span><span class="s2">"..."</span><span class="w"> |
| </span><span class="p">}</span><span class="w"> |
| </span></code></pre></div></div> |
| |
| <p>In combination with <a href="connectivity-mapping.html">payload mapping</a>, the <code class="highlighter-rouge">value1</code> to <code class="highlighter-rouge">value3</code> fields requested by the IFTTT |
| API can be extracted from the <a href="protocol-specification.html">Ditto Protocol</a> and could contain the changed value.</p> |
| |
| <p>In combination with <a href="basic-connections.html#target-topics-and-filtering">filters for targets</a> you can even specify to |
| only publish e.g. <code class="highlighter-rouge">twin events</code>, where the temperature of a twin exceeded a certain threshold:</p> |
| |
| <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>{ |
| "address": "POST:/trigger/&lt;your-event-name&gt;/with/key/&lt;your-key&gt;", |
| "topics": [ |
| "_/_/things/twin/events?filter=gt(features/temperature/properties/value,25)" |
| ], |
| ... |
| } |
| </code></pre></div></div> |
| |
| <p>Get creative on which HTTP APIs to trigger based on twin events. E.g. invoke a “function-as-a-service” API or invoke the |
| Twitter API and let your digital twin tweet whenever it detects, that it is getting too warm in your office.</p> |
| |
| <figure><img class="docimage" src="images/ditto.svg" alt="Ditto" style="max-width: 500px" /></figure> |
| |
| <p>–<br /> |
| The Eclipse Ditto team</p> |
| </description> |
| <pubDate>Thu, 17 Oct 2019 00:00:00 +0000</pubDate> |
| <link>https://www.eclipse.org/ditto/2019-10-17-http-connectivity.html</link> |
| <guid isPermaLink="true">https://www.eclipse.org/ditto/2019-10-17-http-connectivity.html</guid> |
| |
| <category>blog</category> |
| |
| |
| </item> |
| |
| <item> |
| <title>Announcing Ditto Milestone 1.0.0-M1a</title> |
| <description><p>Today the Ditto team is happy to announce the first milestone of the upcoming release |
| <a href="https://projects.eclipse.org/projects/iot.ditto/releases/1.0.0">1.0.0</a>.</p> |
| |
| <p>Have a look at the Milestone <a href="release_notes_100-M1a.html">1.0.0-M1a release notes</a> for what changed in detail.</p> |
| |
| <p>The main changes and new features since the last release <a href="release_notes_090.html">0.9.0</a> are</p> |
| |
| <ul> |
| <li>initial contribution of Java client SDK</li> |
| <li>configurable OpenID Connect authorization servers</li> |
| <li>fine grained access for connections</li> |
| <li>scalable event publishing</li> |
| <li>typed entity IDs</li> |
| </ul> |
| |
| <h2 id="artifacts">Artifacts</h2> |
| |
| <p>The new Java artifacts have been published at the <a href="https://repo.eclipse.org/content/repositories/ditto/">Eclipse Maven repository</a> |
| as well as <a href="https://repo1.maven.org/maven2/org/eclipse/ditto/">Maven central</a>.</p> |
| |
| <p>The Docker images have been pushed to Docker Hub:</p> |
| <ul> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-policies/">eclipse/ditto-policies</a></li> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-things/">eclipse/ditto-things</a></li> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-things-search/">eclipse/ditto-things-search</a></li> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-gateway/">eclipse/ditto-gateway</a></li> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-connectivity/">eclipse/ditto-connectivity</a></li> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-concierge/">eclipse/ditto-concierge</a></li> |
| </ul> |
| |
| <p><br /> |
| <br /></p> |
| <figure><img class="docimage" src="images/ditto.svg" alt="Ditto" style="max-width: 500px" /></figure> |
| |
| <p>–<br /> |
| The Eclipse Ditto team</p> |
| </description> |
| <pubDate>Tue, 17 Sep 2019 00:00:00 +0000</pubDate> |
| <link>https://www.eclipse.org/ditto/2019-09-17-milestone-announcement-100-M1a.html</link> |
| <guid isPermaLink="true">https://www.eclipse.org/ditto/2019-09-17-milestone-announcement-100-M1a.html</guid> |
| |
| <category>blog</category> |
| |
| |
| </item> |
| |
| <item> |
| <title>Eclipse Ditto now supports OpenID Connect</title> |
| <description><p>Eclipse Ditto now supports all OAuth 2.0 providers which implement <a href="https://openid.net/connect/">OpenID Connect</a> out-of-the-box. |
| You can find a list of certified providers at <a href="https://openid.net/developers/certified/">OpenID Connect - Certified OpenID Provider Servers and Services</a>.</p> |
| |
| <p>With this post, we want to give an example of this new feature using the open source provider <a href="https://www.ory.sh">ORY Hydra</a>. |
| Follow their <a href="https://www.ory.sh/docs/next/hydra/configure-deploy#installing-ory-hydra">installation guide</a> for a |
| docker based setup on your development machine.</p> |
| |
| <h4 id="configuration">Configuration</h4> |
| <p>Download the self-signed certificate form the ORY Hydra server: https://localhost:9000/.well-known/openid-configuration</p> |
| |
| <p>Use the downloaded certificate for the akka-http ssl configuration.</p> |
| <pre><code class="language-hocon">ssl-config { |
| trustManager = { |
| stores = [ |
| { type = "PEM", path = "/path/to/cert/globalsign.crt" } |
| ] |
| } |
| } |
| </code></pre> |
| |
| <p>The authentication provider must be added to the ditto-gateway configuration.</p> |
| <pre><code class="language-hocon">ditto.gateway.authentication { |
| oauth { |
| openid-connect-issuers = { |
| ory = "https://localhost:9000/" |
| } |
| } |
| } |
| </code></pre> |
| |
| <p>The configured subject-issuer will be used to prefix the value of the “sub” claim, e.g.</p> |
| <div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w"> |
| </span><span class="s2">"subjects"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w"> |
| </span><span class="s2">"ory:foo@bar.com"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w"> |
| </span><span class="s2">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"generated"</span><span class="w"> |
| </span><span class="p">}</span><span class="w"> |
| </span><span class="p">}</span><span class="w"> |
| </span><span class="p">}</span><span class="w"> |
| </span></code></pre></div></div> |
| |
| <h4 id="authenticate-ditto-api">Authenticate Ditto API</h4> |
| <p>Create an OAuth client with hydra to be able to create ID Tokens.</p> |
| <div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker run <span class="nt">--rm</span> <span class="nt">-it</span> <span class="se">\</span> |
| <span class="nt">-e</span> <span class="nv">HYDRA_ADMIN_URL</span><span class="o">=</span>https://ory-hydra-example--hydra:4445 <span class="se">\</span> |
| <span class="nt">--network</span> hydraguide <span class="se">\</span> |
| oryd/hydra:v1.0.0 <span class="se">\</span> |
| clients create <span class="nt">--skip-tls-verify</span> <span class="se">\</span> |
| <span class="nt">--id</span> eclipse-ditto <span class="se">\</span> |
| <span class="nt">--secret</span> some-secret <span class="se">\</span> |
| <span class="nt">--grant-types</span> authorization_code,refresh_token,client_credentials,implicit <span class="se">\</span> |
| <span class="nt">--response-types</span> token,code,id_token <span class="se">\</span> |
| <span class="nt">--scope</span> openid,offline <span class="se">\</span> |
| <span class="nt">--callbacks</span> http://127.0.0.1:9010/callback |
| </code></pre></div></div> |
| |
| <p>Use the client to generate an ID Token.</p> |
| <div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker run <span class="nt">--rm</span> <span class="nt">-it</span> <span class="se">\</span> |
| <span class="nt">--network</span> hydraguide <span class="se">\</span> |
| <span class="nt">-p</span> 9010:9010 <span class="se">\</span> |
| oryd/hydra:v1.0.0 <span class="se">\</span> |
| token user <span class="nt">--skip-tls-verify</span> <span class="se">\</span> |
| <span class="nt">--port</span> 9010 <span class="se">\</span> |
| <span class="nt">--auth-url</span> https://localhost:9000/oauth2/auth <span class="se">\</span> |
| <span class="nt">--token-url</span> https://ory-hydra-example--hydra:4444/oauth2/token <span class="se">\</span> |
| <span class="nt">--client-id</span> eclipse-ditto <span class="se">\</span> |
| <span class="nt">--client-secret</span> some-secret <span class="se">\</span> |
| <span class="nt">--scope</span> openid |
| </code></pre></div></div> |
| <p>After that perform the OAuth 2.0 Authorize Code Flow by opening the link, as prompted, |
| in your browser, and follow the steps shown there.</p> |
| |
| <p>Use the generated token to authenticate Ditto API.</p> |
| <div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>curl <span class="nt">-X</span> POST <span class="se">\</span> |
| http://localhost:8080/api/2/things <span class="se">\</span> |
| <span class="nt">-H</span> <span class="s1">'Authorization: Bearer &lt;JWT&gt;'</span> <span class="se">\</span> |
| <span class="nt">-H</span> <span class="s1">'Content-Type: application/json'</span> <span class="se">\</span> |
| <span class="nt">-d</span> <span class="s1">'{}'</span> |
| </code></pre></div></div> |
| |
| <p><br /> |
| <br /></p> |
| <figure><img class="docimage" src="images/ditto.svg" alt="Ditto" style="max-width: 500px" /></figure> |
| |
| <p>–<br /> |
| The Eclipse Ditto team</p> |
| </description> |
| <pubDate>Wed, 28 Aug 2019 00:00:00 +0000</pubDate> |
| <link>https://www.eclipse.org/ditto/2019-08-28-openid-connect.html</link> |
| <guid isPermaLink="true">https://www.eclipse.org/ditto/2019-08-28-openid-connect.html</guid> |
| |
| <category>blog</category> |
| |
| |
| </item> |
| |
| <item> |
| <title>Announcing Eclipse Ditto Release 0.9.0</title> |
| <description><p>Today the Eclipse Ditto team proudly presents its second release |
| <a href="https://projects.eclipse.org/projects/iot.ditto/releases/0.9.0">0.9.0</a>.</p> |
| |
| <p>The topics of this release in a nutshell were:</p> |
| |
| <ul> |
| <li>Memory improvements for huge amounts (multi million) of digital twins which are held in memory</li> |
| <li>Adding metrics and logging around the connectivity feature in order to enable being able to operate connections to foreign systems/brokers via APIs</li> |
| <li>Enhancing Ditto’s connectivity feature by additionally being able to connect to Apache Kafka</li> |
| <li>Performance improvements of Ditto’s search functionality</li> |
| <li>Stabilization of cluster bootstrapping</li> |
| <li>Refactoring of how the services configurations are determined</li> |
| <li>Addition of a Helm template in order to simplify Kubernetes based deployments</li> |
| <li>Contributions from Microsoft in order to ease operating Eclipse Ditto on Microsoft Azure</li> |
| </ul> |
| |
| <p>Please have a look at the <a href="release_notes_090.html">0.9.0 release notes</a> for a more detailed information on the release.</p> |
| |
| <h2 id="artifacts">Artifacts</h2> |
| |
| <p>The new Java artifacts have been published at the <a href="https://repo.eclipse.org/content/repositories/ditto/">Eclipse Maven repository</a> |
| as well as <a href="https://repo1.maven.org/maven2/org/eclipse/ditto/">Maven central</a>.</p> |
| |
| <p>The Docker images have been pushed to Docker Hub:</p> |
| <ul> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-policies/">eclipse/ditto-policies</a></li> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-things/">eclipse/ditto-things</a></li> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-things-search/">eclipse/ditto-things-search</a></li> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-gateway/">eclipse/ditto-gateway</a></li> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-connectivity/">eclipse/ditto-connectivity</a></li> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-concierge/">eclipse/ditto-concierge</a></li> |
| </ul> |
| |
| <p><br /> |
| <br /></p> |
| <figure><img class="docimage" src="images/ditto.svg" alt="Ditto" style="max-width: 500px" /></figure> |
| |
| <p>–<br /> |
| The Eclipse Ditto team</p> |
| </description> |
| <pubDate>Wed, 10 Jul 2019 00:00:00 +0000</pubDate> |
| <link>https://www.eclipse.org/ditto/2019-07-10-release-announcement-090.html</link> |
| <guid isPermaLink="true">https://www.eclipse.org/ditto/2019-07-10-release-announcement-090.html</guid> |
| |
| <category>blog</category> |
| |
| |
| </item> |
| |
| <item> |
| <title>Announcing Ditto Milestone 0.9.0-M2</title> |
| <description><p>The second milestone of the upcoming release |
| <a href="https://projects.eclipse.org/projects/iot.ditto/releases/0.9.0">0.9.0</a> was released today.</p> |
| |
| <p>Have a look at the Milestone <a href="release_notes_090-M2.html">0.9.0-M2 release notes</a> for what changed in detail.</p> |
| |
| <p>The main changes and new features since the last milestone <a href="release_notes_090-M1.html">0.9.0-M1</a> are</p> |
| |
| <ul> |
| <li>rewrite of Ditto’s “search” service in order to use the same index and have the same query performance for API v1 and v2</li> |
| <li>several contributions in order to operate Eclipse Ditto on Microsoft Azure</li> |
| </ul> |
| |
| <h2 id="artifacts">Artifacts</h2> |
| |
| <p>The new Java artifacts have been published at the <a href="https://repo.eclipse.org/content/repositories/ditto/">Eclipse Maven repository</a> |
| as well as <a href="https://repo1.maven.org/maven2/org/eclipse/ditto/">Maven central</a>.</p> |
| |
| <p>The Docker images have been pushed to Docker Hub:</p> |
| <ul> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-policies/">eclipse/ditto-policies</a></li> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-things/">eclipse/ditto-things</a></li> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-things-search/">eclipse/ditto-things-search</a></li> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-gateway/">eclipse/ditto-gateway</a></li> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-connectivity/">eclipse/ditto-connectivity</a></li> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-concierge/">eclipse/ditto-concierge</a></li> |
| </ul> |
| |
| <p><br /> |
| <br /></p> |
| <figure><img class="docimage" src="images/ditto.svg" alt="Ditto" style="max-width: 500px" /></figure> |
| |
| <p>–<br /> |
| The Eclipse Ditto team</p> |
| </description> |
| <pubDate>Mon, 29 Apr 2019 00:00:00 +0000</pubDate> |
| <link>https://www.eclipse.org/ditto/2019-04-29-milestone-announcement-090-M2.html</link> |
| <guid isPermaLink="true">https://www.eclipse.org/ditto/2019-04-29-milestone-announcement-090-M2.html</guid> |
| |
| <category>blog</category> |
| |
| |
| </item> |
| |
| <item> |
| <title>Announcing Ditto Milestone 0.9.0-M1</title> |
| <description><p>Today the Ditto team is happy to announce the first milestone of the upcoming release |
| <a href="https://projects.eclipse.org/projects/iot.ditto/releases/0.9.0">0.9.0</a>.</p> |
| |
| <p>Have a look at the Milestone <a href="release_notes_090-M1.html">0.9.0-M1 release notes</a> for what changed in detail.</p> |
| |
| <p>The main changes and new features since the last release <a href="release_notes_080.html">0.8.0</a> are</p> |
| |
| <ul> |
| <li>memory optimizations when working with millions of digital twins</li> |
| <li>enhance connectivity to also be able to establish connections to Apache Kafka</li> |
| <li>providing more detailed metrics for connections</li> |
| <li>cluster bootstrapping stability improvements</li> |
| </ul> |
| |
| <h2 id="artifacts">Artifacts</h2> |
| |
| <p>The new Java artifacts have been published at the <a href="https://repo.eclipse.org/content/repositories/ditto/">Eclipse Maven repository</a> |
| as well as <a href="https://repo1.maven.org/maven2/org/eclipse/ditto/">Maven central</a>.</p> |
| |
| <p>The Docker images have been pushed to Docker Hub:</p> |
| <ul> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-policies/">eclipse/ditto-policies</a></li> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-things/">eclipse/ditto-things</a></li> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-things-search/">eclipse/ditto-things-search</a></li> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-gateway/">eclipse/ditto-gateway</a></li> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-connectivity/">eclipse/ditto-connectivity</a></li> |
| <li><a href="https://hub.docker.com/r/eclipse/ditto-concierge/">eclipse/ditto-concierge</a></li> |
| </ul> |
| |
| <p><br /> |
| <br /></p> |
| <figure><img class="docimage" src="images/ditto.svg" alt="Ditto" style="max-width: 500px" /></figure> |
| |
| <p>–<br /> |
| The Eclipse Ditto team</p> |
| </description> |
| <pubDate>Thu, 28 Mar 2019 00:00:00 +0000</pubDate> |
| <link>https://www.eclipse.org/ditto/2019-03-28-milestone-announcement-090-M1.html</link> |
| <guid isPermaLink="true">https://www.eclipse.org/ditto/2019-03-28-milestone-announcement-090-M1.html</guid> |
| |
| <category>blog</category> |
| |
| |
| </item> |
| |
| <item> |
| <title>Connectivity to Apache Kafka in Eclipse Ditto</title> |
| <description><p>Today we added connectivity to <a href="https://kafka.apache.org/">Apache Kafka</a>. In a first step, it is possible to publish |
| <em>twin events</em>, <em>messages</em>, <em>live commands and events</em> to Kafka topics.</p> |
| |
| <p>Since the last addition to Ditto’s connectivity which <a href="2018-10-16-example-mqtt-bidirectional.html">added MQTT connectivity</a>, |
| the connectivity feature got a lot of stabilization and new smaller features, e.g. the recent addition of |
| <a href="basic-placeholders.html#function-expressions">placeholder functions</a>.</p> |
| |
| <p>Returning to the Kafka integration Ditto can now, for example, whenever a <a href="intro-digitaltwins.html">digital twin</a> is |
| changed (e.g. a device updated some state data), publish a <em>twin event</em> to a Kafka topic.</p> |
| |
| <p>If you already rely on Apache Kafka as a source for your data lake or analytics, integrating Ditto and its digital twins |
| is now super easy.</p> |
| |
| <p>Find out more at our <a href="connectivity-protocol-bindings-kafka2.html">Kafka documentation</a>.</p> |
| |
| <figure><img class="docimage" src="images/ditto.svg" alt="Ditto" style="max-width: 500px" /></figure> |
| |
| <p>–<br /> |
| The Eclipse Ditto team</p> |
| </description> |
| <pubDate>Wed, 13 Mar 2019 00:00:00 +0000</pubDate> |
| <link>https://www.eclipse.org/ditto/2019-03-13-kafka-connectivity.html</link> |
| <guid isPermaLink="true">https://www.eclipse.org/ditto/2019-03-13-kafka-connectivity.html</guid> |
| |
| <category>blog</category> |
| |
| |
| </item> |
| |
| </channel> |
| </rss> |