blob: 06f00e1c266519a699512915994e1b433f1ecdf0 [file] [log] [blame]
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content="">
<meta name="keywords" content="blog, ">
<title> Weak acknowledgments to decouple signal publishers and subscribers </title>
<link rel="stylesheet" href="css/syntax.css">
<link rel="stylesheet" type="text/css" href="//cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css" crossorigin="anonymous">
<link rel="stylesheet" href="css/modern-business.css">
<link rel="stylesheet" href="//cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/3.3.7/css/bootstrap.min.css" crossorigin="anonymous">
<link rel="stylesheet" href="css/customstyles.css">
<link rel="stylesheet" href="css/boxshadowproperties.css">
<link rel="stylesheet" href="css/theme-ditto.css">
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:400,700|Source+Code+Pro:300,600|Titillium+Web:400,600,700">
<script src="//cdnjs.cloudflare.com/ajax/libs/jquery/2.1.4/jquery.min.js" crossorigin="anonymous"></script>
<script src="//cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/3.3.7/js/bootstrap.min.js" crossorigin="anonymous"></script>
<script src="//cdnjs.cloudflare.com/ajax/libs/anchor-js/2.0.0/anchor.min.js" crossorigin="anonymous"></script>
<script src="js/toc.js"></script>
<script src="js/customscripts.js"></script>
<script type="application/ld+json">
{
"@context": "http://schema.org",
"@type": "Organization",
"url": "https://eclipse.org/ditto/",
"logo": "https://eclipse.org/ditto/images/ditto.svg"
}
</script>
<link rel="icon" type="image/png" href="images/favicon-16x16.png" sizes="16x16">
<link rel="icon" type="image/png" href="images/favicon-32x32.png" sizes="32x32">
<link rel="icon" type="image/png" href="images/favicon-96x96.png" sizes="96x96">
<link rel="alternate" type="application/rss+xml" title="Eclipse Ditto Blog" href="https://www.eclipse.org/ditto/feed.xml">
<!-- Eclipse Foundation cookie consent: -->
<link rel="stylesheet" type="text/css" href="//www.eclipse.org/eclipse.org-common/themes/solstice/public/stylesheets/vendor/cookieconsent/cookieconsent.min.css" />
<script src="//www.eclipse.org/eclipse.org-common/themes/solstice/public/javascript/vendor/cookieconsent/default.min.js"></script>
<script>
$(document).ready(function() {
$("#tg-sb-link").click(function() {
$("#tg-sb-sidebar").toggle();
$("#tg-sb-content").toggleClass('col-md-9');
$("#tg-sb-content").toggleClass('col-md-12');
$("#tg-sb-icon").toggleClass('fa-toggle-on');
$("#tg-sb-icon").toggleClass('fa-toggle-off');
});
});
</script>
</head>
<script>
(function(w,d,s,l,i){
w[l]=w[l]||[];
w[l].push({'gtm.start':
new Date().getTime(),event:'gtm.js'});
var f=d.getElementsByTagName(s)[0],
j=d.createElement(s),
dl=l!='dataLayer'?'&l='+l:'';
j.async=true;
j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;
f.parentNode.insertBefore(j,f);
})(window,document,'script','dataLayer','GTM-5WLCZXC');
</script>
<body>
<!-- Navigation -->
<nav class="navbar navbar-inverse navbar-fixed-top">
<div class="container topnavlinks">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1">
<span class="sr-only">Toggle navigation</span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-ditto-home" href="index.html">&nbsp;<img src="images/ditto_allwhite_symbolonly.svg" class="ditto-navbar-symbol" alt="Home"> <img src="images/ditto_allwhite_textonly.svg" class="ditto-navbar-symbol-text" alt="Ditto"></a>
</div>
<div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1">
<ul class="nav navbar-nav navbar-right">
<!-- toggle sidebar button -->
<!--<li><a id="tg-sb-link" href="#"><i id="tg-sb-icon" class="fa fa-toggle-on"></i> Nav</a></li>-->
<!-- entries without drop-downs appear here -->
<li><a href="blog.html">Blog</a></li>
<li><a href="intro-overview.html">Documentation</a></li>
<li><a href="http-api-doc.html">HTTP API</a></li>
<li><a href="sandbox.html">Sandbox</a></li>
<li><a href="https://github.com/eclipse/ditto" target="_blank">GitHub</a></li>
<li><a href="https://github.com/eclipse/ditto-examples" target="_blank">GitHub examples</a></li>
<!-- entries with drop-downs appear here -->
<!-- conditional logic to control which topnav appears for the audience defined in the configuration file.-->
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Links<b class="caret"></b></a>
<ul class="dropdown-menu">
<li><a href="https://projects.eclipse.org/projects/iot.ditto" target="_blank">Eclipse Ditto Project</a></li>
<li><a href="https://www.eclipse.org/forums/index.php/f/364/" target="_blank">Forum</a></li>
<li><a href="https://ci.eclipse.org/ditto/" target="_blank">Jenkins</a></li>
<li><a href="https://dev.eclipse.org/mhonarc/lists/ditto-dev/" target="_blank">Mailing list archives</a></li>
<li><a href="https://gitter.im/eclipse/ditto" target="_blank">Gitter.im chat</a></li>
</ul>
</li>
<!--comment out this block if you want to hide search-->
<li>
<!--start search-->
<div id="search-demo-container">
<input type="text" id="search-input" placeholder="search...">
<ul id="results-container"></ul>
</div>
<script src="//cdnjs.cloudflare.com/ajax/libs/simple-jekyll-search/0.0.9/jekyll-search.js" type="text/javascript"></script>
<script type="text/javascript">
SimpleJekyllSearch.init({
searchInput: document.getElementById('search-input'),
resultsContainer: document.getElementById('results-container'),
dataSource: 'search.json',
searchResultTemplate: '<li><a href="{url}" title="Weak acknowledgments to decouple signal publishers and subscribers">{title}</a></li>',
noResultsText: 'No results found.',
limit: 10,
fuzzy: true,
})
</script>
<!--end search-->
</li>
</ul>
</div>
</div>
<!-- /.container -->
</nav>
<!-- Page Content -->
<div class="container">
<div id="main">
<!-- Content Row -->
<div class="row">
<!-- Content Column -->
<div class="col-md-12" id="tg-sb-content">
<!-- Look the author details up from the site config. -->
<!-- Output author details if some exist. -->
<!-- Output author details if some exist. -->
<!---->
<!--<span>-->
<!--&lt;!&ndash; Mugshot. &ndash;&gt;-->
<!--<img src="https://www.gravatar.com/avatar/d923944b6a8c76e411ff838fb19acfa7?s=135" alt="A photo of Yufei Cai" />-->
<!--&lt;!&ndash; Personal Info. &ndash;&gt;-->
<!--Written by <a href="https://github.com/yufei-cai" target="_blank">Yufei Cai</a>-->
<!--</span>-->
<!---->
<article class="post" itemscope itemtype="http://schema.org/BlogPosting">
<header class="post-header">
<h1 class="post-title" itemprop="name headline">Weak acknowledgments to decouple signal publishers and subscribers</h1>
<p class="post-meta">Published by <img src="https://www.gravatar.com/avatar/d923944b6a8c76e411ff838fb19acfa7?s=135" alt="A photo of Yufei Cai" style="width:50px;border-radius:50%;display:inline-block;margin-right:5px;" /><span itemprop="author" itemscope itemtype="http://schema.org/Person"><span itemprop="name"><a href="https://github.com/yufei-cai" target="_blank">Yufei Cai</a> </span></span> on <time datetime="2020-11-16T00:00:00+00:00" itemprop="datePublished">Nov 16, 2020</time> - Tags:
<a href="tag_blog.html">blog</a>
</p>
</header>
<div class="post-content" itemprop="articleBody">
<!-- this handles the automatic toc. use ## for subheads to auto-generate the on-page minitoc. if you use html tags, you must supply an ID for the heading element in order for it to appear in the minitoc. -->
<script>
$( document ).ready(function() {
// Handler for .ready() called.
$('#toc').toc({ minimumHeaders: 0, listType: 'ul', showSpeed: 0, headers: 'h2,h3,h4' });
/* this offset helps account for the space taken up by the floating toolbar. */
$('#toc').on('click', 'a', function() {
var target = $(this.getAttribute('href'))
, scroll_target = target.offset().top
$(window).scrollTop(scroll_target - 10);
return false
})
});
</script>
<div id="toc"></div>
<h2 id="motivation">Motivation</h2>
<p><a href="2020-08-31-release-announcement-120.html">Ditto 1.2.0</a> introduced at-least-once delivery via
<a href="basic-acknowledgements.html">acknowledgement requests</a>.<br />
It increased coupling between the publisher and the subscriber of signals in that the subscriber is no longer at the
liberty to filter for signals it is interested in. Instead, the subscriber must consume all signals in order to
fulfill acknowledgement requests and prevent endless redelivery.</p>
<p>To combat the problem,
<a href="2020-10-28-release-announcement-140.html">Ditto 1.4.0</a> made acknowledgement labels unique and introduced the requirement
to manage <a href="basic-acknowledgements.html#issuing-acknowledgements"><em>declared acknowledgements</em></a>, identifying of each
subscriber.<br />
It is now possible for Ditto to issue
<a href="basic-acknowledgements.html#weak-acknowledgements-wacks"><em>weak acknowledgements</em></a> on behalf of the subscriber
whenever it decides to not consume a signal. That allows subscribers to configure RQL and namespace filters freely
without causing any futile redelivery.</p>
<div class="alert alert-info" role="alert"><i class="fa fa-info-circle"></i> <b>Note:</b> Weak acknowledgements are available since Ditto 1.5.0.</div>
<h2 id="what-it-is">What it is</h2>
<p>A <a href="basic-acknowledgements.html#weak-acknowledgements-wacks"><em>weak acknowledgement</em></a> is issued by Ditto for any
<a href="basic-acknowledgements.html#requesting-acks">acknowledgement request</a> that will not be fulfilled now or ever without
configuration change.<br />
A weak acknowledgement is identified by the header <code class="highlighter-rouge">ditto-weak-ack: true</code>.</p>
<p>The status code of weak acknowledgements is <code class="highlighter-rouge">200 OK</code>; it signifies that any redelivery is not to be made on their
account.</p>
<p>A weak acknowledgement may look like this in Ditto protocol:</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w">
</span><span class="s2">"topic"</span><span class="p">:</span><span class="w"> </span><span class="s2">"com.acme/xdk_53/things/twin/acks/my-mqtt-connection:my-mqtt-topic"</span><span class="p">,</span><span class="w">
</span><span class="s2">"headers"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="s2">"ditto-weak-ack"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="s2">"path"</span><span class="p">:</span><span class="w"> </span><span class="s2">"/"</span><span class="p">,</span><span class="w">
</span><span class="s2">"value"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Acknowledgement was issued automatically, because the subscriber is not authorized to receive the signal."</span><span class="p">,</span><span class="w">
</span><span class="s2">"status"</span><span class="p">:</span><span class="w"> </span><span class="mi">200</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<h2 id="how-it-works">How it works</h2>
<p>Since Ditto 1.4.0, subscribers of <em>twin events</em> or <em>live signals</em> are required to declare unique acknowledgement labels
they are allowed to send. The labels of acknowledgement requests are then identifying the intended subscribers.<br />
If the intended subscriber exists but does not receive the signal for non-transient reasons, Ditto issues
a weak acknowledgement for that subscriber.<br />
Such reasons may be:</p>
<ul>
<li>The intended subscriber <strong>is not authorized</strong> to receive the signal by policy;</li>
<li>The intended subscriber did not subscribe for the signal type (<em>twin event, live command, live event or live message</em>);</li>
<li>The intended subscriber filtered the signal out by its <a href="basic-changenotifications.html#filtering">namespace or RQL filter</a>;</li>
<li>The intended subscriber dropped the signal because its <a href="connectivity-mapping.html">payload mapper</a> produced nothing.</li>
</ul>
<h2 id="limitation">Limitation</h2>
<p>The distributed nature of cluster pub/sub means that weak acknowledgements are not always issued correctly.<br />
They are only <em>eventually correct</em> in the sense that some time after a change to the publisher-subscriber pair,
the issued weak acknowledgements will reflect the change.<br />
Such changes include:</p>
<ul>
<li>Opening and closing of Websocket or other connections acting as the subscriber;</li>
<li>Subscribing and unsubscribing for different signal types via Websocket;</li>
<li>Modification of connections via the <a href="connectivity-manage-connections.html">connectivity API</a>;</li>
<li>Migration of a connection from one Ditto cluster member to another due to load balancing.</li>
</ul>
<h2 id="feedback">Feedback?</h2>
<p>Please <a href="feedback.html">get in touch</a> if you have feedback or questions towards this new concept of weak
acknowledgements.</p>
<p><br />
<br /></p>
<figure><img class="docimage" src="images/ditto.svg" alt="Ditto" style="max-width: 500px" /></figure>
<p><br />
The Eclipse Ditto team</p>
</div>
</article>
<hr class="shaded"/>
<footer>
<div class="row">
<div class="col-lg-12 footer">
<div class="logo">
<a href="https://eclipse.org"><img src="images/eclipse_foundation_logo.svg" alt="Eclipse logo"/></a>
</div>
<p class="notice">
&copy;2021 Eclipse Ditto.
Site last generated: Apr 7, 2021 <br />
</p>
<div class="quickLinks">
<a href="https://www.eclipse.org/legal/privacy.php" target="_blank">
&gt; Privacy Policy
</a>
<a href="https://www.eclipse.org/legal/termsofuse.php" target="_blank">
&gt; Terms of Use
</a>
<a href="https://www.eclipse.org/legal/copyright.php" target="_blank">
&gt; Copyright Agent
</a>
<a href="https://www.eclipse.org/legal" target="_blank">
&gt; Legal
</a>
<a href="https://www.eclipse.org/legal/epl-2.0/" target="_blank">
&gt; License
</a>
<a href="https://eclipse.org/security" target="_blank">
&gt; Report a Vulnerability
</a>
</div>
</div>
</div>
</footer>
</div>
<!-- /.row -->
</div>
<!-- /.container -->
</div>
<!-- /#main -->
</div>
</body>
</html>