<html><head>
      <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
   <title>Quality of Service Filter</title><link rel="stylesheet" type="text/css" href="css/docbook.css"><meta name="generator" content="DocBook XSL Stylesheets V1.79.1"><meta name="keywords" content="jetty, servlet, servlet-api, cometd, http, websocket, eclipse, maven, java, server, software"><link rel="home" href="index.html" title="Jetty"><link rel="up" href="advanced-extras.html" title="Chapter&nbsp;18.&nbsp;Provided Servlets, Filters, and Handlers"><link rel="prev" href="cgi-servlet.html" title="CGI Servlet"><link rel="next" href="dos-filter.html" title="Denial of Service Filter"><link xmlns:jfetch="java:org.eclipse.jetty.xslt.tools.JavaSourceFetchExtension" xmlns:fetch="java:org.eclipse.jetty.xslt.tools.SourceFetchExtension" xmlns:d="http://docbook.org/ns/docbook" xmlns:l="http://docbook.sourceforge.net/xmlns/l10n/1.0" xmlns:xslthl="http://xslthl.sf.net" xmlns:gcse="http://www.google.com" xmlns:date="http://exslt.org/dates-and-times" rel="shortcut icon" href="images/favicon.ico"><link rel="stylesheet" href="css/highlighter/foundation.css"><script src="js/highlight.pack.js"></script><script>
      hljs.initHighlightingOnLoad();
    </script><link type="text/css" rel="stylesheet" href="css/font-awesome/font-awesome.min.css"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><table xmlns:jfetch="java:org.eclipse.jetty.xslt.tools.JavaSourceFetchExtension" xmlns:fetch="java:org.eclipse.jetty.xslt.tools.SourceFetchExtension" xmlns:d="http://docbook.org/ns/docbook" xmlns:l="http://docbook.sourceforge.net/xmlns/l10n/1.0" xmlns:xslthl="http://xslthl.sf.net" xmlns:gcse="http://www.google.com" xmlns:date="http://exslt.org/dates-and-times"><tr><td style="width: 25%"><a href="http://www.eclipse.org/jetty"><img src="images/jetty-header-logo.png" alt="Jetty Logo"></a><br><span style="font-size: small">
            Version: 9.4.28-SNAPSHOT</span></td><td style="width: 50%"></td></tr></table><div xmlns:jfetch="java:org.eclipse.jetty.xslt.tools.JavaSourceFetchExtension" xmlns:fetch="java:org.eclipse.jetty.xslt.tools.SourceFetchExtension" xmlns:d="http://docbook.org/ns/docbook" xmlns:l="http://docbook.sourceforge.net/xmlns/l10n/1.0" xmlns:xslthl="http://xslthl.sf.net" xmlns:gcse="http://www.google.com" xmlns:date="http://exslt.org/dates-and-times" class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">Quality of Service Filter</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="cgi-servlet.html"><i class="fa fa-chevron-left" aria-hidden="true"></i> Previous</a>&nbsp;</td><th width="60%" align="center">Chapter&nbsp;18.&nbsp;Provided Servlets, Filters, and Handlers<br><a accesskey="p" href="index.html"><i class="fa fa-home" aria-hidden="true"></i> Home</a></th><td width="20%" align="right">&nbsp;<a accesskey="n" href="dos-filter.html">Next <i class="fa fa-chevron-right" aria-hidden="true"></i></a></td></tr></table><hr></div><div xmlns:jfetch="java:org.eclipse.jetty.xslt.tools.JavaSourceFetchExtension" xmlns:fetch="java:org.eclipse.jetty.xslt.tools.SourceFetchExtension" xmlns:d="http://docbook.org/ns/docbook" xmlns:l="http://docbook.sourceforge.net/xmlns/l10n/1.0" xmlns:xslthl="http://xslthl.sf.net" xmlns:gcse="http://www.google.com" xmlns:date="http://exslt.org/dates-and-times" class="jetty-callout"><h5 class="callout"><a href="http://www.webtide.com/">Contact the core Jetty developers at
          <span class="website">www.webtide.com</span></a></h5><p>
 private support for your internal/customer projects ... custom extensions and distributions ... versioned snapshots for indefinite support ...
 scalability guidance for your apps and Ajax/Comet projects ... development services for sponsored feature development
      </p></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="qos-filter"></a>Quality of Service Filter</h2></div></div></div><div class="toc"><dl class="toc"><dt><span class="section"><a href="qos-filter.html#qos-filter-metadata">Info</a></span></dt><dt><span class="section"><a href="qos-filter.html#qos-filter-usage">Usage</a></span></dt><dt><span class="section"><a href="qos-filter.html#qos-understanding">Understanding the Problem</a></span></dt><dt><span class="section"><a href="qos-filter.html#qos-applying">Applying the QoSFilter</a></span></dt></dl></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="qos-filter-metadata"></a>Info</h3></div></div></div><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem">Classname: <code class="literal">org.eclipse.jetty.servlets.QoSFilter</code></li><li class="listitem">Maven Artifact: org.eclipse.jetty:jetty-servlets</li><li class="listitem">Javadoc: <a class="link" href="http://www.eclipse.org/jetty/javadoc/9.4.28-SNAPSHOT/org/eclipse/jetty/servlets/QoSFilter.html" target="_top">http://www.eclipse.org/jetty/javadoc/9.4.28-SNAPSHOT/org/eclipse/jetty/servlets/QoSFilter.html</a></li></ul></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="qos-filter-usage"></a>Usage</h3></div></div></div><p>Jetty supports Continuations, which allow non-blocking handling of HTTP requests, so that threads can be allocated in a managed way to provide application specific Quality of Service (QoS).
The <code class="literal">QoSFilter</code> is a utility servlet filter that implements some QoS features.</p></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="qos-understanding"></a>Understanding the Problem</h3></div></div></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a name="_waiting_for_resources"></a>Waiting for Resources</h4></div></div></div><p>Web applications frequently use JDBC Connection pools to limit the simultaneous load on the database.
This protects the database from peak loads, but makes the web application vulnerable to thread starvation.
Consider a thread pool with 20 connections, being used by a web application that that typically receives 200 requests per second and each request holds a JDBC connection for 50ms.
Such a pool can service on average 200*20*1000/50 = 400 requests per second.</p><p>However, if the request rate rises above 400 per second, or if the database slows down (due to a large query) or becomes momentarily unavailable, the thread pool can very quickly accumulate many waiting requests.
If, for example, the website is "slashdotted" or experiences some other temporary burst of traffic and the request rate rises from 400 to 500 requests per second, then 100 requests per second join those waiting for a JDBC connection.
Typically, a web server&#8217;s thread pool contains only a few hundred threads, so a burst or slow DB need only persist for a few seconds to consume the entire web server&#8217;s thread pool; this is called thread starvation.
The key issue with thread starvation is that it effects the entire web application, and potentially the entire web server.
Even if the requests using the database are only a small proportion of the total requests on the web server, all requests are blocked because all the available threads are waiting on the JDBC connection pool.
This represents non-graceful degradation under load and provides a very poor quality of service.</p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a name="_prioritizing_resources"></a>Prioritizing Resources</h4></div></div></div><p>Consider a web application that is under extreme load.
This load might be due to a popularity spike (slashdot), usage burst (Christmas or close of business), or even a denial of service attack.
During such periods of load, it is often desirable not to treat all requests as equals, and to give priority to high value customers or administrative users.</p><p>The typical behavior of a web server under extreme load is to use all its threads to service requests and to build up a backlog of unserviced requests.
If the backlog grows deep enough, then requests start to timeout and users experience failures as well as delays.</p><p>Ideally, the web application should be able to examine the requests in the backlog, and give priority to high value customers and administrative users.
But with the standard blocking servlet API, it is not possible to examine a request without allocating a thread to that request for the duration of its handling.
There is no way to delay the handling of low priority requests, so if the resources are to be reallocated, then the low priority requests must all be failed.</p></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="qos-applying"></a>Applying the QoSFilter</h3></div></div></div><p>The Quality of Service Filter (QoSFilter) uses Continuations to avoid thread starvation, prioritize requests and give graceful degradation under load, to provide a high quality of service.
When you apply the filter to specific URLs within a web application, it limits the number of active requests being handled for those URLs.
Any requests in excess of the limit are suspended. When a request completes handling the limited URL, one of the waiting requests resumes and can be handled.
You can assign priorities to each suspended request, so that high priority requests resume before lower priority requests.</p><div class="section"><div class="titlepage"><div><div><h4 class="title"><a name="_required_jars"></a>Required JARs</h4></div></div></div><p>To use the QoS Filter, these JAR files must be available in <code class="literal">WEB-INF/lib</code>:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem">$JETTY_HOME/lib/jetty-util.jar</li><li class="listitem">$JETTY_HOME/lib/jetty-servlets.jar &#8211; contains QoSFilter</li></ul></div></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a name="_sample_configuration"></a>Sample Configuration</h4></div></div></div><p>Place the configuration in a webapp&#8217;s <code class="literal">web.xml</code> or <code class="literal">jetty-web.xml</code>.
The default configuration processes ten requests at a time, servicing more important requests first and queuing up the rest.
This example processes fifty requests at a time:</p><pre xmlns:jfetch="java:org.eclipse.jetty.xslt.tools.JavaSourceFetchExtension" xmlns:fetch="java:org.eclipse.jetty.xslt.tools.SourceFetchExtension" xmlns:d="http://docbook.org/ns/docbook" xmlns:l="http://docbook.sourceforge.net/xmlns/l10n/1.0" xmlns:xslthl="http://xslthl.sf.net" xmlns:gcse="http://www.google.com" xmlns:date="http://exslt.org/dates-and-times"><code>&lt;filter&gt;
   &lt;filter-name&gt;QoSFilter&lt;/filter-name&gt;
   &lt;filter-class&gt;org.eclipse.jetty.servlets.QoSFilter&lt;/filter-class&gt;
   &lt;init-param&gt;
     &lt;param-name&gt;maxRequests&lt;/param-name&gt;
     &lt;param-value&gt;50&lt;/param-value&gt;
   &lt;/init-param&gt;
 &lt;/filter&gt;</code></pre></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a name="qos-filter-init"></a>Configuring QoS Filter Parameters</h4></div></div></div><p>A semaphore polices the <code class="literal">maxRequests</code> limit.
The filter waits a short time while attempting to acquire the semaphore.
The <code class="literal">waitMs</code> init parameter controls the wait, avoiding the expense of a suspend if the semaphore is shortly available.
If the semaphore cannot be obtained, Jetty suspends the request for the default suspend period of the container or the value set as the <code class="literal">suspendMs</code> init parameter.</p><p>The QoS filter uses the following init parameters:</p><div class="variablelist"><dl class="variablelist"><dt><span class="term">maxRequests</span></dt><dd>The maximum number of requests to be serviced at a time. The default is 10.</dd><dt><span class="term">maxPriority</span></dt><dd>The maximum valid priority that can be assigned to a request.
A request with a high priority value is more important than a request with a low priority value. The default is 10.</dd><dt><span class="term">waitMs</span></dt><dd>The length of time, in milliseconds, to wait while trying to accept a new request.
Used when the maxRequests limit is reached.
Default is 50 ms.</dd><dt><span class="term">suspendMs</span></dt><dd>Length of time, in milliseconds, that the request will be suspended if it is not accepted immediately.
If not set, the container&#8217;s default suspend period applies. Default is -1 ms.</dd><dt><span class="term">managedAttr</span></dt><dd>If set to true, then this servlet is set as a <code class="literal">ServletContext</code> attribute with the filter name as the attribute name.
This allows a context external mechanism (for example, JMX via <code class="literal">ContextHandler.MANAGED_ATTRIBUTES</code>) to manage the configuration of the filter.</dd></dl></div></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a name="_mapping_to_urls"></a>Mapping to URLs</h4></div></div></div><p>You can use the <code class="literal">&lt;filter-mapping&gt;</code> syntax to map the QoSFilter to a servlet, either by using the servlet name, or by using a URL pattern.
In this example, a URL pattern applies the QoSFilter to every request within the web application context:</p><pre xmlns:jfetch="java:org.eclipse.jetty.xslt.tools.JavaSourceFetchExtension" xmlns:fetch="java:org.eclipse.jetty.xslt.tools.SourceFetchExtension" xmlns:d="http://docbook.org/ns/docbook" xmlns:l="http://docbook.sourceforge.net/xmlns/l10n/1.0" xmlns:xslthl="http://xslthl.sf.net" xmlns:gcse="http://www.google.com" xmlns:date="http://exslt.org/dates-and-times"><code>&lt;filter-mapping&gt;
   &lt;filter-name&gt;QoSFilter&lt;/filter-name&gt;
   &lt;url-pattern&gt;/*&lt;/url-pattern&gt;
 &lt;/filter-mapping&gt;</code></pre></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a name="_setting_the_request_priority"></a>Setting the Request Priority</h4></div></div></div><p>Requests with higher values have a higher priority.
The default request priorities assigned by the QoSFilter are:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem">2&#8201;&#8212;&#8201;For any authenticated request</li><li class="listitem">1&#8201;&#8212;&#8201;For any request with a non-new valid session</li><li class="listitem">0&#8201;&#8212;&#8201;For all other requests</li></ul></div><p>To customize the priority, subclass QoSFilter and then override the <code class="literal">getPriority(ServletRequest request)</code> method to return an appropriate priority for the request.
You can then use this subclass as your QoS filter.
Here&#8217;s an example:</p><pre xmlns:jfetch="java:org.eclipse.jetty.xslt.tools.JavaSourceFetchExtension" xmlns:fetch="java:org.eclipse.jetty.xslt.tools.SourceFetchExtension" xmlns:d="http://docbook.org/ns/docbook" xmlns:l="http://docbook.sourceforge.net/xmlns/l10n/1.0" xmlns:xslthl="http://xslthl.sf.net" xmlns:gcse="http://www.google.com" xmlns:date="http://exslt.org/dates-and-times"><code>public class ParsePriorityQoSFilter extends QoSFilter
 {
     protected int getPriority(ServletRequest request)
     {
         String p = ((HttpServletRequest)request).getParameter("priority");
         if (p!=null)
             return Integer.parseInt(p);
         return 0;
     }
 }</code></pre></div></div></div><script type="text/javascript">
      SyntaxHighlighter.all()
    </script><div class="navfooter"><hr><table width="100%" summary="Navigation footer"><tr><td width="40%" align="left"><a accesskey="p" href="cgi-servlet.html"><i class="fa fa-chevron-left" aria-hidden="true"></i> Previous</a>&nbsp;</td><td width="20%" align="center"><a accesskey="u" href="advanced-extras.html"><i class="fa fa-chevron-up" aria-hidden="true"></i> Top</a></td><td width="40%" align="right">&nbsp;<a accesskey="n" href="dos-filter.html">Next <i class="fa fa-chevron-right" aria-hidden="true"></i></a></td></tr><tr><td width="40%" align="left" valign="top">CGI Servlet&nbsp;</td><td width="20%" align="center"><a accesskey="h" href="index.html"><i class="fa fa-home" aria-hidden="true"></i> Home</a></td><td width="40%" align="right" valign="top">&nbsp;Denial of Service Filter</td></tr></table></div><p xmlns:jfetch="java:org.eclipse.jetty.xslt.tools.JavaSourceFetchExtension" xmlns:fetch="java:org.eclipse.jetty.xslt.tools.SourceFetchExtension" xmlns:d="http://docbook.org/ns/docbook" xmlns:l="http://docbook.sourceforge.net/xmlns/l10n/1.0" xmlns:xslthl="http://xslthl.sf.net" xmlns:gcse="http://www.google.com" xmlns:date="http://exslt.org/dates-and-times"><div class="jetty-callout">
            See an error or something missing?
            <span class="callout"><a href="http://github.com/eclipse/jetty.project">Contribute to this documentation at
                <span class="website"><i class="fa fa-github" aria-hidden="true"></i> Github!</span></a></span><span style="float: right"><i>(Generated: 2020-03-10)</i></span></div></p></body></html>