| <?xml version="1.0" encoding="UTF-8"?> |
| <!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" |
| "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd"> |
| <chapter id="serviceability"> |
| <title>Serviceability and Diagnostics</title> |
| |
| <para> |
| @project.name@ supports two kinds of logging: <emphasis>Event Logging</emphasis> and <emphasis>Trace logging</emphasis> which is usually referred |
| to simply as <emphasis>Logging</emphasis>. The difference between Event Logging and Logging is explained below, but both are configured in the |
| <literal>serviceability.xml</literal> file in the <literal>configuration</literal> directory. This file takes the form of a Logback configuration, @project.name@ |
| uses a Logback implementation behind the SLF4J logging interface. |
| <para> |
| For a description of the syntax and facilities provided by <literal>serviceability.xml</literal> |
| see the <emphasis>Logback</emphasis> documentation (referenced in <xref linkend="furtherreading"/>). |
| </para> |
| </para> |
| |
| <section id="serviceability-info-log"> |
| <title>Event Logging</title> |
| <para> |
| Event logging records important events in @project.name@. Each event is logged to |
| an event log file and is accompanied by a code enclosed in angle brackets. |
| An example is shown below: |
| <screen> |
| [2010-10-25 16:20:45.897] system-artifacts <TC0010I> Creating HTTP/1.1 connector with scheme http on port 8080. |
| </screen> |
| (For a description of the log code syntax, see <xref linkend="log-codes"/>.) |
| The format of event log messages is fully configurable. |
| </para> |
| <para> |
| By default, event log messages are stored in <literal>$SERVER_HOME/serviceability/eventlogs/eventlog.log</literal>. |
| </para> |
| <para> |
| The default behaviour is that, once <literal>eventlog.log</literal> reaches a 10Mb limit, it rolls into a series of files named |
| <literal>eventlog_</literal><emphasis>i</emphasis><literal>.log</literal> where <emphasis>i</emphasis> ranges from 1 to 4, and event logging continues in |
| a new <literal>eventlog.log</literal> file. |
| </para> |
| </section> |
| |
| <section id="serviceability-info-trace"> |
| <title>(Trace) Logging</title> |
| <para> |
| The @project.name@’s (trace) logging support serves two main purposes: |
| <itemizedlist> |
| <listitem> |
| It provides global trace files that capture high-volume information regarding the @project.name@’s internal events. |
| The files are intended for use by support personnel to diagnose runtime problems. |
| </listitem> |
| <listitem> |
| It provides application trace files that contain application-generated output. This includes output generated using popular logging and |
| tracing APIs including the OSGi LogService, as well as output generated by calls to <literal>System.out</literal> and <literal>System.err</literal>. |
| These files are intended for use by application developers and system administrators. An application is defined as a scope so a single bundle will |
| not get its own log file unless it is a Web application Bundle or is included in a scoped plan or a par file. |
| </listitem> |
| </itemizedlist> |
| </para> |
| <para> |
| By default, the @project.name@ trace file is called <literal>$SERVER_HOME/serviceability/logs/log.log</literal>, |
| and, again by default, the application trace files are called <literal>$SERVER_HOME/serviceability/logs/</literal><emphasis>application_name</emphasis> |
| <literal>/log.log</literal>, where <emphasis>application_name</emphasis> is automatically set by @project.name@ for each application artifact |
| installed and run (it is a combination of the artifact name and the version). |
| </para> |
| <para> |
| The default behaviour of these trace files is that, once <literal>log.log</literal> reaches a 10Mb limit, it rolls into a series of files named |
| <literal>log_</literal><emphasis>i</emphasis><literal>.log</literal> where <emphasis>i</emphasis> ranges from 1 to 4, and logging continues in |
| a new <literal>log.log</literal> file. |
| </para> |
| <para> |
| Entries in trace files are by default of the form <timestamp> <thread-name> <source> <level> <entry-text>. For example: |
| <screen> |
| [2008-05-15 09:09:46.940] server-dm-2 org.apache.coyote.http11.Http11Protocol I Initializing Coyote HTTP/1.1 on http-48080 |
| </screen> |
| although this format is completely determined by the Logback configuration file <literal>serviceability.xml</literal>. |
| </para> |
| <section id="serviceability-info-trace-app"> |
| <title>Application Output</title> |
| <para> |
| @project.name@ provides advanced support for capturing and tracing application-generated output by automatically separating trace output on a |
| per-application basis and will also capture any <literal>System.out</literal> and <literal>System.err</literal> output. |
| </para> |
| <section id="per-application-trace"> |
| <title>Per-application trace</title> |
| <para> |
| @project.name@ uses SLF4J interfaces to Logback, and the root logger (by default) captures all logging output |
| and appends it to the application-specific trace files as described above. |
| To modify this, define application-specific loggers in the <literal>serviceability.xml</literal> file in the normal way. |
| </para> |
| </section> |
| <section id="sysout-and-syserr"> |
| <title>System.out and System.err</title> |
| <para> |
| <literal>System.out</literal> and <literal>System.err</literal> output from applications is, by default, captured in the |
| application’s trace file. |
| This happens because the output streams are intercepted and written to the loggers named |
| <literal>System.out</literal> and <literal>System.err</literal> respectively. |
| Since there are no explicit loggers defined with these names in the <literal>serviceability.xml</literal> file, |
| this output is logged by the root logger (which captures <literal>INFO</literal> level and above). |
| </para> |
| <para> |
| The capture of <literal>System.out</literal> and <literal>System.err</literal> output is configured in the |
| <literal>configuration/org.eclipse.virgo.medic.properties</literal> file by the <literal>log.wrapSysOut</literal> and |
| <literal>log.wrapSysErr</literal> properties. By default the properties have a value of <literal>true</literal> |
| and capture is enabled. Capture can be disabled by configuring the properties with a value of <literal>false</literal>. |
| The third and last accepted value is <literal>tee</literal> which captures the System streams output in the logs |
| while at the same time allows printing output to the System streams. Thus this output will appear both in the logs |
| and in the System stream output - an example could be the Equinox console, launched with <literal>-console</literal> startup argument. |
| </para> |
| <important> |
| If you provide value different than 'true | tee | false' then the server will default to 'tee' and print out a warning. |
| </important> |
| <para> |
| The trace entries for <literal>System.out</literal> and <literal>System.err</literal> |
| output are of the form: |
| <screen> |
| [2008-05-16 09:28:45.874] server-tomcat-thread-1 System.out Hello world! |
| [2008-05-16 09:28:45.874] server-tomcat-thread-1 System.err Hello world! |
| </screen> |
| The third column indicates where the output came from (<literal>System.out</literal> or <literal>System.err</literal>). |
| </para> |
| <para> |
| To over-ride this behaviour, simply define explicit loggers named <literal>System.out</literal> |
| and/or <literal>System.err</literal> in the configuration file to send this output to an appender of your choice. |
| Be aware that all applications’ output streams will be caught by these loggers, and that a sifting appender might be useful to separate them. |
| </para> |
| </section> |
| <section id="janino"> |
| <title>Janino</title> |
| <para> |
| Janino can be used to define trace filters as Java expressions. This adds a significant overhead to tracing, so should be used with care. |
| </para> |
| <para> |
| For example, the addition of the following filter element to the sifting appender in <literal>serviceability.xml</literal> |
| suppresses per-application trace output that is not associated with a particular application and is normally written to |
| <literal>serviceability/logs/virgo-kernel/log.log</literal>. |
| <programlisting language="xml"><![CDATA[<appender name="SIFTED_LOG_FILE" class="ch.qos.logback.classic.sift.SiftingAppender"> |
| <discriminator> |
| <Key>applicationName</Key> |
| <DefaultValue>virgo-kernel</DefaultValue> |
| </discriminator> |
| <sift> |
| <appender name="${applicationName}_LOG_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender"> |
| <filter class="ch.qos.logback.core.filter.EvaluatorFilter"> |
| <evaluator class="ch.qos.logback.classic.boolex.JaninoEventEvaluator"> |
| <expression> |
| (mdc == null) || (mdc.get("applicationName") == null) |
| </expression> |
| </evaluator> |
| <OnMismatch>NEUTRAL</OnMismatch> |
| <OnMatch>DENY</OnMatch> |
| </filter> |
| <file>serviceability/logs/${applicationName}/log.log</file> |
| ... |
| </appender> |
| </sift> |
| </appender>]]></programlisting> |
| </para> |
| <para> |
| To enable Janino in @project.name@, place the Janino and commons compiler JARs, converted to OSGi bundles, in <literal>plugins</literal>. |
| For example these bundles are available at v2.6.1 from the SpringSource Enterprise Bundle Repository. |
| Then add the following lines to |
| <literal>configuration/org.eclipse.equinox.simpleconfigurator/bundles.info</literal> |
| (as described in <link linkend="configuring-framework-bundles">Configuring OSGi Framework Bundles</link>): |
| <programlisting><![CDATA[com.springsource.org.codehaus.janino,2.6.1,plugins/com.springsource.org.codehaus.janino-2.6.1.jar,4,false |
| com.springsource.org.codehaus.commons.compiler,2.6.1,plugins/com.springsource.org.codehaus.commons.compiler-2.6.1.jar,4,false]]></programlisting> |
| </para> |
| <important> |
| Current versions of Logback, including 0.9.28 to 1.0, do not set Janino's "parent" class loader correctly. |
| This bug is covered by the Logback issue <ulink url="http://jira.qos.ch/browse/LBCORE-244">LBCORE-244</ulink>. |
| With such versions, it is necessary to attach a fragment bundle to Janino. Place the fragment bundle in <literal>plugins</literal> and list it in |
| <literal>configuration/org.eclipse.equinox.simpleconfigurator/bundles.info</literal>. |
| The fragment's contents are described in <ulink url="https://bugs.eclipse.org/bugs/show_bug.cgi?id=333920#c15">bug 333920</ulink>. |
| </important> |
| </section> |
| </section> |
| </section> |
| <section id="serviceability-info-dump"> |
| <title>Service Dumps</title> |
| <para> |
| A service dump is triggered when one of the following events |
| occurs: |
| </para> |
| <orderedlist numeration="arabic"> |
| <listitem> |
| <para> |
| A failure is detected in the @project.name@ code, or |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| a thread deadlock is detected. |
| </para> |
| </listitem> |
| </orderedlist> |
| <para> |
| A service dump contains a snapshot of all the important state from |
| the running @project.name@ instance. This snapshot is not intended |
| for end user consumption but is useful for service personnel. |
| </para> |
| <para> |
| By default, service dumps are created in |
| <literal>$SERVER_HOME/serviceability/dump</literal>. |
| </para> |
| </section> |
| </chapter> |