317 lines
16 KiB
HTML
317 lines
16 KiB
HTML
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
|
|
<html>
|
|
<head>
|
|
<title>E-MailRelay Developer Guide</title>
|
|
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
|
|
<link rel="stylesheet" href="emailrelay.css" type="text/css">
|
|
</head>
|
|
<body>
|
|
<!-- index:0::::E-MailRelay Developer Guide -->
|
|
<div class="div-main">
|
|
<h1><a class="a-header" name="H_1">E-MailRelay Developer Guide</a></h1> <!-- index:1:H:1::E-MailRelay Developer Guide -->
|
|
<h2><a class="a-header" name="SH_1_1">Principles</a></h2> <!-- index:2:SH:1:1:Principles -->
|
|
<p>
|
|
The main principles in the design of E-MailRelay can be summarised as:
|
|
</p>
|
|
<ul>
|
|
<li>Minimal third-party dependencies</li>
|
|
<li>Windows/Unix portability without #ifdefs</li>
|
|
<li>Event-driven, non-blocking, single-threaded networking code</li>
|
|
<li>Functionality without imposing policy</li>
|
|
</ul>
|
|
<h2><a class="a-header" name="SH_1_2">Dependencies</a></h2> <!-- index:2:SH:1:2:Dependencies -->
|
|
<p>
|
|
E-MailRelay started life at a time when Linux had no decent package manager and
|
|
Windows was in the grip of DLL hell. As a result, a key principle is that it
|
|
has no dependencies other than a decent C++ runtime. Since that time OpenSSL
|
|
has been introduced as a dependency to support TLS encryption, and the optional
|
|
configuration and installation GUI has been developed using the Qt toolkit.
|
|
</p>
|
|
|
|
<p>
|
|
In those early years multi-threading support in C++ libraries was poor, so up
|
|
until version 2.0 the code was single-threaded throughout.
|
|
</p>
|
|
<h2><a class="a-header" name="SH_1_3">Portability</a></h2> <!-- index:2:SH:1:3:Portability -->
|
|
<p>
|
|
The E-MailRelay code is mostly written in C++-1998, but using some features of
|
|
C++-2011. A C++-1998 compiler can be used, but multi-threading will be disabled.
|
|
</p>
|
|
|
|
<p>
|
|
The header files <em class="quote">gdef.h</em> in <em class="quote">src/glib</em> is used to fix up some compiler
|
|
portability issues such as missing standard types, non-standard system headers
|
|
etc. Conditional compilation directives (<em class="quote">#ifdef</em> etc.) are largely confined
|
|
this file in order to improve readability.
|
|
</p>
|
|
|
|
<p>
|
|
Windows/Unix portability is generally addressed by providing a common class
|
|
declaration with two implementations. The implementations are put into separate
|
|
source files with a <em class="quote">_unix</em> or <em class="quote">_win32</em> suffix, and if necessary a 'pimple' (or
|
|
'Bridge') pattern is used to keep the o/s-specific details out of the header.
|
|
If only small parts of the implementation are o/s-specific then there can be
|
|
three source files per header. For example, <em class="quote">gsocket.cpp</em>, <em class="quote">gsocket_win32.cpp</em>
|
|
and <em class="quote">gsocket_unix.cpp</em> in the <em class="quote">src/gnet</em> directory.
|
|
</p>
|
|
|
|
<p>
|
|
Underscores in source file names are used exclusively to indicate build-time
|
|
alternatives.
|
|
</p>
|
|
<h2><a class="a-header" name="SH_1_4">Event model</a></h2> <!-- index:2:SH:1:4:Event model -->
|
|
<p>
|
|
The E-MailRelay server uses non-blocking socket i/o, with a select() event loop.
|
|
This event model means that the server can handle multiple network connections
|
|
simultaneously from a single thread, and even if multi-threading is disabled at
|
|
build-time the only blocking occurs when external programs are executed (see
|
|
<em class="quote">--filter</em> and <em class="quote">--address-verifier</em>).
|
|
</p>
|
|
|
|
<p>
|
|
This event model can make the code more complicated than the equivalent
|
|
multi-threaded approach since (for example) it is not possible to wait for a
|
|
complete line of input to be received from a remote SMTP client because there
|
|
might be other connections that need servicing half way through.
|
|
</p>
|
|
|
|
<p>
|
|
The advantages of a non-blocking event model are discussed in the well-known
|
|
<a class="a-href" href="http://www.kegel.com/c10k.html">C10K Problem</a> document.
|
|
</p>
|
|
|
|
<p>
|
|
At higher levels the C++ slot/signal design pattern is used to propagate events
|
|
between objects (not to be confused with operating system signals). The
|
|
slot/signal implementation has been simplified compared to Qt or boost by not
|
|
supporting signal multicasting, so each signal connects to no more than one
|
|
slot.
|
|
</p>
|
|
<h2><a class="a-header" name="SH_1_5">Module structure</a></h2> <!-- index:2:SH:1:5:Module structure -->
|
|
<p>
|
|
The main C++ libraries in the E-MailRelay code base are as follows:
|
|
</p>
|
|
|
|
<dl>
|
|
<dt><em class="quote">glib</em></dt>
|
|
<dd>
|
|
Low-level classes for file-system abstraction, date and time representation,
|
|
string utility functions, logging, command line parsing etc.
|
|
</dd>
|
|
<dt><em class="quote">gssl</em></dt>
|
|
<dd>
|
|
A thin layer over the third-party TLS libraries.
|
|
</dd>
|
|
<dt><em class="quote">gnet</em></dt>
|
|
<dd>
|
|
Network and event-loop classes.
|
|
</dd>
|
|
<dt><em class="quote">gauth</em></dt>
|
|
<dd>
|
|
Implements various authentication mechanisms.
|
|
</dd>
|
|
<dt><em class="quote">gsmtp</em></dt>
|
|
<dd>
|
|
SMTP protocol and message-store classes.
|
|
</dd>
|
|
<dt><em class="quote">gpop</em></dt>
|
|
<dd>
|
|
POP3 protocol classes.
|
|
</dd>
|
|
</dl>
|
|
<p>
|
|
All of these libraries are portable between Unix-like systems and Windows.
|
|
</p>
|
|
|
|
<p>
|
|
Under Windows there is an additional library <em class="quote">win32</em> for the user interface.
|
|
</p>
|
|
<h2><a class="a-header" name="SH_1_6">SMTP class structure</a></h2> <!-- index:2:SH:1:6:SMTP class structure -->
|
|
<p>
|
|
The message-store functionality uses three abstract interfaces: <em class="quote">MessageStore</em>,
|
|
<em class="quote">NewMessage</em> and <em class="quote">StoredMessage</em>. The <em class="quote">NewMessage</em> interface is used to create
|
|
messages within the store, and the <em class="quote">StoredMessage</em> interface is used for
|
|
reading and extracting messages from the store. The concrete implementation
|
|
classes based on these interfaces are respectively <em class="quote">FileStore</em>, <em class="quote">NewFile</em> and
|
|
<em class="quote">StoredFile</em>.
|
|
</p>
|
|
|
|
<p>
|
|
Protocol classes such as <em class="quote">GSmtp::ServerProtocol</em> receive network and timer
|
|
events from their container and use an abstract <em class="quote">Sender</em> interface to send
|
|
network data. This means that the protocols can be independent of the network
|
|
and event loop framework.
|
|
</p>
|
|
|
|
<p>
|
|
The interaction between the SMTP server protocol class and the message store is
|
|
mediated by the <em class="quote">ProtocolMessage</em> interface. Two main implementations of this
|
|
interface are available: one for normal spooling (<em class="quote">ProtocolMessageStore</em>), and
|
|
another for immediate forwarding (<em class="quote">ProtocolMessageForward</em>). The <em class="quote">Decorator</em>
|
|
pattern is used whereby the forwarding class uses an instance of the storage
|
|
class to do the message storing and filtering, while adding in an instance
|
|
of the <em class="quote">GSmtp::Client</em> class to do the forwarding.
|
|
</p>
|
|
|
|
<p>
|
|
Message filtering (<em class="quote">--filter</em>) is implemented via an abstract <em class="quote">Filter</em>
|
|
interface. Concrete implementations are provided for doing nothing, running an
|
|
external executable program and talking to an external network server.
|
|
</p>
|
|
|
|
<p>
|
|
The protocol, processor and message-store interfaces are brought together by the
|
|
high-level <em class="quote">GSmtp::Server</em> and <em class="quote">GSmtp::Client</em> classes. Dependency injection is
|
|
used to create the concrete instances of the <em class="quote">ProtocolMessage</em> and <em class="quote">Filter</em>
|
|
interfaces.
|
|
</p>
|
|
<h2><a class="a-header" name="SH_1_7">Event handling and exceptions</a></h2> <!-- index:2:SH:1:7:Event handling and exceptions -->
|
|
<p>
|
|
The use of non-blocking i/o in the network library means that most processing
|
|
operates within the context of an i/o event or timeout callback, so the top
|
|
level of the call stack is nearly always the event loop code. This can make
|
|
catching C++ exceptions a bit awkward compared to a multi-threaded approach
|
|
because it is not possible to put a single catch block around a particular
|
|
high-level feature.
|
|
</p>
|
|
|
|
<p>
|
|
The event loop delivers asynchronous socket events to the <em class="quote">EventHandler</em>
|
|
interface, timer events to the <em class="quote">TimerBase</em> interface, and 'future' events to the
|
|
<em class="quote">FutureEventCallback</em> interface. If any of the these event handlers throws an
|
|
exception then the event loop will catch it and deliver it back to an exception
|
|
handler through the <em class="quote">onException()</em> method of an associated <em class="quote">ExceptionHandler</em>
|
|
interface. If an exception is thrown out of _this_ callback then the event loop
|
|
code lets it propagate back to <em class="quote">main()</em>, typically terminating the program.
|
|
</p>
|
|
|
|
<p>
|
|
Every pointer to an event callback interface is associated with an
|
|
<em class="quote">ExceptionHandler</em>. The default <em class="quote">ExceptionHandler</em> is the <em class="quote">EventLoop</em>
|
|
singleton, and a call to its <em class="quote">onException()</em> method terminates the event loop.
|
|
</p>
|
|
|
|
<p>
|
|
This leads to a programming model where key objects are instantiated on the
|
|
heap and these objects delete themselves when they receive certain events from
|
|
the event loop. In the <em class="quote">GNet</em> library it is the <em class="quote">ServerPeer</em> and <em class="quote">HeapClient</em>
|
|
classes that do this lifetime management; instances of these classes delete
|
|
themselves when the associated network connection goes away and they
|
|
implement the <em class="quote">ExceptionHandler</em> interface so that they schedule their own
|
|
deletion when an exception is thrown.
|
|
</p>
|
|
|
|
<p>
|
|
Special smart pointers are sometimes used for these self-deleting classes; the
|
|
smart pointer does not delete the contained object when it is reset, it just
|
|
tells the object to delete itself with a zero-length timer and then releases it
|
|
for garbage collection.
|
|
</p>
|
|
<h2><a class="a-header" name="SH_1_8">Multi-threading</a></h2> <!-- index:2:SH:1:8:Multi-threading -->
|
|
<p>
|
|
Multi-threading can be used as a build-time option to make DNS lookup and the
|
|
execution of helper programs asynchronous; if std::thread is available then it
|
|
is used in a future/promise pattern to wrap up <em class="quote">getaddrinfo()</em> and <em class="quote">waitpid()</em>
|
|
system calls. The shared state comprises only the parameters and return results
|
|
from these system calls, and synchronisation back to the main thread uses the
|
|
event loop (see <em class="quote">GNet::FutureEvent</em>).
|
|
</p>
|
|
<h2><a class="a-header" name="SH_1_9">E-MailRelay GUI</a></h2> <!-- index:2:SH:1:9:E-MailRelay GUI -->
|
|
<p>
|
|
The optional GUI program <em class="quote">emailrelay-gui</em> uses the Qt toolkit for its user
|
|
interface components. The GUI can run as an installer or as a configuration
|
|
helper, depending on whether it can find an installation <em class="quote">payload</em>. Refer to
|
|
the comments in <em class="quote">src/gui/guimain.cpp</em> for more details.
|
|
</p>
|
|
|
|
<p>
|
|
The user interface runs as a stack of dialog-box pages with forward and back
|
|
buttons at the bottom. Once the stack has been completed by the user then each
|
|
page is asked to dump out its state as a set of key-value pairs (see
|
|
<em class="quote">src/gui/pages.cpp</em>). These key-value pairs are processed by an installer class
|
|
into a list of action objects (in the <em class="quote">Command</em> design pattern) and then the
|
|
action objects are run in turn. In order to display the progress of the
|
|
installation each action object is run within a timer callback so that the Qt
|
|
framework gets a chance to update the display between each one.
|
|
</p>
|
|
|
|
<p>
|
|
During development the user interface pages and the installer can be tested
|
|
separately since the interface between them is a simple text stream containing
|
|
key-value pairs.
|
|
</p>
|
|
|
|
<p>
|
|
When run in configure mode the GUI normally ends up simply editing the
|
|
<em class="quote">emailrelay.conf</em> file (or <em class="quote">emailrelay-start.bat</em> on Windows) and/or the
|
|
<em class="quote">emailrelay.auth</em> secrets file.
|
|
</p>
|
|
|
|
<p>
|
|
When run in install mode the GUI expects to unpack all the E-MailRelay files
|
|
from the payload into target directories. (The payload used to be a single
|
|
archive file appended to the executable, but it is now simple directory
|
|
tree that lives alongside the GUI exectuable or inside the Mac application
|
|
bundle.)
|
|
</p>
|
|
<h2><a class="a-header" name="SH_1_10">Windows packaging</a></h2> <!-- index:2:SH:1:10:Windows packaging -->
|
|
<p>
|
|
On Windows E-MailRelay is packaged as a zip file containing the executables
|
|
(including the emailrelay GUI as <em class="quote">emailrelay-setup.exe</em>), documentation, and a
|
|
<em class="quote">payload</em> directory tree. The payload contains many of the same files all over
|
|
again, and while this duplication is not ideal it is at least straightforward.
|
|
</p>
|
|
|
|
<p>
|
|
The Qt tool <em class="quote">windeployqt</em> is used to add run-time dependencies, such as the
|
|
Qt DLLs.
|
|
</p>
|
|
<h2><a class="a-header" name="SH_1_11">Unix packaging</a></h2> <!-- index:2:SH:1:11:Unix packaging -->
|
|
<p>
|
|
On Unix-like operating systems it is more natural to use some sort of package
|
|
derived from the <em class="quote">make install</em> process rather than an installer program, so
|
|
the emailrelay GUI is not normally used.
|
|
</p>
|
|
|
|
<p>
|
|
Top-level makefile targets <em class="quote">dist</em>, <em class="quote">deb</em> and <em class="quote">rpm</em> can be used to create a
|
|
binary tarball, a debian package, and an RPM package respectively.
|
|
</p>
|
|
<h2><a class="a-header" name="SH_1_12">Source control</a></h2> <!-- index:2:SH:1:12:Source control -->
|
|
<p>
|
|
The source code is stored in the SourceForge <em class="quote">svn</em> repository. A working
|
|
copy can be checked out as follows:
|
|
</p>
|
|
|
|
<div class="div-pre">
|
|
<pre>$ svn co https://svn.code.sf.net/p/emailrelay/code/trunk emailrelay</pre>
|
|
</div><!-- div-pre -->
|
|
<h2><a class="a-header" name="SH_1_13">Compile-time features</a></h2> <!-- index:2:SH:1:13:Compile-time features -->
|
|
<p>
|
|
Compile-time features can be selected with options passed to the <em class="quote">configure</em>
|
|
script. These include the following:
|
|
</p>
|
|
|
|
<ul>
|
|
<li>Debug-level logging (<em class="quote">--enable-debug</em>)</li>
|
|
<li>Configuration GUI (<em class="quote">--enable-gui</em>)</li>
|
|
<li>PAM support (<em class="quote">--with-pam</em>)</li>
|
|
</ul>
|
|
|
|
<p>
|
|
Use <em class="quote">./configure --help</em> to see a complete list of options and refer to
|
|
<em class="quote">acinclude.m4</em> for more detailed comments.
|
|
</p>
|
|
|
|
|
|
|
|
<div class="div-footer">
|
|
<p>
|
|
Copyright (C) 2001-2018 Graeme Walker
|
|
</p>
|
|
</div><!-- div-footer -->
|
|
</div> <!-- div-main -->
|
|
</body>
|
|
</html>
|
|
<!-- Copyright (C) 2001-2018 Graeme Walker <graeme_walker@users.sourceforge.net>. All rights reserved. -->
|