emailrelay/doc/developer.md

E-MailRelay Developer Guide

Principles

The main principles in the design of E-MailRelay can be summarised as:

• Minimal third-party dependencies
• Windows/Unix portability without #ifdefs
• Event-driven, non-blocking, single-threaded networking code
• Functionality without imposing policy

Dependencies

E-MailRelay started life at a time when Linux had no decent package manager and Windows was in the grip of DLL hell. As a result, a key principle is that it has no dependencies other than a good C++ runtime. Since that time OpenSSL has been introduced as a dependency to support TLS encryption, and the optional configuration and installation GUI has been developed using the Qt toolkit.

In those early years multi-threading support in C++ libraries was poor, so up until version 2.0 the code was single-threaded throughout.

Portability

The E-MailRelay code is mostly written in C++-1998, but using some features of C++-2011. A C++-1998 compiler can be used, but multi-threading will be disabled.

The header files gdef.h in src/glib is used to fix up some compiler portability issues such as missing standard types, non-standard system headers etc. Conditional compilation directives (#ifdef etc.) are largely confined this file in order to improve readability.

Windows/Unix portability is generally addressed by providing a common class declaration with two implementations. The implementations are put into separate source files with a _unix or _win32 suffix, and if necessary a 'pimple' (or 'Bridge') pattern is used to keep the o/s-specific details out of the header. If only small parts of the implementation are o/s-specific then there can be three source files per header. For example, gsocket.cpp, gsocket_win32.cpp and gsocket_unix.cpp in the src/gnet directory.

Underscores in source file names are used exclusively to indicate build-time alternatives.

Event model

The E-MailRelay server uses non-blocking socket i/o, with a select() event loop. This event model means that the server can handle multiple network connections simultaneously from a single thread, and even if multi-threading is disabled at build-time the only blocking occurs when external programs are executed (see --filter and --address-verifier).

This event model can make the code more complicated than the equivalent multi-threaded approach since (for example) it is not possible to wait for a complete line of input to be received from a remote SMTP client because there might be other connections that need servicing half way through.

The advantages of a non-blocking event model are discussed in the well-known C10K Problem document.

At higher levels the C++ slot/signal design pattern is used to propagate events between objects (not to be confused with operating system signals). The slot/signal implementation has been simplified compared to Qt or boost by not supporting signal multicasting, so each signal connects to no more than one slot.

The synchronous slot/signal pattern needs some care when when the signalling object gets destructed as a side-effect of raising a signal, and that situation can be non-obvious precisely because of the slot/signal code decoupling. In most cases signals are emitted at the end of a function and the stack unwinds back to the event loop immediately afterwards, but in other situations, particularly when emitting more than one signal, defensive measures are required.

Module structure

The main C++ libraries in the E-MailRelay code base are as follows:

• glib

  Low-level classes for file-system abstraction, date and time representation, string utility functions, logging, command line parsing etc.

• gssl

  A thin layer over the third-party TLS libraries.

• gnet

  Network and event-loop classes.

• gauth

  Implements various authentication mechanisms.

• gsmtp

  SMTP protocol and message-store classes.

• gpop

  POP3 protocol classes.

All of these libraries are portable between Unix-like systems and Windows.

Under Windows there is an additional library under src/win32 for the user interface implemented using the Microsoft Win32 API.

SMTP class structure

The message-store functionality uses three abstract interfaces: MessageStore, NewMessage and StoredMessage. The NewMessage interface is used to create messages within the store, and the StoredMessage interface is used for reading and extracting messages from the store. The concrete implementation classes based on these interfaces are respectively FileStore, NewFile and StoredFile.

Protocol classes such as GSmtp::ServerProtocol receive network and timer events from their container and use an abstract Sender interface to send network data. This means that the protocols can be independent of the network and event loop framework.

The interaction between the SMTP server protocol class and the message store is mediated by the ProtocolMessage interface. Two main implementations of this interface are available: one for normal spooling (ProtocolMessageStore), and another for immediate forwarding (ProtocolMessageForward). The Decorator pattern is used whereby the forwarding class uses an instance of the storage class to do the message storing and filtering, while adding in an instance of the GSmtp::Client class to do the forwarding.

Message filtering (--filter) is implemented via an abstract Filter interface. Concrete implementations are provided for doing nothing, running an external executable program and talking to an external network server.

The protocol, processor and message-store interfaces are brought together by the high-level GSmtp::Server and GSmtp::Client classes. Dependency injection is used to create the concrete instances of the ProtocolMessage and Filter interfaces.

Event handling and exceptions

The use of non-blocking i/o in the network library means that most processing operates within the context of an i/o event or timeout callback, so the top level of the call stack is nearly always the event loop code. This can make catching C++ exceptions a bit awkward compared to a multi-threaded approach because it is not possible to put a single catch block around a particular high-level feature.

The event loop delivers asynchronous socket events to the EventHandler interface, timer events to the TimerBase interface, and 'future' events to the FutureEventCallback interface. If any of the these event handlers throws an exception then the event loop catches it and delivers it back to an exception handler through the onException() method of an associated ExceptionHandler interface. If an exception is thrown out of this callback then the event loop code lets it propagate back to main(), typically terminating the program.

However, sometimes there are objects that need to be more resilient to exceptions. In particular, a network server should not terminate just because one of its connections fails unexpectedly. In these cases the owning parent object receives the exception notification together with a pointer that identifies the child object that threw the exception (ie. the exception source). This allows the parent object to absorb the exception and delete the child, without the exception killing the whole server.

The combination of an exception handler and the optional exception source pointer bound to it is known as an ExceptionSink.

Multi-threading

Multi-threading can be used as a build-time option to make DNS lookup and the execution of helper programs asynchronous; if std::thread is available then it is used in a future/promise pattern to wrap up getaddrinfo() and waitpid() system calls. The shared state comprises only the parameters and return results from these system calls, and synchronisation back to the main thread uses the event loop (see GNet::FutureEvent).

E-MailRelay GUI

The optional GUI program emailrelay-gui uses the Qt toolkit for its user interface components. The GUI can run as an installer or as a configuration helper, depending on whether it can find an installation payload. Refer to the comments in src/gui/guimain.cpp for more details.

The user interface runs as a stack of dialog-box pages with forward and back buttons at the bottom. Once the stack has been completed by the user then each page is asked to dump out its state as a set of key-value pairs (see src/gui/pages.cpp). These key-value pairs are processed by an installer class into a list of action objects (in the Command design pattern) and then the action objects are run in turn. In order to display the progress of the installation each action object is run within a timer callback so that the Qt framework gets a chance to update the display between each one.

During development the user interface pages and the installer can be tested separately since the interface between them is a simple text stream containing key-value pairs.

When run in configure mode the GUI normally ends up simply editing the emailrelay.conf file (or emailrelay-start.bat on Windows) and/or the emailrelay.auth secrets file.

When run in install mode the GUI expects to unpack all the E-MailRelay files from the payload into target directories. (The payload used to be a single archive file appended to the executable, but it is now simple directory tree that lives alongside the GUI exectuable or inside the Mac application bundle.)

Windows packaging

On Windows E-MailRelay is packaged as a zip file containing the executables (including the emailrelay GUI as emailrelay-setup.exe), documentation, and a payload directory tree. The payload contains many of the same files all over again, and while this duplication is not ideal it is at least straightforward.

The Qt tool windeployqt is used to add run-time dependencies, such as the Qt DLLs.

Unix packaging

On Unix-like operating systems it is more natural to use some sort of package derived from the make install process rather than an installer program, so the emailrelay GUI is not normally used.

Top-level makefile targets dist, deb and rpm can be used to create a binary tarball, a debian package, and an RPM package respectively.

Source control

The source code is stored in the SourceForge svn repository. A working copy can be checked out as follows:

    $ svn co https://svn.code.sf.net/p/emailrelay/code/trunk emailrelay

Compile-time features

Compile-time features can be selected with options passed to the configure script. These include the following:

• Debug-level logging (--enable-debug)
• Configuration GUI (--enable-gui)
• PAM support (--with-pam)

Use ./configure --help to see a complete list of options and refer to acinclude.m4 for more detailed comments.

---

Copyright (C) 2001-2019 Graeme Walker