emailrelay/doc/reference.txt

E-MailRelay Reference
=====================

Introduction
------------
This is the E-MailRelay reference guide. It contains material which is
supplementary to the user guide.

Command line usage
------------------
The "emailrelay" program supports the following command-line usage:

	emailrelay [<switch> [<switch> ...]]

where <switch> is:
# --admin (-a)
  Enables the administration interface and specifies its listening port number.

# --as-client (-q)
  Runs as a client, forwarding spooled mail to <host>: equivalent to "--log --no-syslog --no-daemon --dont-serve --forward --forward-to".

# --as-proxy (-y)
  Runs as a proxy: equivalent to "--log --close-stderr --immediate --forward-to".

# --as-server (-d)
  Runs as a server: equivalent to "--log --close-stderr".

# --client-auth (-C)
  Enables authentication with remote server, using the given secrets file.

# --close-stderr (-e)
  Closes the standard error stream after start-up.

# --connection-timeout (-U)
  Sets the timeout (in seconds) when connecting to a remote server (default is 40).

# --debug (-g)
  Generates debug-level logging (if compiled-in).

# --domain (-D)
  Sets an override for the host's fully qualified domain name.

# --dont-listen (-X)
  Dont listen for smtp connections (usually used with --admin).

# --dont-serve (-x)
  Dont act as a server (usually used with --forward).

# --filter (-z)
  Defines a mail processor program.

# --forward (-f)
  Forwards stored mail on startup (requires --forward-to).

# --forward-to (-o)
  Specifies the remote smtp server (required by --forward and --admin).

# --help (-h)
  Displays help text and exits.

# --immediate (-m)
  Forwards each message as soon as it is received (requires --forward-to).

# --interface (-I)
  Listen on a specific interface.

# --log (-l)
  Writes log information on standard error and syslog.

# --log-time (-L)
  Adds a timestamp to the logging output.

# --no-daemon (-t)
  Does not detach from the terminal.

# --no-syslog (-n)
  Disables syslog output.

# --pid-file (-i)
  Records the daemon process-id in the given file.

# --port (-p)
  Specifies the smtp listening port number.

# --remote-clients (-r)
  Allows remote clients to connect.

# --response-timeout (-T)
  Sets the response timeout (in seconds) when talking to a remote server (default is 1800).

# --server-auth (-S)
  Enables authentication of remote clients, using the given secrets file.

# --spool-dir (-s)
  Specifies the spool directory (default is "/var/spool/emailrelay").

# --user (-u)
  Names the effective user to switch to when started as root (default is "daemon").

# --verbose (-v)
  Generates more verbose output (works with --help and --log).

# --verifier (-Z)
  Defines an external address verifier program.

# --version (-V)
  Displays version information and exits.

If no command-line switches are supplied at all then the default
behaviour is:
* to run as a daemon, detached from the terminal
* listen on the standard SMTP port (25)
* store e-mail messages in "/usr/local/var/spool/emailrelay"
* reject connections from remote clients
* disable the administration interface
* generate no logging or diagnostic messages

The "--as-server" switch makes sure that logging is enabled and that
the standard error stream is closed.

To foward spooled messages to the ISP the command-line switch "--as-client"
is provided to run the program...
* in foreground, exiting when all spooled mail has been processed
* forwarding spooled mail from "/usr/local/var/spool/emailrelay"
* without listening on any port
* with error, warning and information messages sent to stderr
* without using syslog

Message store
-------------
Mail messages are stored as text files in the configured spool directory. Each
message is represented as an envelope file and a content file. The envelope
file contains parameters relevant to the SMTP dialogue, and the content file
contains the RFC822 headers and body text.

The filenames used in the message store have a prefix of "emailrelay", followed
by a process-id and sequence number, followed by "envelope" or "content". The
envelope files then have an additional suffix to implement a simple locking
scheme.

The envelope suffixes are:
* ".new" -- while the envelope is first being written
* <none> -- while the message is spooled
* ".busy" -- while the message is being forwarded
* ".bad" -- if the message cannot be forwarded
* ".local" -- for copies of the envelope file for delivery to local recipients

Copies of the content file for delivery to local recipients will also have
a "local" suffix.

If a message cannot be forwarded the envelope file is given a "bad" suffix,
and the failure reason is written into the file.

SMTP issues
-----------
# Authentication:

  The AUTH extension is supported, providing the CRAM-MD5 and LOGIN mechanisms.

# Local delivery:

  Recipient addresses like "postmaster", "postmaster@localhost" and "postmaster@fqdn"
  (where "fqdn" is the host's fully qualified domain name) are treated as the local
  postmaster, resulting in local delivery rather than mail forwarding.

  Recipient addresses (other than "postmaster" addresses) with no "at" sign (@) or
  ending in "@localhost" will be rejected at the time the message is submitted by the
  e-mail front-end. This is in line with E-MailRelay's intended purpose as a simple
  mail relay, rather than a fully-fledged routing MTA.

  Delivery of mail to a local "postmaster" is a feature of E-MailRelay which is
  provided for completeness and for comformance to the SMTP specification.
  Note that the E-MailRelay daemon does not actually deliver mail to the postmaster
  mailbox.  All it does is create an envelope and content file in the spool
  directory with a ".local" suffix. Some external system, such as a shell
  script run from cron calling "procmail", should be used to process the
  ".local" files. An example script is provided.

# Timeouts:

  A simple client-side timeout is implemented which will abort the transaction
  if the server fails to respond to any of the client's SMTP commands within the
  given time period.

# Message loops:

  Message loops are not detected.

# Eight bit messages:

  The 8BITMIME SMTP extension is supported, however no attempt is made to
  re-encode 8-bit messages into 7-bit messages if the next-hop server
  does not.

Address validation
------------------
An address supplied to the SMTP commands "RCPT" and "VRFY" is normally accepted
if it contains an at sign, or if it is some sort of local "postmaster" address
(as described above). However, this behaviour can be modified by using an external
verifier program, using the "--verifier" command-line switch.

The verifier program is passed a command-line containing the full address, the
user-name part of the address, the host-name part, the local host's fully
qualified domain name, the current "MAIL" command's "FROM:" address or the
empty string for the "VRFY" command, the IP address of the client connection,
the authentication mechanism used by the client ("NONE" if trusted), and either
the authentication name or the fourth field from authentication secrets file if
a trusted IP address.

For valid local mailbox addresses the verifier is expected to write two lines to
the standard output -- the full name associated with the mailbox, and the
canonical mailbox name -- and then exit with a value of zero. For valid non-local
addresses the first line of output should be empty, the second line should be
copied from the first command-line argument, and the exit value should be one.
For invalid addresses the exit value should be greater than one, and anything
written to the standard output is taken as the failure reason.

(Only the few few thousand characters are read from the verifier's standard
output stream; any more is thrown away.)

In this simple example script all addresses are accepted as long as they contain
an at sign. This has the effect of removing the normal "postmaster" functionality,
which is in any case not very useful when running in proxy mode:

	#!/bin/sh
	# verifier.sh
	# An address verifier script for E-MailRelay.
	address="${1}"
	user="${2}"
	host="${3}"
	if test "${address}" != "${user}@${host}" ; then exit 2 ; fi
	echo "${address}"
	echo "${address}" # again
	exit 1 # accept

As another example, the following address verifier script accepts all recipient
addresses by default, but rejects remote addresses if the client has bypassed
authentication by connecting on a trusted IP address:

	#!/bin/sh
	# verifier.sh
	# An address verifier script for E-MailRelay.
	host="$3"
	local_domain="$4"
	auth_mechanism="$7"
	if test "${auth_mechanism}" = "NONE" -a "${host}" != "${local_domain}"
	then
		echo "cannot relay without authentication"
		exit 2 # reject the recipient address
	fi
	echo "${address}"
	echo "${address}" # again
	exit 1 # accept the recipient address

Administration interface
------------------------
If enabled, the server will provide a network interface for performing
administration tasks. This is a simple command-line interface which is
compatible with "telnet".

Currently the only supported commands are "flush" and "info". The "flush" command
is used to forward spooled mail to the configured next-hop SMTP server. The
next-hop server address must have been defined on the "emailrelay" command
line at start-up using the "--forward-to" switch; it cannot be specified through
the administration interface.

The "flush" command allows you to run a single process which combines the
functionality of storage daemon and forwarding client. This might be useful
in systems which are low on memory: the memory footprint of "telnet" or
"emailrelay-poke" will be much smaller than "emailrelay --as-client".

Mail processing
---------------
The "--filter" command-line switch allows you to specify a mail processor program
which operates on mail messages as they pass through the E-MailRelay system. The
mail processor program is run as soon as the mail message has been stored in the
spool directory, with the full path of the content file specified on the command
line.

This can be combined with the "--immediate" (or "--as-proxy") switch to implement
personal mail processing, without replacing or reconfiguring the system's
default MTA.

For example, the following command will start a proxy server on port 10025 which
processes mail using the specified filter program, and then forwards the mail on
to the system's default MTA (on port 25):

	emailrelay --as-proxy localhost:smtp --port 10025 --no-syslog \
	  --filter ${HOME}/.emailrelay/filter \
	  --spool-dir ${HOME}/.emailrelay/spool

The mail processor program should terminate with an exit code of zero to indicate
success. An exit code of 100 can be used to cancel all further processing of the
message, and any other non-zero exit code is used to indicate an error.

For example, the following mail processor shell script examines the client's
IP address and conditionally dumps the message into "sendmail" (using the
sendmail command-line interface rather than SMTP):

	#!/bin/sh
	#
	content="${1}"
	envelope="`echo \"${content}\" | sed 's/content/envelope.new/'`"
	ip="`awk '/MailRelay-Client:/ {print $2;exit}' \"${envelope}\"`"
	if test "${ip}" = "192.168.0.2"
	then
		cat "${content}" | /usr/sbin/sendmail -t
		rm -f "${envelope}" "${content}"
		exit 100 # <= cancel further processing by emailrelay
	fi
	exit 0

An example mail processor script which does simple rot-13 masking of messages is
also provided in the distribution ("share/emailrelay/emailrelay-process.sh").
This could be used as a template for a more sophisticated message encryption
system.

Security issues
---------------
A major security concern is the use of an external mail processor (using the
"--filter" switch), and so the following precautions are taken:

# effective userid

  Suid privileges are revoked at start-up, switching the effective userid/groupid
  to be the real userid/groupid values. If started as "root" then the effective
  userid/groupid are taken from the "daemon" user. Special privileges are only
  reclaimed when needed to bind sockets and do file i/o. Normally this means
  temporarily switching the userid and groupid back to what they were at start-up.
  However, when writing spool files only the effective userid is changed, not the
  groupid.

# execution environment

  The mail processor runs with an almost empty set of environment variables ("PATH"
  and "IFS"), and with no open file descriptors other than "stdin"/"stdout"/"stderr"
  open onto "/dev/null".

# configuration

  The mail processor filename has to be configured using a full path, so there is
  no dependence on the current working directory or the PATH environment variable.

Security issues which relate to the SMTP protocol itself are beyond the scope of
this document, but RFC2821 makes the following observation: "SMTP mail is
inherently insecure in that it is feasible for even [..] casual users to [..]
create messages that will trick a [..] recipient into believing that they came
from somewhere else. [..] Real [..] security lies [..] in end-to-end methods [..]
such as those which use digital signatures."

The "Authentication" section below also relates to security.

Some other points are:
* The program runs for most of the time with a "umask" of 177, switching to 117 when creating spool files.
* Strings are dynamically allocated, so buffer overflow/truncation issues are avoided.
* By default connections to the SMTP and administrative ports will be rejected if they come from remote machines.
* No configuration parameters can be changed through the administrative interface.
* No exec(), system() or popen() calls are used other than execve() to spawn the mail processor and address verifier.
* The submit utility is installed as set-group-id with group ownership of "daemon".

Authentication
--------------
E-MailRelay supports the ESMTP "AUTH" extension, as defined in RFC2554, on both
the server-side and client-side. The only authentication mechanisms currently
provided are the non-standard but widely used "LOGIN" mechanism, which uses
plaintext passwords, and the "CRAM-MD5" mechanism, as defined in RFC2195.

Authentication is enabled with the "--auth-client" and "--auth-server"
command-line switches. The switch parameter is the name of a "secrets" file,
containing usernames and passwords.

The secrets file has a line-based format: blank lines are ignored and the hash
character (#) is used for comments. Lines have four white-space delimited
fields: "mechanism", "client-or-server", "userid", and "secret". The "mechanism"
field must be "LOGIN" or "CRAM-MD5" (case-insensitive); the "client-or-server"
field must be "client" or "server"; the "userid" field is xtext-encoded user
identifier; and the "secret" field is the xtext-encoded "LOGIN" password or
"CRAM-MD5" key. The "xtext" encoding scheme is defined in RFC1891. The
"CRAM-MD5" keys can be generated using the "emailrelay-passwd" utility.

A client-side secrets file should contain at least one "LOGIN client" or
"CRAM-MD5 client" entry. A server-side secrets file should contain zero or
more "LOGIN server" or "CRAM-MD5 server" entries. The same secrets file may
be specified for both "--auth-client" and "--auth-server" switches.

For example, the following secrets file defines "jsmith" as the username to be
used when E-MailRelay authenticates with a next-hop server, and defines two
usernames ("user1" and "user2") which can be used by clients when they
authenticate with the E-MailRelay server:

	#
	# emailrelay secrets file
	#
	LOGIN client jsmith my+20password
	LOGIN server user1 secret
	LOGIN server user2 e+3Dmc2

A "CRAM-MD5" version would look like this:

	#
	# emailrelay secrets file
	#
	CRAM-MD5 client jsmith 688498119.2977922305.1278051807.3015243256.2216875978.2833592318.2902375592.3156808220
	CRAM-MD5 server user1 4059553961.2316091643.3282746241.1444639637.3735501773.3404060330.2760590371.1201092398
	CRAM-MD5 server user2 2798539199.3144534242.3784876256.2879973305.2327113479.216533878.2436460291.2361831919

When using the LOGIN mechanism you have to store plaintext passwords in a file
and then send them unencypted over a network. This is a bad thing. You should at
least make sure that the secrets file has tight permissions, and that the
passwords in it are not also used for anything important (such as root access).

On the server side authentication is advertised in the response to the SMTP
"EHLO" command if the "--auth-server" command-line switch is used, and
authentication by the client is mandatory unless the client's IP address is
configured as a trusted address. If the client does authenticate then
the authenticated user-id is stored with the message and then passed on to a
next-hop server using an "AUTH=userid" parameter on the SMTP "MAIL FROM"
command. If the client does not to authenticate then the submitted messages
will be forwarded using "AUTH=<>" on the "MAIL FROM" command. Note that any
"AUTH=userid" information on incoming submitted messages is ignored and
discarded: it is the authorised userid from the AUTH command which is
propagated, not the userid from the incoming "MAIL FROM" command's "AUTH="
parameter.

Trusted IP addresses are configured with lines in the secrets file having "NONE"
in the first field, "server" in the second field, a wildcarded IP address in
the third field, and an arbitrary keyword in the fourth field. The keyword
is passed to any external address verifier program specified by the "--verifier"
command-line switch.

On the client side authentication is performed when the client has connected to
a server which supports the AUTH extension with the LOGIN or CRAM-MD5 mechanism.
If client authentication is enabled (with the "--auth-client" switch) but the
server does not support the AUTH extension, or does not support the LOGIN or
CRAM-MD5 mechanism, then the client will fail the first message and terminate
with an error message.

Note that some ISPs require separate POP/IMAP authentication before SMTP access
from a particular IP address is allowed. This type of POP-before-SMTP
authentication can be done outside the E-MailRelay system by POP/IMAP utilities
such as "fetchmail".

Application notes
-----------------
# *Mutt* [http://www.mutt.org]

  Mutt can be configured to use the E-MailRelay 'submit' program rather than
  "sendmail" with an entry in the ".muttrc" file like this:

	set sendmail="/usr/local/libexec/emailrelay-submit"

# *Spam Assassin* [http://spamassassin.taint.org]

  The E-MailRelay server can use Spam Assasin to mark potential spam by having
  a small shell script which calls spamassassin's "spamc" program installed
  as the E-MailRelay mail processor (using the "--filter" command-line switch).

  The shell script would look something like this:

	#!/bin/sh
	tmp="`basename $0`.$$.tmp"
	cat ${1} | spamc > ${tmp} && mv ${tmp} ${1}

Files and directories
---------------------
By default "make install" installs files in the following locations:
* /usr/local/libexec/emailrelay
* /usr/local/libexec/emailrelay-deliver.sh
* /usr/local/libexec/emailrelay-notify.sh
* /usr/local/libexec/emailrelay-poke
* /usr/local/libexec/emailrelay-process.sh
* /usr/local/libexec/emailrelay-resubmit.sh
* /usr/local/man/man1/emailrelay-passwd.1.gz
* /usr/local/man/man1/emailrelay-poke.1.gz
* /usr/local/man/man1/emailrelay-submit.1.gz
* /usr/local/man/man1/emailrelay.1.gz
* /usr/local/sbin/emailrelay
* /usr/local/sbin/emailrelay-passwd
* /usr/local/sbin/emailrelay-submit
* /usr/local/share/emailrelay/doc/NEWS
* /usr/local/share/emailrelay/doc/README
* /usr/local/share/emailrelay/doc/changelog.gz
* /usr/local/share/emailrelay/doc/changelog.html
* /usr/local/share/emailrelay/doc/developer.html
* /usr/local/share/emailrelay/doc/developer.txt
* /usr/local/share/emailrelay/doc/emailrelay.css
* /usr/local/share/emailrelay/doc/index.html
* /usr/local/share/emailrelay/doc/readme.html
* /usr/local/share/emailrelay/doc/reference.html
* /usr/local/share/emailrelay/doc/reference.txt
* /usr/local/share/emailrelay/doc/userguide.html
* /usr/local/share/emailrelay/doc/userguide.txt
* /usr/local/share/emailrelay/doc/windows.html
* /usr/local/share/emailrelay/doc/windows.txt
* /usr/local/var/spool/emailrelay/emailrelay.*.content
* /usr/local/var/spool/emailrelay/emailrelay.*.envelope

This directory structure is constrained by the GNU/"autoconf" conventions rather
than the Filesystem Hierarchy Standard (FHS).

To force FHS compliance you can use the "--enable-fhs" switch when running
"configure". This results in the following file locations:
* /etc/init.d/emailrelay
* /usr/lib/emailrelay/emailrelay-poke
* /usr/sbin/emailrelay
* /usr/sbin/emailrelay-passwd
* /usr/sbin/emailrelay-submit
* /usr/share/doc/emailrelay/NEWS
* /usr/share/doc/emailrelay/README
* /usr/share/doc/emailrelay/changelog.gz
* /usr/share/doc/emailrelay/changelog.html
* /usr/share/doc/emailrelay/developer.html
* /usr/share/doc/emailrelay/developer.txt
* /usr/share/doc/emailrelay/emailrelay.css
* /usr/share/doc/emailrelay/examples/emailrelay-deliver.sh
* /usr/share/doc/emailrelay/examples/emailrelay-notify.sh
* /usr/share/doc/emailrelay/examples/emailrelay-process.sh
* /usr/share/doc/emailrelay/examples/emailrelay-resubmit.sh
* /usr/share/doc/emailrelay/index.html
* /usr/share/doc/emailrelay/readme.html
* /usr/share/doc/emailrelay/reference.html
* /usr/share/doc/emailrelay/reference.txt
* /usr/share/doc/emailrelay/userguide.html
* /usr/share/doc/emailrelay/userguide.txt
* /usr/share/doc/emailrelay/windows.html
* /usr/share/doc/emailrelay/windows.txt
* /usr/share/man/man1/emailrelay-passwd.1.gz
* /usr/share/man/man1/emailrelay-poke.1.gz
* /usr/share/man/man1/emailrelay-submit.1.gz
* /usr/share/man/man1/emailrelay.1.gz
* /var/spool/emailrelay/emailrelay.*.content
* /var/spool/emailrelay/emailrelay.*.envelope

For finer control of the directory structure the following variables
can be specified on the "configure" command-line (but note that the
"--enable-fhs" will override them):
* e_sbindir
* e_libexecdir
* e_docdir
* e_initdir
* e_spooldir
* e_man1dir
* e_examplesdir

For example, running "./configure --prefix=/usr e_spooldir=/tmp/spool" will
create the GNU-style directory structure under "/usr" rather than "/usr/local",
and create the E-MailRelay spool directory as "/tmp/spool" rather than
"/usr/local/var/spool/emailrelay".

The default spool directory path which is built into the executables and scripts
comes from "configure", via the makefiles. So after running "configure" with a
different spool directory do a "make clean" in at least "src/main" and "bin".

Even though the "--enable-fhs" switch overrides all other directory specifiers
during the build process, it is still possible to change the installation
root directory using "make install DESTDIR=<root>" or "DESTDIR=<root> make -e install".
However this will not affect the default spool directory path built into the
scripts and executables; the spool directory path will have to be explicitly
defined at run-time.

For example, building with "configure --enable-fhs ; make ; make install DESTDIR=/root"
would create a spool directory "/root/var/spool/emailrelay", but the server
run as "/root/usr/sbin/emailrelay --as-server" would expect a spool directory
of "/var/spool/emailrelay". This could be fixed by either adding
"--spool-dir /root/var/spool/emailrelay" to the server command-line, or by
creating the "/var/spool/emailrelay" directory manually.

The "emailrelay" script in "init.d" creates a pid file in "/var/run" if the
directory exists, or in "/tmp" otherwise.



Copyright (C) 2001-2003 Graeme Walker <graeme_walker@users.sourceforge.net>. All rights reserved.