Using the link below, download a tarball containing the source code for a web server written in Scheme and C.

A bit of history. The server project was originally started in 2008 to create a web site for an arts organization. Choosing Scheme to implement the server was the direct result of realizing Scheme's template forms were so naturally expressive of HTML's hierarchical structure. I reasoned that it would be straightforward to show "web designers" (already familiar with HTML) how easy it is to write web pages in Scheme to be served up by the server.

Ultimately the organization decided not to proceed with the project. However, a merit of Scheme is that the original version was created in less than two months and already quite usable. It seemed a shame to discard the project, and since I had other ways to use it, development continued.

Over the subsequent four years, the code has been revised and refined several times. In particular, the server's Scheme-implemented bidirectional "iochannels" achieve intuitive usability and efficient performance, but now also allow better interoperability with standard Scheme input/output ports, maximizing programming flexibility and ease of use.

The most recent (and substantial) changes were made from Sep 2012 through April 2013. Many bugs were fixed, and support for web-based email clients and MIME protocol processing was improved. During Mar/Apr 2013 the focus was on major enhancements to security, performance, and operational reliability. But as always, it remains a work in progress.

The server is implemented as a multiplexed, multitasking, compiled, embedded Scheme-in-C application, using Chicken Scheme and its embedding API. This approach provides advantages in stability, low-memory use, and performance, leveraging the capabilities of the Scheme and C implementations, and avoiding some of the potential pitfalls of both. Alas, while it's not yet completed, the plan is still having available a detailed description of its inner workings, a tour through the code.

Development sources are available. As the server project grows, at some point setting up a real VCS distribution would be much more satisfactory . In the interim, comments, feedback, bug-reports are welcome! Write to 'sourcecode at bmedctr dot com'.

• It's intended to make the server portable across implementations of R5RS (and the future R7RS) scheme. The goal is partly achieved, but significant barriers to portability abound. Interfacing with C-language APIs was necessary to achieve good integration with SSL and database services. Arguably, Chicken scheme easily has the best C-FFI among Scheme implementations, and conveniently, Chicken itself is highly portable across platforms. While it may be possible to modify the code for other schemes, that's a big project in it's own right, and we regard the present one as hard enough, and enough to do.
• The server is a multitasking application, allowing full use of multi-core hardware. At startup, the server forks a configurable number of "workers" which listen for connections on ports 80 and 443 (or however configured). Client requests are handled by forked processes, which can handle multiple requests per connection. The inital process stays alive, monitoring the status of the workers. If a worker process terminates, it is promptly replaced to maintain traffic-handling capacity.
• Mail creation and MIME-parsing are built in, which can be used, for example, to build web mail clients in combination with an MTA, and database server.
• Database access procedures for postgresql are included. With portability in mind, the server implements the pgsql "front end/back end" TCP protocol, rather than a wrapper to pgsql's C library interface. While this works well, it proved to have its own modest complexities, possibly impeding Scheme portability. In any case, using it is quite straightforward and convenient.
• A unique feature is that web pages must be loaded as compiled shared (.so) files, which provides advantages in performance, memory usage, and security. Notably, there are no *.html text files whatsoever; every page is dynamically and efficiently generated from Scheme/C code. This feature allows the whole range of scheme progamming techniques to be used; modular style is used throughout. Making use of common, standard client-side capabilities, with Javascript, SQL, and so on, is easily integrated by the Web programmer.
• HTTP virtual hosts are supported, and at least one is required. Virtual sites are very easily set up. The number of virtual hosts is limited only by computer memory, hardly a problem with current-day equipment. Web page source files can be written in any sensible style, easily intermixed with Scheme procedures, macros and other definitions. The full range of web application techniques is available, and requires no external software or dependencies. As a matter of fact, creating any web site, from the simplest to the most intricate requires no more than an unadorned plain text editor.
• Among the numerous Scheme modules in the server, several may be of interest, as potentially applicable in other contexts. common contains myriad utilities, such as a match/search wrapper considerably simplifying irregex use. Module mime-parse and mail-create allow accessing MIME and mail data, including mixed binary/text. Module iochannel is a bidirectional input/output abstraction that operates on binary and text data, essential, for example, for web-based file-upload facilities. A goal of iochannel was having uniform I/O API for files, net, SSL, byte channels. Also, iochannel objects can be converted to Scheme input/output ports, which are then extended to access enhanced functionality iochannel provides.

• C compiler: gcc, clang
• Gnu make: gmake on BSD, etc.
• Chicken Scheme, v. 4.8.0 or above
• Chicken eggs: base64, srfi-99
• OpenSSL v.1.0.0i or above (for ssl/tls library)
• For database access:
  Postgresql-9.1.3 or above (recommended)

Download: Webserver source (multiplex-1.5.1-3.tar.xz)

Of course, Chicken scheme must be installed in order to compile the server. Once the server tarball is expanded, change to the "embed" directory, and at a shell prompt, enter "make" (or "gmake" on BSDs), and then "make install". The default installation prefix is '/usr/local', but of course this can be changed by editing the Makefile.

As an example, this web site is included in the source tarball.

• -c config-file
  The default configuration file is ".multiplexrc" in the directory from which the server is run. The config parameters are quite straightforward--please see the file contents.
• Options beginning with "-:" are Chicken compiled program options. See the Chicken manual for details.
• Otherwise command-line arguments are considered startup files that will be loaded by the embedded scheme program. These should be compiled shared objects. The default is "startup.so", compiled from "startup.scm", which contains virtual host website files to load and initialize the specified hosts. Looking at "startup.scm" will help clarify how startup is organized.

After it's compiled, "make install" will put the multiplex binary in a run path directory (by default /usr/local/bin). The sample web site can be accessed by running the server in the source directory, or optionally, "make deploy" will copy the site files to /usr/local/www, and the server can be started from that directory. The supplied "servcmd" script is a simple tool to run the server.

Edit "servcmd" to make sure user and group ids are set properly. Similarly, check the .multiplexrc configuration file, particularly the uid/gid settings. Setting "workers" to 2 is usually optimum, but it can be set higher, e.g., to the number of cpu cores if more than 2 are present.

You should then be able to start the server with "./servcmd", at a shell prompt, e.g., in an X terminal. Note that under X, "servcmd" calls the "termx" wrapper for xterm. If xterm is not installed you need to install it or modify "termx" to use the terminal emulator present on your system. Problems starting the server are usually due to incorrect directory or file permissions.

For the browser to connect to the server running on the same host, it's necessary to set a line in /etc/hosts:

Once everything is set, this page should appear in your browser when pointed at your locally running virtual host. If you're having trouble, check uid/gid, directory and file permissions, local DNS setup, as these are the most frequent sources of difficulty.