TeleVideo Systems Part VII - Almmmost replacement for MmmOST

Almmmost CP/M file server

Functions

Almmmost is meant to be a general replacement for the TeleVideo MmmOST server software, which ran on Z80-based CP/M systems, and served data to up to 16 clients on a star-based network.  It’s designed to work on more modern hardware and uses a still-available Z85C30 serial controller chip to interface with the TeleVideo client systems.

The main functions I was looking to provide were:

• Communications with the original hardware over an 800K baud RS-422 SDLC link
• Compatibility with the original client OS
• Server software that runs on hardware that’s easy to acquire, develop on, and provides a robust TCP/IP stack (and with a CPU fast enough to do TLS/SSL connections to modern systems)
• Ability to boot the clients off of the RS-422 link
• Private and Shared drive functionality to provide per-client storage, and the ability to share files between clients easily
• Remote floppy drive access, replaced with images, to facilitate moving data into the system
• Multiple client support off of one server
• Support for standard MmmOST interfaces through RS-422 and the MULTI.SYS shared file
• Additional support for special files, like MULTI.SYS, which provide access to dynamic data or the ability to access resources through the server’s TCP/IP connection

Not included yet, but things I’d like to add which MmmOST supported include Print queueing, file/record locking support, and communication using mailboxes/FIFOs between clients on the same server.  Printing and communication should be easy to add, but locking support will require a re-design of some of the server software.
Design

The system is built around a Next Thing Co. CHIP ARM board. It was chosen because it’s cheap and easily available (though, it is discontinued).  It should be adaptable to another Linux-based ARM board with enough GPIOs and a PWM available without too much effort.  It was designed to be modular, to make it easy to add/subtract client ports as desired.

The software is all written in C, with one shell script to set up the PWM for an 800K baud rate.  It consists of a kernel module and a single-threaded server application that uses the kernel driver to talk to clients and can interpret/process requests from clients.  Adding locking support will require moving to either a multi-threaded or state-machine model to track both client interactions and lock availability.

Hardware design

The hardware was gone through two revisions.  The first revision was a “quick and dirty” design, to glue a 5V logic Z85C30 and the RS-422 buffers to a 3.3V set of GPIOs.  It only supported a maximum of two clients (one per port on the Z85C30) but was sufficient to figure out how to interface with the clients and understand the protocol.

picture of hardware rev 1

The hardware was re-designed to use two boards, one which plugs into the ARM board’s GPIO headers, and it does 3.3V to 5V level conversion, generates the chip select lines (allowing for multiple Z85C30 chip connections), and generates the 6MHz “system” clock that is used internally by the Z85C30s.

picture of hardware rev 2

The second board handles I/O to the clients and is connected using a simple bus over a common 34-pin ribbon cable connector, allowing multiple boards to be connected easily.  The bus carries the 8 data bits, a set of chip selects to allow for multiple client-I/O boards, the 6MHz system & 800KHz Tx clock, control lines for the Z85C30s, and 5V power to the boards.

The I/O board contains a Z85C30 serial interface chip, 26LS31/32 RS-422 drivers, and a dip switch to select the board ID (which connects that chip select line to the Z85C30).

I used Eagle to design the boards and used Advanced Circuits and Seeed Studio to manufacture the boards.  I was impressed with the quality of board I could get from Seeed Studio - including fast DHL shipping I spent about $30 total to get 10 of each board.  I also got a lot better with surface mount soldering through the process.  I’ve found that I actually prefer surface mount parts to through-hole parts because they’re much easier and faster to solder as long as you have decent tools.

The board designs are linked at the end of this article in the GitHub repository.

First Gen software

My first attempt at software, I ended up trying to just use the GPIOs in userland software, and ioremap’d them into the process’s I/O space.  I was able to use this for some initial debugging and testing, but I quickly ran into a problem that Linux will steal enough time from the process to underrun or overrun the 1-byte buffer on the Z85C30.  Dropping the baud rate to 666KHz helped quite a bit, but I still had occasional corrupted packets.

This taught me a few things about how the basic I/O routines worked.  Operating SDLC on the Z85C30 was a bit different than the code on the Z80-SIO and needed me to select the correct CRC initial vector.  I was able to use the onboard Baud Rate Generator on the Z85C30, but in the end, decided that it wasn’t flexible enough, and it was easier to just generate the needed baud rate on the system PWM.

I also discovered that the routines on the TeleVideo systems transmitted the bytes in reverse order, and discarded the checksum bytes (though, they did verify that the checksum was received correctly).

Kernel Driver

After giving up on userland code to do this, I wrote a kernel driver (tvi_sdlc).  It still took some effort to make this fast enough to not underflow buffers.  I ended up having to turn off interrupts and preemption while transferring a packet two/from the Z85C30.  Fortunately, the longest packet size was 128 bytes, so the delay doesn’t cause too many problems with the system.  I also discovered that I/O to the GPIO pin memory locations (I think this really only applies to writes, but it’s possible that reads are also affected) have quite a few wait states, which limit you to about 1-2M GPIO changes per second.

With that knowledge, I cut out as many I/Os as I could, trying to do single reads/writes to change as many pins at once as possible.  This is what let me turn the baud rate back up to 800K without dropping any characters when reading/writing a port.

Overall, the driver is a fairly simple character driver, which implements IOCTL’s to change and check the state of various registers in the chip, and direct I/O to read/write packets as SDLC frames.  The application opens the device file once per I/O port it uses, and the kernel and application use that to track which port to talk to.

Application

The application is a C based program with multiple modules.  The main module (almmmost.c) reads a config file, initializes general parameters, and asks the other modules to initialize themselves.  It starts the main loop that waits for clients to make requests.  Once it receives a request, it determines where the request should be handed off to, and dispatches it to the correct modules.  The software uses a round-robin to check for requests to attempt to prevent one client from starving other clients from having requests serviced.

Once the request is received and dispatched, that request is followed through until completion.  This won’t work as written to allow for locking (as any blocking request will prevent the lock from being released), but it allowed for most of the code to be easily implemented.

The interface to the kernel module is written in almmmost_device.c.  It handles config file parsing for device configuration, resetting ports, checking the port for a request (if CTS is received), and reading/writing packets. The config file parser looks at the config file section that configures how many I/O ports should be used, and which number in the program attaches to which physical port.

The OS loading module (almmmost_osload.c) handles sending the bootloader and OS image to the client system. It’s called when the server receives a request packet with ‘L’ in the second byte and sends back the appropriate data for the system that made the request.  The module uses the config file to point at raw OS images and gives parameters that can be modified on a real MmmOST system by GENPARAM.  This data structure provides the location in RAM of the system console buffer, the print spool drive, the MmmOST “general” revision, a bitfield indicating which drives are public drives and the number of shared disks that it handles.  It then passes control to the share disk image handling module to further modify the images.

The Shared Disk Image module (almmmost_image.c) handles requests from the client to access individual 128-byte CP/M records on disks shared by the server.  This is primarily used by Private drives (which aren’t shared between clients), and Public Only drives (which don’t support shared files), but also is used for Public drives to read the disk directory by things like DIR.  It uses parameters supplied by the config file to build up the CP/M Drive Parameter Headers and Drive Parameter Blocks for the disk images.

The Shared File module (almmmost_file.c) handles requests to the shared (Public) drives. It implements a CP/M file system so that the disk is still image-compatible with CP/M.  Most functions line up with BDOS functions and handle things like opening/closing files, reading/writing files, and modifying the file’s FCB to update the current file location.  It tries to return sensible errors for various conditions, and prevent multiple clients from opening the same file to write to at the same time.  This module hands off file access to the Special file module for any files that the module will handle.

The Special File module handles files like MULTI.SYS, which don’t represent data stored on the disk image, but perform some other function. MULTI.SYS is implemented to access the system time, with other functions not yet implemented.  The module implements a simple character generator (CHARGEN.SYS), files to move data to the host filesystem directly (FILEIN.SYS and FILEOUT.SYS), a file to directly access URLs via libcurl on the host (URLGET.SYS), and two functions that use libcurl and a web service running elsewhere to convert HTML to text (LYNXGET.SYS) and to convert images to a format that can be displayed on the TS-803 and TPC-I (IMGGET.SYS).  These both use CGI scripts that are in the cgi source directory.

The Miscellaneous module (almmmost_misc.c) handles requests that didn’t fit in elsewhere.  These include print spool functions, responding to client requests for information, and the LOGON command that changes the current Private drive.

Finally, the Command line module (almmmost_cmdline.c) provides a command line for debugging and changing things on the server while it’s running, by presenting a command line when you press ctrl-C.  This lets you do things like print a list of open files, display generated disk parameters, change out disk images while the server is running, and sync disk image data/shut down the server safely.

Z80 software

Several programs were written in Z80 assembly to work with the special files.

FILEOUT copies a file to B:FILEOUT.SYS.  This was needed because PIP tries to copy to a temporary file, and then rename it, which doesn’t work correctly with special files.

IMAGEGET uses IMGGET.SYS to retrieve and display an image at an entered URL onto the screen on a TS-803 or TPC-I.

Logo displayed on a TS-803, and moon picture on a TPC-I

LYNXGET uses LYNXGET.SYS to retrieve and display a web page on the system console.

URLDISP and URLGET use URLGET.SYS get a file from a URL and display it to the console, or save it to disk, unprocessed.

URLDISP displaying HTML

Several other programs there were written at test programs to figure out how to display graphics on the systems.

Limitations

There are still some bugs in the software. It’s all single threaded, and locking will require some amount of re-writing the code to make it work.  I don’t have a real MmmOST system to compare this against, but all my test so far have produced results which at least seem reasonable to me.

Future work

Locking and print spool handling are the next two things that should be worked on.  Mailboxes/FIFOs to communicate between clients would also be nice.

One thing I noticed while working on this is that the OS images are rather large.  On the TS-803 and TPC-I, there is only about 75-80% of the system memory left for running application programs.  It’d be nice to re-write the OS to make it more efficient.  I’m not yet sure how compatible it would be, but something like TurboDOS compatibility would be nice, as it seems like a faster/better alternative to CP/M, which supported similar features to MmmOST.

Downloads

Everything can be downloaded from my GitHub repository.