Connecting a Real IBM 3270 Terminal to an Emulated Mainframe with OECFollow article
Building the Open Establishment Controller interface and using this to connect a 3270 terminal to an emulated IBM mainframe running VM/370.
In a previous post, we took a fresh look at IBM 3270 and noted how this family of terminals is particularly interesting, working as part of a highly integrated system that allowed a mainframe computer to scale and serve thousands of users. In this post, we now take a look at a rather neat open-source implementation of an “establishment controller” which is used to interface a 3270 series terminal with a mainframe. The OEC hardware is Arduino-based and in our case, we’ll be using this to interface with an emulated mainframe running on a Raspberry Pi 4.
As noted in the previous post, the 3270 data stream (protocol) can be run over a TCP/IP connection — aka TN3270 — and this is commonly used with mainframe computers today; unsurprisingly very few would use a real physical terminal, when much more convenient terminal emulation is available for Linux, Windows and Mac. However, in our case, we actually want to use a classic terminal and so this means interfacing with its coax connection, but on the other side, the OEC will use TN3270 for connecting to the emulated mainframe.
There are currently two versions of the coax interface: the original is an Arduino shield that uses hard-to-find 3270 transmitter and receiver ASICs, while the more recent v2 design uses an STM32 microcontroller together with an iCE40 FPGA. The 3270 ICs had been acquired some time ago before the v2 design was completed, so this is the version that we’ll be building here.
As is customary, we started out by soldering the smaller components, such as resistors.
Working up to larger components, such as the capacitors and pulse transformer. Given that the 3270 ICs are becoming quite rare, it was decided to socket these so that reuse would be possible.
Once assembled the shield was then fitted onto an Arduino Mega 2560.
At this point, we could now program the Arduino and the firmware sources are provided as a PlatformIO project. This makes use of VS Code and since we already have it installed, we just needed to open its Extension Manager, search for the PlatformIO IDE and select to install. Following which we could open the coax interface1 firmware project, select Build and then Upload.
Our coax terminal interface is now complete, but the OEC application itself is written in Python and we’ll run this on a Raspberry Pi 4 4GB, with the interface connected via USB.
OEC requires Python 3.8 or greater, while Raspbian is provided with Python 3.7, so we needed to install a more recent version. We started by installing the build dependencies, along with git.
$ sudo apt install git build-essential tk-dev libncurses5-dev libncursesw5-dev libreadline6-dev libdb5.3-dev libgdbm-dev libsqlite3-dev libssl-dev libbz2-dev libexpat1-dev liblzma-dev zlib1g-dev libffi-dev
Following which we could get the latest Python 3.x sources, build and install these.
$ wget https://www.python.org/ftp/python/3.9.5/Python-3.9.5.tar.xz $ tar xvf Python-3.9.5.tar.xz $ cd Python-3.9.5 $ ./configure --prefix=/usr/local/opt/python-3.9.5 $ make -j4 $ sudo make altinstall
To ensure that this would be in our path, we then edited ~/.bashrc and added:
Now on to installing OEC and we started by cloning the git repository and creating a Python virtual environment, before installing the required modules with pip.
$ git clone https://github.com/lowobservable/oec.git $ cd oec $ python3.9 -m venv VIRTUALENV $ source VIRTUALENV/bin/activate $ pip install -r requirements.txt
That’s it, we now have our 3270 coax interface and open establishment controller set up. Next let’s take a look at installing the Hercules mainframe emulator, followed by VM/370 operating system.
There are two main versions of the Hercules emulator: the original development branch aka Spinhawk, which is maintained by Roger Bowler and others, and the Hyperion fork from SoftDevLabs (SDL). The latter seemingly incorporates numerous new features, while the original might be considered to in more of a maintenance mode these days.
We decided to stick with the original version for now since this can simply be installed via the package manager.
$ sudo apt install hercules
VM/370 — or Virtual Machine Facility/370 — was first released in 1972 and is a reimplementation of the earlier CP/CMS operating system. The “370” part refers to the IBM System/370 range of mainframe computers, which were the successor to the groundbreaking System/360 family that was announced in 1964. The S/360 being notable for numerous features and not least of which that it was available in small and large, lower and high-performance models, which used the same instruction set, plus support for backwards compatibility with previous systems via emulation.
We take such things for granted nowadays, but the flexibility and backwards compatibility of S/360 was incredibly novel at the time. Similarly, VM incorporated numerous advanced features, providing an entire virtual machine, complete with address space and virtual devices, to each user that was logged on. Inside each VM it then being possible to IPL — Initial Processor Load in IBM parlance, i.e. “boot” — an instance of another operating system.
VM very much lives on today and provides a powerful virtualisation platform for IBM z architecture mainframes. However, the version that we are concerned with is of course much older and will have a subset of the current O/S features.
Early releases are considered open source/public domain and recently a community of hobbyists, many of which clearly veteran mainframe programmers, decided to take on maintenance and provide a Community Edition that is based upon an old public release. This may not be of much practical use in real-world applications today, but is certainly a lot of fun for those looking to gain hands-on experience with a bona fide mainframe operating system.
First, we downloaded the VM/370 Community Edition and extracted this.
$ wget http://www.vm370.org/sites/default/files/2021-05/VM370CE.V1.R1.1.zip $ unzip VM370CE.V1.R1.1.zip $ cd VM370CE.V1.R1.1
The VM/370 distribution is provided as a number of virtual disk images and these are expected to be configured within the mainframe, or rather emulator, in a certain way. Hercules is set up via a config file and a ready-made one, vm370ce.conf, is provided. However, it appears that the current release assumes you are using the SDL version of Hercules and this has some config file differences, so we needed to replace the “ARCHLVL” keyword in the config file with “ARCHMODE”. We also needed to change the value of “CNLSPORT” to 23, since this is the TCP port number that OEC will expect to connect to.
We could now start the emulator with:
$ sudo hercules -f vm370ce.conf
This is actually not ideal and we should probably have used netcap to allow the Hercules executable to use privileged ports without being run as root.
Once Hercules starts it then reads a script that IPLs VM.
From the command prompt, we can press ESC to toggle to a virtual operator console as shown above, where amongst other things we can see the emulated device list.
In another window we can then launch OEC, specifying the TN3270 mode and a host address of localhost, since the emulated mainframe is running on the same Raspberry Pi. Here we can see that after an initial error, the terminal has been attached and its capabilities printed out.
Now we see our first output on the actual 3270 terminal!
We hit enter and are greeted with the VM/370 logon screen.
After an initial failed logon attempt, we then successfully logon as CMSUSER. Following which enter “ipl cms” to load the Conversational Monitor System — a single user O/S — inside our VM. The text that follows lists the disks that we have available, plus a help welcome message. From here we can explore the commands available to us, list, copy, edit and delete files etc. and make use of the various compilers which are provided as part of VM/370 CE.
Just for fun
Above can be seen a much more modern, late 1990s, terminal — an IBM 3483, the last produced in the 3270 family. This time OEC has been started with:
$ python -m oec /dev/ttyACM0 vt100 /bin/sh -l
What this does is to operate OEC in VT100 mode and spawn a Linux shell on the Raspberry Pi, attaching this to the terminal session. Above we can see that from the shell we’ve then run “top”.
Just for fun, we decided to try installing the lynx text-based web browser and here this can be seen being used to navigate the RS website, back on the older IBM 3179 terminal. No images obviously, but looks pretty good considering!
A huge thanks to Andrew Kay, the author of OEC and designer of the coax interfaces, for publishing the software and hardware designs, making it possible to connect a real vintage 3270 terminal to an emulated mainframe — or indeed a real one, should you have access to one. It’s been a lot of fun building the interface and finally getting to use a real terminal with Hercules and VM/370, and now I look forward to sitting in front of phosphor for some distraction-free exploration of VM, CMS and classic mainframe architecture.