Benchmarks for PDP-8 Emulators

In an alt.sys.pdp8 posting, Klemens Krause suggested the following benchmarks for PDP-8 emulators:

Benchmarks without I/O
A program which calculates 1500 functions (squareroot, sine and logarithm) and stores the results in a two-dimensional array. Times are stopped between the "START" and "END" message on the screen, without the compile time and the 17 lines at the end of the programs. The program is translated to three languages:
1. BASIC, with 23-bit accuracy and use of the EAE
2. FORTRAN-IV with 27-bit
3. PASCAL with 35-bit (plus 1-bit for the sign)
Benchmarks with file I/O (on a RK05 disk)
The PASCAL.PA file was the last file on DSK: and I squished the disk before assembling or editing, so that the head movement is approximately constant.
1. Assembling the PASCAL-S compiler (source file of 9.000 lines and 144 kB) with PAL-8 without listing:
```
.R PAL8
*PASCAL<PASCAL/K
```
2. Assembling the same compiler, with producing a 328 kB list file:
```
.R PAL8
*PASCAL,PASCAL<PASCAL/K
```
3. Reading and writing the PASCAL.PA source file with the OS/8-editor EDIT.SV.
```
.ED PASCAL.PA
#R
#E
.
```

These are the hand measured times (in seconds) given by Klemens Krause for a hardware PDP-8/E and his PDP-8/E simulator running on PC hardware of the year 1994:

	1.a	1.b	1.c	2.a	2.b	2.c

Hardware PDP-8/E	16	18	44	75	192	91
486/33 (EISA)	6	8	15	28	79	45
486/33 (EISA) with RAMDISK	6	8	15	25	55	35
486DLC/40	5	7	15	30	105	55
486DLC/40 with RAMDISK	5	7	15	24	57	36
386/28	26	23	81	n/a	n/a	n/a
386/40	8	9	20	37	120	65
386/40 with RAMDISK	8	9	20	32	73	50
386SX/25	30	36	78	122	279	183

The benchmarks were performed with the Macintosh PDP-8/E Simulator, and the following hand measured times (in seconds) were perceived. The first column specifies the Macintosh and in parenthesis the system software version, the runtime environment of the simulator (Classic Mac OS with Motorola 68K or PowerPC processor or Mac OS X with Carbon natively on PowerPC-based Macs or under “Rosetta” emulation on Intel-based Macs) and the version of the simulator used for the test:

	1.a¹	1.b	1.c	2.a	2.b	2.c

Macintosh Plus (2.0.1/68K/1.5)	n/a	1070	2385	n/a	n/a	n/a
Colour Classic (7.1/68K/1.1)²	n/a	327	637	930	2060	1265
Colour Classic (7.1/68K/1.2)⁵	n/a	410	840	1150	2550	1635
SE/30 (6.0.4/68K/1.1)	n/a	173	633	540	825	740
SE/30 (7.1/68K/1.1)	n/a	180	633	553	1246	780
IISi (6.0.8/68K/1.1)	n/a	140	315	440	992	615
IISi (7.1/68K/1.1)	n/a	150	401	615	1180	760
Quadra 650 (7.5.1/68K/1.0)	n/a	78³	80	115	200	160
Quadra 650 (7.5.1/68K/1.1)	n/a	25	52	76	170	106
PowerMac 7100/66 (7.5.1/68K/1.1)	n/a	87	187	258	580	348
PowerMac 7100/66 (7.5.1/PPC/1.1)	n/a	11	22	32	71	45
Performa 6200/75 (7.5.1/PPC/1.1)	n/a	11	21	33	73	46
PowerMac 5400/180 (7.5.5/PPC/1.1)	n/a	4	8	10	23	14
PowerMac 5400/180 (8.0/68K/1.2)	n/a	14	24	37	82	52
PowerMac 5400/180 (8.5.1/PPC/1.2)⁴	n/a	4 (13)⁷	8 (26)	10 (41)	23 (85)	15 (55)
PowerMac 5400/180 (8.6/PPC/1.3)⁶	n/a	18	44	79	191	90
PowerMac G3/300 (8.6/PPC/1.2)	n/a	2	4	5	11	7
PowerMac G4/400 (9.0.4/PPC/1.4)	n/a	1	2	4	8 (10)	5 (6)
PowerMac G4 Cube/450 (9.0.4/PPC/1.4)⁸	n/a	1	2	4 (6)	7 (13)	5 (8)
PowerMac G4 Cube/450 (X PB/Carbon/1.4)	n/a	1	2 (3)	4 (6)	7 (12)	5 (8)
PowerMac G4 Cube/450 (10.1/Carbon/1.4)⁹	n/a	1 (3)	2 (9)	4 (14)	8 (26)	5 (15)
iBook/500 (10.1/Carbon/1.4)	n/a	2 (11)	3 (21)	5 (30)	10 (33)	7 (45)
MacBook Pro/2.16 (10.4/Rosetta/Carbon/1.5)¹⁰	1(1)	2(2)	2(3)	4(4)	7(9)	4(5)

Footnotes

The BASIC benchmarks could not be performed due to malfunctioning of the OS/8 BASIC interpreter.
The initial release of the PDP-8/E Simulator was developed on Colour Classic. (Since July 1997, the Power Macintosh 5400 was used.)
This test was performed without use of the EAE because a bug in the EAE implementation of version 1.0 of the simulator prevented the functioning of the FORTRAN mathematical library in EAE mode.
This benchmark was performed using the PowerPC versions of the I/O device plugins. When using the 68K versions of the plugins, the measured times were nearly the same (less than one second longer). The KC8-EA Console plugin was not loaded for this benchmark. With the KC8-EA Console plugin loaded and locked (display lights off), the times are nealy the same (less than one second longer).
A reason for the significant slowdown of version 1.2 of the simulator compared to version 1.1 could not be found. The I/O device plugin interface itself causes nearly no overhead, only the plugins IOT instructions have to handle the A4/A5 world switches and access of the PDP-8 data structure over a pointer. But the .FT and .PS benchmarks involve no I/O operations. Maybe the 68K code optimization of the newer C compiler is not as good as in previous versions.
With the CPU and all I/O devices running with real time speed. Note that these are nearly the same times Klemens Krause measured with a hardware PDP-8/E.
In brackets: times with KC8-EA unlocked (display lights turned on). All other times are measured without KC8-EA or with locked KC8-EA (display lights turned off).
See footnote 7. Since 13. September 2000 (Mac OS X Public Beta release date), the PDP-8/E Simulator is hosted on the Cube. The PDP-8/E with console lights turned on might be slower on the 450 MHz G4 Cube than on a 400 MHz G4 because the Apple 17" Studio Display has a higher screen refresh rate (up to 154 Hz) and colour depth (32 bits) than the VGA monitor of the G4 400 MHz, and the KC8-EA refreshs the lights with the screen refresh rate.
The remarkably slower times with console lights turned on measured with Mac OS X final compared to the public beta may be caused by the fact that screen double buffering was not implemented in Mac OS X Public Beta.
Note that on the Intel-based MacBook Pro, the PDP-8/E Simulator itself is emulated by the “Rosetta” PowerPC emulator of Mac OS X, so the PDP-8/E is “double-emulated” on the Intel Core Duo processor (only one of the two cores is used).