Blinkenlights

This project contains two programs:

blink is a virtual machine that runs x86-64-linux programs on different operating systems and hardware architectures. It's designed to do the same thing as the qemu-x86_64 command, except that

1. Blink is 221kb in size (115kb with optional features disabled), whereas qemu-x86_64 is a 4mb binary.

2. Blink will run your Linux binaries on any POSIX system, whereas qemu-x86_64 only supports Linux.

3. Blink goes 2x faster than qemu-x86_64 on some benchmarks, such as SSE integer / floating point math. Blink is also much faster at running ephemeral programs such as compilers.

blinkenlights is a terminal user interface that may be used for debugging x86_64-linux or i8086 programs across platforms. Unlike GDB, Blinkenlights focuses on visualizing program execution. It uses UNICODE IBM Code Page 437 characters to display binary memory panels, which change as you step through your program's assembly code. These memory panels may be scrolled and zoomed using your mouse wheel. Blinkenlights also permits reverse debugging, where scroll wheeling over the assembly display allows the rewinding of execution history.

Getting Started

We regularly test that Blink is able run x86-64-linux binaries on the following platforms:

• Linux (x86, ARM, RISC-V, MIPS, PowerPC, s390x)
• macOS (x86, ARM)
• FreeBSD
• OpenBSD
• Cygwin

Blink depends on the following libraries:

• libc (POSIX.1-2017 with XSI extensions)

Blink can be compiled on UNIX systems that have:

• A C11 compiler with atomics (e.g. GCC 4.9.4+)
• Modern GNU Make (i.e. not the one that comes with XCode)

The instructions for compiling Blink are as follows:

./configure
make -j4
doas make install  # note: doas is modern sudo
blink -v
man blink

Here's how you can run a simple hello world program with Blink:

blink third_party/cosmo/tinyhello.elf

Blink has a debugger TUI, which works with UTF-8 ANSI terminals. The most important keystrokes in this interface are ? for help, s for step, c for continue, and scroll wheel for reverse debugging.

blinkenlights third_party/cosmo/tinyhello.elf

Alternative Builds

For maximum tinyness, use MODE=tiny, since it makes Blink's binary footprint 50% smaller. The Blink executable should be on the order of 200kb in size. Performance isn't impacted. Please note that all assertions will be removed, as well as all logging. Use this mode if you're confident that Blink is bug-free for your use case.

make MODE=tiny
strip o/tiny/blink/blink
ls -hal o/tiny/blink/blink

Some distros configure their compilers to add a lot of security bloat, which might add 60kb or more to the above binary size. You can work around that by using one of Blink's toolchains. This should produce consistently the smallest possible executable size.

make MODE=tiny o/tiny/x86_64/blink/blink
o/third_party/gcc/x86_64/bin/x86_64-linux-musl-strip o/tiny/x86_64/blink/blink
ls -hal o/tiny/x86_64/blink/blink

If you want to make Blink even tinier (more on the order of 120kb rather than 200kb) than you can tune the ./configure script to disable optional features such as jit, threads, sockets, x87, bcd, xsi, etc.

./configure --disable-all --posix
make MODE=tiny o/tiny/x86_64/blink/blink
o/third_party/gcc/x86_64/bin/x86_64-linux-musl-strip o/tiny/x86_64/blink/blink
ls -hal o/tiny/x86_64/blink/blink

The traditional MODE=rel or MODE=opt modes are available. Use this mode if you're on a non-JIT architecture (since this won't improve performance on AMD64 and ARM64) and you're confident that Blink is bug-free for your use case, and would rather have Blink not create a blink.log or print SIGSEGV delivery warnings to standard error, since many apps implement their own crash reporting.

make MODE=rel
o/rel/blink/blink -h

You can hunt down bugs in Blink using the following build modes:

• MODE=asan helps find memory safety bugs
• MODE=tsan helps find threading related bugs
• MODE=ubsan to find violations of the C standard
• MODE=msan helps find uninitialized memory errors

You can check Blink's compliance with the POSIX standard using the following configuration flags:

./configure --posix  # only use c11 with posix xopen standard

If you want to run a full chroot'd Linux distro and require correct handling of absolute symlinks, displaying of certain values in /proc, and so on, and you don't mind paying a small price in terms of size and performance, you can enable the emulated VFS feature by using the following configuration:

./configure --enable-vfs

Testing

Blink is tested primarily using precompiled binaries downloaded automatically. Blink has more than 700 test programs total. You can check how well Blink works on your local platform by running:

make check

To check that Blink works on 11 different hardware $(ARCHITECTURES) (see Makefile), you can run the following command, which will download statically-compiled builds of GCC and Qemu. Since our toolchain binaries are intended for x86-64 Linux, Blink will bootstrap itself locally first, so that it's possible to run these tests on other operating systems and architectures.

make check2
make emulates

Production Worthiness

Blink passes 194 test suites from the Cosmopolitan Libc project (see third_party/cosmo). Blink passes 350 test suites from the Linux Test Project (see third_party/ltp). Blink passes 108 of Musl Libc's unit test suite (see third_party/libc-test). The tests we haven't included are because either (1) it wanted x87 long double to have 80-bit precision, or (2) it used Linux APIs we can't or won't support, e.g. System V message queues. Blink runs the precompiled Linux test binaries above on other operating systems too, e.g. Apple M1, FreeBSD, Cygwin.

Reference

The Blinkenlights project provides two programs which may be launched on the command line.

blink Flags

The headless Blinkenlights virtual machine command (named blink by convention) accepts command line arguments per the specification:

blink [FLAG...] PROGRAM [ARG...]

Where PROGRAM is an x86_64-linux binary that may be specified as:

1. An absolute path to an executable file, which will be run as-is
2. A relative path containing slashes, which will be run as-is
3. A path name without slashes, which will be $PATH searched

The following FLAG arguments are provided:

• -h shows help on command usage

• -v shows version and build configuration details

• -e means log to standard error (fd 2) in addition to the log file. If logging to only standard error is desired, then -eL/dev/null may be used.

• -j disables Just-In-Time (JIT) compilation, which will make Blink go ~10x slower.

• -m disables the linear memory optimization. This makes Blink memory safe, but comes at the cost of going ~4x slower. On some platforms this can help avoid the possibility of an mmap() crisis.

• -0 allows argv[0] to be specified on the command line. Under normal circumstances, blink cmd arg1 is equivalent to execve("cmd", {"cmd", "arg1"}) since that's how most programs are launched. However if you need the full power of execve() process spawning, you can say blink -0 cmd arg0 arg1 which is equivalent to execve("cmd", {"arg0", "arg1"}).

• -L PATH specifies the log path. The default log path is blink.log in the current directory at startup. This log file won't be created until something is actually logged. If logging to a file isn't desired, then -L /dev/null may be used. See also the -e flag for logging to standard error.

• -s enables system call logging. This will emit to the log file the names of system calls each time a SYSCALL instruction in executed, along with its arguments and result. System calls are logged once they've completed. If this option is specified twice, then system calls which are likely to block (e.g. poll) will be logged at entry too. If this option is specified thrice, then all cancellation points will be logged upon entry. System call logging isn't available in MODE=rel and MODE=tiny builds, in which case this flag is ignored.

• -Z will cause internal statistics to be printed to standard error on exit. Stats aren't available in MODE=rel and MODE=tiny builds, and this flag is ignored.

• -C path will cause blink to launch the program in a chroot'd environment. This flag is both equivalent to and overrides the BLINK_OVERLAYS environment variable. Note: This flag works especially well if you use ./configure --enable-vfs.

blinkenlights Flags

The Blinkenlights ANSI TUI interface command (named blinkenlights by convention) accepts its command line arguments in accordance with the following specification:

blinkenlights [FLAG...] PROGRAM [ARG...]

Where PROGRAM is an x86_64-linux binary that may be specified as:

1. An absolute path to an executable file, which will be run as-is
2. A relative path containing slashes, which will be run as-is
3. A path name without slashes, which will be $PATH searched

The following FLAG arguments are provided:

• -h shows help on command usage

• -v shows version and build configuration details

• -r puts your virtual machine in real mode. This may be used to run 16-bit i8086 programs, such as SectorLISP. It's also used for booting programs from Blinkenlights's simulated BIOS.

• -b ADDR pushes a breakpoint, which may be specified as a raw hexadecimal address, or a symbolic name that's defined by your ELF binary (or its associated .dbg file). When pressing c (continue) or C (continue harder) in the TUI, Blink will immediately stop upon reaching an instruction that's listed as a breakpoint, after which a modal dialog is displayed. The modal dialog may be cleared by ENTER after which the TUI resumes its normal state.

• -w ADDR pushes a watchpoint, which may be specified as a raw hexadecimal address, or a symbolic name that's defined by your ELF binary (or its associated .dbg file). When pressing c (continue) or C (continue harder) in the TUI, Blink will immediately stop upon reaching an instruction that either (a) has a ModR/M encoding that references an address that's listed as a watchpoint, or (b) manages to mutate the memory stored at a watchpoint address by some other means. When Blinkenlights is stopped at a watchpoint, a modal dialog will be displayed which may be cleared by pressing ENTER, after which the TUI resumes its normal state.

• -j enables Just-In-Time (JIT) compilation. This will make Blinkenlights go significantly faster, at the cost of taking away the ability to step through each instruction. The TUI will visualize JIT path formation in the assembly display; see the JIT Path Glyphs section below to learn more. Please note this flag has the opposite meaning as it does in the blink command.

• -m enables the linear memory optimization. This makes blinkenlights capable of faster emulation, at the cost of losing some statistics. It no longer becomes possible to display which percentage of a memory map has been activated. Blinkenlights will also remove the commit / reserve / free page statistics from the status panel on the bottom right of the display. Please note this flag has the opposite meaning as it does in the blink command.

• -t may be used to disable Blinkenlights TUI mode. This makes the program behave similarly to the blink command, however not as good. We're currently using this flag for unit testing real mode programs, which are encouraged to use the SYSCALL instruction to report their exit status.

• -L PATH specifies the log path. The default log path is $TMPDIR/blink.log or /tmp/blink.log if $TMPDIR isn't defined.

• -C path will cause blink to launch the program in a chroot'd environment. This flag is both equivalent to and overrides the BLINK_OVERLAYS environment variable.

• -s enables system call logging. This will emit to the log file the names of system calls each time a SYSCALL instruction in executed, along with its arguments and result. System calls are logged once they've completed. If this option is specified twice, then system calls which are likely to block (e.g. poll) will be logged at entry too. If this option is specified thrice, then all cancellation points will be logged upon entry. System call logging isn't available in MODE=rel and MODE=tiny builds, in which case this flag is ignored.

• -Z will cause internal statistics to be printed to standard error on exit. Each line will display a monitoring metric. Most metrics will either be integer counters or floating point running averages. Most but not all integer counters are monotonic. In the interest of not negatively impacting Blink's performance, statistics are computed on a best effort basis which currently isn't guaranteed to be atomic in a multi-threaded environment. Stats aren't available in MODE=rel and MODE=tiny builds, and this flag is ignored.

• -z [repeatable] may be specified to zoom the memory panels, so they display a larger amount of memory in a smaller space. By default, one terminal cell corresponds to a single byte of memory. When memory has been zoomed the magic kernel is used (similar to Lanczos) to decimate the number of bytes by half, for each -z that's specified. Normally this would be accomplished by using CTRL+MOUSEWHEEL where the mouse cursor is hovered over the panel that should be zoomed. However, many terminal emulators (especially on Windows), do not support this xterm feature and as such, this flag is provided as an alternative.

• -V [repeatable] increases verbosity

• -R disables reactive error mode

• -H disables syntax highlighting

• -N enables natural scrolling

Recommended Environments

Blinkenlights' TUI requires a UTF-8 VT100 / XTERM style terminal to use. We recommend the following terminals, ordered by preference:

• KiTTY (Linux)
• PuTTY (Windows)
• Gnome Terminal (Linux)
• Terminal.app (macOS)
• CMD.EXE (Windows 10+)
• PowerShell (Windows 10+)
• Xterm (Linux)

The following fonts are recommended, ordered by preference:

• PragmataPro Regular Mono (€59)
• Bitstream Vera Sans Mono (a.k.a. DejaVu Sans Mono)
• Consolas
• Menlo

JIT Path Glyphs

When the Blinkenlights TUI is run with JITing enabled (using the -j flag) the assembly dump display will display a glyph next to the address of each instruction, to indicate the status of JIT path formation. Those glyphs are defined as follows:

• or space indicates no JIT path is associated with an address

• S means that a JIT path is currently being constructed which starts at this address. By continuing to press s (step) in the TUI interface, the JIT path will grow longer until it is eventually completed, and the S glyph is replaced by *.

• * (asterisk) means that a JIT path has been installed to the adjacent address. When s (step) is pressed at such addresses within the TUI display, stepping takes on a different meaning. Rather than stepping a single instruction, it will step the entire length of the JIT path. The next assembly line that'll be highlighted will be the instruction after where the path