CHICKEN Scheme 6.0 Bets Its Ecosystem on Unicode and R7RS

CHICKEN Scheme shipped 6.0.0 this week, nearly eight years after 5.0 landed in late 2018. That gap tells you something about the project's temperament: CHICKEN is the Scheme that compiles to portable C — using Henry Baker's "Cheney on the MTA" trick, where the C stack doubles as the garbage collector's nursery — and its maintainers treat major versions as once-a-decade events where they're allowed to break things. This time they broke two big ones: strings are now UTF-8 Unicode all the way down, and R7RS small is in the core rather than bolted on as an extension.

Both changes are overdue, both are done thoughtfully, and both carry the same bill: the egg ecosystem has to be ported, again, by a volunteer community that just finished doing this for CHICKEN 5.

Unicode, the pragmatic way

CHICKEN 5 strings were byte sequences with a Latin-1 squint. If you wanted Unicode you installed the utf8 egg, which shadowed the core string procedures — a leaky abstraction that worked until some other library imported the originals. CHICKEN 6 makes UTF-8 the internal representation for all strings and symbols, with port encodings selectable at open time (UTF-8 by default, Latin-1 and raw binary as options).

The interesting part is what the maintainers didn't do. There's no eager validation and no decode exception: invalid byte sequences are carried through transparently, smuggled in surrogate encodings, so reading a file with a few mangled bytes doesn't blow up your program. If that sounds familiar, it's Python's PEP 383 surrogateescape behavior — except applied everywhere, by default. Felix Winkelmann, CHICKEN's founder, has been explicit that the goal was to avoid costly conversions at the OS boundary and to keep garbage-in-garbage-out workflows working. That's the Go/Emacs school of Unicode pragmatism, not the Python 3 school of purity, and for a language whose main job is writing small portable Unix tools, it's the right call.

The known trade-off: string-ref on a variable-width encoding is O(n). CHICKEN mitigates with a per-string cache of byte-offset/code-point pairs, which handles the common sequential-access pattern fine. Code that does heavy random indexing into large strings will feel it, but idiomatic Scheme leans on ports and SRFI-style traversal anyway. There's a subtler hazard in the FFI: strings and symbols are now passed to C uncopied, so foreign code that scribbles on a buffer mutates the Scheme string in place. That's a real performance win — no marshalling copies — but it converts a class of harmless C bugs into heap corruption on the Scheme side. If you maintain an egg that wraps a C library, audit those call sites first.

R7RS stops being an egg

The r7rs egg is gone; every module in the R7RS small standard now ships in core, and define-library works out of the box. This matters more than it sounds. The portable-Scheme story has been slowly consolidating around R7RS small — Guile, Gauche, Chibi, Cyclone all speak it — and CHICKEN sitting behind an optional extension made it the awkward one at the table. Now a plain R7RS library should build on CHICKEN untouched, which is good news for anyone maintaining cross-implementation Scheme code, and quietly raises the pressure on the remaining holdouts.

The cost is churn in the import graph. A pile of identifiers moved from (chicken base) to (scheme base) — parameterize, vector-copy, case-lambda (now in (scheme case-lambda)), and friends — and blobs were replaced wholesale by R7RS bytevectors, #u8(...) syntax and all. Hex escapes in string literals now require the R7RS semicolon terminator, so "\x1b" must become "\x1b;" — the maintainers themselves admit this one is "particularly bad from a compatibility point of view." Old #ci/#cs case markers are out in favor of #!fold-case/#!no-fold-case. None of this is hard; all of it is tedious. The porting guide's honest summary is that when something's suddenly unbound, adding an import of (scheme base) usually fixes it.

The toolchain quietly grew up

Less headline-worthy but more day-to-day useful: the build system now uses a configure script instead of hand-edited makefile variables, Windows builds standardize on w64devkit (the bare-MinGW path is dead), and — the sleeper feature — the whole system can be built with Zig's zig cc as the C compiler and linker. Since CHICKEN's output is portable C, zig cc's bundled cross-compilation targets make "build my Scheme program for ARM Linux from a Mac" a realistic one-liner rather than a toolchain safari. For the run-Scheme-where-Scheme-doesn't-run crowd — which is CHICKEN's core constituency — that combination is genuinely new capability.

There are also new -merge-reusable-closures/-merge-shareable-closures optimization flags to cut allocations, process APIs now return proper process objects instead of raw PIDs, and the feathers debugger moved out of core into an egg.

Should you upgrade?

If you're starting a new project: yes, immediately. There's no reason to write new code against CHICKEN 5's byte-string semantics, and the R7RS alignment means what you write is more portable than it's ever been.

If you have a working CHICKEN 5 codebase: the language-level port is a day of mechanical fixes, but your real dependency is the egg ecosystem. Eggs must be explicitly ported to 6 and published to the new version's egg index, so the CHICKEN 6 pool starts small and fills in as maintainers get to it — the same J-curve the community climbed after the 4-to-5 transition. Check your dependency list against the ported eggs before scheduling the migration, not after.

The bigger picture: niche language implementations die by freezing, not by breaking. CHICKEN's community is small, its pace is glacial, and yet it keeps making the correct expensive decisions — the module overhaul in 5, now Unicode and standards convergence in 6. That's what three more decades of relevance looks like for a Scheme: not chasing features, just refusing to let the foundations rot.