Dec. 9, 2022, 5:22 a.m.
11.5 KB
You are viewing the rendered version of this paste. Click "Source" to see the source.

Table of Contents

  1. So Long and thanks for all the standard-fish
    1. History
    2. Production Racket: What would have to change?
      1. Cater towards CI/CD
      2. Protect users from themselves
      3. How the heck do signature and units work
      4. Consider the role of documentation
      5. There needs to be transparency
    3. What's next?

TODO So Long and thanks for all the standard-fish

Hey everyone, I wanted to share with you a realization I had about my time with Racket - it's time to move on.


Back in 2015-2016 when attending community college, I had discovered Scheme via the R5RS paper, and was impressed with the conciseness of the language. After trying out a few implementations, it occurred to me the majority of scheme implementations are not beginner friendly, missing documentation, and hard to imagine how to use as a prospective user. Then I discovered Racket. It was like some sort of drug - I couldn't get enough of it. Racket does a lot of things right. Contracts or gradual typing means you usually have unambiguous runtime errors without any typing problems. Pattern matching and for loops reduce code complexity significantly. Most importantly, s-expression syntax is dead simple and consistent - as long as you can google the first form in each list (pair of parenthesis), you can understand a Racket program.

I toiled away for a couple years during university reading Racket documentation, using it for IEEE-CS (Computer Science Club) HackerRank competitions, even building a few tools I still use today. Eventually I started contributing a patch here and there to Racket - mainly to fix weird stuff like build failures or documentation typos. I also found opportunity to write a riposte-like clone in Racket for my capstone project and began uploading packages to

Another couple years pass and finally I understand macros enough to use them on my own. These aren't easy to understand - the docs don't really help in this regard. But I persisted. Soon after I started writing languages in racket such as TinyBasic and subby (Subnautica item description language).

While this was all going on, I also started a project to build out tutorials or howtos for Racket. You see Racket suffers from a bunch of smart people who themselves are deeply in the clutches of "The Curse of Knowledge".1 They don't quite understand what it takes for a beginner to pick up Racket, outside of a pedagogical setting (classroom). I even bought the domain with the idea of calling it the Practical Racket Wiki - I wanted people to have resources to do stuff quickly in racket without toiling through a metric crapton of documentation pages. Sometimes a productive dev needs to focus on the important stuff, not how to use every single API in a language correctly.2

Then I had a realization. It's been 7 years since I first started using Racket and I'm still not being paid for research, writing software, nor making Racket content. I'm not getting rewarded for the time investment. I need to be realistic - Racket in production is a pipe dream.

This is me stating: I've had enough - I might use Racket for toy parsers or other curiousities, but you'll never find me using Racket in a serious setting. It's counterproductive and high risk.

Production Racket: What would have to change?

Racket seems to work well for the research and pedagogical communities. I understand their needs may be fairly different than a production-seeking user. The leadership is set up to cater towards a different community than the one I was seeking to build.

Cater towards CI/CD

Racket runs in docker, but there's more work to be done. Racket does not have a distinction between documentation dependencies and general dependencies. This frequently causes Docker builds to take 10-20 minutes and use 2-5 times the disk space. This is caused by packages bundling documentation. Want to document your package and don't know any better, bam, you've made a package that will ruin some production users day. And the Racket community describes this as a "human problem". The solution is to split out packages so documentation and library code are separated. Alternately one may use binary builds (which are built for a few particular ISA/OS combinations and only for the latest Racket release). The usual root cause is a dependency pulling in racket-doc or racket-index when using a minimal racket distribution. I remember running Racket on a Pentium 4, with the horror of watching raco pkg install --auto some-random-lib causing the laptop to spin for 10 hours, only to fail due to a build failure caused by Racket's poor support for Alpine linux.

I even voiced this and was flatly told contributions welcome. Thanks, I will totally invest a bunch of time guessing how the racket package server works because it has no development instructions nor is there any public documentation on how to automate deploys of this server process (such as an ansible playbook or whatever). So I'd pretty much be at the mercy of asking upstream a bunch of questions and they're too busy to care about these "trivial" problems.

Protect users from themselves

Racket has a blanket policy of never break APIs. This sounds super cool in theory. Unfortunately I've seen how bad this can be in legacy APIs. I worked on OpenEdge Advanced Business Language codebases for about 8 months at one job. Now that, is the epitomy of bad API design and legacy carry over. I see flashes of that in Racket.

Instead of unifying a way to access and manipulate container types, we have APIs for structs, hashes, sets, lists, vectors. There are libraries to do this, but nobody pushes them into the core language. This leads to a ton of frustration for beginners. Please note there are excellent ways to unify these and you can see them on the racket documentation - but nobody is pushing this from a top-down perspective - so everyone is taught the least practical way to write racket code for containers.

Racket currently doesn't respect Unix permission semantics out of the box. This came as a shock because it seemed natural and inherit if you make a programming language for Unix, it works like it should. This seems like a small issue, but what else am I going to discover the more I think about file APIs?

Recently I discovered Racket's web server has a way to lookup headers in a case sensitive fashion and a case insensitive fashion. The former (case sensitive) is abjet to the RFC. The Racket devs made this into a human problems by adding a caveat to the documentation, leaving it at that. It's a techincal problem, just unify the APIs and protect the user from themself.

There's also stale APIs I've never seen used anywhere, such as Racket's generator API. In short it's a way to build iterable objects in Racket without using the standard scheme stream API. This is a distraction to anyone reading the documentation, and may even mislead readers into believing they should use it over stream or a Racket class.

There's countless other sharp edges in the APIs like this. For the most part it's not so bad, however, it shows pattern of complancency of technical debt in API design.

How the heck do signature and units work

It's been 7 years and while I have played around with Racket's signature/units API which is used for dynamic code loading. It's very confusing and hard to reason about. It isn't understandable from the documentation alone. This bites because you'll need this to implement robust code hotloading and plugin support in production.

I can't imagine getting other devs to understand this at all. Maybe complete rewrite is warranted?

Consider the role of documentation

Racket is renowned for its thorough technical documenation. If you need to look up an API, it'll have documentation in most cases. I get confused because there isn't really a guide or tutorial for racket. There's a super short and sweet one and there's a guide which reads more like golang's tour (which is certaintly not a guide either).

Working off this framework for documentation, I wonder if maybe there's a misunderstanding of what it means to write tutorials, howtos, or guides. Using that framework, I believe the vast majority of racket docs are truly reference material, not actually tutorials, guides, or how-tos.

I'd focus here, if there's truly a desire for adoption.

There needs to be transparency

For the most part, without exception, I don't understand the Racket decision making process. There's not many policy documents nor a RFC process for Racket itself. I don't feel like I have a vote in how Racket grows, maybe I would have a say if I joined one of the aformentioned academic cliques.

I don't feel incentivized to contribute upstream for the most part - they aren't particularly easy to work with, and sometimes not the most personable - we all have our days. In other FOSS projects (VLC, FreeBSD, Homebrew, nixpkgs) I've been blessed with a sort of advocate who helps guide new contributors. Perhaps Racket could use some developer advocates - but that runs the risk of giving beginners a voice, which I don't think the Racket people want. They are used to the complancey of complete creative control.

The majority of official racket codebases don't even have a readme or guide on how to contribute. It's a hurdle to know how to get started, let alone put in energy on a PR that might not be well recieved.

What's next?

I will mark most of the code I have in Racket as "no maintenance intended". Here's a badge!

The opportunity cost in my Racket time investment has been significant. I've been operating off the presumption I could either create a SaaS product around Racket or find work doing Racket for somebody else. Neither has come to fruition, so I'm shelving most of my Racket work starting now. If you want to take over maintainership and direction of any of my racket codebases, please reach out, I would be more than willing to transfer ownership.

Now I am presently learning more about the Clojure ecosystem with the express idea of shipping stuff in Clojure. Sure it doesn't have excellent error messages nor a very good contribution process, however, it does have a community of folks who have battle-tested this language and runtime in production. To be fair I have a lot of time spread across many languages so it's not too important to pick up Clojure, however, I think it'd be a fun technology to use in employment.

P.S. I am kind of broke working freelance. If you want some devops work done, backend work done, or R/D I'm your guy.

If you're interested in a blog post detailing some of my concerns with Racket in production and the attitude of the core team with this use case, please let me know on Twitter, Mastodon, or email. Otherwise I'll quietly show myself the door and let the past be in the past.



2 I also argue that if it is possible to misuse an API, it's shit and you should fix it thanks.