May I recommend… understanding Emacs's patterns

May I recommend… understanding Emacs's patterns

May I recommend… understanding Emacs's patterns

1. About

emacs-carnival-may-banner.jpeg

Figure 1: JPEG produced with OpenAI gpt-image-1

What's in a name? That which we call Emacs
By any other name, works just as well;
Our editor could, were it not an Emacs call'd,
Retain those perfect patterns which it owes
Without that title. Emacsen, doff thy names,
And for those names, which are no part of thee,
Take all my code.

The above, unless I've completely forgotten my Shakespeare, was a soliloquy from Juliet Consulet regarding Romeo Montagnu

"Emacs is an Effortless Bloom," or at least, this is what my brain has been screaming into its own void for the past few months….

I have recently been publishing posts to my blog as part of a composite series on the usage patterns that make Emacs powerful to its most fluent users. I don't know if "Emacs is an Effortless Bloom" will be the name of the series when all is said and done….

Maybe it's more of a mantra, or perhaps even a haunting brain spasm at the moment. At least, this phrase has been popping involuntarily into my mind since I started this series, and I think it's worth thinking about why.

In this post, I'll discuss, among others, two core patterns that make Emacs easy and powerful, Incremental Completing Read and recognize-dispatch.

If you find the prose herein flowery, it's because it pertains to flowers πŸ˜‰.

2. "Effortless"

My goal of the series is to promote Emacs usage by conveying how and why it makes computer use easier. I feel Emacs gets a bad rap in the broader society of builders because there is a common notion that it's arcane and complicated and, therefore, difficult to use. I do believe this notion is a false one, and much of my writing and talking about Emacs revolves around precisely why Emacs makes using a computer easy.

When I tell folks "I use Emacs", I typically get some kind of "Wow, good for you" or "that's impressive!" or "I wish I could use Emacs, but I never had the time to grok it". Despite the flattery, I always feel dejected by these comments, especially as a lonely advocate for its use and a firm believer that using Emacs is easy.

It is largely this misconception that motivated me to write my series, and hopefully in doing so I can dispel the notion that Emacs is some nuclear wand only to be brandished by the lisp era's most considerate wizards.

3. "Bloom"

This one is not self-explanatory, and the word came to me before its justification… it just felt 'right'. As I continue to think critically about Emacs's patterns and why they make computer use effortless, I can attribute a few significances to this word.

3.1. Bloom as in Blossom

First, when I started using Emacs, it really did make me feel like I had come into my own as a builder.

I had a somewhat unconventional entry point to Emacs. I had just learned Visual Basic for Applications (VBA) in order to automate clinical research workflows at a research institution. This was my first job, and my first foray into the world of automating via programming. Learning VBA was an almost indescribable stepwise increase in my power as a computer user and builder of clinical research workflows. But that ability lost its leverage quickly….

As the data practice in our research firm outgrew the memory and performance constraints of Excel + VBA, I found that I needed to learn 3 new languages to support our operations: SQL for data extraction from our data warehouse, Python for automated workflows, and the researcher-friendly R for analysis and predictive modelling.

With this polyglot burden came another one: I needed to learn 3 new GUIs to be able to write and execute the programs I'd written in these 3 languages. Not only was this daunting, it was also uncomfortable. My glorious unified experience of doing everything in VBA now had the friction of context switching between different GUIs, all with different usage paradigms, patterns, and keybindings.

I recognized what underpinned my dread in this brief but tumultuous phase: the incidental complexity of adding languages to my toolbelt scaled somewhat worse than linearly with the number of languages. For every 1 new programming language I wanted to speak, I would have to learn at least 1 new interface modality to support that. I knew I was good at learning languages (and bad at learning GUIs), so how could I escape this friction?

As if from some theophany, Emacs came to me in an article I was reading about how Unix works. It was mentioned off-hand, in a single sentence, but with a compelling label as the 'most powerful, programmable editor'.

Huh… I thought "if I (and indeed anyone else) can program the editor, maybe I can work with all these languages and runtimes in a single environment". After watching dozens of Emacs demos on YouTube, I became convicted that this was indeed the case. Shoutouts are well deserved here for Magnar Sveen's Emacs Rocks, Mike Zamansky's Using Emacs, and Aaron Bieber's (no relation) very brave pitch on Emacs that he delivered at a vim conference.

Before I had seen the light of Emacs, I felt uncomfortably bounded and acutely constrained by the possibly infinite number of new tools I would need to grok in order to become a polyglot. This infinite expanse of tooling came with an ironic feeling of claustrophobia. But with Emacs, those bounds seemed to disappear, and I felt like my mind was able to 'blossom' in the fertile soil of Emacs's rich and endlessly configurable environment.

3.2. Bloom as in the Roots Fan-In, the Petals Fan-Out

Picture a rose, a purple one if you could (as we are talking about Emacs here).

On one end, the rose has roots, pulling in nutrients and hydrogen-dioxide through a complex and ever growing tendrillic network. At the other, we see the rose's beautiful flowering of petals: the outcome of all this rose's growth, the attractor of admiring eyes, and the basis for proliferation through pollination. Emacs's patterns, architecturally, remind me of this kind of flowering because we can see that Emacs fans-in and fans-out in the same beautiful and life-giving ways as our purple rose's roots and petals do.

Consider the buffer. Anything you look at in Emacs (no need to enumerate, I literally mean everything) is a buffer of characters with text properties layered on top. I'm simplifying a little bit, but that's pretty much it. That standardization is the core data model from which everything else grows.

The roots of the rose look the same whether they're pulling minerals from clay, soil, or sand (it's all just water and ions in the end). In Emacs, any thing is pulled into just characters and properties in a buffer (fan-in). The corollary is that every editing primitive you learn β€” search, kill, mark, narrow, replace, undo, blah blah blah β€” works against every piece of content you'll ever look at in Emacs, because nothing is special with respect to this data model, and no exceptions will be found. The characters + text props is the unifying idea, and the buffer in which they reside offers a universal vocabulary with which you can speak into Emacs whatever your heart desires, Juliet πŸ˜‰.

On top of that vocabulary sits Elisp, and on top of Elisp sit the applications. Whatever you can likely imagine (Email, RSS, calendar, version control, file management, IRC, music, financial ledger, terminal, etc…) is a small Elisp program that puts text into a buffer (often with a specialized major-mode) and binds keys to commands that transform and navigate that text.

There is no fundamental linguistic difference between something as complex as Magit and something as simple as a mode I may write to depict an ASCII rose in a popup buffer. The elisp entities powering those modes live in the same namespace, share the same APIs, and adhere to the same Emacsien and Elispien conventions. This is the deeper meaning of 'interface unification': you don't merely view different things through one interface (which is valuable by itself). Instead, you build, extend, and rewire them through one interface. And if a use case doesn't have an application yet, building one is easy, because you have the full power of Elisp.

The next pattern lives at the input layer, and is evocative of many an 'at-your-fingertips' metaphor. When Emacs asks you to choose something (files, buffers, commands, or indeed a candidate from any context) it doesn't ask you to remember what you're looking for verbatim. Incremental Completing Read (ICR) drastically reduces the burden of recall when searching for anything when your inside of Emacs, and the flexible filtering offered, for example, by packages like Orderless, make the resulting candidate set small and specific enough that recognition is often trivial.

ICR in Emacs can take whatever set of candidates is in play and filters it live as you type, ranking by recency and frecency, fuzzy-matching, re-sorting by relevance, etc…. You don't have to know the exact command name; you simply have to know, or perhaps more accurately 'intuit', enough of its shape to narrow the field. And because every domain in Emacs eventually exposes its candidates through the same minibuffer mechanism, the entire universe of things-you-might-want pulls through a single, deeply familiar selector (fan-in). At runtime, you can materialize and search this universe (fan-out) with only your tiny, fuzzy notion of what you're looking for. This recall facility is something that makes using Emacs powerful, but also cognitively ergonomic. Like with buffers + text + props, you'll find after a while that most of your Emacs usage funnels through this unified interface of minibuffer completion.

ICR is only part of this story. After you've selected your candidate, what can you do with it? Emacs cheekily answers "what can't you do with it!", and more helpfully adds "let me show you exactly what you can do with it". This comes from an implementation of what I think of as the recognize-dispatch pattern.

In most software, the answer ("what can I do with this thing on the screen") is hardcoded to a single default action: the open-file dialog opens files, the contact picker picks contacts, and so on. Packages like Embark and Hyperbole change the cardinality here for the better.

Any "thing" (which is a funnily loaded word in Emacs land) on which you can place your cursor (file path, a URL, a symbol, a region, etc…) has a type, and that type has a menu of actions associated with it. Because both the types and actions are extensible, Embark offers the ability to assign a specific type (or set of specific types) to anything that appears in Emacs (fan-in) and enables you to take any action on that thing (fan-out).

In this way, for example, a symbol is a thing you can describe, jump to, occur, rename, search the web for, hand to an LLM, ad infinitum…. A file is a thing you can open, diff, copy a path to, rename, attach to an email, ad infinitum…. The "infinitum" part is quite literal, especially when you consider that you can use any parameterized command as an Embark action (you aren't limited to what's in the defined menus). In this way, the actions accrue as you continue to extend Emacs. When you install a new package (fan-in), suddenly all your existing nouns, or types, gain new verbs, or actions (fan-out).

Also consider how, with Emacs, you can achieve the same kind of piping paradigm you would with a Unix shell. In a spiritual recovery of the 90's era CLIM (Common Lisp Interface Manager) implementation of presentation-types, output of some command in Emacs can be input to another via this recognize-dispatch system. The snake eats its tail, somehow in a non self-cannibalistic manner, just like in the all-powerful Unix shell.

An especially enabling set of actions that you can add in this way is provided by language servers in Emacs. For decades, "go to definition," "find references," and "rename symbol" were the exclusive privileges of IDEs that had been hand-coded (and price-tagged) for a specific language. The Language Server Protocol (LSP) externalizes that introspection into a per-language server that speaks a common format. Emacs (via eglot or lsp-mode) talks to the server, the server reports types and locations, and Emacs renders the result into a buffer. The upshot is that Emacs's existing typed-thing machinery inherits the introspective powers of a language server, in every language that has one (fan-in). The cost of literacy in a new language collapses (yay for me!), because the editor doesn't have to learn anything new; only the server (or the implementer of the server) does. In this way, you can effortlessly navigate, code, and execute in any language (fan-out) using the same usage pattern.

There is so much power in this pattern: recognize the noun, dispatch a verb. Pour all instances into a single channel (fan-in); radiate out from that channel with the full force of every extension you've ever installed (fan-out).

This fan-in, fan-out occurs at the application level as well. Take elfeed, for example, the leading RSS reader in Emacs. All the vagaries of any news source you subscribe to funnels its new items into a flat list (fan-in) in an elfeed-search buffer, and all the Emacs facilities that work in any other mode are available to you as you navigate and consume these news sources (fan-out). As another example, take mu4e, a popular email package for Emacs. Emails from all your email addresses, again, are funneled into a single flat list (fan-in), and you have all the power of Emacs at your disposal to navigate mail, read, and respond (fan-out).

The rose has the structure it has because it works, and many years of evolution have brought its beautiful form and function to bear.

The bloom:

Pull broadly, push narrowly; pull narrowly, push broadly.

With recognize-dispatch, pull broadly everything into text buffers, push narrowly into an actionable type system; pull narrowly from that sparse set of typed things, push broadly into any conceivable action.

Any search or selection you need to do inbetween is made effortless by ICR, where, by the way, types and actions are still available (thanks to Embark). Pull broadly any candidate source into Emacs and push narrowly into the minibuffer's ICR system; pull narrowly from a filtered candidate set and push broadly into any conceivable action on your selection.

You can even see the two patterns work gracefully in concert: consider dispatching a huge action menu for a generic target. You don't want to manually scan through the which-key buffer to select that action, so you can use embark-help to run ICR and select your action that way. Beautiful!

Emacs's patterns aren't incidental. They're something closer to the right answer to a question every aspirationally generic tool eventually asks, which is "how do you make one interface stand in for many, without losing the specificity of any"?

4. Emacs by any other name

Emacs is not magic, and exactly zero of the patterns I've described are intellectually proprietary to it.

A few patterns have already begun to leak out. The Language Server Protocol, in fact, wasn't invented for Emacs. It was invented at Microsoft for Visual Studio Code, and the rest of the editor world (Emacs included, via eglot) caught up. ICR-style recognition-over-recall pickers have shown up as command palettes in VS Code and Sublime Text. And plugin systems of course exist in nearly every modern editor. So the question is fair: What's in a rose?. Could another editor be Emacs, given enough effort? In principle, yes.

But in practice, the difficulty isn't any one pattern, but instead in the composition of those patterns. The gander is much greater than the sum of its geese, and in Emacs, the patterns synergize with each other, while somehow staying out of each other's way.

The patterns all 'play nice in the sandbox' if you will, but they can play more fortuitously as a team. Embark's typed-thing dispatch is only as good as the ubiquity of buffers and text properties beneath it. If half your "things" live in special-purpose UI widgets that aren't presented as text-in-buffers, then Embark woefully can't see them. ICR is only as good as the standardization of candidates. Even LSP β€” the most ubiquitous pattern leveraged outside of Emacs β€” only delivers its full potential when the editor already has a uniform "thing under point" model to which the server's responses can attach, and from which Emacs's fan-out action system can be dispatched.

There's also a sociological aspect to this. The patterns crystallized out of the Emacs community precisely because the community is philosophically aligned with discovering them. Everything is Lisp; every interaction is inspectable, hookable, redefinable. New patterns surface, get implemented as packages, and interestingly, many of these new patterns get factored upstream into the Emacs source. While any other editor could, in theory, be another Emacs, those editors that want to import the patterns wholesale would have to import the conditions that produced them and promote the culture that supported them.

But none of that is to say it's impossible. If a future editor actually delivered the full stack of patterns (text buffers as universal substrate, a Lisp-equivalent extension language with shared namespace, ICR, Embark/Hyperbole-style typed dispatch, LSP-style introspection, the fan-in/fan-out logic of one channel for everything), then it would definitively be powerful in the same way Emacs is powerful. Whether we called it Emacs or not would be arbitrary nomenclature.

Which brings me, predictably, to Juliet:

What's in a name? That which we call Emacs
By any other name, works just the same;
Our editor could, were it not an Emacs call'd,
Retain those perfect patterns which it owes
Without that title. Emacsen, doff thy names,
And for those names, which are no part of thee,
Take all my code.