Annotate-in-Place Notes with Emacs and org-remark

Annotate-in-Place Notes with Emacs and org-remark

Annotate-in-Place Notes with Emacs and org-remark

1. About   notes annotations emacs orgRemark

annotate-in-place-banner.jpeg

Figure 1: JPEG produced with DALL-E 3

I'm like most people when it comes to reading and note-taking.

Whether I'm new to a subject or fluent in it, I find myself devouring massive helpings of the seemingly infinite corpus of relevant text online, clamouring for some way to annotate, reason about, and synthesize that information.

Regardless of my level of mastery in a given subject, I've found that the rate of my information consumption has outpaced the capabilities of my note-taking system.

My dysfunctional note-taking process has been plagued by 2 main pain points:

  1. context switching (from the source content to the notebook)
  2. tenuous connections (between the source content and the notebook).

These issues have imposed an unnecessary friction on my learning process, and so I feel compelled to demonstrate my current workflow, which I find to be simple and cognitively ergonomic. In this post I want to introduce the specific but very generalizable pattern that enables my new workflow, annotate-in-place, and one elegant implementation of it in Emacs via nobiot's org-remark.

What makes this pattern so elegant to me is the familiarity of its experience. I don't know about you, but I've been annotating books and taking notes with pencils and pens for almost my entire life, and this is often the most engaging and soul-lifting experience. There is a je ne sais quoi in this interaction that makes me feel closer to, if not part of, the thing I'm reading. This is a physical annotate-in-place, and it works beatifully.

I've been long searching for a cognitive bridge between the ergonomics of putting pen to source text with the infinite flexibility of a software solution. annotate-in-place is the pattern that provides that bridge, and org-remark in Emacs is one implementation of that pattern. With it, digital note taking feel as intuitive and ergonmic to me as note taking on a physical medium.

As a more cognitive benefit, I've found that annotate-in-place's reduction on the memory and organization burden of note-taking let's my mind expand into the internet, unincumbered by any friction or worry. That's a great feeling!

The first half of the post is about the pattern, in the abstract, of annotate-in-place. The meat and potatoes is a demo of org-remark in Emacs across a handful of contexts. I wrap up with some caveats of my current approach, and some hypotheses on why this approach is relevant beyond Emacs.

2. The Video: org-remark in Action   video demo

The upper-right of my Emacs (in the tab-bar) shows the keybindings and command names I am invoking, so you can map what you see onto your own configuration.

3. The Problem with Modern Digital Note-Taking   notes problem

We builders are often taught that coupling is bad, bad, bad…. But in some rare cases it makes perfect sense.

Most digital note-taking approaches decouple the note from the source. That's bad decoupling in my opinion. Whether digital or physical, I typically read in one app and write in another. Pure on-book annotation is an exception, but once I factor in something like a 'knowledge base' (a personal notes graph — think Obsidian, Roam, Logseq, org-roam, denote), I'm dealing with decoupled artifacts: the source artifact (abstractly, the 'site') and an annotation artifact (abstractly, the 'notebook').

In this context 'sites' can be a webpage, a book, or even your own notebook; and 'notebooks' can be a digital note app, or a physical piece of paper.

Everyone reading this (I hope) understands the pickle in this decoupling, but here are a few issues that stand out for me:

3.1. The Context-Switching Tax

The friction of this is small on a per-action basis, but it adds up, and tragically punishes the virtuous act of noticing. Noticing, as passive as it sounds, is out intellectual leverage as special humans, and I believe it should be met with as little friction as possible. I want to let my 'noticing' perform its magic, unbounded by the friction or imposition of my tooling or note-taking workflow.

3.2. Source Amnesia and Note Amnesia

I often find myself accumulating notes completely detached from the source they came from. When I revisit my notes months later, I find myself hunting for the source the notes were taken from. When I revisit the source, I similarly find myself in search for the notes that I took on that source, sometimes struggling to remember if I ever recorded those notes in the first place.

I find this is a more significant problem when I revisit a source (note amnesia). In the best case, if I did take notes then it's possible that I wrote down the title of the source, or perhaps even the URL. That unidirectional link /can save me when I'm reading notes. But when I'm revisiting a source, there's nothing embedded in the source itself to let me know that some highlights or notes exist for it, and so in those cases, I never recover the precious memories I recorded.

3.3. No Signal on Revisit

When I return to the source (potentially months or years later) nothing tells me what past me found interesting. The blank source looks the same as it did the first time I visited it. I've likely lost familiarity with the source material or failed to review the relevant notes for that source before revisiting, so my brain has to do significant work just to resurface what I already found salient and recorded. And that's before the even harder, more rewarding, work of refamiliarizing myself with its concepts or synthesizing it with something else I learned or realized in the meantime.

3.4. Memory Anxiety

If you're like me, you probably carry a low-grade worry about whether you can ever review what you covered across the many daily hours of reading complex source material.

Most second-brain systems compound this by encouraging capture without robust bi-directional links between the source and the notes. org-mode's capture offers a uni-directional link (in the notes), but the captured data never surfaces when I revisit the source from which I captured that information. I basically have to know somehow that the notes exist.

By contrast, when I annotate a physical book, the marks stay on the page. For me personally, this makes note taking on physical media worry-free because I know that all my highlights and marginalia will remain there and will shine brightly back at me when I re-crack that book or paper open. On re-reads, my eye is drawn to exactly the passages that mattered to me the last time.

4. Annotate-in-Place: The Pattern   pattern

This pattern, like most truly useful ones, is simple and elegant.

With annotate-in-place, when I find something worth marking, I highlight it in a single easy motion, as if it were a physical document. The highlight is visible the next time I visit the source, and a separate notes file is updated in the background with an entry containing three core fields plus optional metadata:

  1. The excerpt (so the verbatim source quote survives if the source disappears).
  2. A link back to the source, with enough location information to find the exact passage.
  3. Any commentary I choose to attach.
  4. Optional (configurable) metadata (timestamps, tags, etc…).

Everything else I might want to do with these notes (search, tag, daily review, integration with a knowledge base) is made possible by those recorded fields. The pattern's elegance comes from how little it asks of me at note-taking time, and how much it makes available to me at review or search time.

I've already expounded upon the complexity of reading and note-taking. Annotate-in-place comes to my rescue by making sure that the extra work of 'noticing' on my part is astronomically close to zero. All I have to do is point and highlight, as if the document were as ergonomic as physical entity.

5. What Existing Tools Get Right (and Wrong)   readwise hypothesis

A few tools already implement annotate-in-place, but each is bounded by its environment. A couple I've used in the past:

  • Readwise offers in-browser highlighting (via browser official browser extensions) that syncs to Obsidian, Roam, Notion, and others. Its companion read-it-later app, Readwise Reader, extends the same model to PDFs, EPUBs, emails, and YouTube within Readwise's own apps, with strong mobile support and OCR that Emacs cannot match out of the box. This works really nicely (and was my first adventure into the annotate-in-place pattern), but the annotation environments are still Readwise's own, and scoped to the browser. What's more, to annotate any non-web-page content, I have to upload it to Readwise's servers for processing before I can mark it up. Readwise lacks support for annotating documents on your personal computer, like code files, org files, etc….
  • Hypothes.is does the same thing in the browser with shared annotations and a public web layer that no Emacs tool offers (yet) but is similarly scoped: highlighting only happens on web pages.

An interesting observation I made after using both is that, for me, the plurality of what I read is not a web page. Often times, it's RSS items in a feed reader, code in my editor, transcripts of chats with an LLM, Emacs's info pages, ebooks, mail, dictated notes, papers in PubMed, articles in my read-it-later tool (wombag), etc…. None of these polyhedral pegs fit into the round hole of Readwise or Hypothes.is, so I find myself pushed back into brittle copy-paste workflows unless I'm reading in an app they support.

What I'm really after is an implementation of the annotate-in-place pattern that accommodates all these sources.

6. org-remark in Brief   orgRemark emacs

org-remark is the closest Emacs gets to a faithful implementation of the annotate-in-place pattern. It works on any text buffer. In other words, it works anywhere Emacs renders text (so everywhere) and it stores each highlight in an org-mode notes file that you own (no third-party intermediary like Readwise or Hypothes.is).

The command surface to achieve this is pleasingly small:

  • org-remark-mark-default marks the active region in the source (wherever that is) with a highlight overlay (the command name comes from the pen — see the next section). An entry appears in the corresponding notes file with the excerpt, a link to the source, and any metadata the pen has been configured to attach.
  • org-remark-open jumps from a highlight to the corresponding entry in the notes file, where you can write commentary using the full power of org-mode.

Of course, the notes file is an ordinary org file and presents in an ordinary Emacs buffer. As such, every existing org or Emacs tool (org-agenda, org-roam, denote, ripgrep, your own scripts, etc…) is available to help you search, navigate, and digest the information therein.

The killer feature of annotate-in-place for me is that when I reopen the source buffer later, org-remark displays the overlays at the correct positions. This is the thing that bridges the gap between the ergonomics of annotating physical media and the feature-richness of note-taking in a digital knowledge base. It also makes the link between the source and the notes bi-directional. So I get the benefit of decoupling the note from the source, yet I retain the intuitive behavior of highlighting and annotating source material in-place.

6.1. Pens and Metadata   pens metadata

A pen in org-remark is basically a named highlighter, and each pen can attach arbitrary properties to its highlights. In my config of org-remark I define a default pen that timestamps every new highlight with a date link in Logseq's preferred format: 

(org-remark-create "default"
                   'org-remark-highlighter
                   `(CATEGORY "important"
                     org-remark-highlight-date ,(my/org-remark-get-date)))

Calling org-remark-create defines a marker command named after the pen (here, org-remark-mark-default), which is what I then bind to a key. The date in the snippet is computed dynamically at mark time, so every new highlight automatically carries a link to its day. This, among other things, turns my notes files into a reviewable timeline (more on that in the daily review section).

Note that you can define as many pen commands as you like using org-remark-create. I get by with just one, but I can imagine use cases for having many.

7. Extending org-remark to New Modes   extension elfeed wombag pubmed

org-remark ships built-in support for a few major modes, but it can be easily extended to support any arbitrary new mode. To do this, you implement two functions, register them on hooks, and optionally arrange for org-remark-auto-on to fire when the mode opens a buffer (that's what makes the highlights reappear when you revisit a source). Concretely, for a mode foo-show-mode (think elfeed-show-mode for RSS items, wombag-show-mode for read-it-later articles, pubmed-show-mode for PubMed papers), you need:

  1. A function that returns a stable file name for the buffer's content. This is what org-remark uses to find the right notes file when the same content is reopened. Hook it onto org-remark-source-find-file-name-functions.
  2. A function that returns a link back to the source. Hook it onto org-remark-highlight-link-to-source-functions.
  3. A way to turn org-remark on when a buffer of this mode appears. This is either an :after advice or a mode hook calling org-remark-auto-on.

Some modes can be tricky, but in general, I've found that this is a lot simpler than it sounds. Here is the elfeed integration, from my config. 

(define-minor-mode org-remark-elfeed-mode
  "Enable Org-remark to work with elfeed.el"
  :global t
  :group 'org-remark-elfeed
  (if org-remark-elfeed-mode
      ;; Enable
      (progn
        (add-hook 'org-remark-source-find-file-name-functions
                  #'org-remark-elfeed-find-file-name)
        (add-hook 'org-remark-highlight-link-to-source-functions
                  #'org-remark-elfeed-highlight-link-to-source))
    ;; Disable
    (remove-hook 'org-remark-source-find-file-name-functions
                 #'org-remark-elfeed-find-file-name)
    (remove-hook 'org-remark-highlight-link-to-source-functions
                 #'org-remark-elfeed-highlight-link-to-source)))

(defun my-advice-elfeed-show-mode-org-remark (&rest _args)
  (org-remark-auto-on))

(advice-add #'elfeed-show-entry
            :after #'my-advice-elfeed-show-mode-org-remark)

(defun org-remark-elfeed-find-file-name ()
  (when (equal major-mode 'elfeed-show-mode)
    (my-org-remark-transform-org-link-to-filename)))

(defun org-remark-elfeed-highlight-link-to-source (filename _point)
  (when (equal major-mode 'elfeed-show-mode)
    (org-store-link nil)))

(org-remark-elfeed-mode)

The PubMed and Wombag integrations look the same (feel free to explore my configuration). This is the part of org-remark I find most enabling: the protocol for adding a new mode is small enough that the juice is typically worth the squeeze.

8. A Daily Review Workflow   review

Because every highlight is stamped with a date link, reviewing what I read today is just a simple grep for that timestamp. I search my highlights directory for the current day's link and get back exactly the passages I found interesting today, each with any commentary I wrote and a working link to the source. My video demonstrates this workflow.

For me, this completely eliminates the anxiety of reviewing anything I've read, and allows me to read more expansively and feverishly. I don't have to remember anything I'm reading because the timestamped entry created by org-remark does the remembering for me.

There are two adjustments you may want to consider to make this work well:

  • Persist highlights in a stable directory. Mine live under ~/logseq/pages/, but a single directory of any kind is enough. rg or grep against the date link in this directory is the review query. As a bonus, note that if you sync your knowledge base across machines, the persistent in-place highlights will show up on all the machines to which you synced your knowledge base.
  • Pick a date link format your knowledge base understands. I use Logseq's [[Mar 4th, 2026]] form so the same link doubles as a backlink in the rest of my graph, but [[2026-03-04]] works equally well in org-roam or denote.

9. What org-remark Gives You   benefits orgRemark

Beyond the pattern-level wins already covered, org-remark itself contributes some specifics worth naming:

  • Notes are cited and attributed by default. The link to the source is automatic.
  • Capture is cheap. The action is "select region, hit a key". The excerpt, timestamp, link, and notes-file entry are all created without my involvement, and my in-place highlights show up the next time I visit that source.
  • Annotations are first-class org data. Anything you can do in org (which is a whole lot) you can do under any highlight.
  • Notes integrate with the rest of your knowledge base. Because the output is plain org, denote, org-roam, and Logseq all treat my notes files like any other node in the knowledge graph.
  • Configurable everything. Pens, metadata, the on-disk location of the notes file, the link format, the highlight face, and almost any behaviour of the setup are all configurable. See org-remark's documentation for more details.

10. Caveats   caveats

There are some caveats to going whole-hog on this pattern in Emacs with org-remark.

  • JavaScript-heavy web pages. eww cannot render modern SPA-style pages, so I cannot highlight inside them directly. My fallback is to clip the page to a read-it-later tool (in my config, Wombag) that produces a clean text view org-remark can annotate. Readwise also remains a reasonable browser-side fallback for the cases where neither approach works.
  • PDFs. org-remark does not support PDFs. For PDFs, the annotate-in-place option in Emacs is pdf-tools, which lets you add annotations directly inside the PDF buffer. org-noter is the spirit-equivalent for tying org notes to PDF/EPUB locations side-by-side rather than in place, and its daily-review workflow transfers with minimal adjustment.
  • Drifting source text. org-remark relocates highlights using the persisted excerpt when the underlying buffer changes, but heavily-edited files (or web pages that change between visits) can still produce orphans. The notes file always has the excerpt, but the overlay may need re-anchoring.
  • Emacs lock-in. The org-remark tool is Emacs-only. The pattern is not (read on). If you have not moved into Emacs, see the closing section: Readwise and Hypothes.is together implement the annotate-in-place pattern really well, and the workflow you build around them will transfer if you ever do switch to org-remark in Emacs.

11. Closing: The Pattern Is the Point   closing

To wrap up, I want to close not on org-remark specifically, but the annotate-in-place pattern.

In Emacs the implementation is especially complete because Emacs treats every reading surface as a text buffer. Outside Emacs, those same four fields (excerpt, link, commentary, optional metadata) are the right rubric for evaluating any annotate-in-place tool you adopt. I personally think these fields are necessary if you want to get leverage out of an annotate-in-place solution.

If you are not in Emacs, Readwise and Hypothes.is implement the pattern within the browser. I've used both and think they're great and worth using (in fact Readwise is still my fallback for when I can't do annotate-in-place within Emacs). If you are in Emacs, org-remark is worth thirty minutes of setup and a small amount of elisp per mode.

In either case, you still benefit from annotate-in-place.

12. TLDR   tldr

Most digital note-taking detaches notes from their sources. Annotate-in-place is the pattern of marking content where it lives, with the highlight visible on revisit and an excerpt+link persisted in a separate notes file. org-remark is the Emacs implementation: built-in support for eww, Info, nov.el ebooks, and plain files, plus a small extension protocol for adding new surfaces (elfeed, pubmed-show-mode, wombag, gptel chats, etc.). A timestamped default pen turns the notes file into a reviewable timeline: a daily review is a grep for today's date link. Caveats are JS-heavy pages (fall back to Wombag or Readwise), PDFs (use pdf-tools for in-place, org-noter for side-by-side), and Emacs lock-in itself. The tool is Emacs-only, but the pattern is portable.