r/Rag • u/hyeonsang010716 • 7h ago
Showcase PDFs don't have footnotes. How I link them for RAG without guessing
I maintain an open-source document-to-Markdown converter. This post is about one problem I spent far too long on, and I'd like to hear how others handle it.
The problem
Here's a line from page 14 of a (made-up) annual report, as pdfplumber extracts it:
Net debt fell to NOK 512 million, and the net debt ratio to 1.4 times4 at year end, down from 1.9 aThat "4" is a footnote mark. The note at the bottom of the page says the ratio excludes lease liabilities under IFRS 16, and that it would be 2.9 if you included them.
So "1.4" is true, but only with a caveat that roughly doubles the number. Once the page is chunked, the sentence and the note can easily land in different chunks. Retrieval returns "net debt ratio 1.4", the model states it with full confidence, and nothing in the chunk says it's conditional. "times4" is just noise.
This isn't an edge case. Financial reports, legal documents and papers put the important exceptions in footnotes on purpose.
Why it's hard
A PDF has no concept of a footnote. You get glyphs at positions. A small raised "2" could be a footnote mark, the 2 in m², an exponent, or a figure label. The notes are just smaller text near the bottom of the page. When I compared marker, Docling, MinerU, pymupdf4llm and MarkItDown, none of them connected the mark to its note.
And guessing fails in a bad way. A missing link costs you a caveat. A wrong link attaches someone else's caveat to your sentence, and the model will repeat it.
What I ended up doing: link only when the page proves it
A page gets linked only if all of these hold:
there's a short horizontal rule near the bottom, with smaller text right under it starting at its left end
every note starts with a raised number, and the numbers run on (1, 2, 3, or continuing from the previous page)
each note number appears as a raised mark in the body exactly once, in the same order as the notes
nothing on the page is ambiguous: if there's a note 2 and also an m² anywhere on the page, the whole page is left alone
If any check fails, the page comes out exactly as it did before. Linked pages get standard Markdown footnotes:
Net debt fell to NOK 512 million, and the net debt ratio to 1.4 times[^4] at year end, ...
[^4]: Net debt is interest-bearing debt minus cash and cash equivalents. The ratio divides it by EBITDA
for the last twelve months and excludes lease liabilities under IFRS 16; including them, the ratio would be 2.9.The "exactly once, anywhere on the page" rule is there because the clever version broke. My first attempt used reading order: if the m² comes after the mark for note 3, it can't be the mark for note 2. Then LaTeX happened. LaTeX floats figures to the top or bottom of the page, so the mark in a figure caption can be read before mark 1 or after mark 3, and an m² sitting in between took note 2. After that I stopped being clever and just skip the page.
The numbers, honestly
120 PDFs I generated from DOCX files with real footnotes (printed through LibreOffice), so the ground truth is known: 1,580 marks, 864 linked, 0 linked to the wrong note. Everything else was left as it was.
On a messier set of 1,103 real-world PDFs (17,708 pages: arXiv papers, a public corpus, about a thousand business documents, mostly Korean), an earlier version linked only 40 marks on 25 pages. I checked every one by hand and none was wrong. Most business documents don't qualify at all, because their marks are inside table cells or there's no footnote rule.
So recall on real documents is low. That's the trade I chose: I'd rather miss a footnote than invent one. I'm honestly not sure it's the right call for every use case.
Using it at retrieval time
The point of [^4] is that the link survives chunking. Labels are unique within a document, so you can re-attach the note to any chunk that cites it:
import re
from opendockit import convert_file
doc = convert_file("annual-report.pdf")
# Each note is its own paragraph ("[^4]: ..."), possibly wrapped over several lines.
notes = dict(re.findall(r"^\[\^([\w-]+)\]: (.+?)(?=\n\n|\Z)", doc.text, flags=re.M | re.S))
def with_notes(chunk: str) -> str:
"""Append the note of every mark the chunk cites but doesn't contain."""
marks = dict.fromkeys(re.findall(r"\[\^([\w-]+)\](?!:)", chunk))
extra = [f"[^{m}]: {notes[m]}" for m in marks if m in notes and f"[^{m}]:" not in chunk]
return "\n\n".join([chunk, *extra])With page-sized chunks the note is usually in the chunk already. With small fixed-size chunks it often isn't, and that's where this helps.
Questions for you
How do you handle footnotes today? Ignore them, chunk by page, run a VLM-based parser?
Would you accept higher recall at the cost of an occasional wrong link?
If you know public PDFs where footnotes really matter (10-Ks, court opinions, standards), I'd love to test on them. Real-world cases are what I'm short on.
The library is OpenDocKit (MIT, pure Python, no models, runs offline): https://github.com/hyeonsang010716/OpenDocKit
The full footnote rules and their known gaps are written up here: https://github.com/hyeonsang010716/OpenDocKit/blob/main/docs/decisions/pdf-flow.md#pdf-footnotes
The same idea runs through the rest of it. For example, merged table cells are repeated in every row they cover, so a row still makes sense on its own in a chunk. It also reads HWP, which mostly matters if you work with Korean documents.