Chunking

1 posts

Chunking for a Knowledge Base: One Markdown Convention, Chunked by Heading

What the knowledge base maintains is Markdown: PDFs, web pages, manual material and Q&A all follow one convention, and the chunker reads only the Markdown heading tree and block structure. This post covers the convention, PDF transcription and checks, the heading-level chunking rules, and the retrieval evaluation showing why one heading per chunk works best.

  • AI
  • RAG
  • Chunking
  • Markdown
November 15, 2026 · 16 min · 3316 words · Sam Peng
Different sources become Markdown with a heading hierarchy, then are cut into chunks by heading