Somewhere along the way, the AI developer community reached a quiet consensus. If you want to feed web content to a large language model, you convert the HTML to Markdown first. Scraping libraries advertise it as a headline feature. "Reader" APIs strip away the stylesheets and hand you clean-looking Markdown. The assumption slides right in. This is the optimal format for LLMs. It looks readable. It removes the CSS junk. So it must be perfect.
Not quite. If you're building a production RAG pipeline, that assumption quietly costs you accuracy and hours of debugging.
Here's the uncomfortable truth about markdown for RAG. John Gruber invented Markdown in 2004 as a way for humans to write formatted text easily in a plain text editor. It was never designed as a serialization format for machine learning models or vector databases. It optimizes for human readability, not for machine retrieval or token efficiency. Weigh that against what your pipeline actually needs before you commit.
This post makes a direct argument. Converting raw HTML to Markdown genuinely improves on dumping raw HTML into your pipeline. No one is disputing that. But relying solely on Markdown for ingestion introduces severe token waste and brittle retrieval. For production-grade RAG, structured JSON is the superior standard for ingestion, chunking, and metadata filtering. Let's break down exactly why it wins.
The Failure Modes of HTML-to-Markdown Converters
Before we talk about what to use instead, we should be honest about where generic HTML-to-Markdown conversion fails. These aren't edge cases. They show up on nearly every real-world web page you'll try to scrape. Keep them in mind as you audit your own ingestion path.
Layout Noise and Scrambled Context
Web pages are not linear documents. They're two-dimensional layouts full of sidebars, cookie banners, and social share buttons. A human reading the page instinctively ignores all of that. The focus lands on the main content column.
A basic Markdown converter has no such instinct. It walks the DOM tree and flattens everything into a single linear stream of text. So your "Subscribe to our newsletter" call-to-action and the 40-item footer sitemap get injected straight into the main text flow, often in the middle of the article body.
For RAG, this is poison. When you chunk and embed that content, the boilerplate pollutes your semantic vectors. A chunk that should represent "how OAuth2 token refresh works" now also carries the embedding weight of navigation labels. Your retrieval precision drops because the vector no longer cleanly represents a single concept. Strip the noise before you embed, and watch that precision recover.
The "Broken Table" Problem
This is the failure mode that catches the most teams off guard. Tables render beautifully on a web page. To a human, they're the clearest way to present structured comparisons like pricing tiers and specification sheets.
When a converter turns an HTML table into Markdown, it produces something like this:
| Plan | Requests/mo | Price | Overage ||------|-------------|-------|---------|| Free | 1,000 | $0 | N/A || Pro | 100,000 | $49 | $0.50/k || Enterprise | Unlimited | Custom | Custom |This works fine if the entire table stays intact. But here's what actually happens in a RAG pipeline. Your chunker splits text by character count or token count. If a chunk boundary lands in the middle of that table, you get a fragment like:
| Pro | 100,000 | $49 | $0.50/k || Enterprise | Unlimited | Custom | Custom |The header row, Plan | Requests/mo | Price | Overage, now sits in a different chunk. The embedding model and the generator have no idea that "100,000" means "requests per month" or that "$49" is a price. The column-to-row relationship is gone. The tabular data becomes gibberish. When a user asks "what's the price of the Pro plan?", the retriever might surface that orphaned row, and the LLM guesses which number is the price. Extract tables as typed objects instead, so the relationships survive chunking.
Interactive and Dynamic Content
Modern web pages lean heavily on JavaScript-rendered components. Think collapsible accordions and nested tabs-within-tabs. These components encode relationships. This content belongs under this tab. That answer belongs to that question.
When flattened into basic Markdown, all of that structural meaning evaporates. An FAQ accordion becomes a wall of questions and answers with no clear pairing. A tabbed code example collapses into three unlabeled code blocks stacked together. The reader, human or machine, can no longer tell which snippet belongs to which language. The HTML to Markdown for RAG path silently discards exactly the structure your model needs to reason correctly. Preserve those component boundaries during extraction to keep the logic intact.
The Math of Token Waste: 200KB Markdown vs. 14KB Cleaned JSON
If the accuracy arguments don't move you, the economics should. Let's run real numbers on a common scenario. We're scraping a dense page, maybe a product detail page or a financial report.
The Raw Markdown Route
You scrape the page and convert it to "clean" Markdown. Even after stripping stylesheets and scripts, a complex page routinely produces a file hovering around 200KB. Why so large? Because the Markdown still contains:
- Repeating header navigation on every page
- Full footer link farms (privacy, terms, careers, dozens of category links)
- Inline link URLs embedded throughout the prose
- Layout artifacts and duplicated widget text
- The actual content you wanted, buried somewhere in the middle
In token terms, 200KB of text is roughly 150,000+ tokens. If you send that to a context window, or chunk it blindly and embed every piece, you pay to process an enormous volume of noise. Multiply that across thousands of pages. Your indexing bill balloons for data that's actively degrading your retrieval quality. Measure your current token spend per page first, then decide.
The Structured JSON Route
Now flip the approach. Instead of scraping and dumping the entire page, you pass a schema describing exactly what you want:
{ "product_name": "string", "price": "number", "specifications": "object", "author": "string", "publish_date": "date", "main_body_text": "string"}The extraction engine returns only those typed fields. The same page that produced 200KB of Markdown now yields a tight, typed 14KB JSON response. No footer farms. No repeated navigation. Just the fields you asked for, correctly typed. Define your schema once and reuse it across every similar page.
The ROI
That's roughly a 93% reduction in tokens. The downstream effects compound:
- Lower API and embedding costs. you pay to process signal, not noise.
- Faster inference. smaller, cleaner context windows mean quicker responses.
- Mitigation of "Lost in the Middle". LLMs are documented to lose track of information buried in the center of long contexts. Feed tight, relevant JSON instead of a sprawling Markdown dump. You keep the critical facts front and center where the model actually attends to them.
Clean input isn't just cheaper. It's measurably more accurate. Run the comparison on one page today and confirm the savings for yourself.
RAG Data Preparation: Why Embedding Markdown Is a "Blind" Vector Search
Good RAG data preparation is about more than shrinking token counts. It's about how you query the data later. This is where the markdown vs JSON RAG debate gets decisive.
The Blind Spot of Semantic-Only Search
If your vector store contains nothing but unstructured Markdown chunks, you rely 100% on semantic embeddings to retrieve anything. That works beautifully for fuzzy conceptual questions. It falls apart for anything requiring precision.
Consider the query: "Show me security policies updated after Q3 2025."
A pure vector search will semantically match "security policies" and even "Q3 2025" as text. But a vector database cannot natively perform mathematical date comparisons on prose. It has no reliable way to enforce date > 2025-09-30. You'll get documents that mention those words, not documents that actually satisfy the constraint. For anything involving dates or version numbers, semantic-only search is flying blind. Test a date-bound query against your current index and see how many false positives come back.
How Structured JSON Enables Hybrid Retrieval
Structured JSON for RAG solves this by cleanly separating your content from your metadata.
Deterministic pre-filtering. Because your dates, categories, and ratings live in explicit typed fields, you run a precise SQL-like filter before you ever touch the vector index:
WHERE publish_date > '2025-09-30' AND category = 'security'Only after narrowing the candidate set do you run semantic vector search on the remaining clean text. You get the best of both worlds. That's deterministic precision on structured attributes and fuzzy matching on natural language. This hybrid approach is impossible when your dates are buried inside a Markdown blob.
Field-level chunking. Instead of blindly slicing a Markdown document by character count, you chunk only the specific JSON fields that contain body text. And because you have the metadata in hand, you automatically append the exact metadata keys to every single chunk. Each chunk becomes self-describing. That sharply improves both retrieval and the model's ability to ground its answers. Start by tagging each chunk with its source and date.
Citation, Grounding, and AI Agent Validation
There's one more argument that matters enormously once you move from a demo to something users actually trust: provenance.
The Hallucination Problem
When an LLM generates an answer from a messy Markdown document, tracing where a specific fact came from is fragile at best. Which page? Which section? Reconstructing that from a flattened text blob usually means fuzzy string-matching back to the source, and often failing. Without reliable citations, users can't verify the output. Unverifiable output is where trust goes to die. Bake provenance into your data before it ever reaches the model.
How Structured JSON Solves It
Every extracted JSON block carries explicit provenance metadata alongside its content:
{ "source_url": "https://example.com/docs/auth", "last_updated": "2026-02-15", "extracted_entity": "OAuth2 Flow", "content_chunk": "The token refresh endpoint accepts a grant_type of refresh_token..."}When the retriever surfaces this object, your AI agent has deterministic variables it parses programmatically. It doesn't guess the source. The source_url is right there. It generates bulletproof, clickable inline citations and even surfaces the last_updated timestamp so users know how current the information is. That's the difference between a demo and an enterprise-grade system. Attach these fields at extraction time and your citations write themselves.
Implementation: Moving from Markdown to Schema-Guided Extraction
So how do you actually make this shift without building a mountain of fragile parsing code?
The wrong answer is to scrape to Markdown and then write custom regex parsers to reverse-engineer it back into JSON. That's slower and more delicate. It breaks the moment a site changes its layout. The right answer is to extract structured data directly at the point of ingestion.
This is exactly what the DataBlue Extract API is built for. You define your desired output as a standard JSON schema and point it at a URL. The API handles the hard parts in a single step. That covers JavaScript rendering, DOM parsing, and structured extraction into your typed schema. No Markdown middle-man. No regex archaeology.
The workflow difference is stark:
Legacy Markdown workflow:Scrape URL → HTML → Convert to Markdown → Guess chunk boundaries → Embed everything → Query (and hope)
DataBlue workflow:Scrape URL + Schema → Typed JSON → Precise field ingestion & automatic metadata tagging → Accurate hybrid search
The second pipeline is shorter and cheaper to run. It produces cleaner, filterable, citable data at every stage. Sketch your target schema and try the second path on a single URL.
Conclusion: The Right Tool for the Right Job
None of this means Markdown is useless. It's excellent as a presentation layer. That's exactly the format you want your LLM to use when formatting its final answer back to the user. Headings, bullet lists, and code blocks make responses easy to read.
But presentation and pipeline are two different jobs. For the ingestion, storage, and retrieval phases of RAG, structured JSON is the undisputed winner. It slashes token waste and keeps tables intact. It enables deterministic metadata filtering and makes every fact traceable back to its source.
Retrieving clean web text is great for real-time workflows, as explored in our guide on Integrating Google Search with LLMs for Real-Time Context. Structuring that data into typed JSON is what transforms a simple search wrapper into a robust, enterprise-grade RAG pipeline.
Stop dumping lazy Markdown into your vector store. Build cleaner and more accurate pipelines by defining target schemas and extracting typed JSON from the start. Head over to the DataBlue Extract API and define your first schema. Your retrieval accuracy and your token budget will thank you.