Don't Feed PDFs Directly to AI: Use MarkItDown to Convert to Markdown First, Save Tokens and Avoid Mistakes

What this article solves: Many people upload PDFs directly to AI for summarization, only to have headers, footers, page numbers, and layout noise eat up massive tokens, while the output is still inaccurate. MarkItDown (Microsoft open source) converts PDFs to clean Markdown--clean it first, then feed it to AI. More importantly--conversion is only the first step. The real edge lies in the workflow that follows: "check structure first, then read step by step."

Who this is for: Readers who frequently use AI to read PDF reports, papers, contracts, and white papers. Requires a Python environment. Follow the tutorial through 7 steps to complete the full loop.

Table of Contents:

1. What Is MarkItDown
2. What Scenarios Is It Suitable For
3. Quick Start (For Those Who Don't Want to Read Everything)
4. Step 1: Prepare the Python Environment
5. Step 2: Install MarkItDown
6. Step 3: Convert PDF to Markdown
7. Step 4: Check If the Markdown Is Clean
8. Step 5: Let AI Do Structured Reading First (For Diagnosis)
9. Batch Convert Multiple PDFs
10. FAQ
11. Summary

---

Many people's first instinct when using AI to read PDFs is: upload directly, summarize, extract key points.

But real PDFs have headers, footers, page numbers, two-column layouts, table fragments, and broken word wraps--all of this noise gets ingested by AI as body text. Noise counts as tokens too.

That is why many PDF summaries look "wordy" but miss the key points. It is not that the model is bad--it is that the input is too dirty.

Convert the PDF to Markdown first, then hand the clean text to AI.

The tool is Microsoft's open-source MarkItDown: https://github.com/microsoft/markitdown

---

What Is MarkItDown

MarkItDown is an open-source Python tool by Microsoft whose goal is to convert various file formats into Markdown for easier downstream use by LLMs.

It supports more than just PDFs--Word, PPT, Excel, HTML, CSV, JSON, and more can also be handled depending on the dependency set. But this article only covers one minimal closed loop:

PDF → Markdown → AI summarization / Q&A / rewriting / extraction

---

What Scenarios Is It Suitable For

1. You have a PDF report and want AI to summarize it
2. You have a batch of papers and want to extract readable text first
3. You have product manuals, contracts, or white papers and do not want AI distracted by formatting
4. You want to archive materials into Obsidian, Notion, or a knowledge base
5. You want to reduce token waste and let AI read only the body text

But it is not a universal OCR. PDFs with copyable text → suitable for MarkItDown.

---

Quick Start

For those who do not want to read everything--if you already have Python 3.10+ installed:

pip install "markitdown[all]"
markitdown file.pdf -o file.md

Open file.md, check if the body text order and tables look normal, then jump directly to Step 5 to start collaborating with AI.

Below is the full 7-step breakdown.

---

Step 1: Prepare the Python Environment

MarkItDown requires Python 3.10 or higher.

First, check the version:

python --version

If the version is below 3.10, upgrade Python first.

---

Step 2: Install MarkItDown

Microsoft officially recommends installing all optional dependencies:

pip install "markitdown[all]"

If you only handle PDF, Word, and PPT, you can be more restrained:

pip install "markitdown[pdf,docx,pptx]"

I recommend beginners start with [all]. Today you are processing PDFs, tomorrow it might be PPTs, Excel, or HTML. Getting the tool running first matters more than optimizing dependencies upfront.

After installation, check:

markitdown --help

If it says the command does not exist, close and reopen the terminal, or confirm whether the virtual environment is activated.

---

Step 3: Convert PDF to Markdown

Assuming your file is called file.pdf:

markitdown file.pdf -o file.md

You can also use redirection:

markitdown file.pdf > file.md

I recommend -o. It is more intuitive and less prone to strange results caused by terminal encoding or redirection issues.

After conversion, you will get file.md. Do not rush to feed it to AI--open it and take a look first.

---

Step 4: Check If the Markdown Is Clean

Open file.md and focus on five things:

1. Whether heading levels are basically normal
2. Whether body text order matches the original PDF
3. Whether headers and footers are heavily mixed into the body text
4. Whether tables are still readable
5. Whether there is obvious garbled text, broken words, or repeated page numbers

Do not aim for 100% perfection. What we want is "more suitable for AI reading than the raw PDF." If the Markdown has already sorted out the body text order and reduced page numbers and layout noise significantly, you can move on to the next step.

Note: If the document contains images, MarkItDown will not display image content in the Markdown.

---

Step 5: Let AI Do Structured Reading First

Label: For Diagnosis -- use this prompt first to let AI check input quality

Do not start by asking "summarize this PDF." That phrase is too vague. A better approach: let AI check the material structure first.

Paste file.md to AI, then use this prompt:

I will give you a Markdown file converted from a PDF.

Please do not summarize directly.

First, check the structural quality of this Markdown and output:
1. Document topic;
2. Main chapters/sections;
3. Whether there is noise such as headers, footers, page numbers, tables of contents, or copyright notices;
4. Which paragraphs may be table or layout artifacts;
5. What tasks this material is suitable for next.

Requirements:
- Base your response only on the Markdown I provide;
- Do not supplement with external information;
- Mark uncertain areas as "uncertain."

This step is equivalent to having AI survey the scene before getting to work. If it finds a lot of noise in the Markdown, you can manually clean it up before proceeding to summarization.

---

Step 6: Then Let AI Summarize

Label: For Summarization -- use this prompt for deep extraction once the structure is confirmed good

Please produce a structured summary based on this Markdown.

Output format:
1. One-sentence conclusion;
2. 5 core points;
3. Original text evidence corresponding to each point;
4. Key data or case studies;
5. The author's reasoning chain;
6. 5 follow-up questions suitable for further inquiry.

Requirements:
- Do not fabricate information not present in the original text;
- If a conclusion lacks supporting evidence, mark it as "insufficient basis";
- Preserve important terminology;
- Output in English.

Key requirement: each point must correspond to original text evidence. This forces AI to go back into the material to find support, rather than freely elaborating based on headings alone.

---

Batch Convert Multiple PDFs

If a folder contains many PDFs, you can batch convert using PowerShell:

Get-ChildItem -Filter *.pdf | ForEach-Object {
  $out = [System.IO.Path]::ChangeExtension($_.FullName, ".md")
  markitdown $_.FullName -o $out
}

After execution: a.pdf → a.md, b.pdf → b.md, c.pdf → c.md.

After batch conversion, do not feed all Markdown files to AI at once. Spot-check 2-3 files first to confirm stable conversion quality, then proceed with the batch.

---

FAQ

1. markitdown command does not exist

Check:

pip show markitdown

If you can see package info but the command is still unavailable, reopen the terminal and try again. On Windows, sometimes the Scripts directory is not in the current session's PATH.

2. Output is garbled

Try a different PDF first. If only one particular file produces garbled output, the PDF may have special internal font encoding. In this case, try:

1. Use a PDF reader to save as a new PDF first
2. Or use another tool to export the text first
3. Then feed it to MarkItDown

3. Tables are messed up

PDF tables are inherently difficult. If tables are core information, do not rely on a single conversion. You can have AI handle the table areas separately:

The Markdown below may come from PDF table conversion and the formatting is somewhat messy. Please do not add new data--only based on the existing content, organize it into a Markdown table. If a cell cannot be determined, fill in "uncertain."

4. The file is too large and AI still cannot ingest it

Do not feed the entire thing at once. Have AI process it in sections based on the table of contents or headings first:

I will provide the Markdown of a long document in batches. Please take local notes for each batch only, and keep "conclusions for this batch" and "questions to verify against later sections." Only produce a full summary after I say "all batches sent."

This is far more stable than stuffing the context window full all at once.

---

Summary

When it comes to AI reading materials, it is often not about having a stronger model--it is about having cleaner input.

PDF is a format designed for humans to read. Markdown is more like a format designed for AI to read.

Spending 30 seconds converting a PDF to Markdown upfront can save you thousands or tens of thousands of tokens later, and spare you from summaries that seem plausible but are actually skewed by layout noise.

MarkItDown is not a flashy tool--it is a cleaning step before feeding AI.

---

Related Reading

• AI Tools Overview
• US Apple ID Registration
• Register .edu Email