--- title: "Large Markdown files: what to do when a viewer struggles" slug: large-markdown-files excerpt: Find what makes a large Markdown file heavy with wc, awk and grep, then move pasted images out and split the file by heading. quickAnswer: 'A large Markdown file can get heavy from an image pasted in as base64 text, a table with thousands of rows, or a few very long single lines. Measure the file with wc -c and wc -l, find the longest lines with awk, and count data:image links and table rows with grep -c. Then move pasted images out to real image files with a short Python script, and split what is left into one file per ## heading with a one-line awk command. Read the file in less -S meanwhile, which chops long lines instead of wrapping them.' keyTakeaways: - A viewer that lays out the whole file before you see it has to wrap every huge line and draw every table row first. - Start with wc -c for size, wc -l for lines and awk for the longest lines. - An image pasted in as a base64 data:image link can be most of a file's weight, and a short Python 3 script moves it out to an image file without overwriting anything. - 'A one-line awk command splits a file into one part per ## heading, so you can see which section is heavy and open it alone.' - Sashmark is a Markdown reader and editor in development and not released; nothing on this page depends on it. publishedDate: 2026-10-20 dateModified: 2026-10-20 author: Bobby Huang cluster: F role: post status: published faq: - q: Why is my Markdown file so big when it has so little text? a: Check for pasted images first. Run grep -c 'data:image' on the file. An image pasted as a base64 data link is stored as one very long line of text inside the file, and that one line can outweigh everything else you wrote. - q: Will moving images out of the file change how it looks? a: It shouldn't. The script writes the image bytes out unchanged and swaps each data link for a normal image link to a file in an images folder next to the Markdown file, with the same alt text. A viewer that shows local images reads the same bytes from the file. - q: Is it safe to run the image script twice? a: Yes. The second run finds no data links left, prints moved 0 image(s) and writes nothing. The first run writes a .bak.md copy of the original before it touches anything, and the script stops if that copy already exists. related: - markdown-images - markdown-tables - get-through-a-pile-of-agent-markdown-files --- Your agent wrote a report, and now it won't open. Or it opens, but the scroll bar crawls and the viewer freezes each time you search. The file looked harmless in the folder listing: one `.md` file with a plain name. Inside, it might hold a pasted screenshot, a log the agent kept appending to, or a table with thousands of rows. This page shows you how to find out what makes a large Markdown file heavy and how to fix it with free terminal tools. You'll measure the file, find the heavy part, move pasted images out to real files, and split what's left by heading. All of it works in a Mac or Linux terminal, and the one script needs only Python 3. If you just need to open a normal file, [opening and reading a Markdown file on Mac, Windows and Linux](open-and-read-a-markdown-file-on-mac-windows-linux.md) covers the basics. ## Why a Markdown file gets heavy Markdown is plain text, so size comes from characters. These add a lot of them fast: - **A pasted image.** Some tools paste a screenshot as text: a `data:image` link with the whole picture encoded in base64. That's one line, and it can be enormous. - **A giant table.** Every row is a line, usually starting with `|`. An agent logging each task as a row can write thousands of them. - **Very long single lines.** Minified data, a pasted log without line breaks, or the image case above. A viewer that lays out the whole file before you see it has more to do. Wrapping one huge line, or drawing a table with thousands of rows, is work done up front, and you feel it as a slow open or a frozen scroll. The examples below use a sample file called `notes.md`: a release notes draft with a title and four sections, Summary, Screenshot (one pasted image), Task log (a 20,000-row table) and Next steps. ## Step 1: measure the file Start with bytes, lines and a human-readable size. ```sh wc -c notes.md wc -l notes.md ls -lh notes.md ``` ```text 1460271 notes.md 20020 notes.md 1.4M notes.md ``` `wc -c` counts bytes and `wc -l` counts newline characters, which is the line count when the last line ends with one. `ls -lh` shows the size the way a file browser would; its other columns (permissions, owner, date) are trimmed here. If you don't know which file is the problem, find every Markdown file over a megabyte in the current folder and below: ```sh find . -name '*.md' -size +1M ``` ```text ./notes.md ``` ## Step 2: find the heavy part ### The longest lines This prints each line's length and line number, sorts by length and keeps the top three: ```sh awk '{ print length, NR }' notes.md | sort -rn | head -3 ``` ```text 800040 11 42 3 32 9999 ``` Line 11 is 800,040 characters long. The next longest is 42. That gap tells you the file has one monster line, not a general size problem. ### Pasted images Count lines with a `data:image` link, then see where they are. The `cut` keeps the output to 60 characters so your terminal doesn't fill with base64: ```sh grep -c 'data:image' notes.md grep -n 'data:image' notes.md | cut -c1-60 ``` ```text 1 11:![Settings page](data:image/png;base64,O9ia6vmWfiE77stWiB ``` There's the monster: line 11 is a pasted image. For how normal image links and relative paths work, see [Markdown images](markdown-images.md). ### Table rows Count lines that start with a pipe. Tables can be written without a leading pipe, so treat this as a quick check, not an exact count: ```sh grep -c '^|' notes.md ``` ```text 20002 ``` That's 20,002 table lines (the rows plus the header and its divider line). For table syntax itself, see [Markdown tables](markdown-tables.md). ### The headings List the `##` headings with their line numbers to see how the file is laid out: ```sh grep -n '^## ' notes.md ``` ```text 5:## Summary 9:## Screenshot 13:## Task log 20018:## Next steps ``` The Task log starts at line 13 and runs to line 20018. Almost the whole file is that one table, plus the image on line 11. ## Fix 1: move pasted images out to files Save this as `extract_images.py`. It finds base64 PNG, JPEG, GIF and WebP images written as plain `![alt](data:...)` links, writes each one to an `images/` folder next to the file, and swaps the link for a normal one. It uses only Python 3, with no extra packages. It is careful about your files. It writes a `.bak.md` copy of the original before it touches anything, and stops if that copy already exists. It skips any image name already taken in `images/`, so a later run never replaces an earlier picture. In the image name, any character other than a letter, digit, dot, dash or underscore becomes a dash, so the new link works. One limit: it also rewrites matching links inside code blocks, so check the result if the file shows Markdown examples. ```python import base64, pathlib, re, sys src = pathlib.Path(sys.argv[1]) out_dir = src.parent / "images" backup = src.with_name(src.stem + ".bak.md") pattern = re.compile(r"!\[([^\]]*)\]\(data:image/(png|jpeg|gif|webp);base64,([A-Za-z0-9+/=]+)\)") text = src.read_text(encoding="utf-8") count = len(pattern.findall(text)) if count and backup.exists(): sys.exit(f"stopped: {backup.name} already exists, move it away first") n = 0 def save(match): global n alt, ext, data = match.groups() ext = "jpg" if ext == "jpeg" else ext while True: # skip names already in images/, so nothing is overwritten n += 1 name = f"{re.sub(r'[^A-Za-z0-9._-]', '-', src.stem)}-{n}.{ext}" if not (out_dir / name).exists(): break (out_dir / name).write_bytes(base64.b64decode(data)) return f"![{alt}](images/{name})" if count: backup.write_text(text, encoding="utf-8") out_dir.mkdir(exist_ok=True) src.write_text(pattern.sub(save, text), encoding="utf-8") print(f"moved {count} image(s) to {out_dir.name}/") ``` Run it on the file: ```sh python3 extract_images.py notes.md ``` ```text moved 1 image(s) to images/ ``` Compare the new file with the `notes.bak.md` copy: ```sh wc -c notes.md notes.bak.md ``` ```text 660267 notes.md 1460271 notes.bak.md 2120538 total ``` The file dropped from 1,460,271 bytes to 660,267. Line 11 is now a short, ordinary image link: ```sh grep -n 'images/' notes.md ``` ```text 11:![Settings page](images/notes-1.png) ``` Run the script a second time and it finds nothing left to move, so it writes nothing: ```text moved 0 image(s) to images/ ``` The longest-lines check now looks normal: ```text 42 3 36 11 32 9999 ``` Once you've checked the new file looks right, you can delete the `.bak.md` copy yourself. ## Fix 2: split the file by heading If the file is still too big, split it into one file per `##` section, in a new `parts` folder: ```sh mkdir parts && awk '/^## /{n++} {print > sprintf("parts/part-%02d.md", n)}' notes.md ``` `mkdir` fails if `parts` already exists, and the `&&` then stops awk, so an earlier split is never overwritten. Each time awk sees a line starting with `## `, it bumps a counter and starts writing to the next part file. It does not know about code blocks, so a `## ` line inside one also starts a new part. Then check the sizes. This run was on the original `notes.md`, before the image fix: ```sh wc -c parts/part-*.md ``` ```text 67 parts/part-00.md 46 parts/part-01.md 800057 parts/part-02.md 660060 parts/part-03.md 41 parts/part-04.md 1460271 total ``` Everything before the first `##` heading, here the `# Release notes draft` title, lands in `part-00.md`. Then each section gets its own file: Summary in `part-01.md`, Screenshot in `part-02.md`, Task log in `part-03.md` and Next steps in `part-04.md`. The sizes point right at the two heavy sections. You can open the small parts on their own and deal with the big ones separately. If you split after Fix 1, a link such as `images/notes-1.png` inside a part now points into `parts/images/`, so move the `images` folder into `parts` or fix the links. If you end up with a folder full of parts and other agent output, [getting through a pile of agent-written Markdown](get-through-a-pile-of-agent-markdown-files.md) has a triage routine for many files. ## Read it in the meantime You can read a heavy file right now in `less`, which comes with macOS and is common on Linux: ```sh less -S notes.md less -N notes.md ``` - `-S` chops lines longer than the screen width instead of wrapping them. The rest of a long line stays hidden until you press the right arrow key, so one huge line takes up one row. - `-N` shows a line number at the start of each line, which helps when grep told you "line 11". - Type `/` and a pattern, then Enter, to search forward for it. Press `q` to quit. ## Habits that keep files small A few lines in your agent's instructions stop the problem at the source: 1. **Save images as files and link them.** Ask the agent to write screenshots to an `images/` folder and link them with a relative path, never paste them into the Markdown as data. 2. **Start a new file per run.** An agent that appends to the same log forever will grow it without limit. A new file per run, named with the date, keeps each one readable. ## Where Sashmark fits Sashmark is a Markdown reader and editor for people whose AI agents write Markdown, being built for Mac, Windows and Linux. It is in development and has not been released, and the app itself has no AI features. It is planned to render headings, lists, check-box lists, tables, code blocks, links, local images and block quotes, and to show front matter as a table through the planned Frontmatter Table. A good reader should show you what changed since you last looked, so you don't reread a whole report to find the new part. Sashmark is being built for this, with a planned Live Reload and Changes View; the [roadmap](/roadmap) shows where it stands. See the [planned features](/features) for the full list. Sashmark is in development. See what it is at [sashmark.com](https://www.sashmark.com/).