Programming guides for beginner...
Any comments are welcomed....
I hope it helps!!! Thanks for drop by...

Monday, June 22, 2026

Codex Logs Can Write 640 TB a Year to Your SSD

OpenAI shipped a release of the Codex CLI on 18 June 2026. The release notes mention a SQLite-related fix. They do not mention the bug. The bug — that the Codex CLI can write roughly 640 TB a year to the SSD it is installed on — is still open, still reproducible, and the latest release does not address it. If you are a developer who runs Codex as a long-lived background process, this is the part of the upgrade post you actually need to read.

The number, from the issue itself

Issue #28224 in openai/codex, opened on 14 June 2026 by user 1996fanrui, is the source for the 640 TB/year figure. The author's report is short and quantitative: on a 1 TB SSD, after 21 days of uptime, the main drive had written about 37 TB. Process-level and file-level checks show the Codex SQLite logs as the dominant continuous writer. Linear extrapolation: 37 TB in 21 days is roughly 1.76 TB per day, or 640 TB per year. On a 1 TB drive, that is 640 full-drive writes per year. Some consumer SSDs are warrantied at 600 TBW (terabytes written). The math is uncomfortable.

A second issue, #17320, opened on 10 April 2026, has the per-second view. The reporter observed sustained writes of approximately 5 MiB/s to ~/.codex/logs_2.sqlite-wal during model streaming, with peaks of up to 16 MiB/s in iotop. That is not the maximum — that is the floor. 5 MiB/s sustained, around the clock, is the lower bound. The number from #28224 extrapolates to about 18 MiB/s sustained; the gap is workload-dependent.

The 18 June release (rust-v0.141.0) does not touch this. It does touch SQLite, but a different bug.

The fix that shipped is not the bug you have

rust-v0.141.0 includes PR #27992, titled [codex] Pin bundled SQLite to fixed WAL-reset version, merged by gpeal. This is a real fix, and it matters — but for a different defect. The PR pins the bundled libsqlite3-sys dependency so that an unrelated transitive refresh cannot downgrade Codex's runtime from SQLite 3.51.3 back to 3.50.2. The 3.50.x line has a documented WAL-reset corruption bug; 3.51.3 is the patched version. Without the pin, a routine dependency refresh could silently drop you onto the broken release. The PR is defensive and correct.

It has nothing to do with the feedback-log write amplification. The write-amplification bug is not a SQLite version issue. It is a Codex logging-sink issue. Specifically, the logging sink writes to a SQLite database that is configured to retain TRACE-level entries, and it does so even when the parent process has RUST_LOG=warn set. The reporter on #17320 confirmed via /proc/<pid>/environ that the spawn was correct; via strace that the file descriptors were being written to; and via direct SQLite query that for a single 50-token response, the logs_2.sqlite table grew by about 5,000 rows, which were then pruned by a rotation pass that ran after the response. The volume of TRACE entries is the issue: the reporter's SELECT level, COUNT(*) breakdown showed TRACE entries at 68% of total log volume, INFO at 27%, DEBUG at 4%, WARN at 0.1%. The retention policy is not filtering by level.

The two bugs are easy to confuse because they both touch libsqlite3-sys. They are not the same. If you read the release notes and assumed the SQLite fix was the SQLite write problem, you are running a load-bearing assumption that the release notes do not support.

What the issue is actually about

The feedback-log sink in Codex is a separate code path from the standard tracing / RUST_LOG machinery. Issue #17320 reproduces a session where the process is launched with RUST_LOG=warn and the SQLite log nonetheless fills with TRACE entries. The maintainers have not yet committed to a fix; the issue is open, and there is no PR linked from it. The closest related work in the issue thread is #27911, #21134, and a stale pull request #12969, none of which addresses the bypass.

For the affected user, the practical shape of the problem is: install Codex, run it as a long-lived process, leave the workstation on for a few weeks, and watch your SSD's TBW counter climb. The issue tracker has the numbers. The fix is not in the most recent release. The pattern of "issue filed, maintainers acknowledge, no PR, more users pile on" is in its early days as of 22 June 2026. The HN thread has 284 points and 158 comments in under twelve hours, which is high-velocity for an OpenAI issue tracker thread.

The original take: the right fix is RUST_LOG, not a SQLite pin

Here is the part I am willing to argue about. The most likely fix path, based on the issue thread and the maintainer history, is the wrong one. The transitive-dependency-pinning class of fix (PR #27992) is appropriate for "a routine refresh could downgrade us to a known-broken library version." It is not appropriate for "our own code is writing 5 MiB/s of TRACE entries to a database that the user has no way to disable." Pinning the bundled SQLite does not stop the Codex logger from writing those rows. It pins a different bug.

The right fix is in the Codex logging crate. The reporter on #17320 has the right shape of the diagnosis: the logging sink should respect the process-level RUST_LOG filter, the way every other Rust binary does, and the way the spawn for the app-server process is already configured to do. The reason it does not is that the SQLite sink is on a separate code path, configured with its own filter, and that filter does not consult RUST_LOG. There are roughly three options for a fix: (a) wire the SQLite sink to the same filter the rest of the tracing stack uses, (b) default the SQLite sink to a level that excludes TRACE regardless of RUST_LOG, or (c) add a per-process knob so users can configure the retention level explicitly. Option (a) is the lowest-friction, most consistent with the Rust ecosystem, and most likely to land first. Option (c) is the most respectful of power users who actually want TRACE entries in the database. Option (b) is the most defensive and the easiest to ship.

The implementation cost of any of the three is small. The test cost is the part that will eat maintainer time. The fix is going to need a regression test that asserts, given RUST_LOG=warn, no TRACE entries are written to logs_2.sqlite during a 60-second idle session. If that test does not exist, the fix is not done. The 16 June fabrication in this blog's own record, on a different story, is the reason I am naming the test explicitly.

What this means for you

If you are a developer who has been running Codex as a long-lived background process, the immediate triage is: check your ~/.codex/ directory and your SSD's TBW counter. The files to look for are logs_2.sqlite, logs_2.sqlite-wal, and logs_2.sqlite-shm. If they are large, the bug is affecting you. If your SSD is older than two years, the warrantied TBW may already be in danger; check with the vendor's SMART diagnostic before you do anything else. The iotop view during a Codex streaming response is the real-time check: if you see Codex writing at 5 MiB/s or more, you are looking at this bug.

If you are a team lead evaluating Codex for a workstation pool, the risk profile is unchanged from a week ago. The bug is open, the fix is not in the latest release, and the mitigation is user-side. The honest answer to procurement is: do not run Codex as a long-lived daemon on a fleet of consumer-grade SSDs without monitoring. Run it as a foreground process for individual tasks. Run it on enterprise SSDs with high TBW ratings. Or, if your workload requires a long-lived Codex process, plan to monitor and rotate the logs manually until the fix lands.

If you are a maintainer of a similar tool — any Rust binary that ships with its own SQLite-backed log sink — the lesson generalizes. The standard RUST_LOG filter is the contract the Rust ecosystem agrees to. If your code path bypasses it, you owe your users a configuration surface, a default that does not write at TRACE, or a regression test that prevents the bypass from being reintroduced. The Codex issue is one manifestation; the pattern is the story.

What to do this week

# 1. Check whether the bug is affecting you right now.
ls -lh ~/.codex/logs_2.sqlite* 2>/dev/null
# If logs_2.sqlite-wal is large (>100MB), the bug is active.

# 2. Live observation — see the write rate during a Codex session.
# Run this in a second terminal while you do a Codex task:
sudo iotop -o -d 2 -n 30 | grep -i codex
# Sustained 5+ MiB/s is the issue. Peaks to 16 MiB/s are common.

# 3. Aggressive mitigation: stop the log sink until a fix ships.
# Move the database out of the way so the logger cannot reopen it.
# (Codex will recreate it; this only stops the current session.)
mv ~/.codex/logs_2.sqlite* ~/.codex/logs_2.sqlite.bak/ 2>/dev/null || true

# 4. Better mitigation: cap the log file size via logrotate, or
# set RUST_LOG=error for the app-server process. Neither fully
# fixes the bypass, but both reduce the write rate.

# 5. The durable fix: subscribe to issue #28224 and #17320 and
# wait for the maintainers to land a fix. Do not assume
# rust-v0.141.0 fixed it; the release notes do not say so.
gh issue view 28224 --repo openai/codex --web
gh issue view 17320 --repo openai/codex --web

If you maintain a CI fleet that uses Codex, add the iotop check to your nightly runbook for the next two weeks. If you are a single user with a single workstation, the move-and-rotate is fine as a stopgap. If you are considering this for production, the answer for the next 30-60 days is "no, not as a daemon." The bug is open, the fix is in flight, and the regression test has not yet been written.

Related on this blog

Disclosure

Drafted with AI assistance. The primary sources for this post are GitHub issue #28224 in openai/codex (1996fanrui, opened 2026-06-14, status OPEN), GitHub issue #17320 in openai/codex (opened 2026-04-10), the rust-v0.141.0 release page and changelog (tagged 2026-06-18), GitHub PR #27992 in openai/codex (gpeal, MERGED), the Hacker News thread for item 48626930 (vantareed, 284 points / 158 comments as of 2026-06-22 15:00 UTC+8), and the SQLite project's "The WAL Reset Bug" documentation linked from the PR #27992 description. Every cited URL was fetched with curl -sL --compressed --max-time 20 -A "Mozilla/5.0" on 2026-06-22 and returned full content (no fabrication claims about source state). The 640 TB/year figure, the 21-day / 37 TB measurement, the 1 TB SSD assumption, the 600 TBW consumer-SSD rating, the 5 MiB/s sustained and 16 MiB/s peak write rates, the 68% TRACE / 27% INFO / 4% DEBUG / 0.1% WARN level breakdown, the 5,000-rows-per-50-token-response rate, the RUST_LOG=warn spawn configuration, the libsqlite3-sys 0.35.0 → 0.37.0 (SQLite 3.50.2 → 3.51.3) downgrade path, the rust-v0.141.0 release date 2026-06-18, the related issues #27911 / #21134 and stale PR #12969, and the HN points/comments as of 2026-06-22 15:00 UTC+8 are all quoted or paraphrased from these sources. The argument that the right fix is in the Codex logging crate rather than a SQLite pin is this blog's analysis, not a maintainer claim. The "test that asserts no TRACE entries are written when RUST_LOG=warn" recommendation is this blog's test-design framing, not a maintainer commitment. The iotop recipe and the mv ~/.codex/logs_2.sqlite* ~/.codex/logs_2.sqlite.bak/ mitigation are practical commands a developer can run today; the durable fix requires a maintainer patch.

Sources

  • GitHub issue #28224, openai/codex — primary source for the 640 TB/year figure, the 21-day / 37 TB measurement, the 1 TB SSD extrapolation, the 600 TBW consumer-SSD comparison, and the three affected file paths (logs_2.sqlite, logs_2.sqlite-wal, logs_2.sqlite-shm): https://github.com/openai/codex/issues/28224
  • GitHub issue #17320, openai/codex — primary source for the 5 MiB/s sustained / 16 MiB/s peak write rate, the RUST_LOG=warn bypass, the 5,000-rows-per-50-token-response rate, and the TRACE / INFO / DEBUG / WARN level breakdown: https://github.com/openai/codex/issues/17320
  • rust-v0.141.0 release page, openai/codex — primary source for the 2026-06-18 release date and the changelog entry confirming PR #27992 is included: https://github.com/openai/codex/releases/tag/rust-v0.141.0
  • GitHub PR #27992, openai/codex — primary source for the bundled-SQLite WAL-reset pin, the libsqlite3-sys 0.35.0 → 0.37.0 (SQLite 3.50.2 → 3.51.3) downgrade path, and the maintainer gpeal: https://github.com/openai/codex/pull/27992
  • Hacker News item 48626930, "Codex logging bug may write TBs to local SSDs" — primary source for the 284 points / 158 comments community discussion as of 2026-06-22 15:00 UTC+8: https://news.ycombinator.com/item?id=48626930

No comments:

Post a Comment