| Package | Affected versions | Patched versions |
|---|---|---|
| nltk | < 3.10.3 | 3.10.3 |
| nltk | < 3.10.3 | 3.10.3 |
Several corpus readers still step outside NLTK's symlink-aware trusted-root model. They derive in-root paths from trusted corpus state, convert those paths back into plain strings, and reopen them with built-in open() rather than nltk.pathsec.open().
nltk.corpus.reader.ipipan, nltk.corpus.reader.crubadan, nltk.corpus.reader.lin3.9.4 and current source v3.10.0-rc2 both reproduced.open() without preserving the trusted-root boundary.IPIPANCorpusReader opens header.xml derived from morph.xml, CrubadanCorpusReader opens table.txt directly, and LinThesaurusCorpusReader opens simN.lsp paths returned from its own root helpers. Under pathsec.ENFORCE=True, a symlink placed inside the trusted corpus root can point outside the root and still be parsed successfully. It was confirmed parsed outside-root content is returned through public methods such as channels(), domains(), categories(), langs(), crubadan_to_iso(), synonyms(), and scored_synonyms().
Preconditions
Steps
pathsec.ENFORCE=True with that root allowlisted.header.xml, table.txt, or simN.lsp inside the root and point them to external files.pathsec.open() blocks the same symlink targets.Minimal reproducible excerpt
Running one of these products? Scan your site to see whether it exposes a vulnerable version.
Scan my site free{'ipipan': ['LEAK', 'TOPSECRET', 'CLASSIFIED'], 'crubadan': ['LEAK'], 'lin': [('LEAK', 9.5)]}
An attacker who can stage corpus files or symlinks under a trusted data root can disclose outside-root content through normal corpus-reader results, defeating the boundary NLTK documents for shared and untrusted-input environments.
Preserve PathPointer and required_root semantics end to end. Replace direct open() calls with nltk.pathsec.open() or a reader helper that keeps the trusted-root boundary intact.
I swept every raw file open in the corpus readers, not just the three the umbrella named:
| Reader | Site | Advisory | Root scoping |
|---|---|---|---|
| crubadan | table.txt + <code>-3grams.txt | p4rw / j5pw | required_root=self.root |
| lin | simN.lsp | p4rw | required_root=self.root |
| xmldocs | XMLCorpusView bare-string fileid | 934p (base reader) | global fallback (view has no root) |
| pl196x | textids index | found by audit | required_root=self._root |
| mte | MTEFileReader | mvf5 | required_root threaded through 8 call sites |
| toolbox | StandardFormat.open codecs.open | cr8c | global sandbox (low-level parser) |
| named_entity | load_ace_file ann/text | 7qj2 | global sandbox |
| nkjp | XML_Tool source file | p4rw class | required_root=self._root |
ipipan already validates via the earlier #3727 fix — unchanged.
Each site now calls nltk.pathsec.validate_path(path, required_root=…) before opening. Where the reader has a concrete corpus root, the check is scoped with required_root (rejects any escape outside that root). XMLCorpusView carries no root, so it falls back to the global data-root sandbox via getattr(self, "_root", None) — which also avoids an AttributeError on the bare-string path.
raw open(symlink) reads: 'TOPSECRET_OUTSIDE_ROOT' <- the bypass
validate_path(symlink, required_root): ValueError -> BLOCKS the escape
validate_path(legit in-root): PASSED <- loads normally
The global-sandbox fallback (toolbox, named_entity, xmldocs-view) is only as tight as the allowed-roots list, which currently includes the system temp dir. Scoping every reader with required_root and removing the temp dir from the allowed roots would harden it further (separate advisory / task).
test_corpus_reader_pathsec.py — symlink escape rejected, in-root file allowed, XMLCorpusView string-fileid no AttributeError, MTEFileReader out-of-root rejected. 46 existing corpus/toolbox tests pass; all edited modules import (no circular import). pre-commit (black/isort/ruff) clean.
validate_path blocks every symlink escape variant (verified) and equals pathsec.open()'s guarantee, but does NOT block hardlinks (no symlink to resolve; tracked separately as GHSA-f794-5jv7-7672) or the validate-then-open TOCTOU race (shared by pathsec.open; needs O_NOFOLLOW/openat).