NLTK: FileSystemPathPointer.open() sandbox check is dead code — arbitrary file read via file:// protocol

Summary

There's a logic bug in FileSystemPathPointer.open() inside nltk/data.py that makes the sandbox check permanently inert. The guard condition is always False — meaning any file the process can read is accessible by passing a file:// URL to nltk.data.load().

---

Details

In nltk/data.py, FileSystemPathPointer.open() was patched at some point with a comment saying "SECURITY PATCH ENFORCING SANDBOX", but the check doesn't work:

def open(self, encoding=None):
    path = os.path.normpath(self._path)

    # Block raw absolute reads such as "/" "C:\\Windows" etc.
    if os.path.isabs(path) and path != os.path.normpath(self._path):
        raise ValueError(f"Direct absolute file access blocked: {path}")

    stream = open(self._path, "rb")

path is set to os.path.normpath(self._path) on line 1, then compared against os.path.normpath(self._path) again in the condition. They are always equal. The ValueError never fires.

On top of that, __init__ already calls os.path.abspath() before storing self._path, so it's normalized before open() is even called. Running normpath on it again changes nothing.

The stream = open(self._path, "rb") line is always reached regardless of what path was passed in.

---

PoC

Tested on Python 3.11, NLTK 3.9.1, Ubuntu 22.04.

import nltk
from nltk.data import FileSystemPathPointer

# direct construction
ptr = FileSystemPathPointer("/etc/passwd")
with ptr.open() as f:
    print(f.read(300))

# via load() using file:// URL
data = nltk.data.load("file:///etc/passwd", format="raw")
print(data[:300])

Both print file contents. No exception is raised.

---

Impact

Any app that lets users influence the string passed to nltk.data.load() or nltk.data.find() is exposed — web APIs, notebook servers, multi-tenant pipelines. An attacker can read any file the process user has access to: /etc/passwd, .env files, private keys, ~/.aws/credentials, etc.

Suggested Fix

File: nltk/data.py — FileSystemPathPointer.open() (lines 378–390)

What's wrong

Line 387 compares normpath(self._path) against itself — always equal, so the ValueError never fires. The check is dead code. __init__ already calls abspath() on construction, so re-running normpath inside open() changes nothing either.

---

Fix

Validate against the actual list of permitted data directories instead:

def open(self, encoding=None):
    import nltk.data as _d
    allowed = [os.path.abspath(p) for p in _d.path if p]
    if allowed and not any(
        os.path.commonpath([self._path, r]) == r for r in allowed
    ):
        raise ValueError(
            f"Access outside nltk_data blocked: {self._path!r}"
        )
    stream = open(self._path, "rb")
    if encoding is not None:
        stream = SeekableUnicodeStreamReader(stream, encoding)
    return stream

---

Why commonpath not startswith

startswith is bypassable by a path that shares a prefix:

/tmp/nltk_data_evil".startswith("/tmp/nltk_data") → True  ✗
commonpath(["/tmp/nltk_data_evil", "/tmp/nltk_data"]) → "/tmp"  ✓

---

Diff

-    path = os.path.normpath(self._path)
-    if os.path.isabs(path) and path != os.path.normpath(self._path):
-        raise ValueError(f"Direct absolute file access blocked: {path}")
-
+    import nltk.data as _d
+    allowed = [os.path.abspath(p) for p in _d.path if p]
+    if allowed and not any(
+        os.path.commonpath([self._path, r]) == r for r in allowed
+    ):
+        raise ValueError(f"Access outside nltk_data blocked: {self._path!r}")
     stream = open(self._path, "rb")