NLTK: Unsafe Pickle Deserialization in TransitionParser Allows Remote Code Execution

Summary

The NLTK library's TransitionParser.parse() method deserializes model files using pickle_load() with the default restricted=False parameter, allowing arbitrary Python code execution when loading a malicious model file. The library provides a RestrictedUnpickler class for safe deserialization, but it is never used by production code paths, leaving the vulnerability unpatched.

Root Cause

File: nltk/parse/transitionparser.py (lines 542-557)

The parse() method calls pickle_load(f) without restricted=True, routing through WarningUnpickler which inherits from pickle.Unpickler and does NOT override find_class(). This allows arbitrary class/function resolution during unpickling, enabling RCE via standard pickle gadgets (e.g., os.system, subprocess.Popen).

Vulnerability chain in nltk/picklesec.py:

def pickle_load(file, *, context=None, restricted=False):
    if restricted:
        return RestrictedUnpickler(file).load()  # Safe: blocks all globals
    return WarningUnpickler(file, context=context).load()  # VULNERABLE PATH

WarningUnpickler only emits a warning but does NOT block unsafe class loading — it calls super().load() which is standard pickle.Unpickler.load().

Why this is not by design:

• NLTK intentionally created RestrictedUnpickler to block unsafe deserialization
• The restricted=True parameter exists in the API but is never used by any production code path
• All call sites use the default restricted=False: transitionparser.py:557, parse/chartparser_app.py:816, parse/chartparser_app.py:2273, parse/chartparser_app.py:2311

Attack Surface

Entry point: TransitionParser().parse(depgraphs, modelFile) receives a filesystem path with no validation.

Exploitation path:

1. Attacker places a malicious pickle file at a known or attacker-controlled location
2. Victim calls parser.parse(sentences, "/path/to/malicious_model.pkl")
3. pickle_load() deserializes the file with restricted=False (default)
4. Standard pickle gadget chain executes arbitrary Python code with victim's privileges

Impact: Remote code execution with the privileges of the user running the NLTK-dependent application. Affects researchers, data scientists, and automated ML pipelines using NLTK for parsing tasks.

Steps to Reproduce

Environment

• NLTK version: 3.8.1+ (all versions with transitionparser.py)
• Python 3.6+
• No special dependencies required

Reproduction

1. Create a malicious pickle file that uses __reduce__ to execute a system command during deserialization.

2. Call TransitionParser().parse([], '/path/to/malicious_model.pkl').

3. The pickle_load(f) call at transitionparser.py:557 uses restricted=False by default, routing through WarningUnpickler, which does not override find_class() and permits full class resolution — executing the embedded gadget.

4. Arbitrary code executes with the victim's privileges.

Proof That the Fix Works

Changing line 557 in transitionparser.py from:

model = pickle_load(f)

to:

model = pickle_load(f, restricted=True)

causes RestrictedUnpickler to raise an UnpicklingError and block execution, confirming the safe path prevents the attack.

Working PoC

import pickle
import os
from nltk.parse.transitionparser import TransitionParser

# Create malicious pickle with RCE payload
class Exploit:
    def __reduce__(self):
        return (os.system, ('touch /tmp/nltk_poc_triggered',))

with open('/tmp/malicious_model.pkl', 'wb') as f:
    pickle.dump(Exploit(), f)

# Trigger the vulnerable code path (requires algorithm argument in ≤ 3.9.4)
parser = TransitionParser('arc-standard')      # or 'arc-eager'
parser.parse([], '/tmp/malicious_model.pkl')   # loads and unpickles unsafely

# Exploit succeeds: file /tmp/nltk_poc_triggered is created

On NLTK ≥ 3.10.0 (patched), the same code fails with:

_pickle.UnpicklingError: global 'posix.system' is not in the pickle allowlist

This proves the vulnerability exists in versions ≤ 3.9.4 and is fixed in 3.10.0+.

Recommended Fix

Change all call sites to use restricted=True:

| File | Line | Before | After | |------|------|--------|-------| | nltk/parse/transitionparser.py | 557 | pickle_load(f) | pickle_load(f, restricted=True) | | nltk/parse/chartparser_app.py | 816 | pickle_load(model_data_file) | pickle_load(model_data_file, restricted=True) | | nltk/parse/chartparser_app.py | 2273 | pickle_load(file) | pickle_load(file, restricted=True) | | nltk/parse/chartparser_app.py | 2311 | pickle_load(fp) | pickle_load(fp, restricted=True) |

Note: This fix may affect loading older sklearn models. A more robust approach would implement a module allowlist in RestrictedUnpickler.find_class().