DPO/SafeDPO/OPAD training + eval for teaching tool-using LLMs to refuse falsely-benign MCP exploits
- Halloran, John. "MCP Safety Training: Learning to Refuse Falsely Benign MCP Exploits using Improved Preference Alignment." arXiv:2505.23634 (2025). - Halloran, John T. "Leveraging RAG for Training-Free Alignment of LLMs." arXiv:2605.11217 (2026).
DPO / SafeDPO training and evaluation code for aligning tool-using LLMs against falsely-benign MCP exploits (FBAs) — CVE-derived Model Context Protocol tool-use attacks phrased as ordinary, harmless-sounding requests.
No safety-tuned model (1B–14B params) refused more than 35% of FBAs out of the box. Standard DPO/SafeDPO alignment only pushed that to 48% at best. RAG-Pref, the training-free retrieval-based method proposed alongside this code, gets ~3x refusal-rate improvement alone and ~3.7x combined with DPO/SafeDPO — see arXiv:2505.23634 and arXiv:2605.11217 for full numbers.
Figure from arXiv:2605.11217. All bars shown are implemented in this repo: Base/DPO/SafeDPO via dpo.py/safedpo.py/mcptestcache.py, Vanilla RAG/RAG-Pref via makeragdbs.py/ragpref.py (see Safety Alignment Methods).
- [x] DPO / SafeDPO training + evaluation - [x] OPAD (on-the-fly, training-free principle-guided decoding) - [x] RAG-Pref (training-free retrieval-based alignment) - [x] Vanilla RAG baseline (ragpref.py --vanilla-rag)
File Purpose ------ dpo.py Standard DPO training entry point (TRL's DPOTrainer), 4-bit QLoRA. safedpo.py SafeDPO training entry point; swaps in SafeDPOTrainer for TRL's DPOTrainer. safedpotrainer.py SafeDPOTrainer, a DPOTrainer subclass adding a safety-penalty term for preference pairs flagged betterisunsafe/worseisunsafe, and a
From the project README.
Add the radar badge to your README — it shows your project was picked up by MCP Radar and links to this page:
[](https://mcp.liqiwa.com/s/johnhalloran321--mcp_safety_training.html)
Indirect prompt injection sanitizer and delimiter boundary firewall neutralizing embedded command overrides
Alpha-Park/genpark-adversarial-prompt-jailbreak-detector-skillAdversarial prompt jailbreak detector analyzing framing evasion, hypothetical bypasses, and token smuggling
alphaparkinc/genpark-adversarial-prompt-jailbreak-detector-skillAdversarial prompt jailbreak detector analyzing framing evasion, hypothetical bypasses, and token smuggling
alphaparkinc/genpark-covert-data-exfiltration-firewall-skillEgress data exfiltration firewall detecting obfuscated credentials, private keys, and API tokens in payloads
Alpha-Park/genpark-indirect-prompt-injection-sanitizer-skillIndirect prompt injection sanitizer and delimiter boundary firewall neutralizing embedded command overrides
alphaparkinc/genpark-agent-tool-permission-rbac-validator-skillRole-based access control and tool privilege attenuator preventing unauthorized tool invocations
The top new MCP servers of the week, every Monday. No spam, unsubscribe anytime.