jayeshpandey01/RLHF

Production-grade RLHF & post-training pipeline built from first principles in PyTorch. Features causal transformers, LoRA SFT, Bradley-Terry reward modeling, DPO, IPO, PPO with GAE, DeepSeek-R1 style

About RLHF

Autonomous Continual Learning LLM Prototype

An end-to-end, closed-loop system prototype for autonomous continual post-training in Large Language Models. The system integrates live web curation, hybrid retrieval, factual citation grounding, experience replay, parameter-efficient fine-tuning (LoRA), preference alignment (DPO/PPO/GRPO), and non-parametric statistical hypothesis release gating with zero-downtime atomic rollbacks.

[!NOTE] Implementation Scope: The default FastAPI application performs live web collection, curation, persistent hybrid indexing, and Ollama-backed grounded inference. Curated documents feed a separate Hugging Face Qwen LoRA post-training worker; candidate checkpoints are promoted into Ollama only when the statistical benchmark release gate approves them. For complete theoretical foundations, mathematical derivations, and architecture specifications, consult the 11-Chapter Academic Research Monograph.

Pipeline Stage Module Engineering Specification :--- :--- :--- 1. Web Ingestion ingestion/ Multi-source scraping via Trafilatura DOM parser, HTML text density ratios, SHA-256 provenance tracking, and rate-limited async fetching. 2. Curation & LSH curation/ MinHash Locality-Sensitive Hashing (LSH) for near-duplicate elimination ($s = 0.80$, 32 bands $ imes$ 4 rows), 13-gram benchmark contamination filtering, and lexical quality scoring. 3. Hybrid Retrieval retrieval/ Sparse Okapi BM25 + dense sentence embeddings fused via Reciprocal Rank Fusion (RRF, $k=60$) with bounded in-memory caching. 4. Citation Grounding evaluation/ Sentence-level ALCE citation r

From the project README.

Maintaining this server?

Add the radar badge to your README — it shows your project was picked up by MCP Radar and links to this page:

[![On MCP Radar](https://mcp.liqiwa.com/badge.svg)](https://mcp.liqiwa.com/s/jayeshpandey01--RLHF.html)

Related MCP servers