Himanshu Pathak

Hi, I’m Himanshu Pathak.

I am a Reinforcement Learning research intern working on sequential decision-making: training agents that learn from reward signals, and studying why they behave the way they do.

My current interests span policy gradient methods, exploration strategies, and applying RL ideas to language model post-training. I share models and experiments on Hugging Face and open-source my code on GitHub.

Portrait of Himanshu Pathak

Posts

All posts →
  • The World Models Today

    How world models represent hidden state, learn dynamics, and turn predictions into decisions, with five animated explanations.

  • Policy gradients and the meaning of credit

    I gave a learner an action that always helped, then watched it learn from luck. An experiment about what an update can—and cannot—tell us.

  • Reward shaping without changing the task

    I changed the rewards, kept the better route, and watched the learner choose the worse one. A small experiment about what it means to give an agent advice.

  • What uncertainty can tell a planner

    I kept giving a model more evidence. Its intervals narrowed around the wrong answer. The useful question became which observation could change its mind.

An illustrated rover explores an isometric maze, with dotted trial paths and a green route toward a golden reward.
Learning a policy, one interaction at a time.

Projects & Open Source

All projects →
  • Let Execution Do the Teaching

    Execution-based post-training experiments for code models: supervised fine-tuning, GRPO reward pipelines, and reproducible C++ evaluations with test feedback.

    Model post-training · SFT · GRPO · C++ evaluation

    Fork of wootzapp/let-execution-do-the-teaching.

  • Supervisory Analytics for SOC Assessment

    A supervisory analytics project focused on assessing Security Operations Centre (SOC) operations.

    Security operations · Supervisory analytics

  • VaporWall

    A Rust network-security project combining eBPF/XDP packet filtering with asynchronous TCP stream analysis and local ONNX threat classification.

    Rust · eBPF / XDP · Tokio · ONNX

  • Fraud Radar — Real-Time Fraud Detection

    A transaction anomaly-detection pipeline using Isolation Forest, Kafka streaming, and a FastAPI service with live WebSocket updates. Includes synthetic data generation and Docker Compose orchestration.

    Python · Isolation Forest · Kafka · FastAPI

More code and experiments on GitHub.

A world of working systems An original architectural illustration of one floating island. A workshop and looping test track evoke execution-based learning; an observatory evokes security assessment; a translucent gateway evokes packet filtering; and a river of transaction lights passes an observation station. Their connections are artistic metaphors, not shared project infrastructure. N a small geography of experiments
Experiments in learning, observation, and trust. Explore the drawing ↗

Models & Datasets

Hugging Face →
  • GLM-4.7-Flash — Generalized C++ GRPO LoRA adapters · Training data · GRPO

    Portable checkpoints from a C++ post-training experiment, warm-started from the Phone Number GRPO adapter. The release includes the six-task training set, manifests, and reproduction evidence, with iteration 19 selected for release.

    Access conditions apply. The training tasks overlap Fixed26; reported evaluations measure assisted regression.

    Learning from consequenceAn ink sketch of code attempts branching toward tests. A failed attempt receives feedback and a revised attempt passes. This is a conceptual illustration, not a measured training trajectory. I. learning from consequence f(x) {try;return y;} a conjecture EXECUTE EXECUTEa useful failure revise, then test again What did the failure teach us?
    An execution-feedback sketch; not a measured training curve.
  • Phone Number — Kernel12 GRPO20 Training data · Adapter · WootzappLab

    The Phone Number post-training experiment, with links to its training JSONL, scored adapter, launch configuration, and evaluation method in the execution-teaching repository.

    The rules beneath the symbolsA fictional North American telephone number, +1 (202) 555-0147, becomes 2025550147 after removing permitted formatting and the leading country code. Letters must be rejected; area and exchange prefixes must begin from 2 through 9. II. the rules beneath the symbols +1 (202) 555-0147 a fictional number, many ways to write it ( ) -+ space 12025550147 drop leading 1 2025550147 abcreject letters 10 digits · prefixes start at 2–9 What may change? What must remain?
    A fictional example using the Phone Number task’s validation rules.
  • Stack v2 — C++ Task Candidates Dataset · Benchmark development

    Independently authored C++ task candidates developed from functionality discovered through Stack v2 metadata and Software Heritage. The collection preserves provenance and validation evidence for reviewing each candidate.

    Candidate stage; not approved for training or evaluation. Access conditions apply.

    What counts as knowing?An open research notebook presents the conjecture that every odd number is prime. The counterexample 9 equals 3 times 3 disproves it. A revised statement notes that every prime greater than 2 is odd. This is an illustrative reasoning example, not a claim about the dataset contents. III. what counts as knowing? CONJECTUREevery odd nis prime? 3 5 7 … 9= 3 × 3 one counterexample revised: prime n > 2 ⇒ n is odd Evidence can change the question.
    A reasoning sketch: one counterexample can undo a conjecture.
  • SimHim Model repository

    My SimHim model repository on Hugging Face.

Reading List

All resources →
  • Reinforcement Learning: An Introduction Richard S. Sutton & Andrew G. Barto

    The foundations: rewards, value functions, policy gradients, and learning through interaction.

  • Spinning Up in Deep RL OpenAI

    A practical introduction to deep reinforcement learning, with algorithm explanations and implementations.

  • The RLHF Book Nathan Lambert

    A guide to reinforcement learning from human feedback and language model post-training.