Preference Model
RL environments for training capable, better-aligned frontier models
Community-submitted · content unverified. This organization has not verified ownership or the information in this profile.
About
Preference Model is a superintelligence data research company that builds reinforcement learning environments for frontier AI labs, treating reward hacking as a security problem by hardening its harnesses and graders against models probing them across millions of training rollouts. It was founded by Jennifer Zhou, previously an early member of Anthropic's data team who worked on Claude's pretraining data infrastructure, and Ning Cao, an early DatologyAI employee. The company is backed by a $16M seed round led by a16z, and on October 7, 2026 it open-sourced Karotte, its framework for building robust RL environments.
- Website
- preferencemodel.com
- Added via
- web
- Ownership
- unclaimed
Where to go
Products
- Karotte — Open-source framework for building robust, hack-resistant RL environments · Open Source