Karotte
Open-source framework for building robust, hack-resistant RL environments
github.com/preferencemodel/karotte
Community-submitted · content unverified. This organization has not verified ownership or the information in this profile.
About
Karotte is Preference Model's MIT-licensed framework for building reinforcement learning environments to train aligned AI. It ships secure defaults and primitives aimed at preventing reward hacking, sandbox escapes and grader crashes, such as killing stray processes before grading and rejecting files designed to crash the grader. Preference Model used it internally for about a year and says it was hardened through more than a million evaluation runs and controlled red-teaming before being open-sourced on October 7, 2026.
- Added via
- web
- Ownership
- unclaimed
Where to go
Product details
- Product kind
- Open Source