Clean, curate, create and compose training data inside your own environment
Datology's data curation platform runs a four-stage pipeline in the customer's environment: Clean (source ingestion, heuristic filtering and benchmark decontamination at petabyte scale), Curate (quality and taxonomy annotation, redundancy reduction and quality-based resampling), Create (synthetic data generation for task relevance and diversity) and Compose (optimal mixing of sources into staged training datasets). It is aimed at teams mid-training open models or pre-training their own foundation models on proprietary, public and licensed data.