Meta paper outlines Agent design and lessons learned from large-scale AI-assisted A/B experimentation

Meta released the paper 'Learnings and Pitfalls from AI-Assisted A/B Experimentation at Scale', introducing its ABTest Agent: The LLM handles only selecting and ordering experimental steps; all numerical data is calculated by verified deterministic tools. Report templates require every table to be traceable to code, and the knowledge base includes only expert-reviewed documents.