Klaviyo CEO on Building AI Products
How a 2,300-person company develops AI products
1Klaviyo's AI Product Development Process
Klaviyo co-founder and CEO Andrew Bialecki shared insights at SaaStr AI on how a 2,300-person company builds AI products. Klaviyo stood out in e-commerce by showing merchants the revenue generated by campaigns, not just email open rates. This approach helped Klaviyo capture 80% market share in the Shopify ecosystem and become one of the most beloved apps in 2023.
2Levels of AI Autonomy
Klaviyo defined AI autonomy levels similar to autonomous vehicles. L1 is using AI for search. L2 is starting a session and running an agent. L3 is constantly running multiple sessions or a team of agents. By the end of June, all employees, including PMs, designers, and sales teams, had to reach L3.
3Dark Factory: Agents Building Agents
Klaviyo's internal system, 'Dark Factory,' takes a prompt, acts as a PM, writes specifications, decomposes problems into engineering subsystems, and creates API interfaces between them. Subagents then build against each contract. The first Composer prototype was completed in a weekend.
4Composer's Success
Composer, Klaviyo's marketing agent, reached over 95,000 users in its first month, with about a quarter returning weekly. Credit consumption grew by 30% weekly. Composer includes agents for creativity and design, orchestration, and analysis, pulling from tools like Canva and Figma.
5AI Redefining Job Roles
At Klaviyo, the average PM, who previously wrote wireframes and specs, now has to reach L3. This redefinition applies to the entire organization. Andrew emphasized that this transformation is for the team's benefit, enabling them to succeed in the AI era, whether they stay at Klaviyo or not.
6Headless Approach and APIs
The headless approach is now the default, meaning products are treated as infrastructure. Anything requiring human login needs APIs. Klaviyo is building a path for users to sign up, configure, and pay without touching the UI, with a dedicated engineer focused on this experience.
7AI Changing User Onboarding
Software follows a power law of user sophistication: a few experts and a long tail of novices. Agents skip this curve, making onboarding less critical. What matters is what the agent asks you to build next. Agents become power users from day one, sitting alongside your best human users.
8Continuous Human Oversight in Dark Factory
Dark Factory raises flags when it encounters uncertainty. Throughout Friday, Saturday, and Sunday, it streams small questions: your requirements were unclear here, specify this. This continuous human oversight ensures the system works smoothly and makes human code reviews manageable.
9Composer's First Prototype
Composer's first working prototype was built in a weekend by other agents. This prototype laid the foundation for Klaviyo's marketing agent, reaching over 95,000 users in its first month. About a quarter of users returned weekly, and credit consumption grew by 30% weekly.
10AI's Organizational Impact
AI adoption at Klaviyo significantly changed the organizational structure. Everyone now writes code, from Andrew to summer interns. This is not just tool usage but a transformation redefining how the entire organization operates, serving as an important example of AI's role in business.
11AI's Future and Klaviyo
Klaviyo's AI strategy serves as a model for future business models. Integrating AI into product development processes increases efficiency and creates new opportunities. Klaviyo's success is a clear indicator of the transformative potential of AI in business.
12Conclusion and Importance
Klaviyo's AI product development process demonstrates AI's potential role in business. Innovations like AI autonomy, Dark Factory, and Composer have positioned Klaviyo as a leader in e-commerce. This transformation serves as an important example for the entire business world.
Original article: SaaStr — Klaviyo CEO'sundan Yapay Zeka Ürünleri İnşa Süreci