Measuring Incremental Lift in Customer Value at PayPal
Delta CV Analysis Using Synthetic Control
1Understanding Customer Value Lift
PayPal developed the Delta CV concept to measure the impact of customers adopting new products or performing specific actions. This metric represents the increase in profit margin in the first year. For example, customers adopting Crypto are expected to bring an additional $20 in profit on average.
2The Importance of Delta CV
Delta CV differs from Customer Lifetime Value (CLV). While CLV estimates the total profit generated by a customer over their relationship with PayPal, Delta CV focuses on how new adoptions affect the value of existing users. This provides critical data for strategic decisions.
3Direct and Indirect Effects
New product adoptions can increase customer value directly or indirectly. Direct effects occur when the product itself generates profit. Indirect effects occur when actions facilitate the use of other profitable products.
4Synthetic Control Method
PayPal uses causal inference and synthetic control to measure Delta CV. The treatment group consists of users adopting the product for the first time. The synthetic control group is selected from users who did not adopt the product and matched based on features.
5Measuring Treatment Effect
The synthetic control group is used as the counterfactual for the treatment group. When an intervention is applied to the treatment group, the difference in profit margin between the two groups measures the impact of the intervention. This is achieved using the KNN algorithm.
6Interpreting Delta CV
Delta CV metrics can often be skewed. Some users may bring significantly more value after adopting new products than others. Median Delta CV reflects the expected lift from a typical user, while mean Delta CV shows the financial impact.
7Model Limitations
Delta CV estimates are subject to variance and bias. It is important to consider the accuracy of predictions when making decisions based on the model's output. Additionally, the impact of adoptions is usually measured after 12 months, causing a delay.
8Applications of Delta CV
Delta CV is used in various areas such as calculating campaign ROI, determining opportunity sizing for new efforts, and making decisions on product launches. This metric enables PayPal to make data-driven strategic decisions.
Original article: PayPal Technology Blog — PayPal'da Müşteri Değerindeki Artışı Ölçmek