Joint Work with Brett Gordon, Mitchell J. Lovett, and Bowen Luo
Management Science, 2022
We study the effects of positive and negative advertising in presidential elections. We develop a model to disentangle these effects on voter turnout and candidate choice. The central empirical challenges are highly correlated and endogenous advertising quantities that are measured with error. To address these challenges, we construct a large set of potential instruments, including interactions with incumbency that we demonstrate provide the critical identifying variation, and apply machine-learning causal inference methods. Using data from the 2000 and 2004 U.S. presidential elections, we find that positive and negative ads play fundamentally different roles. Negative ads are more effective at driving relative candidate shares, whereas positive ads stimulate turnout. These results indicate that a candidate geographically targeting tone trades off local relative share gains and local increases in turnout for localities with a strong base. Counterfactual simulations, where the candidates adjust the quantity of positive and negative advertising while budgets remain fixed, indicate that ad tone alone can impact the outcome of close elections. Our analysis also provides potential explanations as to why past studies have produced mixed findings on both ad-tone and turnout effects.
Joint Work with Paul B. Ellickson and Wreetabrata Kar
Marketing Science, 2022
We estimate the causal effects of different targeted email promotions on the opening and purchase decisions of the consumers who receive them. To do so, we synthesize and extend recent advances in causal machine learning techniques to capture heterogeneity in the content of the email subject line itself, as well as heterogeneous consumer responses to the promotional offers and semantic choices contained therein. We find that content and framing are important for driving performance. We identify precise causal estimates of the effects of individual deal components, personalized content, and various semantic choices on consumer outcomes all the way down the conversion funnel. The decompositional nature of our methodology allows us to show how different combinations of keywords and promotional inducements produce significantly different outcomes, both within a given stage and across all stages of the funnel. Notably, discounts framed as clearance events sharply outperform those tied to particular products. We also find that components that drive engagement at the top of the funnel don't always lead to conversion at the bottom: their efficacy, across the funnel, is significantly moderated by the engagement levels of the consumers who receive them. Finally, leveraging both aspects of heterogeneity, we use off-policy evaluation to demonstrate the potential for significant gains from improved targeting.
Joint Work with Wreetabrata Kar and Gary L. Lilien
Revise and Resubmit at the Journal of Marketing Research
Most B2B buyer-seller relationships involve a connection between a sales rep and a customer. If the relationship is suboptimal, the selling firm can move customers from one sales rep to another, a process we call proactive reassignment. Historically, geographic considerations severely constrained feasible reassignments. Technological advancements have made virtual connections between a customer and a rep the norm, removing geographic constraints on proactive reassignment. Using data from a quasi-field experiment in a partner seller firm and machine learning methodology, we develop a decision support system (DSS) to guide sales managers in revenue-enhancing customer-rep reassignments. We find that reassigning a customer to a long-standing rep does not significantly affect post-period sales, but reassigning to a new rep typically leads to a decline in sales. Significantly, our methodology identifies situations when proactive replacement, even with new reps, can be beneficial. For instance, customers with past relationship disruptions perform better when reassigned to new reps than to other, existing reps. Integrating these findings into the DSS, we identify numerous situations where future proactive reassignments can increase the focal firm’s revenue. Our counterfactual analysis shows a potential 14% improvement in post-period revenue compared to the selling firm's actual actions.
Joint Work with Candace Jens and T. Beau Page
Causal forest is part of a growing class of doubly-robust machine learning based estimators that non-parametrically recovers heterogeneity in treatment effects. However, causal forest's usefulness is currently limited because the group-level heterogeneity present in many economics settings violates a key assumption of causal forest. We provide a solution: estimate group-level fixed effects in a regression, create a vector of estimated group-level coefficients, and include this vector in the casual forest estimation. We use a series of Monte Carlo experiments and two applications to demonstrate our solution's success and the shortcomings of alternatives. We compare techniques' recovery of both average and heterogeneity in treatment effects. Examining bias in observation-level effects is important because a key benefit of forests is their ability to recover heterogeneity. We show that a key driver of differences between techniques' performances is the number of groups relative to sample size. In simulated settings with fewer, larger groups, several alternative techniques (e.g., binary group-level fixed effects and a vector of group-level average outcome) recover relatively unbiased average effects. However, in no scenario do these alternatives recover average effects with lower bias than our method. Moreover, these alternatives struggle to capture heterogeneity in effects, recovering more biased observation-level treatment effects than our solution across all simulated settings. We confirm our Monte Carlo results that show the effectiveness of our solution in two applications, one with randomized treatment and the other using observational data. Our study greatly increases the number of settings in which unbiased, heterogeneous treatment effects are recoverable.
Joint Work with Daniel Kebede
Revise and Resubmit at the Journal of Marketing
This paper examines the extent that political polarization moderates the effectiveness of promotions in driving public policy outcomes. Specifically, heterogeneous effects of COVID-19 vaccination promotions on county-level vaccination rates are studied to quantify the effect. A central challenge of this work is both accounting for the endogenous action of state-level officials and modeling the underlying diffusion pattern associated with vaccination rates. Blending the utility-based Bass Diffusion model of Cosguner and Seetharaman (2022) with counterfactual estimation techniques and machine learning solves the aforementioned challenges. Though other studies show a positive effect of promotions on vaccinations, this paper finds the opposite. Studying the estimated heterogeneous effects show that the underlying negative response is driven by vaccination hesitation and political ideology causing a backfire effect, which enhances an individual's resistance to vaccination. For example, counterfactual analysis shows that in regions where the promotions had a negative impact, the result is a 1.28% reduction in the vaccination rate per period. The net impact of these broad vaccination promotions is approximately 195,000 individuals delayed or forgo vaccination altogether. This paper shows that a backfire effect can manifest even with small incentives in a politically charged environment.
Joint Work with Mohammad S. Rahman
Preparing for Submission to ISR
The digital revolution has transformed the way we interact with each other. It is unknown how effective these digital interactions are compared to in-person interaction within a high-touch environment. This study delves into this question by comparing the relative value of in-person and virtual interactions (e.g., WebEx, Facetime) in the context of direct sales. Using a pseudo-random experiment with thousands of clients overseen by agents from a leading medical device firm, this paper explores the effects of switching from in-person visits (IPV) to a combination of remote visits (RV) and IPV. The findings reveal that, on average, IPV is nearly one-and-a-half times more valuable than RV. By leveraging machine learning techniques, this study uncovers factors that can either widen or narrow the gap between IPV and RV. For example, clients with stronger ties to the company are less responsive to RV, thus increasing the gap. Conversely, clients in technology-friendly regions are more responsive to RV, closing the gap to near parity with IPV. This study provides valuable insights for businesses looking to optimize their sales strategies in the digital age.
Joint Work with Arie Beresteanu, Paul B. Ellickson, and Sanjog Misra
Preparing for Submission to Management Science
This paper examines competition between supermarkets as a dynamic discrete game between heterogeneous players. We focus on the overall impact of Wal-Mart’s entry on incumbent supermarket firms, quantifying the effects on prices, producer surplus, consumer welfare and overall competitive structure. Employing a thirteen-year panel dataset of store level observations that includes every supermarket firm operating in the United States alongside the rapid proliferation of Wal-Mart Supercenters, we propose and estimate a dynamic structural model of chain level competition in which incumbent firms choose each period whether to add or subtract stores or exit the market entirely, and potential entrants choose whether or not to enter. Product market competition is modeled as a discrete-choice demand system, incorporating detailed information on prices and characteristics of chains, as well as unobserved heterogeneity in chain-level quality. Our estimation approach combines two-step estimation techniques with a novel random forest based value function approximation algorithm to accommodate the high-dimensional structure of the underlying state space.
Joint Work with Paul B. Ellickson and Wreetabrata Kar
Finalizing the Working Paper
In this paper, we propose a novel framework that leverages firm-level data and generative AI to develop and assess new email promotions. To account for the large number of possible emails, we leverage the method proposed in Ellickson, Kar, and Reeder (2023) to account for observational data and obtain the heterogeneous effects of heterogeneous treatments. To then link our outcome model with ChatGPT, we use the Ada embeddings from OpenAI. We show the benefit of using Ada embeddings in describing textual data in prediction exercises. Next, we highlight methods to assess the needed feature saturation to conduct off-policy evaluation. Last, we provide ChatGPT with selected features and use our framework to generate the predicted heterogeneous outcomes. We find that a naïve manager using ChatGPT to develop email promotions may, in fact, be selecting suboptimal choices. By using our framework, a marketing manager can calibrate the responses of ChatGPT to make a more optimal selection.
Joint Work with Jia Li and Paul Nelson
Joint Work with Mohammad S. Rahman
Estimation in Progress
Joint Work with Paul Ellickson and Wreetabrata Kar
Finalizing Estimation and Preparing Manuscript
Copyright © 2023 James C. Reeder, III - All Rights Reserved.
Updated Sept. 2023