Question 1: Proposal critique
The Commerce School has a huge alumni base, but only recently has been working to engage them in a lifelong relationship with the school. It is not a good idea to bombard alumni with every possible fundraising opportunity. There are various different sorts of opportunities and engagements, and Stern wants to match them with the alumni for whom they seem to be best aligned. There are many positive advantages to such relationships; right now the School is interested specifically in increasing alumni giving. The Stern School administration has learned that you studied data mining for business analytics, and has asked you to help them assess a proposal from Blue Moon Consulting, to help increase alumni giving.
As a trial, Blue Moon has been asked to consider one fundraising engagement: the Undergraduate Scholarship Drive (the USD). Critique their proposal, below. Find the four most serious flaws in the proposal, and suggest how to rectify them. Your answer should comprise two sentences for each flaw: one stating the flaw, and one stating your suggestion for fixing it. You should accept as true any factual statement Blue Moon makes about what has happened in the past.
"We will mine the data from the prior USD campaign that was delivered to a random sample of 10,000 alumni. We propose to build a model to predict how much each alumnus will give, and then target those who will give the most. Stern has collected various data points on each alum, including demographic, geographic, major, year, interest, and first-job data, and stored it in the Alumni Database. We will use the amount donated as the target variable, and the data from the Alumni Database as the features. We will build a classification tree and a logistic regression from the data from the random campaign to estimate the amount donated. We will compare the models built based on the area under the ROC curve. The Stern administration has told us that they would like to target another 5000 alumni in the next test. The 5000 alumni with the highest area under the ROC curve will be targeted."
Flaw 1:
Solution:
Flaw 2:
Solution:
Flaw 3:
Solution:
Flaw 4:
Solution:
Question 2: Expected Utility Analysis for Data-Driven Decision Making
You're responsible for reducing fraud in the online store, among the 100 million customer accounts. Fraudsters break into customer accounts and buy digital goods, then they profit by selling these goods elsewhere. This is costly for us because we have to pay our suppliers the wholesale cost of the stolen goods. Plus we might even lose business if customers buy the stolen goods rather than purchasing from us.
The fraudsters have gotten pretty good at knowing how much or how little they should steal using a particular account, in order to get past the initial fraud screening. Nonetheless, you have gotten to the point that you can model pretty well the probability that an account has been compromised. You also can model pretty well how much you're likely to lose from a particular account, if it actually were compromised.
Management is worried about bothering and alarming the customers, so you should be careful in using your model to target corrective action (verifying transactions; forcing customers to change their password).
Design and explain how you will use models and modeling to decide on which accounts you will take corrective action. Do not worry about what modeling algorithms you will use; instead focus on what the models will be predicting and how to use those predictions to make decisions. Define clearly any additional things you need for your design. Describe clearly any simplifying assumptions you make.