Assignment:
1. Continue from Quiz #2 diagram and follow the instructions.(Make sure all variables are in as interval except for the churn). Make sure training is set to 70% and validation is set to 30% in the data partition node.
2. Drag Neural Network node and attach it to partition (leave it as default)
a. What is ASE for training and validation set and at what iteration does the neural network produces lowest ASE?
b. What is the total number of inputs and weights in the neural network and briefly explain how weights are calculated.
c. Change the Neural network to 2 hidden units.
d. What difference do you notice? Explain your answer with evidence.
3. Drag variable selection node and attach it to partition (leave it as default).
a. Which variables did the "variable selection" node select?
b. Explain why other variables were rejected?
4. Drag Auto Neural node and attach it to variable selection node (leave it as default) and run it.
5. Out of the five models you created thus far.
a. Which one is the best in predicting customer churn?
b. How did you come up with this answer and based on what criteria?
c. Do you see signs of overfitting in the selected model? Please explain why or why not?
d. What Is the AUC value for validation data set for the selected model?
6. Score with the following file:
a. Subset of churn -
b. Please post a screenshot of the score results identifying row level output.
c. From your screen shot, identify if the following ID's will churn or not
|
ID
|
Churn or not (1 or 0)
|
Probability of Predicted Churn = 1
|
|
8
|
|
|
|
13
|
|
|
|
21
|
|
|
|
25
|
|
|
|
28
|
|
|
d. Please perform a sanity check! Does the probabilities and churn decision make sense to you? Please explain why or why not! (1 paragraph minimum)
7. Paste screenshot of the complete diagram here:
8. In this course we reviewed number of models. Since decision trees are relatively easy to interpret, management wants to implement the decision tree model to score new customers.
a. Score with the decision tree model.
b. Which ID's are predicted to churn from the small_Phone_churn_Score.xlsx data set?
c. What is the probability of churn and why is the probability same for all observations that are predicted to churn?
d. Post screenshot of the diagram: