Written Assignment #2B requires hands-on practice of XLNfiner, and you are expected to use XLIVIiner to mine the Boston Housing data in the file BostonHousing.xls posted in Written Assignment #2B entry.
Your task is to use run k-nearest neighbors algorithm in XLIVfiner for both prediction and classification tasks describe below, and submit your answer with your XLMiner execution result files attached in your submission. Since the k-nearest neighbor algorithm can be used for both classification and prediction, there are two menus under XLIVfiner, Classify and Predict.
The file BostonHousing.xls contains information on over 500 census tracts in Boston, where for each tract 14 variable values are recorded. The last column (CAT.MEDV) was derived from MEDV, such that it obtains the value 1 if IVIEDV>30 and 0 otherwise. Consider the goal of predicting and classifying the median value NEE:40 of a tract, given the information in the first 12 columns. Partition the data into training (60%) and validation (40%) sets. (For description of the column names in BostonHousing.xls, please make reference to Table 2.2 on page 33 of the textbook, the 3.1 edition, on page 27 of the 2"' edition)
1. Under Predict menu in XLMiner, perform a k-nearest neighbors prediction with all 13 predictors (excluding the CAT.NIEDV column is the outcome or decision variable) for both training data set and validation data set, trying values of k from 1 to 10. Make sure to normalize the data (click "normalize input data'). What is the best k chosen? What does it mean? Also attach the execution result file including RIVIS Errors in your submission. (10 points)
2. Under Classify menu in XLIVfiner, perform k-nearest neighbors classification to generate classification results with all 13 predictors for both training data set and validation data set and find the best K for validation data set, trying values of k from 1 to 10. Also attach the execution file including confusion matrix, lift chart, and ROC chart in your submission.
Note
1. The file BostonHousing.xls is posted along Written Assignment #2B, and description of columns are given in the file.
2. The cloud based XLIVIiner is accessible by the URL: https://xlminer.com/
3. For the Windows based XLIVfiner, please check the XLIVIiner download instruction posted in Discussion in Blackboard