Ames Housing Price Prediction using Multi-Model Regression & Advanced Feature Selection

This repository contains a production-grade Machine Learning Regression pipeline evaluated on the comprehensive Ames Housing dataset (2,930 samples, 79 structural features)[cite: 6]. The framework implements non-linear target scaling, systematic missing-value imputation, multi-model evaluation (probabilistic, neural, and maximum-margin), and a deep impact analysis of embedded vs. filter-based feature selection methods[cite: 6, 7].

---

🚀 System Architecture & Methodology

1. Robust Preprocessing Pipeline

To counter data heterogeneity, multicollinearity, and missing records, a ColumnTransformer architecture was engineered:

• Target Transmutation: Applied a log-transformation ($y = \log(1+x)$) to smooth the positive skewness of SalePrice, realigning the feature boundaries with standard statistical regression assumptions.
• Numerical Processing: Missing values are resolved using median imputation to negate outlier effects, followed by standard scaling (StandardScaler).
• Categorical Processing: Handled via most-frequent mode imputation and converted using one-hot encoding, built to dynamically handle unknown categories during cross-testing.

2. Regression Algorithms Under Review

The dataset was benchmarked across 6 distinct algorithmic topologies[cite: 6, 7]:

• Classical & Accelerated: Linear Regression & Stochastic Gradient Descent (SGD) Regressor[cite: 6, 7].
• Probabilistic & Kernel: Bayesian Ridge Regression & Gaussian Process Regression (equipped with DotProduct and WhiteKernel structures)[cite: 6, 7].
• Non-Linear Classifiers: Support Vector Regression (SVR with RBF kernel) & Multi-Layer Perceptron (MLP Neural Network with a 100, 50 structural shape)[cite: 6, 7].

---

📊 Performance Benchmarking & Metrics

All algorithms were evaluated using four complementary dimensional error metrics: MAE, MSE, RMSE, and $R^2$ Score[cite: 6, 7].

Base Model Performance Summary

• SVR (Kernel RBF) and Bayesian Ridge demonstrated clear empirical excellence, with SVR reaching a peak $R^2$ Score of 0.9295 and an exceptionally minimized RMSE (~0.1142)[cite: 8].
• Linear Regression performed with high stability (~0.8857 $R^2$), proving that linear assumptions hold strongly after proper log-scaling[cite: 8].
• SGD and MLP architectures captured the structural variance well but showed higher sensitivity to global hyperparameters[cite: 8].

---

📉 Visualizing Error Boundaries

The regression plots below contrast actual vs. predicted values against a theoretical perfect prediction boundary ($y = x$). SVR and Bayesian Ridge showcase tight convergence clusters, while SGD highlights sparse residuals across lower valuation spaces.

---

📈 Impact Analysis of Feature Selection

A core focus of this project is analyzing how filtering and embedding reduction models influence structural regression performance[cite: 6]. We compared:

1. Chi-Square ($Key=50$): High statistical correlation filtering[cite: 7].
2. Lasso Penalty: Embedded dynamic reduction dropping zero coefficients[cite: 7].
3. Adaptive Lasso: Weighted penalization utilizing Ridge regression coefficients to avoid biased estimations[cite: 7].

Key Takeaway

Adaptive Lasso achieved the most optimal equilibrium[cite: 8]. It compressed feature density by over 40% while preserving or marginally improving baseline $R^2$ metrics[cite: 7, 8]. Chi-Square reduction decreased the parameters drastically but introduced minor predictive drop-offs due to its inability to map latent non-linear cross-interactions[cite: 8].

---

💻 How to Run This Project

1. Clone the Workspace

git clone [https://github.com/mrhashx/ames-housing-regression-analysis.git](https://github.com/mrhashx/ames-housing-regression-analysis.git)
cd ames-housing-regression-analysis

2. Set Up Environment Packages

pip install numpy pandas matplotlib seaborn scikit-learn xgboost

3. Run the Evaluation Suite

python housing_regression.py

🛠️ Tech Stack & Dependencies

Language Framework: Python 3.x[cite: 7]

Data Wrangling Suite: Pandas, NumPy[cite: 7]

Machine Learning Suite: Scikit-Learn[cite: 7]

Visual Engines: Matplotlib, Seaborn[cite: 7]

Note: This predictive analytics pipeline was successfully designed, optimized, and cataloged in January 2026.