机器学习 / A LEAF IN THE GARDEN

加州2020房价预测

机器学习 · 动手深度学习 / 课程竞赛。来自 Daily Learning 的学习笔记。

2026/08/1911 分钟阅读更新于 2026/08/21

代码 2 · In [1]

import kagglehub
from pathlib import Path
import pandas as pd
import torch
import util

已保存输出

/root/autodl-tmp/envs/daily_learning/lib/python3.11/site-packages/requests/__init__.py:92: RequestsDependencyWarning: Unable to find acceptable character detection dependency (chardet or charset_normalizer).
  warnings.warn(
/root/autodl-tmp/envs/daily_learning/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm

数据处理

数据加载和读取
数据检查和分析
数据清洗

加载数据

代码 5 · In [2]

# kagglehub.login()
path = kagglehub.competition_download('california-house-prices', output_dir='./data/california-house-prices')
print("Path to competition files:", path)

已保存输出

Path to competition files: ./data/california-house-prices

读取数据并查看形状

数据	train	test
样本	47439	31626
特征	41	40

代码 7 · In [3]

path = Path(path)
train_data = pd.read_csv(path / 'train.csv')
test_data = pd.read_csv(path / 'test.csv')

del path
train_data.shape, test_data.shape

已保存输出

((47439, 41), (31626, 40))

查看数据结构

代码 9 · In [4]

train_data.dtypes

已保存输出

Id                               int64
Address                            str
Sold Price                     float64
Summary                            str
Type                               str
Year built                     float64
Heating                            str
Cooling                            str
Parking                            str
Lot                            float64
Bedrooms                           str
Bathrooms                      float64
Full bathrooms                 float64
Total interior livable area    float64
Total spaces                   float64
Garage spaces                  float64
Region                             str
Elementary School                  str
Elementary School Score        float64
Elementary School Distance     float64
Middle School                      str
Middle School Score            float64
Middle School Distance         float64
High School                        str
High School Score              float64
High School Distance           float64
Flooring                           str
Heating features                   str
Cooling features                   str
Appliances included                str
Laundry features                   str
Parking features                   str
Tax assessed value             float64
Annual tax amount              float64
Listed On                          str
Listed Price                   float64
Last Sold On                       str
Last Sold Price                float64
City                               str
Zip                              int64
State                              str
dtype: object

第 1 列为 ID 列，售价在第 3 列，提出所有特征继续分析，分离售价

代码 11 · In [5]

all_features = pd.concat((train_data.drop(columns='Sold Price'), test_data))
all_features.head()

已保存输出

Id Address Summary \ 0 0 540 Pine Ln 540 Pine Ln, Los Altos, CA 94022 is a single f... 1 1 1727 W 67th St HURRY, HURRY.......Great house 3 bed and 2 bat... 2 2 28093 Pine Ave 'THE PERFECT CABIN TO FLIP! Strawberry deligh... 3 3 10750 Braddock Dr Rare 2-story Gated 5 bedroom Modern Mediterran... 4 4 7415 O Donovan Rd Beautiful 200 acre ranch land with several pas...

       Type  Year built                                       Heating  \

0 SingleFamily 1969.0 Heating - 2+ Zones, Central Forced Air - Gas
1 SingleFamily 1926.0 Combination
2 SingleFamily 1958.0 Forced air
3 SingleFamily 1947.0 Central
4 VacantLand NaN NaN

                                         Cooling  \

0 Multi-Zone, Central AC, Whole House / Attic Fan
1 Wall/Window Unit(s), Evaporative Cooling, See …
2 NaN
3 Central Air
4 NaN

                                     Parking     Lot  \

0 Garage, Garage - Attached, Covered 1.0
1 Detached Carport, Garage 4047.0
2 0 spaces 9147.0
3 Detached Carport, Driveway, Garage - Two Door NaN
4 0 spaces NaN

                                        Bedrooms  ...  \

0 Ground Floor Bedroom, Master Bedroom on Ground… …
1 3 …
2 2 …
3 5 …
4 NaN …

                            Parking features  Tax assessed value  \

0 Garage, Garage - Attached, Covered 886486.0
1 Detached Carport, Garage 505000.0
2 NaN 49627.0
3 Detached Carport, Driveway, Garage - Two Door 1775000.0
4 NaN NaN

Annual tax amount Listed On Listed Price Last Sold On Last Sold Price
0 12580.0 2019-10-24 4198000.0 NaN NaN
1 6253.0 2019-10-16 525000.0 2019-08-30 328000.0
2 468.0 2019-08-25 180000.0 NaN NaN
3 20787.0 2019-10-24 1895000.0 2016-08-30 1500000.0
4 NaN 2019-06-07 1595000.0 2016-06-27 900000.0

      City    Zip State

0 Los Altos 94022 CA
1 Los Angeles 90047 CA
2 Strawberry 95375 CA
3 Culver City 90230 CA
4 Creston 93432 CA

[5 rows x 40 columns]

Address 全部为唯一值，得想办法提取
Summary 是广告语，可以删除
Lot 建地面积，数值范围 $[0, 10^{10}]$ 提示数据中存在异常值
Cooling、Heating 等是多分段字符串，不宜 one-hot
Bedrooms 应该是数值型，但有字符串列举，需要转换
School 名称应删除，分数、距离可保留但缺失值要特别处理
一些年月日字符串要转换为数值型
邮政编码数值意义不大，但可以提取地理信息

丢弃不需要的列

代码 14 · In [6]

include_cols = all_features.columns.difference(
    ['Id', 'Address', 'Summary'],
    sort=False)
all_features = all_features[include_cols]

del include_cols

能直接转整数的转

代码 16 · In [7]

# bedrooms 字符串列举的数出数量
def count_nums(x):
    return len(x.split(',')) if pd.notna(x) and isinstance(x, str) else x
all_features['Bedrooms'] = all_features['Bedrooms'].apply(count_nums)
# 日期类转为时间戳
neet_time_stamp = ['Last Sold On', 'Listed On']
def to_time_stamp(x):
    referance_date = pd.Timestamp('2000-01-01')
    return (pd.to_datetime(x) - referance_date).dt.days
all_features[neet_time_stamp] = all_features[neet_time_stamp].apply(to_time_stamp)

del neet_time_stamp

唯一值不多的，one-hot 编码

代码 18 · In [8]

needs_one_hot = ['State']
all_features = pd.get_dummies(all_features, columns=needs_one_hot, dummy_na=True)

del needs_one_hot

唯一值较多的，频率编码

代码 20 · In [9]

needs_frequency = ['City', 'Region',
                   'Zip', 'Type',
                   'Elementary School',
                   'High School',
                   'Middle School']
frequency = {i : all_features[i].value_counts(normalize=True) for i in needs_frequency}
for i in needs_frequency:
    all_features[i] = all_features[i].map(frequency[i]).fillna(0)

del frequency, needs_frequency, i

一列多值的字符串，multi-hot 编码

代码 22 · In [10]

need_multi_hot = ['Cooling', 'Cooling features',
                  'Heating', 'Heating features',
                  'Parking', 'Parking features',
                  'Flooring', 'Laundry features',
                  'Appliances included']
# 预处理多标签列，严格 tag1|tag2 格式小写
def preprocess_col_tags(x: pd.Series):
    def preprocess_tags(x):
        return '|'.join([tag.strip().lower() for tag in x.split(',')]) if pd.notna(x) else x
    return x.apply(preprocess_tags)
all_features[need_multi_hot] = all_features[need_multi_hot].apply(preprocess_col_tags)
# 多标签列进行 multi-hot 编码
for col in need_multi_hot:
    dummies = all_features[col].str.get_dummies()
    dummies = dummies.loc[:, dummies.mean() > 0.01] # 只保留出现频率大于 1% 的特征
    all_features = pd.concat([all_features.drop(columns=col), dummies], axis=1)

del dummies, need_multi_hot, col

用中位数填充缺失值，删除常量，标准化数值特征

代码 24 · In [11]

# 填充
all_features = all_features.fillna(all_features.median())
# 删除常量
std = all_features.std()
constant_columns = std[std == 0].index
all_features = all_features.drop(columns=constant_columns)
# 标准化
all_features = (all_features - all_features.mean()) / all_features.std()

del std, constant_columns

转换为 tensor，补充标签

代码 26 · In [12]

n_train = train_data.shape[0]
train_features = torch.tensor(all_features[:n_train].values, dtype=torch.float32, device='cuda' if torch.cuda.is_available() else 'cpu')
test_features = torch.tensor(all_features[n_train:].values, dtype=torch.float32, device='cuda' if torch.cuda.is_available() else 'cpu')
train_labels = torch.tensor(train_data['Sold Price'].values, dtype=torch.float32, device='cuda' if torch.cuda.is_available() else 'cpu').view(-1, 1)

# 下次应该封装函数的，省得到处 del 垃圾变量，这是 python 又不是 c :(
del all_features, n_train

模型

模型定义
数据加载
训练和验证

代码 28 · In [13]

from util import tuning

代码 29 · In [14]

in_features = train_features.shape[1]
loss = torch.nn.MSELoss() # 均方误差
k = 5
num_epochs = 100
batch_size = 512
supparams = {
    'lr': [0.01],
    'weight_decay': [0.1, 1],
    'kind': ['mlp'],
    'layers': [[32], [128], [64, 256]]
}

def log_rmse(net, features, labels):
    clipped_preds = torch.clamp(net(features), 1, float('inf')) # 将预测值限制在 [1, +∞) 区间，避免取对数时出现负数
    rmse = torch.sqrt(loss(torch.log(clipped_preds), torch.log(labels)))
    return rmse.item()

代码 30 · In [15]

tuning.tunning(
    train_features, train_labels,
    k=k, num_epochs=num_epochs, batch_size=batch_size,
    in_features=in_features, loss=loss,
    display_loss=log_rmse, supparams=supparams)

已保存输出

lr=0.01, weight_decay=0.1, kind=mlp, layers=[32]

已保存输出 · 错误记录

---------------------------------------------------------------------------KeyboardInterrupt                         Traceback (most recent call last)Cell In[15], line 1
----> 1 tuning.tunning(
      2     train_features, train_labels,
      3     k=k, num_epochs=num_epochs, batch_size=batch_size,
      4     in_features=in_features, loss=loss,
File ~/autodl-tmp/sync/Daily_Learning/机器学习自学/动手深度学习/util/tuning.py:141, in tunning(train_features, train_labels, k, supparams, num_epochs, batch_size, loss, display_loss, in_features)
    139     records.loc[len(records)] = [lr, weight_decay, kind, layers, train_l, valid_l]
    140 # 循环训练
--> 141 loop_tune(train_attempt, supparams, loss, display_loss)
    142 return records
File ~/autodl-tmp/sync/Daily_Learning/机器学习自学/动手深度学习/util/tuning.py:115, in loop_tune(train_attempt, supparams, loss, display_loss)
    113 else:
    114     for layers in supparams['layers']:
--> 115         train_attempt(loss, display_loss, lr, weight_decay, kind, layers)
File ~/autodl-tmp/sync/Daily_Learning/机器学习自学/动手深度学习/util/tuning.py:126, in tunning.<locals>.train_attempt(loss, display_loss, lr, weight_decay, kind, layers)
    124 def train_attempt(loss, display_loss, lr, weight_decay, kind, layers=None):
    125     print(f'lr={lr}, weight_decay={weight_decay}, kind={kind}, layers={layers}')
--> 126     train_l, valid_l = k_fold(k=k,
    127                               X_train=train_features,
    128                               y_train=train_labels,
    129                               num_epochs=num_epochs,
    130                               batch_size=batch_size,
    131                               loss=loss,
    132                               display_loss=display_loss,
    133                               learning_rate=lr,
    134                               weight_decay=weight_decay,
    135                               kind=kind,
    136                               layers=layers,
    137                               in_features=in_features)
    138     print(f'{k}-折验证: 平均训练 log rmse {float(train_l):f}, 平均验证 log rmse {float(valid_l):f}\n')
    139     records.loc[len(records)] = [lr, weight_decay, kind, layers, train_l, valid_l]
File ~/autodl-tmp/sync/Daily_Learning/机器学习自学/动手深度学习/util/tuning.py:93, in k_fold(k, X_train, y_train, num_epochs, batch_size, loss, display_loss, learning_rate, weight_decay, in_features, kind, layers)
     91 net = get_net(in_features=in_features, kind=kind, layers=layers)
     92 # 训练并记录误差
---> 93 train_ls, valid_ls = train_ones(net, *data, num_epochs, batch_size, loss, display_loss, learning_rate, weight_decay)
     94 train_l_sum += train_ls[-1] # 记录最后一个 epoch 的误差
     95 valid_l_sum += valid_ls[-1]
File ~/autodl-tmp/sync/Daily_Learning/机器学习自学/动手深度学习/util/tuning.py:53, in train_ones(net, train_features, train_labels, test_features, test_labels, num_epochs, batch_size, loss, display_loss, learning_rate, weight_decay)
     51 optimizer = torch.optim.Adam(net.parameters(), lr=learning_rate, weight_decay=weight_decay)
     52 for epoch in range(num_epochs):
---> 53     for X, y in train_iter:
     54         optimizer.zero_grad()
     55         # 这里用 MSELoss 计算损失，log_rmse 只在记录误差时使用
File ~/autodl-tmp/envs/daily_learning/lib/python3.11/site-packages/torch/utils/data/dataloader.py:721, in _BaseDataLoaderIter.__next__(self)
    720 def __next__(self) -> Any:
--> 721     with torch.autograd.profiler.record_function(self._profile_name):
    722         if self._sampler_iter is None:
    723             # TODO(https://github.com/pytorch/pytorch/issues/76750)
    724             self._reset()  # type: ignore[call-arg]
File ~/autodl-tmp/envs/daily_learning/lib/python3.11/site-packages/torch/autograd/profiler.py:932, in record_function.__exit__(self, exc_type, exc_value, traceback)
    927             self._cupti_monitor_external_id = monitor.push_user_annotation(
    928                 self.name
    929             )
    930     return self
--> 932 def __exit__(self, exc_type: Any, exc_value: Any, traceback: Any):
    933     if not torch.jit.is_scripting():
    934         if self._cupti_monitor_external_id is not None:
KeyboardInterrupt:

代码 31 · In [17]

tuning.train_and_pred(
    train_features, test_features,
    train_labels, test_data,
    num_epochs=num_epochs, batch_size=batch_size,
    loss=loss, display_loss=log_rmse,
    kind='mlp', layers=[256, 32],
    lr=0.01, weight_decay=0.5,
    keep_col=['Id'],
    target_col='Sold Price',
    path='./data/california-house-prices/prediction.csv')

已保存输出

训练 log rmse 0.419862