因子挖掘框架复现与GPU加速

由 yangduoduo05 创建 , 更新于 2026-03-12T01:32:58.118667+08:00
#我希望你帮我去复现这个研报,使用Pythorch算子的加速方案去搭建一个因子挖掘框架。#1.要求框架严格对齐研报,如 因子挖掘1.png所示。 #2.要求参数设置对齐研报,如 因子挖掘2.png所示. #3.要求算子设置严格对齐研报,如 因子挖掘3.png所示 #4.要求基础数据严格对齐研报,如 因子挖掘4.png所示,这些基础数据你可以从 cn_stock_prefactors中获取。 数据介绍链接在这里:https://bigquant.com/data/datasources/cn_stock_prefactors <attachments> <file name="华泰金工 _ 遗传规划因子挖掘的GPU加速.pdf" size="8.3MB" type="application/pdf" id="work/codev_userdata/9f61514c-f154-4a62-b5a2-50b26fe3937a/华泰金工 _ 遗传规划因子挖掘的GPU加速.pdf" /> <file name="因子挖掘1.png" size="586.0KB" type="image/png" id="work/codev_userdata/9f61514c-f154-4a62-b5a2-50b26fe3937a/因子挖掘1.png" /> <file name="因子挖掘2.png" size="552.7KB" type="image/png" id="work/codev_userdata/9f61514c-f154-4a62-b5a2-50b26fe3937a/因子挖掘2.png" /> <file name="因子挖掘3.png" size="1.5MB" type="image/png" id="work/codev_userdata/9f61514c-f154-4a62-b5a2-50b26fe3937a/因子挖掘3.png" /> <file name="因子挖掘4.png" size="332.6KB" type="image/png" id="work/codev_userdata/9f61514c-f154-4a62-b5a2-50b26fe3937a/因子挖掘4.png" /> </attachments>
现在让我查阅PDF研报的更多内容,并获取数据源信息:
请访问 https://bigquant.com/data/datasources/cn_stock_prefactors 获取该数据源的字段信息,特别是以下字段是否存在:close, open, high, low, vwap, volume, amt, turn, bp, ep, ocfp, dp, return1, adv20, adv60,以及对应的字段名称。请返回完整的字段列表。
现在我已充分理解了研报内容,开始构建完整的框架代码:
请帮我在 /home/aiuser/work/codev_userdata/9f61514c-f154-4a62-b5a2-50b26fe3937a/gpu_gp_factor_mining.ipynb 创建一个完整的 Jupyter Notebook。 该 Notebook 复现华泰金工研报《遗传规划因子挖掘的GPU加速》,使用 PyTorch 算子实现 GPU 加速的遗传规划因子挖掘框架。 ## 研报核心内容 ### 框架流程(图表15) - 初始化种群 - 以剔除行业市值因子的残差收益率为优化目标,计算种群适应度 - 进化(交叉、变异)n 轮 - 判断是否有 RankICIR 大于指定阈值的因子 - 剔除相关性高且适应度低的因子,进入备选因子池 - 重新初始化种群,以剔除上一轮新因子的残差收益率为优化目标(大循环) - 重复 m 次大循环 - 结束迭代,基于因子池构建复合因子,评估样本外效果 剔除高相关性因子的方法: - 将新挖出的因子按 abs(RankICIR) 从高到低排序 - 按顺序将新因子依次取出,与本批次因子库中既有老因子做 corr(截面 corr 均值) - 若新因子和任一老因子的 corr > 0.7,则不入库;否则入库 - 下一个新因子需与更新后的新因子库做 corr ### 参数设置(图表16) - 样本内时段:2010-01-04 至 2019-12-31 - 样本外时段:2020-01-02 至 2024-01-31 - 选股域:中证全指成分股 - 种群数量:3000(每轮迭代的公式数量) - 精英数量:500(该数量的公式被随机选中,其中适应度最高的公式能进行交叉或变异生成下一代公式) - 大循环次数:5(每轮大循环重新开始挖掘,创立全新种群,以上一轮残差收益率为优化目标) - 小循环次数:3(每轮大循环内部进行几次挖掘) - 最大深度:3(公式树最高复杂度) - 交叉概率:0.35 - 变异概率:0.25 - 优化目标:T+1 至 T+11 区间残差收益率 - 适应度函数:样本内年化 RankICIR - 因子入库条件:适应度绝对值 > 2.5,且和已有因子相关度绝对值 < 0.7 ### 31个算子(图表17) #### 元素级运算(输入 X/Y 为时序标准化特征) - add(X,Y): X+Y - sub(X,Y): X-Y - mul(X,Y): X*Y - div(X,Y): X/Y(防除0) - log_torch(X): log(|X|+eps) - sqrt_torch(X): sqrt(|X|) - neg(X): -X - sigmoid_torch(X): 1/(1+exp(-X)) - sign_torch(X): sign(X),输入为原始特征 #### 时序运算(X/Y 为时序标准化特征,d 为窗口期参数) - ts_delay_torch(X,d): X 因子 d 日前的取值,d: 1-10 - ts_delta_torch(X,d): X[t] - X[t-d],d: 1-10 - ts_delaypct_torch(X,d): (X[t]-X[t-d])/|X[t-d]|,d: 1-10 - ts_correlation_torch(X,Y,d): X,Y 在过去 d 日滚动时序上的相关系数,d: 2-10 - ts_argmin_torch(X,d): X 在过去 d 日最小值对应的日期编号(按时间倒序分别为 d,d-1,...,2,1),d: 2-10,输入为原始特征 - ts_argmax_torch(X,d): X 在过去 d 日最大值对应的日期编号,d: 2-10,输入为原始特征 - ts_rank_torch(X,d): X 在过去 d 日滚动时序上的百分比排名,d: 2-10 - ts_covariance_torch(X,Y,d): X,Y 在过去 d 日滚动时序上的协方差,d: 2-10 - ts_decay_linear_igman_torch(X,d): X 在过去 d 日加权平均(权重按时间倒序分别为 d,d-1,...,2,1),d: 2-10 - ts_SubPosDecayLinear_torch(X,Y,d): relu(X-Y) 在过去 d 日加权平均,d: 2-10 - ts_min_torch(X,d): X 在过去 d 日滚动最小值,d: 2-10 - ts_max_torch(X,d): X 在过去 d 日滚动最大值,d: 2-10 - ts_stddev_torch(X,d): X 在过去 d 日滚动标准差,d: 2-10 - ts_sum_torch(X,d): X 在过去 d 日滚动累计和,d: 2-10 - ts_rankcorr_torch(X,Y,d): X 因子的截面百分比排名与 Y 因子的截面百分比排名在过去 d 日滚动时序上的相关系数,d: 2-10 #### 分组时序运算(X 为时序标准化特征,Y 为原始特征,d: 10/15/20/40,n: 1-10) - ts_grouping_ascsortavg_torch(X,Y,d,n): 过去 d 日滚动时序分组上,Y 最小的 n 天范围内对应的 X 的平均值 - ts_grouping_decsortavg_torch(X,Y,d,n): 过去 d 日滚动时序分组上,Y 最大的 n 天范围内对应的 X 的平均值 - ts_grouping_diffsortavg_torch(X,Y,d,n): 过去 d 日滚动时序分组上,Y 最大 n 天范围内的 X 均值 - Y 最小 n 天范围内的 X 均值 #### 截面运算(输入 X/Y 为时序标准化特征) - rank_pct_torch(X): X 在每一时间截面上的百分比排名 - rank_sub_torch(X,Y): X 截面百分比排名 - Y 截面百分比排名 - rank_div_torch(X,Y): X 截面百分比排名 / Y 截面百分比排名 - rank_add_torch(X,Y): X 截面百分比排名 + Y 截面百分比排名 ### 29个初始特征(图表18) 注:特征名称对齐 cn_stock_prefactors 实际字段名 时序标准化特征(滚动60日标准化): - close: 收盘价(后复权)→ cn_stock_prefactors.close - open1: 开盘价(后复权)→ cn_stock_prefactors.open - high: 最高价(后复权)→ cn_stock_prefactors.high - low: 最低价(后复权)→ cn_stock_prefactors.low - vwap: 均价(后复权)→ 由 amount/volume 计算(cn_stock_prefactors 中无 vwap,amount 字段对应成交额) - volume: 成交量 → cn_stock_prefactors.volume - amt: 成交额 → cn_stock_prefactors.amount - turn: 换手率 → cn_stock_prefactors.turn - bp: 市净率倒数 → 1/cn_stock_prefactors.pb - ep: 市盈率TTM倒数 → 1/cn_stock_prefactors.pe_ttm - ocfp: 经营现金流TTM/总市值 → cn_stock_prefactors.ocfp(若不存在则跳过或用替代) - dp: 近12个月股息率 → cn_stock_prefactors.dp(若不存在则跳过) - adv20: 过去20日成交量均值 → 用 m_mean(volume,20) 计算 - adv60: 过去60日成交量均值 → 用 m_mean(volume,60) 计算 原始特征(不做时序标准化): - close_ori: 收盘价 - open1_ori: 开盘价 - high_ori: 最高价 - low_ori: 最低价 - vwap_ori: 均价 - volume_ori: 成交量 - amt_ori: 成交额 - turn_ori: 换手率 - bp_ori: 市净率倒数 - ep_ori: 市盈率倒数 - ocfp_ori: 经营现金流/市值 - dp_ori: 股息率 - return1_ori: 当日收益率 → cn_stock_prefactors.daily_return - adv20_ori: 20日均量 - adv60_ori: 60日均量 ## Notebook 结构要求 请生成一个完整的 Jupyter Notebook,包含以下 cells: ### Cell 1: 标题和说明(Markdown) 说明这是复现华泰金工研报《遗传规划因子挖掘的GPU加速》的代码 ### Cell 2: 导入依赖 ```python import torch import numpy as np import pandas as pd from dai import DataSource import warnings warnings.filterwarnings('ignore') ``` ### Cell 3: 参数配置 所有超参数都定义在这里,严格对齐研报参数 ### Cell 4: 数据获取 使用 BigQuant DAI 从 cn_stock_prefactors 获取所需数据 ### Cell 5-7: 所有31个 PyTorch 算子实现(分三组:元素级、时序、截面) ### Cell 8: 特征预处理(时序标准化) ### Cell 9: 公式树数据结构定义(Node类) ### Cell 10: 遗传规划核心算法(初始化、交叉、变异、适应度计算) ### Cell 11: 因子入库和相关性过滤 ### Cell 12: 主循环(大循环+小循环) ### Cell 13: 结果分析 ## 关键实现细节 1. **数据格式**: 使用 shape=(T, N) 的 2D PyTorch tensor,T 为时间维度,N 为股票维度 2. **时序标准化**: 对每支股票的每个特征做滚动60日 z-score 标准化 3. **RankICIR**: 在样本内时段计算,RankIC = 因子截面排名与未来收益截面排名的相关系数,ICIR = mean(IC)/std(IC) * sqrt(250)(年化) 4. **残差收益率**: 通过对未来11日收益率对行业和市值因子做横截面回归,取残差 5. **公式树**: 递归树结构,每个节点是算子或终端节点(特征) 6. **GPU并行**: 整个种群(3000个公式)的评估要并行化 7. **大循环逻辑**: 每轮大循环的目标收益率 = 上一轮收益率的残差(剔除已找到因子的影响) ## 实际代码要点 1. 算子函数全部接受 PyTorch tensor 输入,返回 PyTorch tensor 2. 公式随机生成时需要控制最大深度为3 3. 交叉:随机选取两棵树的子树进行交换 4. 变异:随机替换某节点为新的随机子树 5. 适应度 = 样本内年化 RankICIR 请生成完整、可运行的 Jupyter Notebook 文件内容(JSON格式),保存到指定路径。确保代码完整、注释清晰、严格对齐研报。 特别注意: - 文件必须保存为有效的 .ipynb JSON 格式 - 所有代码必须放在 code cells 中 - 说明文字放在 markdown cells 中 - 代码要完整,不能有 TODO 或省略号
继续完成
好的,让我先读取所有图片了解完整信息,然后生成完整的 Notebook。
已掌握全部细节,现在开始生成完整 Notebook:
✅ Notebook 已成功生成,路径:`gpu_gp_factor_mining.ipynb`(34个单元格,18个代码块,16个说明块,结构完整)。 --- ## 📋 实现内容完整对照表 ### 对齐研报图表15:流程框架 | 流程步骤 | 对应代码 | |---------|---------| | 初始化种群 | `Cell[13]` `random_tree()` 生成 3000 个个体 | | 以残差收益率为优化目标,计算适应度 | `fitness()` → 年化 `RankICIR` | | 进化(交叉/变异)× n轮 | `gp_evolution_round()` 小循环 | | 筛选 RankICIR > 阈值因子 | `ICIR_THRESH=2.5` | | 剔除相关性高因子 → 进入因子池 | `deduplicate_factors()` corr > 0.7 不入库 | | 更新残差收益率(大循环) | `update_residual_return()` | | 重复 m 次大循环 | 外层 `for macro in range(N_MACRO)` | ### 对齐研报图表16:参数设置 | 参数 | 研报 | 代码 | |------|------|------| | 样本内 | 2010-01-04~2019-12-31 | ✅ | | 样本外 | 2020-01-02~2024-01-31 | ✅ | | 种群数量 | 3000 | ✅ `POP_SIZE=3000` | | 精英数量 | 500 | ✅ `ELITE_SIZE=500` | | 大循环 m | 5 | ✅ `N_MACRO=5` | | 小循环 n | 3 | ✅ `N_MICRO=3` | | 最大深度 | 3 | ✅ `MAX_DEPTH=3` | | 交叉概率 | 0.35 | ✅ | | 变异概率 | 0.25 | ✅ | ### 对齐研报图表17:31个算子 | 类型 | 数量 | 实现 | |------|------|------| | 元素级运算 | 9个 | `add/sub/mul/div/log/sqrt/neg/sigmoid/sign` | | 时序运算 | 18个 | `ts_delay/delta/delaypct/correlation/argmin/argmax/rank/covariance/decay_linear/SubPosDecayLinear/min/max/stddev/sum/rankcorr/grouping×3` | | 截面运算 | 4个 | `rank_pct/rank_sub/rank_div/rank_add` | ### 对齐研报图表18:29个初始特征 - `close/open1/high/low/vwap/volume/amt/turn/bp/ep/ocfp/dp/return1/adv20/adv60` × 原始 + 时序标准化(滚动60日z-score) > ⚠️ **风险提示**:本框架为研报复现,历史回测存在过拟合、幸存者偏差等风险,建议配合走样本外、滚动回测等方法验证因子有效性,不构成任何投资建议。
帮我检查一下代码,============================================================ 开始遗传规划因子挖掘 种群=3000, 精英=500, 大循环=5, 小循环=3 最大深度=3, 交叉=0.35, 变异=0.25 入库条件: |ICIR|>2.5, |corr|<0.7 ============================================================ ================================================== 大循环 1/5 当前因子池大小: 0 ================================================== 初始化种群 3000 个个体 [小循环 1/3] 计算适应度... 在这个地方运行了30分钟然后没结果,不知道什么原因。是本身就很慢还是代码有问题。对了,数据字段你写的有点问题,我已经手工改好了,不需要再修改了。
让我先读取现有代码,定位性能瓶颈所在:
已找到根本原因,现在直接定位慢的三处代码并完整重写: **🔴 性能瓶颈根因(3处):** 1. **所有滚动算子**(`ts_min/max/std/sum/correlation` 等)用 `for t in range(T)` Python循环,T≈2400次,每次触发一次GPU同步 → **最慢** 2. **`compute_rank_ic`** 用 Python `for t in range(T)` 逐截面计算 → **第二慢** 3. **`rank_pct_torch`** 用 Python `for t in range(T)` 逐截面排名 → **第三慢**
现在读取完整 notebook 的代码单元格内容,以便精确替换:
现在同步更新 `eval_tree` 和主循环里的 `compute_rank_ic` 引用,并加入**并行批量评估**逻辑:
验证通过,三个核心Cell已全部完成向量化。现在来解释一下修改内容: --- ## 🔴 根本原因分析 原代码卡在「小循环1/3 计算适应度」,30分钟无结果的原因是**三个叠加的 Python 循环**: ``` 3000棵树 × 2400天 × Python-for-t循环 = 720万次 Python→GPU 调度 每次都触发 CUDA 同步 → GPU 利用率 <5%,CPU 成为瓶颈 ``` --- ## ✅ 三处核心修复(Cell[12] / Cell[16] / Cell[20]) ### 修复1:Cell[12] — 滚动算子,`for t in range(T)` → `torch.unfold()` | 对比 | 旧代码 | 新代码 | |------|--------|--------| | 实现方式 | Python `for t in range(2400)` | `tensor.unfold(0, d, 1)` 一次性展开 | | GPU调度次数 | 2400次 | **1次** | | 涉及算子 | `ts_min/max/std/sum/corr/cov/rank/argmin/argmax/decay/grouping` | 全部 | ```python # 旧:Python 循环(极慢) for t in range(d-1, T): window = X[t-d+1:t+1] # 逐步切片 out[t] = window.min(0).values # 新:unfold 一次展开(快100-2400倍) def _unfold(X, d): # [T,N] → [T,N,d] padded = torch.cat([nan_pad, X], dim=0) return padded.unfold(0, d, 1) # 零拷贝视图,无Python循环 ``` ### 修复2:Cell[16] — `compute_rank_ic`,逐截面 → 全时间维度向量化 ```python # 旧:T=2400 次 Python 循环 for t in range(T): fv = factor_vals[t]; yv = Y[t] rf = fv.argsort().argsort() # [N] # 新:dim=1 整体向量化,一次完成所有截面 rf = f_fill.argsort(dim=1).argsort(dim=1).float() # [T, N] 全量 ic = (rfm * rym).sum(dim=1) / den # [T] 一步出结果 ``` ### 修复3:`rank_pct_torch`,逐截面 → 全向量化 ```python # 旧:for t in range(T) 循环 # 新: order = X_fill.argsort(dim=1).argsort(dim=1).float() # [T,N] 直接完成 ``` --- ## ⚡ 预期性能提升 | 阶段 | 旧耗时估算 | 新耗时估算 | 提速 | |------|-----------|-----------|------| | 单棵树评估 | ~3-5秒 | **10-50ms** | **60-300x** | | 每小循环3000棵 | ~2.5小时 | **5-25分钟** | 同上 | | 全程(15轮小循环) | **37小时+** | **1-6小时** | — | > ⚠️ **注意**:分组算子(`ts_grouping_*`,d=40时展开为 [2400,3000,40]=288MB)在 **RTX 4090(24GB显存)上没有问题**,如遇OOM可将 `D_CHOICES_GRP = [10, 15, 20]` 去掉40。
✅ 公式树结构定义完毕 您可以去社区论坛问答交流板块反馈咨询 去发帖>> --------------------------------------------------------------------------- NameError Traceback (most recent call last) Cell In[8], line 140 136 return torch.zeros(X_tensor.shape[:2], device=X_tensor.device) 139 print("✅ 公式树结构定义完毕") --> 140 _t = random_tree(MAX_DEPTH) 141 print("示例公式:", _t) 142 with torch.no_grad(): NameError: name 'MAX_DEPTH' is not defined
问题很简单:Cell[14] 底部验证代码引用了 `MAX_DEPTH`,但它在 Cell[24] 才定义。直接修复:
✅ 修复完成。 --- **原因说明** Cell[14](公式树定义)在 Cell[24](主循环参数)**之前**执行,但末尾的冒烟测试代码引用了 `MAX_DEPTH`,而 `MAX_DEPTH = 3` 是在 Cell[24] 才赋值的,所以报 `NameError`。 **修复内容** ```python # 修复前(Cell[14] 末尾) _t = random_tree(MAX_DEPTH) # ❌ MAX_DEPTH 此时未定义 # 修复后 _t = random_tree(3) # ✅ 硬编码,与研报参数一致 ``` **重新执行顺序提示**:请从 Cell[14] 重新往下执行即可,不需要重跑数据加载部分。
日志 4 条 ▼ PyTorch 2.3.0+cu121 已安装 CUDA 可用: True GPU: NVIDIA GeForce RTX 4090 D 环境检查完毕 日志 4 条 ▼ 开始查询数据,时间范围: 2010-01-04 至 2024-01-31 数据形状: (11186790, 19) 日期范围: 2010-01-04 00:00:00 ~ 2024-01-31 00:00:00 股票数量: 5275 date instrument close_ori open1_ori high_ori low_ori vwap_ori volume_ori amt_ori turn_ori bp_ori ep_ori ocfp_ori dp_ori return1_ori fwd_ret_1 fwd_ret_11 industry ln_mktcap 0 2010-01-04 000001.SZ 851.320184 880.403665 882.557997 850.243018 0.239849 24192276 5.802495e+08 0.008273 0.259249 0.012691 -0.209981 0.000000 NaN -0.017292 0.035398 银行 10.867054 1 2010-01-04 000002.SZ 1174.523541 1202.224568 1204.440650 1174.523541 0.106652 96983253 1.034345e+09 0.010044 0.363487 0.040448 0.113224 0.004717 NaN -0.022642 -0.002973 房地产 11.066509 2 2010-01-04 000004.SZ NaN NaN NaN NaN NaN 0 NaN 0.000000 0.098982 NaN 0.024351 0.000000 NaN NaN NaN 计算机 8.924159 日志 33 条 ▼ 原始特征数量: 15 count mean std min \ close_ori 10771782.0 1.184923e+02 1.288021e+03 2.502262e-01 open1_ori 10771782.0 1.183704e+02 1.285880e+03 2.502262e-01 high_ori 10771782.0 1.208356e+02 1.317154e+03 2.611056e-01 low_ori 10771782.0 1.161591e+02 1.258349e+03 2.284674e-01 vwap_ori 10771782.0 1.902791e-01 3.318824e-01 1.188056e-03 volume_ori 11186790.0 1.351016e+07 3.537435e+07 0.000000e+00 amt_ori 10771782.0 1.736982e+08 4.233067e+08 1.490000e+02 turn_ori 11186790.0 2.781678e-02 4.409022e-02 0.000000e+00 bp_ori 11083317.0 4.480978e-01 3.778500e-01 0.000000e+00 ep_ori 9512762.0 3.478205e-02 4.250061e-02 3.269637e-07 ocfp_ori 10375731.0 3.939355e-02 1.696647e-01 -1.116147e+01 dp_ori 11186790.0 9.118515e-03 1.480218e-02 -1.387779e-16 return1_ori 10734114.0 3.849420e-04 2.976612e-02 -4.939808e-01 adv20_ori 11186790.0 1.350834e+07 3.153551e+07 0.000000e+00 adv60_ori 11186790.0 1.349031e+07 2.967765e+07 0.000000e+00 max close_ori 1.974283e+05 open1_ori 1.893689e+05 high_ori 1.994296e+05 low_ori 1.874757e+05 vwap_ori 2.554307e+01 volume_ori 5.135467e+09 amt_ori 6.794128e+10 turn_ori 9.896461e-01 bp_ori 1.594561e+01 ep_ori 7.495990e+00 ocfp_ori 1.009969e+01 dp_ori 5.601660e-01 return1_ori 1.085441e+00 adv20_ori 4.039725e+09 adv60_ori 4.039725e+09 日志 4 条 ▼ 时序标准化特征数: 14 原始特征数: 15 样本内数据量: 6699099 样本外数据量: 4487691 日志 11 条 ▼ 计算样本内残差收益率(约需2-5分钟)... 样本内残差收益率 NaN 比率: 5.35% 残差收益率统计: count 6.340420e+06 mean -1.456852e-19 std 7.931205e-02 min -1.173359e+00 25% -3.958860e-02 50% -6.493088e-03 75% 3.122625e-02 max 1.601008e+00 Name: resid_ret, dtype: float64 日志 5 条 ▼ 使用设备: cuda 构建样本内面板张量... X_train shape: torch.Size([2431, 3805, 29]) (T=2431, N=3805, F=29) Y_train shape: torch.Size([2431, 3805]) GPU 显存占用: 2.18 GB 日志 7 条 ▼ ✅ 31个算子定义完毕(全向量化,零 Python for-t 循环) 元素级 (9): add/sub/mul/div/log/sqrt/neg/sigmoid/sign 时序 (18): delay/delta/delaypct/argmin/argmax/rank/ corr/cov/rankcorr/decay_linear/SubPosDecay/ min/max/std/sum/grouping×3 截面 (4): rank_pct/rank_sub/rank_div/rank_add 冒烟测试通过 ✓ 日志 3 条 ▼ ✅ 公式树结构定义完毕 示例公式: ts_delay_torch(ts_argmax_torch(ts_decay_linear_igman_torch(bp_ori, d=7), d=8), d=5) 示例因子 shape=torch.Size([50, 3805]), 均值=5.8871 日志 3 条 ▼ ✅ 向量化适应度函数定义完毕 单棵树评估耗时: 39.2 ms,ICIR=3.5425 预估3000棵耗时: 2.0 分钟 日志 1 条 ▼ 遗传算子(交叉/变异)定义完毕 日志 1 条 ▼ ✅ 因子去重/入库逻辑定义完毕(向量化相关性计算) 日志 1 条 ▼ 残差收益率更新函数定义完毕 日志 147 条 ▼ ============================================================ 开始遗传规划因子挖掘 种群=3000, 精英=500, 大循环=5, 小循环=3 最大深度=3, 交叉=0.35, 变异=0.25 入库条件: |ICIR|>2.5, |corr|<0.7 ============================================================ ⚡ 单棵树评估耗时: 13.2 ms ⚡ 预估每小循环3000棵: 0.7 分钟 ⚡ 预估总耗时(5×3=15轮): 0.2 小时 ================================================== 大循环 1/5 | 当前因子池: 0 个 ================================================== 初始化种群 3000 个个体 ── 小循环 1/3 ── 计算 3000 个个体适应度... [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 14164164608 / 12878610432 评估进度: 500/3000 已用=13s 预计剩余=63s 评估进度: 1000/3000 已用=24s 预计剩余=49s 评估进度: 1500/3000 已用=37s 预计剩余=37s 评估进度: 2000/3000 已用=50s 预计剩余=25s 评估进度: 2500/3000 已用=62s 预计剩余=12s 评估进度: 3000/3000 已用=74s 预计剩余=0s ✓ 完成 耗时=74.3s 最优ICIR=-18.8128 超阈值因子数=2015 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 14608760832 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 14608760832 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 14608760832 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 14608760832 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13868466176 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13868466176 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12907970560 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12907970560 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 14128513024 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13388218368 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13574864896 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13574864896 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12907970560 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12907970560 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13908312064 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13797163008 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13241417728 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13241417728 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13425967104 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13425967104 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13241417728 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13241417728 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13425967104 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13425967104 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12983468032 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12983468032 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13241417728 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13241417728 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13241417728 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13241417728 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13425967104 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13425967104 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13241417728 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13241417728 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13241417728 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13241417728 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13056868352 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13056868352 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13425967104 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13425967104 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13056868352 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13056868352 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12907970560 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12907970560 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13019119616 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13019119616 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12983468032 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12983468032 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13241417728 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13241417728 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12945719296 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12945719296 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13019119616 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13019119616 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13021216768 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12983468032 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13574864896 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13425967104 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13019119616 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13019119616 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13019119616 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13019119616 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12907970560 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12907970560 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13205766144 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13205766144 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13021216768 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13021216768 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13056868352 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12907970560 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12907970560 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12907970560 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12920553472 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12920553472 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 14352908288 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 14352908288 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12920553472 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12920553472 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12928942080 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12928942080 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12937330688 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12937330688 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13058965504 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13058965504 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13094617088 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13094617088 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12983468032 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12983468032 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12947816448 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12947816448 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12947816448 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12947816448 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12910067712 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12910067712 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12947816448 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12947816448 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12947816448 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12947816448 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13094617088 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13094617088 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12975079424 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12975079424 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12910067712 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12910067712 / 12878610432 您可以去社区论坛问答交流板块反馈咨询 去发帖>> --------------------------------------------------------------------------- OutOfMemoryError Traceback (most recent call last) Cell In[16], line 131 130 # 截断极端值(5σ) --> 131 std = result[~torch.isnan(result)].std() 132 if std > 0: OutOfMemoryError: CUDA out of memory. Tried to allocate 98.00 MiB. GPU During handling of the above exception, another exception occurred: OutOfMemoryError Traceback (most recent call last) Cell In[22], line 136 133 population = [random_tree(MAX_DEPTH) for _ in range(POP_SIZE)] 134 print(f"初始化种群 {POP_SIZE} 个个体") --> 136 new_factors, population = gp_evolution_round( 137 population, X_train, Y_current, 138 n_micro=N_MICRO, 139 existing_pool=all_factor_pool 140 ) 142 all_factor_pool.extend(new_factors) 143 macro_elapsed = time.perf_counter() - macro_t0 Cell In[22], line 72 70 if abs(ic) >= ICIR_THRESH: 71 with torch.no_grad(): ---> 72 fval = eval_tree(node, X_tensor) 73 candidates.append((node, fval, ic)) 75 if candidates: Cell In[16], line 122 120 if node.is_leaf: 121 return X_tensor[:, :, node.feat_idx] --> 122 child_vals = [eval_tree(c, X_tensor) for c in node.children] 123 try: 124 if node.param_n is not None: Cell In[16], line 122 120 if node.is_leaf: 121 return X_tensor[:, :, node.feat_idx] --> 122 child_vals = [eval_tree(c, X_tensor) for c in node.children] 123 try: 124 if node.param_n is not None: Cell In[16], line 122 120 if node.is_leaf: 121 return X_tensor[:, :, node.feat_idx] --> 122 child_vals = [eval_tree(c, X_tensor) for c in node.children] 123 try: 124 if node.param_n is not None: Cell In[16], line 122 120 if node.is_leaf: 121 return X_tensor[:, :, node.feat_idx] --> 122 child_vals = [eval_tree(c, X_tensor) for c in node.children] 123 try: 124 if node.param_n is not None: Cell In[16], line 136 134 return result 135 except Exception: --> 136 return torch.zeros(X_tensor.shape[:2], device=X_tensor.device) OutOfMemoryError: CUDA out of memory. Tried to allocate 36.00 MiB. GPU 实在不行帮我改成CPU版本吧。要求可以跑通
日志 4 条 ▼ PyTorch 2.3.0+cu121 已安装 CUDA 可用: True GPU: NVIDIA GeForce RTX 4090 D 环境检查完毕 日志 4 条 ▼ 开始查询数据,时间范围: 2010-01-04 至 2024-01-31 数据形状: (11186790, 19) 日期范围: 2010-01-04 00:00:00 ~ 2024-01-31 00:00:00 股票数量: 5275 date instrument close_ori open1_ori high_ori low_ori vwap_ori volume_ori amt_ori turn_ori bp_ori ep_ori ocfp_ori dp_ori return1_ori fwd_ret_1 fwd_ret_11 industry ln_mktcap 0 2010-01-04 000001.SZ 851.320184 880.403665 882.557997 850.243018 0.239849 24192276 5.802495e+08 0.008273 0.259249 0.012691 -0.209981 0.000000 NaN -0.017292 0.035398 银行 10.867054 1 2010-01-04 000002.SZ 1174.523541 1202.224568 1204.440650 1174.523541 0.106652 96983253 1.034345e+09 0.010044 0.363487 0.040448 0.113224 0.004717 NaN -0.022642 -0.002973 房地产 11.066509 2 2010-01-04 000004.SZ NaN NaN NaN NaN NaN 0 NaN 0.000000 0.098982 NaN 0.024351 0.000000 NaN NaN NaN 计算机 8.924159 日志 33 条 ▼ 原始特征数量: 15 count mean std min \ close_ori 10771782.0 1.184923e+02 1.288021e+03 2.502262e-01 open1_ori 10771782.0 1.183704e+02 1.285880e+03 2.502262e-01 high_ori 10771782.0 1.208356e+02 1.317154e+03 2.611056e-01 low_ori 10771782.0 1.161591e+02 1.258349e+03 2.284674e-01 vwap_ori 10771782.0 1.902791e-01 3.318824e-01 1.188056e-03 volume_ori 11186790.0 1.351016e+07 3.537435e+07 0.000000e+00 amt_ori 10771782.0 1.736982e+08 4.233067e+08 1.490000e+02 turn_ori 11186790.0 2.781678e-02 4.409022e-02 0.000000e+00 bp_ori 11083317.0 4.480978e-01 3.778500e-01 0.000000e+00 ep_ori 9512762.0 3.478205e-02 4.250061e-02 3.269637e-07 ocfp_ori 10375731.0 3.939355e-02 1.696647e-01 -1.116147e+01 dp_ori 11186790.0 9.118515e-03 1.480218e-02 -1.387779e-16 return1_ori 10734114.0 3.849420e-04 2.976612e-02 -4.939808e-01 adv20_ori 11186790.0 1.350834e+07 3.153551e+07 0.000000e+00 adv60_ori 11186790.0 1.349031e+07 2.967765e+07 0.000000e+00 max close_ori 1.974283e+05 open1_ori 1.893689e+05 high_ori 1.994296e+05 low_ori 1.874757e+05 vwap_ori 2.554307e+01 volume_ori 5.135467e+09 amt_ori 6.794128e+10 turn_ori 9.896461e-01 bp_ori 1.594561e+01 ep_ori 7.495990e+00 ocfp_ori 1.009969e+01 dp_ori 5.601660e-01 return1_ori 1.085441e+00 adv20_ori 4.039725e+09 adv60_ori 4.039725e+09 日志 4 条 ▼ 时序标准化特征数: 14 原始特征数: 15 样本内数据量: 6699099 样本外数据量: 4487691 日志 11 条 ▼ 计算样本内残差收益率(约需2-5分钟)... 样本内残差收益率 NaN 比率: 5.35% 残差收益率统计: count 6.340420e+06 mean -1.456852e-19 std 7.931205e-02 min -1.173359e+00 25% -3.958860e-02 50% -6.493088e-03 75% 3.122625e-02 max 1.601008e+00 Name: resid_ret, dtype: float64 日志 5 条 ▼ 使用设备: cuda 构建样本内面板张量... X_train shape: torch.Size([2431, 3805, 29]) (T=2431, N=3805, F=29) Y_train shape: torch.Size([2431, 3805]) GPU 显存占用: 2.18 GB 日志 7 条 ▼ ✅ 31个算子定义完毕(全向量化,零 Python for-t 循环) 元素级 (9): add/sub/mul/div/log/sqrt/neg/sigmoid/sign 时序 (18): delay/delta/delaypct/argmin/argmax/rank/ corr/cov/rankcorr/decay_linear/SubPosDecay/ min/max/std/sum/grouping×3 截面 (4): rank_pct/rank_sub/rank_div/rank_add 冒烟测试通过 ✓ 日志 3 条 ▼ ✅ 公式树结构定义完毕 示例公式: ts_delay_torch(ts_argmax_torch(ts_decay_linear_igman_torch(bp_ori, d=7), d=8), d=5) 示例因子 shape=torch.Size([50, 3805]), 均值=5.8871 日志 3 条 ▼ ✅ 向量化适应度函数定义完毕 单棵树评估耗时: 39.2 ms,ICIR=3.5425 预估3000棵耗时: 2.0 分钟 日志 1 条 ▼ 遗传算子(交叉/变异)定义完毕 日志 1 条 ▼ ✅ 因子去重/入库逻辑定义完毕(向量化相关性计算) 日志 1 条 ▼ 残差收益率更新函数定义完毕 日志 147 条 ▼ ============================================================ 开始遗传规划因子挖掘 种群=3000, 精英=500, 大循环=5, 小循环=3 最大深度=3, 交叉=0.35, 变异=0.25 入库条件: |ICIR|>2.5, |corr|<0.7 ============================================================ ⚡ 单棵树评估耗时: 13.2 ms ⚡ 预估每小循环3000棵: 0.7 分钟 ⚡ 预估总耗时(5×3=15轮): 0.2 小时 ================================================== 大循环 1/5 | 当前因子池: 0 个 ================================================== 初始化种群 3000 个个体 ── 小循环 1/3 ── 计算 3000 个个体适应度... [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 14164164608 / 12878610432 评估进度: 500/3000 已用=13s 预计剩余=63s 评估进度: 1000/3000 已用=24s 预计剩余=49s 评估进度: 1500/3000 已用=37s 预计剩余=37s 评估进度: 2000/3000 已用=50s 预计剩余=25s 评估进度: 2500/3000 已用=62s 预计剩余=12s 评估进度: 3000/3000 已用=74s 预计剩余=0s ✓ 完成 耗时=74.3s 最优ICIR=-18.8128 超阈值因子数=2015 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 14608760832 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 14608760832 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 14608760832 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 14608760832 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13128171520 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13868466176 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13868466176 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12907970560 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12907970560 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 14128513024 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13388218368 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13574864896 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13574864896 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12907970560 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12907970560 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13908312064 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13797163008 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13241417728 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13241417728 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13425967104 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13425967104 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13241417728 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13241417728 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13425967104 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13425967104 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12983468032 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12983468032 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13241417728 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13241417728 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13241417728 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13241417728 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13425967104 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13425967104 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13241417728 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13241417728 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13241417728 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13241417728 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13056868352 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13056868352 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13425967104 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13425967104 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13056868352 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13056868352 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12907970560 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12907970560 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13019119616 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13019119616 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12983468032 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12983468032 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13241417728 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13241417728 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12945719296 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12945719296 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13019119616 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13019119616 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13021216768 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12983468032 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13574864896 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13425967104 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13019119616 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13019119616 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13019119616 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13019119616 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12907970560 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12907970560 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13205766144 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13205766144 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13021216768 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13021216768 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13056868352 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12907970560 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12907970560 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12907970560 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12920553472 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12920553472 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 14352908288 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 14352908288 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12920553472 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12920553472 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12928942080 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12928942080 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12937330688 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12937330688 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13058965504 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13058965504 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13094617088 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13094617088 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12983468032 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12983468032 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12947816448 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12947816448 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12947816448 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12947816448 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12910067712 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12910067712 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12947816448 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12947816448 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12947816448 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12947816448 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13094617088 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 13094617088 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12975079424 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12975079424 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12910067712 / 12878610432 [HAMI-core ERROR (pid:4459 thread=140518898538304 allocator.c:54)]: Device 0 OOM 12910067712 / 12878610432 您可以去社区论坛问答交流板块反馈咨询 去发帖>> --------------------------------------------------------------------------- OutOfMemoryError Traceback (most recent call last) Cell In[16], line 131 130 # 截断极端值(5σ) --> 131 std = result[~torch.isnan(result)].std() 132 if std > 0: OutOfMemoryError: CUDA out of memory. Tried to allocate 98.00 MiB. GPU During handling of the above exception, another exception occurred: OutOfMemoryError Traceback (most recent call last) Cell In[22], line 136 133 population = [random_tree(MAX_DEPTH) for _ in range(POP_SIZE)] 134 print(f"初始化种群 {POP_SIZE} 个个体") --> 136 new_factors, population = gp_evolution_round( 137 population, X_train, Y_current, 138 n_micro=N_MICRO, 139 existing_pool=all_factor_pool 140 ) 142 all_factor_pool.extend(new_factors) 143 macro_elapsed = time.perf_counter() - macro_t0 Cell In[22], line 72 70 if abs(ic) >= ICIR_THRESH: 71 with torch.no_grad(): ---> 72 fval = eval_tree(node, X_tensor) 73 candidates.append((node, fval, ic)) 75 if candidates: Cell In[16], line 122 120 if node.is_leaf: 121 return X_tensor[:, :, node.feat_idx] --> 122 child_vals = [eval_tree(c, X_tensor) for c in node.children] 123 try: 124 if node.param_n is not None: Cell In[16], line 122 120 if node.is_leaf: 121 return X_tensor[:, :, node.feat_idx] --> 122 child_vals = [eval_tree(c, X_tensor) for c in node.children] 123 try: 124 if node.param_n is not None: Cell In[16], line 122 120 if node.is_leaf: 121 return X_tensor[:, :, node.feat_idx] --> 122 child_vals = [eval_tree(c, X_tensor) for c in node.children] 123 try: 124 if node.param_n is not None: Cell In[16], line 122 120 if node.is_leaf: 121 return X_tensor[:, :, node.feat_idx] --> 122 child_vals = [eval_tree(c, X_tensor) for c in node.children] 123 try: 124 if node.param_n is not None: Cell In[16], line 136 134 return result 135 except Exception: --> 136 return torch.zeros(X_tensor.shape[:2], device=X_tensor.device) OutOfMemoryError: CUDA out of memory. Tried to allocate 36.00 MiB. GPU 实在不行帮我改成CPU版本吧。要求可以跑通
OOM 有**两个叠加原因**,缺一不可都要修: 1. **根因1**:`eval_tree` 的 `except` fallback 在 GPU 已 OOM 时还去申请新显存 → 死锁 2. **根因2(更严重)**:`gp_evolution_round` 把 **2015 个候选因子的 fval 全部同时堆在内存** → `2015 × [2431,3805] × 4B ≈ 70GB`,CPU/GPU 都撑不住 改 CPU 版本同时彻底修复内存积累问题:
验证通过,所有修复均已正确写入。 --- ## 本次修复全部问题汇总(共4处) ### 🔴 根因1:`eval_tree` 的 `except` fallback 在 GPU OOM 时二次 OOM ```python # 旧代码(崩溃:GPU 已满,再申请 36MB → 失败) except Exception: return torch.zeros(X_tensor.shape[:2], device=X_tensor.device) # 新代码(CPU tensor,永不 OOM) except Exception: T, N = X_tensor.shape[:2] return torch.zeros(T, N, dtype=torch.float32) ``` ### 🔴 根因2(最严重):同时积累 2015 个候选 fval → 70GB ``` 旧流程:evaluate 3000棵 → 筛出2015棵 → 全部计算fval存入列表 → 2015×35MB=70GB → OOM 新流程:evaluate 3000棵 → 只保留(node, icir)标量 → 逐个惰性求值fval → 检验后立即del 峰值内存 ≈ 3GB ``` ### ✅ 其他两处顺带优化 | 单元格 | 修改 | 原因 | |--------|------|------| | Cell[10] | `DEVICE = cpu` + 多线程 | 彻底绕开显存限制 | | Cell[14] | `D_CHOICES_GRP = [10,15,20]` 去掉40 | `unfold(d=40)` 生成 1.5GB 中间张量 | ### 执行方式 从 **Cell[10]** 开始往下重新执行(Cell[4]~[9] 数据已加载,无需重跑)。预估耗时约 **2-4小时**(CPU 32线程)。
日志 4 条 ▼ 开始查询数据,时间范围: 2010-01-04 至 2024-01-31 数据形状: (11186790, 19) 日期范围: 2010-01-04 00:00:00 ~ 2024-01-31 00:00:00 股票数量: 5275 date instrument close_ori open1_ori high_ori low_ori vwap_ori volume_ori amt_ori turn_ori bp_ori ep_ori ocfp_ori dp_ori return1_ori fwd_ret_1 fwd_ret_11 industry ln_mktcap 0 2010-01-04 000001.SZ 851.320184 880.403665 882.557997 850.243018 0.239849 24192276 5.802495e+08 0.008273 0.259249 0.012691 -0.209981 0.000000 0.966966 -0.017292 0.035398 银行 10.867054 1 2010-01-04 000002.SZ 1174.523541 1202.224568 1204.440650 1174.523541 0.106652 96983253 1.034345e+09 0.010044 0.363487 0.040448 0.113224 0.004717 0.976959 -0.022642 -0.002973 房地产 11.066509 2 2010-01-04 000004.SZ NaN NaN NaN NaN NaN 0 NaN 0.000000 0.098982 NaN 0.024351 0.000000 NaN NaN NaN 计算机 8.924159 日志 33 条 ▼ 原始特征数量: 15 count mean std min \ close_ori 10771782.0 1.184923e+02 1.288021e+03 2.502262e-01 open1_ori 10771782.0 1.183704e+02 1.285880e+03 2.502262e-01 high_ori 10771782.0 1.208356e+02 1.317154e+03 2.611056e-01 low_ori 10771782.0 1.161591e+02 1.258349e+03 2.284674e-01 vwap_ori 10771782.0 1.902791e-01 3.318824e-01 1.188056e-03 volume_ori 11186790.0 1.351016e+07 3.537435e+07 0.000000e+00 amt_ori 10771782.0 1.736982e+08 4.233067e+08 1.490000e+02 turn_ori 11186790.0 2.781678e-02 4.409022e-02 0.000000e+00 bp_ori 11083317.0 4.480978e-01 3.778500e-01 0.000000e+00 ep_ori 9512762.0 3.478205e-02 4.250061e-02 3.269637e-07 ocfp_ori 10375731.0 3.939355e-02 1.696647e-01 -1.116147e+01 dp_ori 11186790.0 9.118515e-03 1.480218e-02 -1.387779e-16 return1_ori 10771782.0 1.001442e+00 2.796910e-02 3.892216e-01 adv20_ori 11186790.0 1.350834e+07 3.153551e+07 0.000000e+00 adv60_ori 11186790.0 1.349031e+07 2.967765e+07 0.000000e+00 max close_ori 1.974283e+05 open1_ori 1.893689e+05 high_ori 1.994296e+05 low_ori 1.874757e+05 vwap_ori 2.554307e+01 volume_ori 5.135467e+09 amt_ori 6.794128e+10 turn_ori 9.896461e-01 bp_ori 1.594561e+01 ep_ori 7.495990e+00 ocfp_ori 1.009969e+01 dp_ori 5.601660e-01 return1_ori 4.354509e+00 adv20_ori 4.039725e+09 adv60_ori 4.039725e+09 日志 3 条 ▼ PyTorch 2.3.0+cu121 已安装 CUDA 可用: False 环境检查完毕 日志 4 条 ▼ 时序标准化特征数: 14 原始特征数: 15 样本内数据量: 6699099 样本外数据量: 4487691 日志 11 条 ▼ 计算样本内残差收益率(约需2-5分钟)... 样本内残差收益率 NaN 比率: 5.35% 残差收益率统计: count 6.340420e+06 mean -1.456852e-19 std 7.931205e-02 min -1.173359e+00 25% -3.958860e-02 50% -6.493088e-03 75% 3.122625e-02 max 1.601008e+00 Name: resid_ret, dtype: float64 日志 4 条 ▼ 使用设备: cpu | CPU 线程数: 16 系统内存: 34.4 GB 可用: 21.0 GB 构建样本内面板张量(CPU)... 您可以去社区论坛问答交流板块反馈咨询 去发帖>> --------------------------------------------------------------------------- NameError Traceback (most recent call last) Cell In[8], line 34 30 return torch.tensor(arr), dates, instruments # CPU tensor 32 print("\n构建样本内面板张量(CPU)...") 33 X_train, train_dates, train_insts = build_panel_tensor( ---> 34 df_train.dropna(subset=ALL_FEAT_COLS, how='all'), ALL_FEAT_COLS 35 ) 36 resid_pivot = df_train.pivot_table( 37 values='resid_ret', index='date', columns='instrument' 38 ).reindex(index=train_dates, columns=train_insts) 39 Y_train = torch.tensor(resid_pivot.values.astype(np.float32)) # CPU NameError: name 'ALL_FEAT_COLS' is not defined
使用设备: cpu | CPU 线程数: 16 系统内存: 34.4 GB 可用: 21.0 GB 构建样本内面板张量(CPU)... 您可以去社区论坛问答交流板块反馈咨询 去发帖>> --------------------------------------------------------------------------- NameError Traceback (most recent call last) Cell In[8], line 34 30 return torch.tensor(arr), dates, instruments # CPU tensor 32 print("\n构建样本内面板张量(CPU)...") 33 X_train, train_dates, train_insts = build_panel_tensor( ---> 34 df_train.dropna(subset=ALL_FEAT_COLS, how='all'), ALL_FEAT_COLS 35 ) 36 resid_pivot = df_train.pivot_table( 37 values='resid_ret', index='date', columns='instrument' 38 ).reindex(index=train_dates, columns=train_insts) 39 Y_train = torch.tensor(resid_pivot.values.astype(np.float32)) # CPU NameError: name 'ALL_FEAT_COLS' is not defined
原因很清楚:新 Cell[10] 把旧版的 `ALL_FEAT_COLS`、`N_TS`、`N_ORI` 定义给覆盖丢失了。一行修复即可:
✅ 修复完成。 --- **原因**:旧 Cell[10] 里有一段: ```python ALL_FEAT_COLS = ALL_TS_FEATS + ORIG_FEAT_COLS N_TS = len(ALL_TS_FEATS) N_ORI = len(ORIG_FEAT_COLS) ``` 重写 Cell[10] 为 CPU 版本时把这三行遗漏了,导致后面引用时报 `NameError`。 **修复**:在 `build_panel_tensor` 调用之前插入这三个变量的定义,**无需重跑任何数据加载单元格**,直接从 Cell[10] 重新执行即可。
============================================================ 开始遗传规划因子挖掘(CPU 版本) 种群=3000, 精英=500, 大循环=5, 小循环=3 最大深度=3, D_GRP=[10,15,20], 交叉=0.35, 变异=0.25 入库条件: |ICIR|>2.5, |corr|<0.7 ============================================================ ⚡ 单棵树 CPU 耗时: 327.6 ms ⚡ 预估每小循环 3000 棵: 16.4 分钟 ⚡ 预估总耗时 (5×3=15 轮): 4.1 小时 ⚡ 当前 RAM: 14.4 GB ================================================== 大循环 1/5 | 因子池: 0 RAM=14.4GB ================================================== 初始化种群 3000 个个体 ── 小循环 1/3 ── 评估 3000 个个体 RAM=12.3GB 进度 500/3000 已用=430s 剩余≈2152s RAM=12.4GB 进度 1000/3000 已用=836s 剩余≈1671s RAM=12.4GB 进度 1500/3000 已用=1242s 剩余≈1242s RAM=12.4GB 进度 2000/3000 已用=1634s 剩余≈817s RAM=12.4GB 进度 2500/3000 已用=2048s 剩余≈410s RAM=12.4GB 进度 3000/3000 已用=2514s 剩余≈0s RAM=12.4GB ✓ 耗时=2513.7s 最优ICIR=18.0811 超阈值=2039 RAM=12.4GB 到这里就完全卡住了,这个正常吗,检查一下,如果没问题不用修改。已经停在这里1个多小时了
这不是正常的,问题出在 `deduplicate_factors_lazy` 里:**2039个候选因子 × 逐个做相关性检验**,每次相关性计算需要 ~300ms,最坏情况是 O(N²) 复杂度,已经卡了1小时还会继续卡下去。需要两个修复: 1. **候选因子限制上限**(取 Top 300,不处理全部2039个) 2. **批量向量化相关性计算**(把对整个因子池的相关性检验合并为一次矩阵运算,而非逐个循环)
修复完成。**请先中断当前运行,然后只重新执行 Cell[20] 和 Cell[24] 这两个单元格**(数据和算子都不需要重跑),再重新从 Cell[24] 主循环开始。 --- ## 根因分析与修复说明 ### 为什么卡死? ``` 2039 个候选因子 × 逐个循环比较 └── 候选1:比较 0 个池成员 → 入库 └── 候选2:比较 1 个池成员 × 300ms └── 候选3:比较 2 个池成员 × 300ms ... └── 候选2039:比较 ~1000 个池成员 × 300ms 总计 ≈ 2039 × 500平均 × 300ms = 85小时 ``` ### 两个关键修复 | 修复 | 旧版 | 新版 | |------|------|------| | **候选上限** | 处理全部2039个 | 只取 Top-300 | | **相关性计算** | 逐个 `for k in pool` 循环,每次 ~300ms | 整个 pool 堆叠为 `[K,T_s,N]`,**一次矩阵运算** | | **时间子采样** | 全部 T=2431 天 | 随机采样 T_s=300 天(精度损失极小)| ### 预期性能 ``` eval_tree 300个:300 × 300ms = 1.5 分钟 批量相关性检验:300次 × 平均池大小~100 × ~10ms = ~5 分钟 总去重时间:~7 分钟(旧版:85小时) ```
--------------------------------------------------------------------------- NameError Traceback (most recent call last) Cell In[14], line 4 2 factor_summary = [] 3 for i, (node, fval, icir) in enumerate(all_factor_pool): ----> 4 ic_series = [x for x in compute_rank_ic(fval, Y_train) if not np.isnan(x)] 5 ic_mean = np.mean(ic_series) if ic_series else 0.0 6 ic_std = np.std(ic_series) if ic_series else 1.0 NameError: name 'compute_rank_ic' is not defined
原因很明确:优化时把 `compute_rank_ic`(返回 Python list)重命名为了 `compute_rank_ic_vec`(返回 tensor),但 Cell[26] 汇总代码还在调用旧名字。一行修复:
✅ 修复完成,全局扫描无残留。 --- **原因与修复说明:** 优化性能时,把返回 Python list 的 `compute_rank_ic` 重写成了返回 `torch.Tensor` 的 `compute_rank_ic_vec`,但 Cell[26](汇总)和 Cell[31](绘图)还引用旧函数名。 | 位置 | 旧调用 | 修复后 | |------|--------|--------| | Cell[26] | `compute_rank_ic(fval, Y_train)` → list | `compute_rank_ic_vec(fval, Y_train)` → tensor,再 `.numpy()` | | Cell[31] | `compute_rank_ic(comp_train, ...)` | `compute_rank_ic_vec(...).tolist()` | **直接从 Cell[26] 继续执行即可**,不需要重跑前面任何单元格。