回测正常 模拟交易运行失败
由bqgt6c9n创建，最终由bqgt6c9n 被浏览 25 用户
2.0环境 可视化AI stockranker策略回测成功 提交模拟交易后运行失败 数据抽取开始及结束时间都有绑定交易日实盘数据
任务ID：125d87e5-8b40-4426-b0d6-f76066b840ab
运行报错记录如下 ：
2024-05-23 17:45:30 任务运行开始调度 state=trigger event=20240523 125d87e5-8b40-4426-b0d6-f76066b840ab ..
2
2024-05-23 19:25:00 任务运行状态更新 state=scheduled event=20240523 125d87e5-8b40-4426-b0d6-f76066b840ab
3
2024-05-23 19:25:12 任务运行状态更新 state=running event=20240523 125d87e5-8b40-4426-b0d6-f76066b840ab
4
2024-05-23 19:25:12 [info ] run task_id='125d87e5-8b40-4426-b0d6-f76066b840ab' ..
5
2024-05-23 19:25:12 [info ] fetch task 125d87e5-8b40-4426-b0d6-f76066b840ab ..
6
2024-05-23 19:25:12 [info ] start 125d87e5-8b40-4426-b0d6-f76066b840ab.ipynb ..
7
/home/aiuser/.ipython/profile_default/startup/000-aistudio.py:234: DeprecationWarning: This module is deprecated. Please use `from bigdatasource.api import D`.
8
return original_import(name, globals, locals, fromlist, level)
9
/home/aiuser/.ipython/profile_default/startup/000-aistudio.py:234: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
10
return original_import(name, globals, locals, fromlist, level)
11
/home/aiuser/.ipython/profile_default/startup/000-aistudio.py:234: MovedIn20Warning: Deprecated API features detected! These feature(s) are not compatible with SQLAlchemy 2.0. To prevent incompatible upgrades prior to updating applications, ensure requirements files are pinned to "sqlalchemy<2.0". Set environment variable SQLALCHEMY_WARN_20=1 to show all deprecation warnings. Set environment variable SQLALCHEMY_SILENCE_UBER_WARNING=1 to silence this message. (Background on SQLAlchemy 2.0 at: https://sqlalche.me/e/b8d9)
12
return original_import(name, globals, locals, fromlist, level)
13
[2024-05-23 19:25:32.551697] INFO: moduleinvoker:4212423387.py:111:<module> input_features_dai.v6 开始运行..
14
[2024-05-23 19:25:32.860756] INFO: moduleinvoker:4212423387.py:111:<module> 命中缓存
15
[2024-05-23 19:25:32.872004] INFO: moduleinvoker:4212423387.py:111:<module> input_features_dai.v6 运行完成[0.320341s].
16
[2024-05-23 19:25:33.010839] INFO: moduleinvoker:4212423387.py:156:<module> input_features_dai.v6 开始运行..
17
[2024-05-23 19:25:33.054236] INFO: moduleinvoker:4212423387.py:156:<module> 命中缓存
18
[2024-05-23 19:25:33.073221] INFO: moduleinvoker:4212423387.py:156:<module> input_features_dai.v6 运行完成[0.062413s].
19
2024-05-23 19:25:33 [warning ] cache disabled for paper/live trading for bound date
20
[2024-05-23 19:25:33.253528] INFO: moduleinvoker:4212423387.py:210:<module> extract_data_dai.v7 开始运行..
21
2024-05-23 19:25:34 [info ] start_date='2024-05-23', end_date='2024-05-23', query_start_date='2023-07-28' ..
22
2024-05-23 19:26:03 [info ] data extracted: (71, 15)
23
[2024-05-23 19:26:04.624450] INFO: moduleinvoker:4212423387.py:210:<module> extract_data_dai.v7 运行完成[31.370882s].
24
[2024-05-23 19:26:04.658846] INFO: moduleinvoker:4212423387.py:221:<module> sql_join_2.v1 开始运行..
25
[2024-05-23 19:26:04.688068] INFO: moduleinvoker:4212423387.py:221:<module> 命中缓存
26
[2024-05-23 19:26:04.694003] INFO: moduleinvoker:4212423387.py:221:<module> sql_join_2.v1 运行完成[0.035161s].
27
2024-05-23 19:26:04 [warning ] cache disabled for paper/live trading for bound date
28
[2024-05-23 19:26:04.764506] INFO: moduleinvoker:4212423387.py:238:<module> extract_data_dai.v7 开始运行..
29
2024-05-23 19:26:05 [info ] start_date='2024-05-23', end_date='2024-05-23', query_start_date='2023-07-28' ..
30
2024-05-23 19:26:54 [info ] data extracted: (0, 16)
31
[2024-05-23 19:26:56.121310] INFO: moduleinvoker:4212423387.py:238:<module> extract_data_dai.v7 运行完成[51.356728s].
32
[2024-05-23 19:26:56.265858] INFO: moduleinvoker:4212423387.py:249:<module> extract_data_dai.v7 开始运行..
33
2024-05-23 19:26:56 [warning ] cache disabled for paper/live trading for bound date
34
2024-05-23 19:26:56 [info ] start_date='2024-05-23', end_date='2024-05-23', query_start_date='2023-07-28' ..
35
2024-05-23 19:27:36 [info ] data extracted: (0, 16)
36
[2024-05-23 19:27:37.964938] INFO: moduleinvoker:4212423387.py:249:<module> extract_data_dai.v7 运行完成[41.699095s].
37
[2024-05-23 19:27:40.493962] INFO: moduleinvoker:4212423387.py:260:<module> stock_ranker_dai_train.v2 开始运行..
38
2024-05-23 19:27:41 [info ] data loaded: train=(0, 16), validation=(0, 16)
39
2024-05-23 19:27:41 [info ] 数据预处理 ..
40
2024-05-23 19:27:41 [info ] 特征预处理 ..
41
[2024-05-23 19:27:41.669887] ERROR: moduleinvoker:4212423387.py:260:<module> module name: stock_ranker_dai_train, module version: v2, trackeback: Exception: 数据 training 为空，请检查输入数据
42
43
2024-05-23 19:27:47 任务运行状态更新 state=failed event=20240523 125d87e5-8b40-4426-b0d6-f76066b840ab
44
Traceback (most recent call last):
45
File "/var/app/enabled/aiflowtaskrun.py", line 150, in <module>
46
run(run_id=sys.argv[1], task_id=sys.argv[2], event_name=sys.argv[3], dry_run=sys.argv[4])
47
File "/var/app/enabled/aiflowtaskrun.py", line 145, in run
48
raise ex from ex
49
File "/var/app/enabled/aiflowtaskrun.py", line 132, in run
50
do_run(task_id, event_name)
51
File "/var/app/enabled/aiflowtaskrun.py", line 109, in do_run
52
pm.execute_notebook(
53
File "/usr/local/python3/lib/python3.8/site-packages/papermill/execute.py", line 128, in execute_notebook
54
raise_for_execution_errors(nb, output_path)
55
File "/usr/local/python3/lib/python3.8/site-packages/papermill/execute.py", line 232, in raise_for_execution_errors
56
raise error
57
papermill.exceptions.PapermillExecutionError:
58
---------------------------------------------------------------------------
59
Exception encountered at "In [1]":
60
---------------------------------------------------------------------------
61
Exception Traceback (most recent call last)
62
Cell In[1], line 260
63
249 m7 = M.extract_data_dai.v7(
64
250 sql=m3.data,
65
251 start_date='2022-01-01',
66
(...)
67
256 debug=False
68
257 )
69
259 # @module(position="-462,22", comment='5. 使用StockRanker算法训练', comment_collapsed=False)
70
--> 260 m5 = M.stock_ranker_dai_train.v2(
71
261 data=m4.data,
72
262 validation_ds=m7.data,
73
263 learning_algorithm='排序',
74
264 number_of_leaves=30,
75
265 min_docs_per_leaf=1000,
76
266 number_of_trees=20,
77
267 learning_rate=0.1,
78
268 max_bins=1023,
79
269 feature_fraction=1,
80
270 data_row_fraction=1,
81
271 plot_charts=True,
82
272 ndcg_discount_base=1
83
273 )
84
275 # @module(position="-222,121", comment='7. 预测', comment_collapsed=False)
85
276 m14 = M.stock_ranker_dai_predict.v3(
86
277 model=m5.model,
87
278 data=m6.data
88
279 )
89
90
File module2/common/modulemanagerv2.py:88, in biglearning.module2.common.modulemanagerv2.BigQuantModuleVersion.call()
91
92
File module2/common/moduleinvoker.py:370, in biglearning.module2.common.moduleinvoker.module_invoke()
93
94
File module2/common/moduleinvoker.py:292, in biglearning.module2.common.moduleinvoker._invoke_with_cache()
95
96
File module2/common/moduleinvoker.py:253, in biglearning.module2.common.moduleinvoker._invoke_with_cache()
97
98
File module2/common/moduleinvoker.py:210, in biglearning.module2.common.moduleinvoker._module_run()
99
100
File /var/app/data/custom_module/publicmodules/stock_ranker_dai_train/v2/init.py:958, in run(data, validation_ds, learning_algorithm, number_of_leaves, min_docs_per_leaf, number_of_trees, learning_rate, max_bins, feature_fraction, data_row_fraction, plot_charts, ndcg_discount_base, base_model)
101
956 validation_data = validation_ds.read() if validation_ds is not None else None
102
957 logger.info(f"data loaded: train={data.shape}, validation={validation_data.shape if validation_data is not None else None}")
103
--> 958 result = train(
104
959 features=None,
105
960 data=data,
106
961 validation_data=validation_data,
107
962 learning_algorithm=learning_algorithm,
108
963 learning_rate=learning_rate,
109
964 max_bins=max_bins,
110
965 number_of_trees=number_of_trees,
111
966 number_of_leaves=number_of_leaves,
112
967 min_docs_per_leaf=min_docs_per_leaf,
113
968 feature_fraction=feature_fraction,
114
969 data_row_fraction=data_row_fraction,
115
970 ndcg_discount_base=ndcg_discount_base,
116
971 base_model_id=None,
117
972 # fai_cluster=None,
118
973 )
119
974 event.set()
120
975 return Outputs(
121
976 model=dai.DataSource.write_text(json.dumps(result["model"])),
122
977 feature_gains=dai.DataSource.write_bdb(result["feature_gains"]),
123
(...)
124
980 plot_charts=plot_charts,
125
981 )
126
127
File /var/app/data/custom_module/publicmodules/stock_ranker_dai_train/v2/init.py:879, in train(features, data, validation_data, learning_algorithm, learning_rate, max_bins, number_of_trees, number_of_leaves, min_docs_per_leaf, feature_fraction, data_row_fraction, ndcg_discount_base, base_model_id, fai_cluster, **kwargs)
128
877 result = fai.get(fai.remote(_train).remote(**kwargs))
129
878 else:
130
--> 879 result = _train(**kwargs)
131
880 return result
132
133
File /var/app/data/custom_module/publicmodules/stock_ranker_dai_train/v2/init.py:694, in train.<locals>._train(features, data, validation_data, learning_algorithm, learning_rate, max_bins, number_of_trees, number_of_leaves, min_docs_per_leaf, feature_fraction, data_row_fraction, ndcg_discount_base, base_model_id)
134
692 # TODO: 优化日志
135
693 logger.info("数据预处理 ..")
136
--> 694 bin_data = df_to_bin(
137
695 working_root=working_root,
138
696 dcg_trunc=10,
139
697 na_label=None,
140
698 sort=True,
141
699 bins="training",
142
700 max_bins=max_bins,
143
701 selected_columns=feature_names,
144
702 training=data,
145
703 validation=validation_data,
146
704 )
147
705 logger.info(f"训练集预处理完成 rows={bin_data.training_rows}, cols={bin_data.training_cols}")
148
706 if validation_data is not None:
149
150
File /var/app/data/custom_module/publicmodules/stock_ranker_dai_train/v2/init.py:532, in train.<locals>.df_to_bin(working_root, dcg_trunc, na_label, bins, max_bins, selected_columns, sort, **kwargs)
151
530 logger.info("特征预处理 ..")
152
531 if len(data[bins]) == 0:
153
--> 532 raise Exception("数据 %s 为空，请检查输入数据" % bins)
154
533 if selected_columns is not None:
155
534 feature_cols = set(selected_columns)
156
157
Exception: 数据 training 为空，请检查输入数据
训练集代码：
@module(position="-630,-117", comment='4. 训练数据，设置训练开始时间和结束时间', comment_collapsed=False)
m4 = M.extract_data_dai.v7( sql=m3.data, start_date='2018-01-01', start_date_bound_to_trading_date=True, end_date='2021-12-31', end_date_bound_to_trading_date=True, before_start_days=300, debug=False
\