滚动训练出现报错,不知道报的这个是什么错误?

策略分享
用户成长系列
标签: #<Tag:0x00007fa1a19748f8> #<Tag:0x00007fa1a19747b8>

(BOLO) #1

滚动训练出现报错,时间区间为2011-01-01到2020-08-01
更新天数为120天,最大最小数据天数为750天。
另外,滚动训练中的bq_graph勾选与否,有何区别呀?

克隆策略

    {"Description":"实验创建于2017/8/26","Summary":"","Graph":{"EdgesInternal":[{"DestinationInputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-15:instruments","SourceOutputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-8:data"},{"DestinationInputPortId":"-106:instruments","SourceOutputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-8:data"},{"DestinationInputPortId":"-773:input_1","SourceOutputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-15:data"},{"DestinationInputPortId":"-106:features","SourceOutputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-24:data"},{"DestinationInputPortId":"-113:features","SourceOutputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-24:data"},{"DestinationInputPortId":"-122:features","SourceOutputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-24:data"},{"DestinationInputPortId":"-129:features","SourceOutputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-24:data"},{"DestinationInputPortId":"-243:features","SourceOutputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-24:data"},{"DestinationInputPortId":"-251:features","SourceOutputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-24:data"},{"DestinationInputPortId":"-768:input_2","SourceOutputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-24:data"},{"DestinationInputPortId":"-778:input_2","SourceOutputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-24:data"},{"DestinationInputPortId":"-243:input_data","SourceOutputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-53:data"},{"DestinationInputPortId":"-122:instruments","SourceOutputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-62:data"},{"DestinationInputPortId":"-141:instruments","SourceOutputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-62:data"},{"DestinationInputPortId":"-113:input_data","SourceOutputPortId":"-106:data"},{"DestinationInputPortId":"-768:input_1","SourceOutputPortId":"-113:data"},{"DestinationInputPortId":"-129:input_data","SourceOutputPortId":"-122:data"},{"DestinationInputPortId":"-2431:input_2","SourceOutputPortId":"-129:data"},{"DestinationInputPortId":"-778:input_1","SourceOutputPortId":"-129:data"},{"DestinationInputPortId":"-168:inputs","SourceOutputPortId":"-160:data"},{"DestinationInputPortId":"-682:inputs","SourceOutputPortId":"-160:data"},{"DestinationInputPortId":"-224:inputs","SourceOutputPortId":"-168:data"},{"DestinationInputPortId":"-231:inputs","SourceOutputPortId":"-196:data"},{"DestinationInputPortId":"-196:inputs","SourceOutputPortId":"-224:data"},{"DestinationInputPortId":"-238:inputs","SourceOutputPortId":"-231:data"},{"DestinationInputPortId":"-682:outputs","SourceOutputPortId":"-238:data"},{"DestinationInputPortId":"-1098:input_model","SourceOutputPortId":"-682:data"},{"DestinationInputPortId":"-1540:trained_model","SourceOutputPortId":"-1098:data"},{"DestinationInputPortId":"-2431:input_1","SourceOutputPortId":"-1540:data"},{"DestinationInputPortId":"-141:options_data","SourceOutputPortId":"-2431:data_1"},{"DestinationInputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-53:data2","SourceOutputPortId":"-768:data"},{"DestinationInputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-53:data1","SourceOutputPortId":"-773:data"},{"DestinationInputPortId":"-251:input_data","SourceOutputPortId":"-778:data"},{"DestinationInputPortId":"-1098:training_data","SourceOutputPortId":"-243:data"},{"DestinationInputPortId":"-1540:input_data","SourceOutputPortId":"-251:data"}],"ModuleNodes":[{"Id":"287d2cb0-f53c-4101-bdf8-104b137c8601-8","ModuleId":"BigQuantSpace.instruments.instruments-v2","ModuleParameters":[{"Name":"start_date","Value":"2010-01-01","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"end_date","Value":"2015-12-31","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"market","Value":"CN_STOCK_A","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"instrument_list","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"max_count","Value":"0","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"rolling_conf","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-8"}],"OutputPortsInternal":[{"Name":"data","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-8","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":1,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"287d2cb0-f53c-4101-bdf8-104b137c8601-15","ModuleId":"BigQuantSpace.advanced_auto_labeler.advanced_auto_labeler-v2","ModuleParameters":[{"Name":"label_expr","Value":"# #号开始的表示注释\n# 0. 每行一个,顺序执行,从第二个开始,可以使用label字段\n# 1. 可用数据字段见 https://bigquant.com/docs/data_history_data.html\n# 添加benchmark_前缀,可使用对应的benchmark数据\n# 2. 可用操作符和函数见 `表达式引擎 <https://bigquant.com/docs/big_expr.html>`_\n\n# 计算收益:5日收盘价(作为卖出价格)除以明日开盘价(作为买入价格)\nshift(close, -5) / shift(open, -1)-1\n\n# 极值处理:用1%和99%分位的值做clip\nclip(label, all_quantile(label, 0.01), all_quantile(label, 0.99))\n\n# 过滤掉一字涨停的情况 (设置label为NaN,在后续处理和训练中会忽略NaN的label)\nwhere(shift(high, -1) == shift(low, -1), NaN, label)\n","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"start_date","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"end_date","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"benchmark","Value":"000300.SHA","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"drop_na_label","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"cast_label_int","Value":"False","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_functions","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"instruments","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-15"}],"OutputPortsInternal":[{"Name":"data","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-15","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":2,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"287d2cb0-f53c-4101-bdf8-104b137c8601-24","ModuleId":"BigQuantSpace.input_features.input_features-v1","ModuleParameters":[{"Name":"features","Value":"(close_0-mean(close_0,12))/mean(close_0,12)*100\nrank(std(amount_0,15))\nrank_avg_amount_0/rank_avg_amount_8\nts_argmin(low_0,20)\nrank_return_30\n(low_1-close_0)/close_0\nta_bbands_lowerband_14_0\nmean(mf_net_pct_s_0,4)\namount_0/avg_amount_3\nreturn_0/return_5\nreturn_1/return_5\nrank_avg_amount_7/rank_avg_amount_10\nta_sma_10_0/close_0\nsqrt(high_0*low_0)-amount_0/volume_0*adjust_factor_0\navg_turn_15/(turn_0+1e-5)\nreturn_10\nmf_net_pct_s_0\n(close_0-open_0)/close_1","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"features_ds","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-24"}],"OutputPortsInternal":[{"Name":"data","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-24","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":3,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"287d2cb0-f53c-4101-bdf8-104b137c8601-53","ModuleId":"BigQuantSpace.join.join-v3","ModuleParameters":[{"Name":"on","Value":"date,instrument","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"how","Value":"inner","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"sort","Value":"False","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"data1","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-53"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"data2","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-53"}],"OutputPortsInternal":[{"Name":"data","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-53","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":7,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"287d2cb0-f53c-4101-bdf8-104b137c8601-62","ModuleId":"BigQuantSpace.instruments.instruments-v2","ModuleParameters":[{"Name":"start_date","Value":"2016-01-01","ValueType":"Literal","LinkedGlobalParameter":"交易日期"},{"Name":"end_date","Value":"2019-04-20","ValueType":"Literal","LinkedGlobalParameter":"交易日期"},{"Name":"market","Value":"CN_STOCK_A","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"instrument_list","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"max_count","Value":"0","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"rolling_conf","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-62"}],"OutputPortsInternal":[{"Name":"data","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-62","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":9,"IsPartOfPartialRun":null,"Comment":"预测数据,用于回测和模拟","CommentCollapsed":false},{"Id":"-106","ModuleId":"BigQuantSpace.general_feature_extractor.general_feature_extractor-v7","ModuleParameters":[{"Name":"start_date","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"end_date","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"before_start_days","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"instruments","NodeId":"-106"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"features","NodeId":"-106"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-106","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":15,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-113","ModuleId":"BigQuantSpace.derived_feature_extractor.derived_feature_extractor-v3","ModuleParameters":[{"Name":"date_col","Value":"date","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"instrument_col","Value":"instrument","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"drop_na","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"remove_extra_columns","Value":"False","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_functions","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_data","NodeId":"-113"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"features","NodeId":"-113"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-113","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":16,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-122","ModuleId":"BigQuantSpace.general_feature_extractor.general_feature_extractor-v7","ModuleParameters":[{"Name":"start_date","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"end_date","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"before_start_days","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"instruments","NodeId":"-122"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"features","NodeId":"-122"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-122","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":17,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-129","ModuleId":"BigQuantSpace.derived_feature_extractor.derived_feature_extractor-v3","ModuleParameters":[{"Name":"date_col","Value":"date","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"instrument_col","Value":"instrument","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"drop_na","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"remove_extra_columns","Value":"False","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_functions","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_data","NodeId":"-129"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"features","NodeId":"-129"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-129","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":18,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-141","ModuleId":"BigQuantSpace.trade.trade-v4","ModuleParameters":[{"Name":"start_date","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"end_date","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"initialize","Value":"# 回测引擎:初始化函数,只执行一次\ndef bigquant_run(context):\n # 加载预测数据\n context.ranker_prediction = context.options['data'].read_df()\n\n # 系统已经设置了默认的交易手续费和滑点,要修改手续费可使用如下函数\n context.set_commission(PerOrder(buy_cost=0.0003, sell_cost=0.0013, min_cost=5))\n # 预测数据,通过options传入进来,使用 read_df 函数,加载到内存 (DataFrame)\n # 设置买入的股票数量,这里买入预测股票列表排名靠前的5只\n stock_count = 10\n # 每只的股票的权重,如下的权重分配会使得靠前的股票分配多一点的资金,[0.339160, 0.213986, 0.169580, ..]\n context.stock_weights = T.norm([1 / math.log(i + 2) for i in range(0, stock_count)])\n # 设置每只股票占用的最大资金比例\n context.max_cash_per_instrument = 0.1\n context.options['hold_days'] = 5\n","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"handle_data","Value":"# 回测引擎:每日数据处理函数,每天执行一次\ndef bigquant_run(context, data):\n # 按日期过滤得到今日的预测数据\n ranker_prediction = context.ranker_prediction[\n context.ranker_prediction.date == data.current_dt.strftime('%Y-%m-%d')]\n\n # 1. 资金分配\n # 平均持仓时间是hold_days,每日都将买入股票,每日预期使用 1/hold_days 的资金\n # 实际操作中,会存在一定的买入误差,所以在前hold_days天,等量使用资金;之后,尽量使用剩余资金(这里设置最多用等量的1.5倍)\n is_staging = context.trading_day_index < context.options['hold_days'] # 是否在建仓期间(前 hold_days 天)\n cash_avg = context.portfolio.portfolio_value / context.options['hold_days']\n cash_for_buy = min(context.portfolio.cash, (1 if is_staging else 1.5) * cash_avg)\n cash_for_sell = cash_avg - (context.portfolio.cash - cash_for_buy)\n positions = {e.symbol: p.amount * p.last_sale_price\n for e, p in context.perf_tracker.position_tracker.positions.items()}\n\n # 2. 生成卖出订单:hold_days天之后才开始卖出;对持仓的股票,按机器学习算法预测的排序末位淘汰\n if not is_staging and cash_for_sell > 0:\n equities = {e.symbol: e for e, p in context.perf_tracker.position_tracker.positions.items()}\n instruments = list(reversed(list(ranker_prediction.instrument[ranker_prediction.instrument.apply(\n lambda x: x in equities and not context.has_unfinished_sell_order(equities[x]))])))\n # print('rank order for sell %s' % instruments)\n for instrument in instruments:\n context.order_target(context.symbol(instrument), 0)\n cash_for_sell -= positions[instrument]\n if cash_for_sell <= 0:\n break\n\n # 3. 生成买入订单:按机器学习算法预测的排序,买入前面的stock_count只股票\n buy_cash_weights = context.stock_weights\n buy_instruments = list(ranker_prediction.instrument[:len(buy_cash_weights)])\n max_cash_per_instrument = context.portfolio.portfolio_value * context.max_cash_per_instrument\n for i, instrument in enumerate(buy_instruments):\n cash = cash_for_buy * buy_cash_weights[i]\n if cash > max_cash_per_instrument - positions.get(instrument, 0):\n # 确保股票持仓量不会超过每次股票最大的占用资金量\n cash = max_cash_per_instrument - positions.get(instrument, 0)\n if cash > 0:\n context.order_value(context.symbol(instrument), cash)\n","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"prepare","Value":"# 回测引擎:准备数据,只执行一次\ndef bigquant_run(context):\n pass\n","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"before_trading_start","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"volume_limit","Value":0.025,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"order_price_field_buy","Value":"open","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"order_price_field_sell","Value":"close","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"capital_base","Value":1000000,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"auto_cancel_non_tradable_orders","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"data_frequency","Value":"daily","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"price_type","Value":"后复权","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"product_type","Value":"股票","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"plot_charts","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"backtest_only","Value":"False","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"benchmark","Value":"000300.SHA","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"instruments","NodeId":"-141"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"options_data","NodeId":"-141"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"history_ds","NodeId":"-141"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"benchmark_ds","NodeId":"-141"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"trading_calendar","NodeId":"-141"}],"OutputPortsInternal":[{"Name":"raw_perf","NodeId":"-141","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":19,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-160","ModuleId":"BigQuantSpace.dl_layer_input.dl_layer_input-v1","ModuleParameters":[{"Name":"shape","Value":"18","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"batch_shape","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"dtype","Value":"float32","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"sparse","Value":"False","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"name","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"inputs","NodeId":"-160"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-160","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":6,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-168","ModuleId":"BigQuantSpace.dl_layer_dense.dl_layer_dense-v1","ModuleParameters":[{"Name":"units","Value":"256","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activation","Value":"relu","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_activation","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"use_bias","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_initializer","Value":"glorot_uniform","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_initializer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_initializer","Value":"Zeros","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_initializer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_activity_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_constraint","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_constraint","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_constraint","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_constraint","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"name","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"inputs","NodeId":"-168"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-168","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":8,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-196","ModuleId":"BigQuantSpace.dl_layer_dense.dl_layer_dense-v1","ModuleParameters":[{"Name":"units","Value":"128","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activation","Value":"relu","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_activation","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"use_bias","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_initializer","Value":"glorot_uniform","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_initializer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_initializer","Value":"Zeros","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_initializer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_activity_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_constraint","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_constraint","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_constraint","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_constraint","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"name","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"inputs","NodeId":"-196"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-196","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":20,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-224","ModuleId":"BigQuantSpace.dl_layer_dropout.dl_layer_dropout-v1","ModuleParameters":[{"Name":"rate","Value":"0.1","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"noise_shape","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"seed","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"name","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"inputs","NodeId":"-224"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-224","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":21,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-231","ModuleId":"BigQuantSpace.dl_layer_dropout.dl_layer_dropout-v1","ModuleParameters":[{"Name":"rate","Value":"0.1","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"noise_shape","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"seed","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"name","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"inputs","NodeId":"-231"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-231","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":22,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-238","ModuleId":"BigQuantSpace.dl_layer_dense.dl_layer_dense-v1","ModuleParameters":[{"Name":"units","Value":"1","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activation","Value":"linear","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_activation","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"use_bias","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_initializer","Value":"glorot_uniform","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_initializer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_initializer","Value":"Zeros","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_initializer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_activity_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_constraint","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_constraint","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_constraint","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_constraint","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"name","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"inputs","NodeId":"-238"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-238","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":23,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-682","ModuleId":"BigQuantSpace.dl_model_init.dl_model_init-v1","ModuleParameters":[],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"inputs","NodeId":"-682"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"outputs","NodeId":"-682"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-682","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":4,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-1098","ModuleId":"BigQuantSpace.dl_model_train.dl_model_train-v1","ModuleParameters":[{"Name":"optimizer","Value":"Adam","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_optimizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"loss","Value":"mean_squared_error","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_loss","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"metrics","Value":"mse","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"batch_size","Value":"1024","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"epochs","Value":"5","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"n_gpus","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"verbose","Value":"2:每个epoch输出一行记录","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_model","NodeId":"-1098"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"training_data","NodeId":"-1098"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"validation_data","NodeId":"-1098"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-1098","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":5,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-1540","ModuleId":"BigQuantSpace.dl_model_predict.dl_model_predict-v1","ModuleParameters":[{"Name":"batch_size","Value":"1024","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"n_gpus","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"verbose","Value":"2:每个epoch输出一行记录","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"trained_model","NodeId":"-1540"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_data","NodeId":"-1540"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-1540","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":11,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-2431","ModuleId":"BigQuantSpace.cached.cached-v3","ModuleParameters":[{"Name":"run","Value":"# Python 代码入口函数,input_1/2/3 对应三个输入端,data_1/2/3 对应三个输出端\ndef bigquant_run(input_1, input_2, input_3):\n # 示例代码如下。在这里编写您的代码\n pred_label = input_1.read_pickle()\n df = input_2.read_df()\n df = pd.DataFrame({'pred_label':pred_label[:,0], 'instrument':df.instrument, 'date':df.date})\n df.sort_values(['date','pred_label'],inplace=True, ascending=[True,False])\n return Outputs(data_1=DataSource.write_df(df), data_2=None, data_3=None)\n","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"post_run","Value":"# 后处理函数,可选。输入是主函数的输出,可以在这里对数据做处理,或者返回更友好的outputs数据格式。此函数输出不会被缓存。\ndef bigquant_run(outputs):\n return outputs\n","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"input_ports","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"params","Value":"{}","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"output_ports","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_1","NodeId":"-2431"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_2","NodeId":"-2431"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_3","NodeId":"-2431"}],"OutputPortsInternal":[{"Name":"data_1","NodeId":"-2431","OutputType":null},{"Name":"data_2","NodeId":"-2431","OutputType":null},{"Name":"data_3","NodeId":"-2431","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":24,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-768","ModuleId":"BigQuantSpace.standardlize.standardlize-v8","ModuleParameters":[{"Name":"columns_input","Value":"[]","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_1","NodeId":"-768"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_2","NodeId":"-768"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-768","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":14,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-773","ModuleId":"BigQuantSpace.standardlize.standardlize-v8","ModuleParameters":[{"Name":"columns_input","Value":"label","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_1","NodeId":"-773"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_2","NodeId":"-773"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-773","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":13,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-778","ModuleId":"BigQuantSpace.standardlize.standardlize-v8","ModuleParameters":[{"Name":"columns_input","Value":"[]","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_1","NodeId":"-778"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_2","NodeId":"-778"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-778","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":25,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-243","ModuleId":"BigQuantSpace.dl_convert_to_bin.dl_convert_to_bin-v2","ModuleParameters":[{"Name":"window_size","Value":1,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"feature_clip","Value":5,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"flatten","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"window_along_col","Value":"instrument","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_data","NodeId":"-243"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"features","NodeId":"-243"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-243","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":26,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-251","ModuleId":"BigQuantSpace.dl_convert_to_bin.dl_convert_to_bin-v2","ModuleParameters":[{"Name":"window_size","Value":1,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"feature_clip","Value":5,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"flatten","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"window_along_col","Value":"instrument","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_data","NodeId":"-251"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"features","NodeId":"-251"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-251","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":27,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-613","ModuleId":"BigQuantSpace.hyper_rolling_train.hyper_rolling_train-v1","ModuleParameters":[{"Name":"run","Value":"def bigquant_run(\n bq_graph,\n inputs,\n trading_days_market='CN', # 使用那个市场的交易日历\n train_instruments_mid='m1', # 训练数据 证券代码列表 模块id\n test_instruments_mid='m9', # 测试数据 证券代码列表 模块id\n predict_mid='m24', # 预测 模块id\n trade_mid='m19', # 回测 模块id\n start_date='2011-01-01', # 数据开始日期\n end_date=T.live_run_param('trading_date', '2020-08-01'), # 数据结束日期\n train_update_days=120, # 更新周期,按交易日计算,每多少天更新一次\n train_update_days_for_live=None,#模拟实盘模式下的更新周期,按交易日计算,每多少天更新一次。如果需要在模拟实盘阶段使用不同的模型更新周期,可以设置这个参数\n train_data_min_days=750, # 最小数据天数,按交易日计算,所以第一个滚动的结束日期是 从开始日期到开始日期+最小数据天数\n train_data_max_days=750, # 最大数据天数,按交易日计算,0,表示没有限制,否则每一个滚动的开始日期=max(此滚动的结束日期-最大数据天数, 开始日期\n rolling_count_for_live=1, #实盘模式下滚动次数,模拟实盘模式下,取最后多少次滚动。一般在模拟实盘模式下,只用到最后一次滚动训练的模型,这里可以设置为1;如果你的滚动训练数据时间段很短,以至于期间可能没有训练数据,这里可以设置大一点。0表示没有限制\n):\n def merge_datasources(input_1):\n df_list = [ds.read_df() for ds in input_1]\n df = pd.concat(df_list)\n instrument_data = {\n 'start_date': df['date'].min().strftime('%Y-%m-%d'),\n 'end_date': df['date'].max().strftime('%Y-%m-%d'),\n 'instruments': list(set(df['instrument'])),\n }\n return Outputs(data=DataSource.write_df(df), instrument_data=DataSource.write_pickle(instrument_data))\n\n def gen_rolling_dates(trading_days_market, start_date, end_date, train_update_days, train_update_days_for_live, train_data_min_days, train_data_max_days, rolling_count_for_live):\n # 是否实盘模式\n tdays = list(D.trading_days(market=trading_days_market, start_date=start_date, end_date=end_date)['date'])\n is_live_run = T.live_run_param('trading_date', None) is not None\n\n if is_live_run and train_update_days_for_live:\n train_update_days = train_update_days_for_live\n\n rollings = []\n train_end_date = train_data_min_days\n while train_end_date < len(tdays):\n if train_data_max_days is not None:\n train_start_date = max(train_end_date - train_data_max_days, 0)\n else:\n train_start_date = start_date\n rollings.append({\n 'train_start_date': tdays[train_start_date].strftime('%Y-%m-%d'),\n 'train_end_date': tdays[train_end_date - 1].strftime('%Y-%m-%d'),\n 'test_start_date': tdays[train_end_date].strftime('%Y-%m-%d'),\n 'test_end_date': tdays[min(train_end_date + train_update_days, len(tdays)) - 1].strftime('%Y-%m-%d'),\n })\n train_end_date += train_update_days\n\n if not rollings:\n raise Exception('没有滚动需要执行,请检查配置')\n\n if is_live_run and rolling_count_for_live:\n rollings = rollings[-rolling_count_for_live:]\n\n return rollings\n\n g = bq_graph\n\n rolling_dates = gen_rolling_dates(\n trading_days_market, start_date, end_date, train_update_days, train_update_days_for_live, train_data_min_days, train_data_max_days, rolling_count_for_live)\n\n # 训练和预测\n results = []\n for rolling in rolling_dates:\n parameters = {}\n # 先禁用回测\n parameters[trade_mid + '.__enabled__'] = False\n parameters[train_instruments_mid + '.start_date'] = rolling['train_start_date']\n parameters[train_instruments_mid + '.end_date'] = rolling['train_end_date']\n parameters[test_instruments_mid + '.start_date'] = rolling['test_start_date']\n parameters[test_instruments_mid + '.end_date'] = rolling['test_end_date']\n # print('------ rolling_train:', parameters)\n results.append(g.run(parameters))\n\n # 合并预测结果并回测\n mx = M.cached.v3(run=merge_datasources, input_1=[result[predict_mid].data_1 for result in results])\n parameters = {}\n parameters['*.__enabled__'] = False\n parameters[trade_mid + '.__enabled__'] = True\n parameters[trade_mid + '.instruments'] = mx.instrument_data\n parameters[trade_mid + '.options_data'] = mx.data\n\n trade = g.run(parameters)\n\n return {'rollings': results, 'trade': trade}\n","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"run_now","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bq_graph","Value":"False","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"bq_graph_port","NodeId":"-613"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_1","NodeId":"-613"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_2","NodeId":"-613"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_3","NodeId":"-613"}],"OutputPortsInternal":[{"Name":"result","NodeId":"-613","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":10,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true}],"SerializedClientData":"<?xml version='1.0' encoding='utf-16'?><DataV1 xmlns:xsd='http://www.w3.org/2001/XMLSchema' xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'><Meta /><NodePositions><NodePosition Node='287d2cb0-f53c-4101-bdf8-104b137c8601-8' Position='207,58,200,200'/><NodePosition Node='287d2cb0-f53c-4101-bdf8-104b137c8601-15' Position='131,205,200,200'/><NodePosition Node='287d2cb0-f53c-4101-bdf8-104b137c8601-24' Position='766,2,200,200'/><NodePosition Node='287d2cb0-f53c-4101-bdf8-104b137c8601-53' Position='301,459,200,200'/><NodePosition Node='287d2cb0-f53c-4101-bdf8-104b137c8601-62' Position='1012,160,200,200'/><NodePosition Node='-106' Position='412,184,200,200'/><NodePosition Node='-113' Position='428,278,200,200'/><NodePosition Node='-122' Position='1006,271,200,200'/><NodePosition Node='-129' Position='1004,352,200,200'/><NodePosition Node='-141' Position='406,1056,200,200'/><NodePosition Node='-160' Position='-202,33,200,200'/><NodePosition Node='-168' Position='-201,148,200,200'/><NodePosition Node='-196' Position='-203,311,200,200'/><NodePosition Node='-224' Position='-203,239,200,200'/><NodePosition Node='-231' Position='-201,395,200,200'/><NodePosition Node='-238' Position='-198,470,200,200'/><NodePosition Node='-682' Position='-194,560,200,200'/><NodePosition Node='-1098' Position='-50,765,200,200'/><NodePosition Node='-1540' Position='249,845,200,200'/><NodePosition Node='-2431' Position='424,950,200,200'/><NodePosition Node='-768' Position='454,369,200,200'/><NodePosition Node='-773' Position='137,329,200,200'/><NodePosition Node='-778' Position='1000,438,200,200'/><NodePosition Node='-243' Position='294,543,200,200'/><NodePosition Node='-251' Position='1004,518,200,200'/><NodePosition Node='-613' Position='350,-127,200,200'/></NodePositions><NodeGroups /></DataV1>"},"IsDraft":true,"ParentExperimentId":null,"WebService":{"IsWebServiceExperiment":false,"Inputs":[],"Outputs":[],"Parameters":[{"Name":"交易日期","Value":"","ParameterDefinition":{"Name":"交易日期","FriendlyName":"交易日期","DefaultValue":"","ParameterType":"String","HasDefaultValue":true,"IsOptional":true,"ParameterRules":[],"HasRules":false,"MarkupType":0,"CredentialDescriptor":null}}],"WebServiceGroupId":null,"SerializedClientData":"<?xml version='1.0' encoding='utf-16'?><DataV1 xmlns:xsd='http://www.w3.org/2001/XMLSchema' xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'><Meta /><NodePositions></NodePositions><NodeGroups /></DataV1>"},"DisableNodesUpdate":false,"Category":"user","Tags":[],"IsPartialRun":true}
    In [2]:
    # 本代码由可视化策略环境自动生成 2020年8月20日 15:13
    # 本代码单元只能在可视化模式下编辑。您也可以拷贝代码,粘贴到新建的代码单元或者策略,然后修改。
    
    
    # Python 代码入口函数,input_1/2/3 对应三个输入端,data_1/2/3 对应三个输出端
    def m24_run_bigquant_run(input_1, input_2, input_3):
        # 示例代码如下。在这里编写您的代码
        pred_label = input_1.read_pickle()
        df = input_2.read_df()
        df = pd.DataFrame({'pred_label':pred_label[:,0], 'instrument':df.instrument, 'date':df.date})
        df.sort_values(['date','pred_label'],inplace=True, ascending=[True,False])
        return Outputs(data_1=DataSource.write_df(df), data_2=None, data_3=None)
    
    # 后处理函数,可选。输入是主函数的输出,可以在这里对数据做处理,或者返回更友好的outputs数据格式。此函数输出不会被缓存。
    def m24_post_run_bigquant_run(outputs):
        return outputs
    
    # 回测引擎:初始化函数,只执行一次
    def m19_initialize_bigquant_run(context):
        # 加载预测数据
        context.ranker_prediction = context.options['data'].read_df()
    
        # 系统已经设置了默认的交易手续费和滑点,要修改手续费可使用如下函数
        context.set_commission(PerOrder(buy_cost=0.0003, sell_cost=0.0013, min_cost=5))
        # 预测数据,通过options传入进来,使用 read_df 函数,加载到内存 (DataFrame)
        # 设置买入的股票数量,这里买入预测股票列表排名靠前的5只
        stock_count = 10
        # 每只的股票的权重,如下的权重分配会使得靠前的股票分配多一点的资金,[0.339160, 0.213986, 0.169580, ..]
        context.stock_weights = T.norm([1 / math.log(i + 2) for i in range(0, stock_count)])
        # 设置每只股票占用的最大资金比例
        context.max_cash_per_instrument = 0.1
        context.options['hold_days'] = 5
    
    # 回测引擎:每日数据处理函数,每天执行一次
    def m19_handle_data_bigquant_run(context, data):
        # 按日期过滤得到今日的预测数据
        ranker_prediction = context.ranker_prediction[
            context.ranker_prediction.date == data.current_dt.strftime('%Y-%m-%d')]
    
        # 1. 资金分配
        # 平均持仓时间是hold_days,每日都将买入股票,每日预期使用 1/hold_days 的资金
        # 实际操作中,会存在一定的买入误差,所以在前hold_days天,等量使用资金;之后,尽量使用剩余资金(这里设置最多用等量的1.5倍)
        is_staging = context.trading_day_index < context.options['hold_days'] # 是否在建仓期间(前 hold_days 天)
        cash_avg = context.portfolio.portfolio_value / context.options['hold_days']
        cash_for_buy = min(context.portfolio.cash, (1 if is_staging else 1.5) * cash_avg)
        cash_for_sell = cash_avg - (context.portfolio.cash - cash_for_buy)
        positions = {e.symbol: p.amount * p.last_sale_price
                     for e, p in context.perf_tracker.position_tracker.positions.items()}
    
        # 2. 生成卖出订单:hold_days天之后才开始卖出;对持仓的股票,按机器学习算法预测的排序末位淘汰
        if not is_staging and cash_for_sell > 0:
            equities = {e.symbol: e for e, p in context.perf_tracker.position_tracker.positions.items()}
            instruments = list(reversed(list(ranker_prediction.instrument[ranker_prediction.instrument.apply(
                    lambda x: x in equities and not context.has_unfinished_sell_order(equities[x]))])))
            # print('rank order for sell %s' % instruments)
            for instrument in instruments:
                context.order_target(context.symbol(instrument), 0)
                cash_for_sell -= positions[instrument]
                if cash_for_sell <= 0:
                    break
    
        # 3. 生成买入订单:按机器学习算法预测的排序,买入前面的stock_count只股票
        buy_cash_weights = context.stock_weights
        buy_instruments = list(ranker_prediction.instrument[:len(buy_cash_weights)])
        max_cash_per_instrument = context.portfolio.portfolio_value * context.max_cash_per_instrument
        for i, instrument in enumerate(buy_instruments):
            cash = cash_for_buy * buy_cash_weights[i]
            if cash > max_cash_per_instrument - positions.get(instrument, 0):
                # 确保股票持仓量不会超过每次股票最大的占用资金量
                cash = max_cash_per_instrument - positions.get(instrument, 0)
            if cash > 0:
                context.order_value(context.symbol(instrument), cash)
    
    # 回测引擎:准备数据,只执行一次
    def m19_prepare_bigquant_run(context):
        pass
    
    
    g = T.Graph({
    
        'm1': 'M.instruments.v2',
        'm1.start_date': '2010-01-01',
        'm1.end_date': '2015-12-31',
        'm1.market': 'CN_STOCK_A',
        'm1.instrument_list': '',
        'm1.max_count': 0,
    
        'm2': 'M.advanced_auto_labeler.v2',
        'm2.instruments': T.Graph.OutputPort('m1.data'),
        'm2.label_expr': """# #号开始的表示注释
    # 0. 每行一个,顺序执行,从第二个开始,可以使用label字段
    # 1. 可用数据字段见 https://bigquant.com/docs/data_history_data.html
    #   添加benchmark_前缀,可使用对应的benchmark数据
    # 2. 可用操作符和函数见 `表达式引擎 <https://bigquant.com/docs/big_expr.html>`_
    
    # 计算收益:5日收盘价(作为卖出价格)除以明日开盘价(作为买入价格)
    shift(close, -5) / shift(open, -1)-1
    
    # 极值处理:用1%和99%分位的值做clip
    clip(label, all_quantile(label, 0.01), all_quantile(label, 0.99))
    
    # 过滤掉一字涨停的情况 (设置label为NaN,在后续处理和训练中会忽略NaN的label)
    where(shift(high, -1) == shift(low, -1), NaN, label)
    """,
        'm2.start_date': '',
        'm2.end_date': '',
        'm2.benchmark': '000300.SHA',
        'm2.drop_na_label': True,
        'm2.cast_label_int': False,
    
        'm13': 'M.standardlize.v8',
        'm13.input_1': T.Graph.OutputPort('m2.data'),
        'm13.columns_input': 'label',
        'm13.m_cached': False,
    
        'm3': 'M.input_features.v1',
        'm3.features': """(close_0-mean(close_0,12))/mean(close_0,12)*100
    rank(std(amount_0,15))
    rank_avg_amount_0/rank_avg_amount_8
    ts_argmin(low_0,20)
    rank_return_30
    (low_1-close_0)/close_0
    ta_bbands_lowerband_14_0
    mean(mf_net_pct_s_0,4)
    amount_0/avg_amount_3
    return_0/return_5
    return_1/return_5
    rank_avg_amount_7/rank_avg_amount_10
    ta_sma_10_0/close_0
    sqrt(high_0*low_0)-amount_0/volume_0*adjust_factor_0
    avg_turn_15/(turn_0+1e-5)
    return_10
    mf_net_pct_s_0
    (close_0-open_0)/close_1""",
        'm3.m_cached': False,
    
        'm15': 'M.general_feature_extractor.v7',
        'm15.instruments': T.Graph.OutputPort('m1.data'),
        'm15.features': T.Graph.OutputPort('m3.data'),
        'm15.start_date': '',
        'm15.end_date': '',
        'm15.before_start_days': 0,
    
        'm16': 'M.derived_feature_extractor.v3',
        'm16.input_data': T.Graph.OutputPort('m15.data'),
        'm16.features': T.Graph.OutputPort('m3.data'),
        'm16.date_col': 'date',
        'm16.instrument_col': 'instrument',
        'm16.drop_na': True,
        'm16.remove_extra_columns': False,
    
        'm14': 'M.standardlize.v8',
        'm14.input_1': T.Graph.OutputPort('m16.data'),
        'm14.input_2': T.Graph.OutputPort('m3.data'),
        'm14.columns_input': '[]',
    
        'm7': 'M.join.v3',
        'm7.data1': T.Graph.OutputPort('m13.data'),
        'm7.data2': T.Graph.OutputPort('m14.data'),
        'm7.on': 'date,instrument',
        'm7.how': 'inner',
        'm7.sort': False,
    
        'm26': 'M.dl_convert_to_bin.v2',
        'm26.input_data': T.Graph.OutputPort('m7.data'),
        'm26.features': T.Graph.OutputPort('m3.data'),
        'm26.window_size': 1,
        'm26.feature_clip': 5,
        'm26.flatten': True,
        'm26.window_along_col': 'instrument',
    
        'm9': 'M.instruments.v2',
        'm9.start_date': T.live_run_param('trading_date', '2016-01-01'),
        'm9.end_date': T.live_run_param('trading_date', '2019-04-20'),
        'm9.market': 'CN_STOCK_A',
        'm9.instrument_list': '',
        'm9.max_count': 0,
    
        'm17': 'M.general_feature_extractor.v7',
        'm17.instruments': T.Graph.OutputPort('m9.data'),
        'm17.features': T.Graph.OutputPort('m3.data'),
        'm17.start_date': '',
        'm17.end_date': '',
        'm17.before_start_days': 0,
    
        'm18': 'M.derived_feature_extractor.v3',
        'm18.input_data': T.Graph.OutputPort('m17.data'),
        'm18.features': T.Graph.OutputPort('m3.data'),
        'm18.date_col': 'date',
        'm18.instrument_col': 'instrument',
        'm18.drop_na': True,
        'm18.remove_extra_columns': False,
    
        'm25': 'M.standardlize.v8',
        'm25.input_1': T.Graph.OutputPort('m18.data'),
        'm25.input_2': T.Graph.OutputPort('m3.data'),
        'm25.columns_input': '[]',
    
        'm27': 'M.dl_convert_to_bin.v2',
        'm27.input_data': T.Graph.OutputPort('m25.data'),
        'm27.features': T.Graph.OutputPort('m3.data'),
        'm27.window_size': 1,
        'm27.feature_clip': 5,
        'm27.flatten': True,
        'm27.window_along_col': 'instrument',
    
        'm6': 'M.dl_layer_input.v1',
        'm6.shape': '18',
        'm6.batch_shape': '',
        'm6.dtype': 'float32',
        'm6.sparse': False,
        'm6.name': '',
    
        'm8': 'M.dl_layer_dense.v1',
        'm8.inputs': T.Graph.OutputPort('m6.data'),
        'm8.units': 256,
        'm8.activation': 'relu',
        'm8.use_bias': True,
        'm8.kernel_initializer': 'glorot_uniform',
        'm8.bias_initializer': 'Zeros',
        'm8.kernel_regularizer': 'None',
        'm8.kernel_regularizer_l1': 0,
        'm8.kernel_regularizer_l2': 0,
        'm8.bias_regularizer': 'None',
        'm8.bias_regularizer_l1': 0,
        'm8.bias_regularizer_l2': 0,
        'm8.activity_regularizer': 'None',
        'm8.activity_regularizer_l1': 0,
        'm8.activity_regularizer_l2': 0,
        'm8.kernel_constraint': 'None',
        'm8.bias_constraint': 'None',
        'm8.name': '',
    
        'm21': 'M.dl_layer_dropout.v1',
        'm21.inputs': T.Graph.OutputPort('m8.data'),
        'm21.rate': 0.1,
        'm21.noise_shape': '',
        'm21.name': '',
    
        'm20': 'M.dl_layer_dense.v1',
        'm20.inputs': T.Graph.OutputPort('m21.data'),
        'm20.units': 128,
        'm20.activation': 'relu',
        'm20.use_bias': True,
        'm20.kernel_initializer': 'glorot_uniform',
        'm20.bias_initializer': 'Zeros',
        'm20.kernel_regularizer': 'None',
        'm20.kernel_regularizer_l1': 0,
        'm20.kernel_regularizer_l2': 0,
        'm20.bias_regularizer': 'None',
        'm20.bias_regularizer_l1': 0,
        'm20.bias_regularizer_l2': 0,
        'm20.activity_regularizer': 'None',
        'm20.activity_regularizer_l1': 0,
        'm20.activity_regularizer_l2': 0,
        'm20.kernel_constraint': 'None',
        'm20.bias_constraint': 'None',
        'm20.name': '',
    
        'm22': 'M.dl_layer_dropout.v1',
        'm22.inputs': T.Graph.OutputPort('m20.data'),
        'm22.rate': 0.1,
        'm22.noise_shape': '',
        'm22.name': '',
    
        'm23': 'M.dl_layer_dense.v1',
        'm23.inputs': T.Graph.OutputPort('m22.data'),
        'm23.units': 1,
        'm23.activation': 'linear',
        'm23.use_bias': True,
        'm23.kernel_initializer': 'glorot_uniform',
        'm23.bias_initializer': 'Zeros',
        'm23.kernel_regularizer': 'None',
        'm23.kernel_regularizer_l1': 0,
        'm23.kernel_regularizer_l2': 0,
        'm23.bias_regularizer': 'None',
        'm23.bias_regularizer_l1': 0,
        'm23.bias_regularizer_l2': 0,
        'm23.activity_regularizer': 'None',
        'm23.activity_regularizer_l1': 0,
        'm23.activity_regularizer_l2': 0,
        'm23.kernel_constraint': 'None',
        'm23.bias_constraint': 'None',
        'm23.name': '',
    
        'm4': 'M.dl_model_init.v1',
        'm4.inputs': T.Graph.OutputPort('m6.data'),
        'm4.outputs': T.Graph.OutputPort('m23.data'),
    
        'm5': 'M.dl_model_train.v1',
        'm5.input_model': T.Graph.OutputPort('m4.data'),
        'm5.training_data': T.Graph.OutputPort('m26.data'),
        'm5.optimizer': 'Adam',
        'm5.loss': 'mean_squared_error',
        'm5.metrics': 'mse',
        'm5.batch_size': 1024,
        'm5.epochs': 5,
        'm5.n_gpus': 0,
        'm5.verbose': '2:每个epoch输出一行记录',
    
        'm11': 'M.dl_model_predict.v1',
        'm11.trained_model': T.Graph.OutputPort('m5.data'),
        'm11.input_data': T.Graph.OutputPort('m27.data'),
        'm11.batch_size': 1024,
        'm11.n_gpus': 0,
        'm11.verbose': '2:每个epoch输出一行记录',
    
        'm24': 'M.cached.v3',
        'm24.input_1': T.Graph.OutputPort('m11.data'),
        'm24.input_2': T.Graph.OutputPort('m18.data'),
        'm24.run': m24_run_bigquant_run,
        'm24.post_run': m24_post_run_bigquant_run,
        'm24.input_ports': '',
        'm24.params': '{}',
        'm24.output_ports': '',
    
        'm19': 'M.trade.v4',
        'm19.instruments': T.Graph.OutputPort('m9.data'),
        'm19.options_data': T.Graph.OutputPort('m24.data_1'),
        'm19.start_date': '',
        'm19.end_date': '',
        'm19.initialize': m19_initialize_bigquant_run,
        'm19.handle_data': m19_handle_data_bigquant_run,
        'm19.prepare': m19_prepare_bigquant_run,
        'm19.volume_limit': 0.025,
        'm19.order_price_field_buy': 'open',
        'm19.order_price_field_sell': 'close',
        'm19.capital_base': 1000000,
        'm19.auto_cancel_non_tradable_orders': True,
        'm19.data_frequency': 'daily',
        'm19.price_type': '后复权',
        'm19.product_type': '股票',
        'm19.plot_charts': True,
        'm19.backtest_only': False,
        'm19.benchmark': '000300.SHA',
    })
    
    # g.run({})
    
    
    def m10_run_bigquant_run(
        bq_graph,
        inputs,
        trading_days_market='CN', # 使用那个市场的交易日历
        train_instruments_mid='m1', # 训练数据 证券代码列表 模块id
        test_instruments_mid='m9', # 测试数据 证券代码列表 模块id
        predict_mid='m24', # 预测 模块id
        trade_mid='m19', # 回测 模块id
        start_date='2011-01-01', # 数据开始日期
        end_date=T.live_run_param('trading_date', '2020-08-01'), # 数据结束日期
        train_update_days=120, # 更新周期,按交易日计算,每多少天更新一次
        train_update_days_for_live=None,#模拟实盘模式下的更新周期,按交易日计算,每多少天更新一次。如果需要在模拟实盘阶段使用不同的模型更新周期,可以设置这个参数
        train_data_min_days=750, # 最小数据天数,按交易日计算,所以第一个滚动的结束日期是 从开始日期到开始日期+最小数据天数
        train_data_max_days=750, # 最大数据天数,按交易日计算,0,表示没有限制,否则每一个滚动的开始日期=max(此滚动的结束日期-最大数据天数, 开始日期
        rolling_count_for_live=1, #实盘模式下滚动次数,模拟实盘模式下,取最后多少次滚动。一般在模拟实盘模式下,只用到最后一次滚动训练的模型,这里可以设置为1;如果你的滚动训练数据时间段很短,以至于期间可能没有训练数据,这里可以设置大一点。0表示没有限制
    ):
        def merge_datasources(input_1):
            df_list = [ds.read_df() for ds in input_1]
            df = pd.concat(df_list)
            instrument_data = {
                'start_date': df['date'].min().strftime('%Y-%m-%d'),
                'end_date': df['date'].max().strftime('%Y-%m-%d'),
                'instruments': list(set(df['instrument'])),
            }
            return Outputs(data=DataSource.write_df(df), instrument_data=DataSource.write_pickle(instrument_data))
    
        def gen_rolling_dates(trading_days_market, start_date, end_date, train_update_days, train_update_days_for_live, train_data_min_days, train_data_max_days, rolling_count_for_live):
            # 是否实盘模式
            tdays = list(D.trading_days(market=trading_days_market, start_date=start_date, end_date=end_date)['date'])
            is_live_run = T.live_run_param('trading_date', None) is not None
    
            if is_live_run and train_update_days_for_live:
                train_update_days = train_update_days_for_live
    
            rollings = []
            train_end_date = train_data_min_days
            while train_end_date < len(tdays):
                if train_data_max_days is not None:
                    train_start_date = max(train_end_date - train_data_max_days, 0)
                else:
                    train_start_date = start_date
                rollings.append({
                    'train_start_date': tdays[train_start_date].strftime('%Y-%m-%d'),
                    'train_end_date': tdays[train_end_date - 1].strftime('%Y-%m-%d'),
                    'test_start_date': tdays[train_end_date].strftime('%Y-%m-%d'),
                    'test_end_date': tdays[min(train_end_date + train_update_days, len(tdays)) - 1].strftime('%Y-%m-%d'),
                })
                train_end_date += train_update_days
    
            if not rollings:
                raise Exception('没有滚动需要执行,请检查配置')
    
            if is_live_run and rolling_count_for_live:
                rollings = rollings[-rolling_count_for_live:]
    
            return rollings
    
        g = bq_graph
    
        rolling_dates = gen_rolling_dates(
            trading_days_market, start_date, end_date, train_update_days, train_update_days_for_live, train_data_min_days, train_data_max_days, rolling_count_for_live)
    
        # 训练和预测
        results = []
        for rolling in rolling_dates:
            parameters = {}
            # 先禁用回测
            parameters[trade_mid + '.__enabled__'] = False
            parameters[train_instruments_mid + '.start_date'] = rolling['train_start_date']
            parameters[train_instruments_mid + '.end_date'] = rolling['train_end_date']
            parameters[test_instruments_mid + '.start_date'] = rolling['test_start_date']
            parameters[test_instruments_mid + '.end_date'] = rolling['test_end_date']
            # print('------ rolling_train:', parameters)
            results.append(g.run(parameters))
    
        # 合并预测结果并回测
        mx = M.cached.v3(run=merge_datasources, input_1=[result[predict_mid].data_1 for result in results])
        parameters = {}
        parameters['*.__enabled__'] = False
        parameters[trade_mid + '.__enabled__'] = True
        parameters[trade_mid + '.instruments'] = mx.instrument_data
        parameters[trade_mid + '.options_data'] = mx.data
    
        trade = g.run(parameters)
    
        return {'rollings': results, 'trade': trade}
    
    
    m10 = M.hyper_rolling_train.v1(
        run=m10_run_bigquant_run,
        run_now=True,
        bq_graph=g
    )
    
    Train on 1632631 samples
    Epoch 1/5
    1632631/1632631 - 27s - loss: 0.9859 - mse: 0.9859
    Epoch 2/5
    1632631/1632631 - 25s - loss: 0.9823 - mse: 0.9823
    Epoch 3/5
    1632631/1632631 - 25s - loss: 0.9815 - mse: 0.9815
    Epoch 4/5
    1632631/1632631 - 25s - loss: 0.9807 - mse: 0.9807
    Epoch 5/5
    1632631/1632631 - 25s - loss: 0.9800 - mse: 0.9800
    
    231143/231143 - 1s
    DataSource(6b3823439852459eb8bbb3a6b16c922dT, v3)
    
    Train on 1669590 samples
    Epoch 1/5
    1669590/1669590 - 28s - loss: 0.9852 - mse: 0.9852
    Epoch 2/5
    1669590/1669590 - 25s - loss: 0.9819 - mse: 0.9819
    Epoch 3/5
    1669590/1669590 - 26s - loss: 0.9810 - mse: 0.9810
    Epoch 4/5
    1669590/1669590 - 25s - loss: 0.9803 - mse: 0.9803
    Epoch 5/5
    1669590/1669590 - 26s - loss: 0.9796 - mse: 0.9796
    
    228671/228671 - 1s
    DataSource(2c3b6be31efe4151a67d4f3659a353e4T, v3)
    
    Train on 1686118 samples
    Epoch 1/5
    1686118/1686118 - 32s - loss: 0.9865 - mse: 0.9865
    Epoch 2/5
    1686118/1686118 - 31s - loss: 0.9832 - mse: 0.9832
    Epoch 3/5
    1686118/1686118 - 30s - loss: 0.9823 - mse: 0.9823
    Epoch 4/5
    1686118/1686118 - 30s - loss: 0.9816 - mse: 0.9816
    Epoch 5/5
    1686118/1686118 - 28s - loss: 0.9809 - mse: 0.9809
    
    221127/221127 - 1s
    DataSource(8661454041da479ab2201a2f990569c7T, v3)
    
    Train on 1681880 samples
    Epoch 1/5
    1681880/1681880 - 19s - loss: 0.9849 - mse: 0.9849
    Epoch 2/5
    1681880/1681880 - 19s - loss: 0.9815 - mse: 0.9815
    Epoch 3/5
    1681880/1681880 - 18s - loss: 0.9805 - mse: 0.9805
    Epoch 4/5
    1681880/1681880 - 17s - loss: 0.9795 - mse: 0.9795
    Epoch 5/5
    1681880/1681880 - 16s - loss: 0.9789 - mse: 0.9789
    
    235540/235540 - 0s
    DataSource(55c5de96db294543b0002125f9675d90T, v3)
    
    Train on 1680704 samples
    Epoch 1/5
    1680704/1680704 - 17s - loss: 0.9823 - mse: 0.9823
    Epoch 2/5
    1680704/1680704 - 16s - loss: 0.9789 - mse: 0.9789
    Epoch 3/5
    1680704/1680704 - 16s - loss: 0.9778 - mse: 0.9778
    Epoch 4/5
    1680704/1680704 - 16s - loss: 0.9767 - mse: 0.9767
    Epoch 5/5
    1680704/1680704 - 16s - loss: 0.9760 - mse: 0.9760
    
    251779/251779 - 0s
    DataSource(52d964badc124fe5b3672d553134c542T, v3)
    
    Train on 1697673 samples
    Epoch 1/5
    1697673/1697673 - 17s - loss: 0.9789 - mse: 0.9789
    Epoch 2/5
    1697673/1697673 - 17s - loss: 0.9757 - mse: 0.9757
    Epoch 3/5
    1697673/1697673 - 17s - loss: 0.9745 - mse: 0.9745
    Epoch 4/5
    1697673/1697673 - 17s - loss: 0.9736 - mse: 0.9736
    Epoch 5/5
    1697673/1697673 - 16s - loss: 0.9728 - mse: 0.9728
    
    269568/269568 - 0s
    DataSource(0f2c7f095ab84f5c837f76af91d2cb60T, v3)
    
    Train on 1738241 samples
    Epoch 1/5
    1738241/1738241 - 17s - loss: 0.9744 - mse: 0.9744
    Epoch 2/5
    1738241/1738241 - 17s - loss: 0.9713 - mse: 0.9713
    Epoch 3/5
    1738241/1738241 - 17s - loss: 0.9703 - mse: 0.9703
    Epoch 4/5
    1738241/1738241 - 17s - loss: 0.9695 - mse: 0.9695
    Epoch 5/5
    1738241/1738241 - 17s - loss: 0.9688 - mse: 0.9687
    
    290116/290116 - 1s
    DataSource(eebf1c0fd4944f288cf121acb3816482T, v3)
    
    Train on 1803442 samples
    Epoch 1/5
    1803442/1803442 - 18s - loss: 0.9694 - mse: 0.9694
    Epoch 2/5
    1803442/1803442 - 18s - loss: 0.9665 - mse: 0.9665
    Epoch 3/5
    1803442/1803442 - 21s - loss: 0.9653 - mse: 0.9653
    Epoch 4/5
    1803442/1803442 - 20s - loss: 0.9644 - mse: 0.9644
    Epoch 5/5
    1803442/1803442 - 20s - loss: 0.9637 - mse: 0.9637
    
    313815/313815 - 1s
    DataSource(170e1b944aa44971bb965da19dfb865dT, v3)
    
    Train on 1901533 samples
    Epoch 1/5
    1901533/1901533 - 22s - loss: 0.9655 - mse: 0.9655
    Epoch 2/5
    1901533/1901533 - 21s - loss: 0.9627 - mse: 0.9627
    Epoch 3/5
    1901533/1901533 - 19s - loss: 0.9615 - mse: 0.9615
    Epoch 4/5
    1901533/1901533 - 19s - loss: 0.9606 - mse: 0.9606
    Epoch 5/5
    1901533/1901533 - 18s - loss: 0.9599 - mse: 0.9599
    
    324882/324882 - 1s
    DataSource(8f7efb9bc07c444ca363b248ef8c050bT, v3)
    
    Train on 2015408 samples
    Epoch 1/5
    2015408/2015408 - 20s - loss: 0.9670 - mse: 0.9670
    Epoch 2/5
    2015408/2015408 - 19s - loss: 0.9638 - mse: 0.9638
    Epoch 3/5
    2015408/2015408 - 19s - loss: 0.9628 - mse: 0.9628
    Epoch 4/5
    2015408/2015408 - 19s - loss: 0.9620 - mse: 0.9620
    Epoch 5/5
    2015408/2015408 - 19s - loss: 0.9613 - mse: 0.9613
    
    348525/348525 - 1s
    DataSource(216d2bdd59f64b6e8f954d468d98054bT, v3)
    
    Train on 2154523 samples
    Epoch 1/5
    2154523/2154523 - 21s - loss: 0.9705 - mse: 0.9705
    Epoch 2/5
    2154523/2154523 - 21s - loss: 0.9674 - mse: 0.9674
    Epoch 3/5
    2154523/2154523 - 21s - loss: 0.9661 - mse: 0.9661
    Epoch 4/5
    2154523/2154523 - 21s - loss: 0.9653 - mse: 0.9653
    Epoch 5/5
    2154523/2154523 - 21s - loss: 0.9647 - mse: 0.9647
    
    360021/360021 - 1s
    DataSource(c879615953cd4ef1a8040f9e8f4f6dfeT, v3)
    
    Train on 2276806 samples
    Epoch 1/5
    2276806/2276806 - 23s - loss: 0.9706 - mse: 0.9706
    Epoch 2/5
    2276806/2276806 - 22s - loss: 0.9676 - mse: 0.9676
    Epoch 3/5
    2276806/2276806 - 22s - loss: 0.9666 - mse: 0.9666
    Epoch 4/5
    2276806/2276806 - 24s - loss: 0.9655 - mse: 0.9655
    Epoch 5/5
    2276806/2276806 - 25s - loss: 0.9649 - mse: 0.9649
    
    368781/368781 - 1s
    DataSource(17195c2606a14a62959afb001e9affc5T, v3)
    
    Train on 2393096 samples
    Epoch 1/5
    2393096/2393096 - 27s - loss: 0.9724 - mse: 0.9724
    Epoch 2/5
    2393096/2393096 - 27s - loss: 0.9697 - mse: 0.9697
    Epoch 3/5
    2393096/2393096 - 26s - loss: 0.9688 - mse: 0.9688
    Epoch 4/5
    2393096/2393096 - 24s - loss: 0.9679 - mse: 0.9679
    Epoch 5/5
    2393096/2393096 - 23s - loss: 0.9675 - mse: 0.9675
    
    380427/380427 - 1s
    DataSource(040caa34647b4b4581adb8b785a04136T, v3)
    
    Train on 2495404 samples
    Epoch 1/5
    2495404/2495404 - 25s - loss: 0.9765 - mse: 0.9765
    Epoch 2/5
    2495404/2495404 - 24s - loss: 0.9737 - mse: 0.9737
    Epoch 3/5
    2495404/2495404 - 24s - loss: 0.9729 - mse: 0.9729
    Epoch 4/5
    2495404/2495404 - 24s - loss: 0.9719 - mse: 0.9719
    Epoch 5/5
    2495404/2495404 - 25s - loss: 0.9713 - mse: 0.9713
    
    ---------------------------------------------------------------------------
    ValueError                                Traceback (most recent call last)
    <ipython-input-2-a075eb9e92ec> in <module>()
        430     run=m10_run_bigquant_run,
        431     run_now=True,
    --> 432     bq_graph=g
        433 )
    
    <ipython-input-2-a075eb9e92ec> in m10_run_bigquant_run(bq_graph, inputs, trading_days_market, train_instruments_mid, test_instruments_mid, predict_mid, trade_mid, start_date, end_date, train_update_days, train_update_days_for_live, train_data_min_days, train_data_max_days, rolling_count_for_live)
        412         parameters[test_instruments_mid + '.end_date'] = rolling['test_end_date']
        413         # print('------ rolling_train:', parameters)
    --> 414         results.append(g.run(parameters))
        415 
        416     # 合并预测结果并回测
    
    ValueError: Structure is a scalar but len(flat_sequence) == 0 > 1
    In [ ]:
    m5 = M.stock_ranker_train_rolling.v2(
        data=m4.data,
        evaluation_start_date=conf.split_date,
        features=conf.features,
        model_update_days=180,)
    
    In [ ]:
    # 第一次训练的训练集
    print(m10.result['rollings'][0]['m1'].data.read_pickle()['start_date'])
    print(m10.result['rollings'][0]['m1'].data.read_pickle()['end_date'])
    # 第一次训练的预测集
    print(m10.result['rollings'][0]['m9'].data.read_pickle()['start_date'])
    print(m10.result['rollings'][0]['m9'].data.read_pickle()['end_date'])
    # 训练的第一个模型的预测结果
    m10.result['rollings'][0]['m11'].data.read_pickle()