神经网络交易个股运行错误

策略分享
标签: #<Tag:0x00007f6099197d90>

(a1750271086) #1

老师您好,请问这个错误应该怎么处理?同时,为什么社区CNN神经网络预测个股的模板用的都是Covn2D但是CNN选股用的是Covn1D?

克隆策略

使用深度学习技术预测股票价格

版本 v1.0

目录

  • ### 深度学习策略的交易规则

  • ### 策略构建步骤

  • ### 策略的实现

正文

一、深度学习策略的交易规则

  • 买入条件:预测的上涨概率>0.5,则买入或保持已有持仓。
  • 卖出条件 :预测的上涨概率<0.5,则卖出已有股票。

二、策略构建步骤

1、确定股票池和数据起止时间

  • 在证券代码列表m24和m28模块中输入要回测的单只股票,以及数据的起止日期(分别为训练集和验证集)。

2、确定因子

  • 在输入特征列表m8模块中输入用于预测的N个因子表达式。

3、获取基础数据

  • 通过基础特征数据抽取模块m22和m16获取指定股票池的基础数据,如收盘价等字段。

4、确定并计算模型标注

  • 通过自动标注股票模块m21计算需要的标注指标,本例中首先计算未来10天收益,然后根据其正负来给每日数据标注1或0,来标识涨跌。

5、抽取因子数据

  • 通过衍生数据抽取模块m23和m26计算因子数据。

6、合并标注与因子数据

  • 通过连接数据m17模块合并因子数据和标注数据。

7、生成序列窗口滚动数据集

  • 通过序列窗口滚动(深度学习)模块m25和m27将训练集和预测集的数据生成固定窗口长度的数据序列,为后续模型训练和预测做准备。

8、构建LSTM + CNN模型构架

  • 在画布左侧模块列表中依次拖入输入层模块、Reshape层模块、Conv2D层模块、Reshape层模块、LSTM层模块、Dropout层模块和全连接层模块(两组),构成深度学习网络构架,

    最后通过“构建(深度学习)”模块组装各层。这里需要注意:

    输入层的shape参数是 窗口滚动数据集的大小 X 因子数量 , 本例为 50 行 X 5个因子

    ReShape层的参数是 窗口滚动数据集的大小 X 因子数量 X 1 ,本例为 50 行 X 5个因子 X1

    Conv2D层中的 kernel_size参数是滑动窗口的尺寸,本例中使用 3行 X 5列 的窗口, 每次滑动的步长为 1行 X 1列 , 卷积核数目为32,这里的窗口设置决定了后面ReShape层的参数

    ReShape层中的target_shape 参数,这是由 窗口滚动数据集 X 因子数量 和 Conv2D层中设置的窗口尺寸以及步长决定的。本例中 50行 X 5因子 的输入数据,使用 3行 X5列 的窗口滑动取数据,

    每次移动1行,共计可以得到48次数据(即可以通过滑动3行 X 5列的窗口48次来获取完整的数据),因此target_shape= 48 X 卷积核数32

    LSTM层的输出空间维度设置为卷积核数32,并设置激活函数

    Dropout层是防止过度拟合采用的主动裁剪数据技术,这里设置rate 为0.8

    全连接层共两层,第一层的输出空间维度与LSTM的输出维度保持一致为32,第二层将第一层的32维数据转变为1维数据输出,即获取预测的label值,此例为0到1之间的连续值,可以认为是上涨的概率。

9、训练深度学习模型

  • 在画布左侧模块列表中拖入“训练(深度学习)”模块m6,设置属性中的优化器、目标函数、评估指标、每次训练的数据量batch_size、迭代次数epochs和GPU的数量以及日志输出频率。

10、使用深度学习模型预测

  • 在画布左侧模块列表中拖入“预测(深度学习)”模块m7,并将“训练(深度学习)”模块m6的模型输出和验证集的序列窗口滚动数据集传给预测模块,通过预测模块即根据股票验证集的数据预测上涨的概率。

11、将预测结果与时间拼接

  • 通过自定义模块m2将预测的每个滚动序列窗口的最后一个值最为当日的预测结果,并与预测集数据的时间列拼接,形成最终的每日预测结果。

12、根据模型预测结果构建策略

  • 如果当日预测的上涨概率大于0.5,则保持持仓或买入

  • 如果当日预测的上涨概率小于0.5,则卖出股票或保持空仓。

13、模拟回测

  • 通过 trade 模块中的初始化函数定义交易手续费和滑点,通过 context.prediction 获取每日的上涨概率预测结果;

  • 通过 trade 模块中的主函数(handle函数)查看每日的买卖交易信号,按照买卖原则执行相应的买入/卖出操作。

三、策略的实现

可视化策略实现如下:

    {"Description":"实验创建于2017/11/15","Summary":"","Graph":{"EdgesInternal":[{"DestinationInputPortId":"-692:input_data","SourceOutputPortId":"-316:data"},{"DestinationInputPortId":"-332:trained_model","SourceOutputPortId":"-320:data"},{"DestinationInputPortId":"-2431:input_1","SourceOutputPortId":"-332:data"},{"DestinationInputPortId":"-692:features","SourceOutputPortId":"-2295:data"},{"DestinationInputPortId":"-333:features","SourceOutputPortId":"-2295:data"},{"DestinationInputPortId":"-341:features","SourceOutputPortId":"-2295:data"},{"DestinationInputPortId":"-300:features","SourceOutputPortId":"-2295:data"},{"DestinationInputPortId":"-316:features","SourceOutputPortId":"-2295:data"},{"DestinationInputPortId":"-307:features","SourceOutputPortId":"-2295:data"},{"DestinationInputPortId":"-300:instruments","SourceOutputPortId":"-620:data"},{"DestinationInputPortId":"-2140:input_data","SourceOutputPortId":"-692:data"},{"DestinationInputPortId":"-320:training_data","SourceOutputPortId":"-333:data"},{"DestinationInputPortId":"-332:input_data","SourceOutputPortId":"-341:data"},{"DestinationInputPortId":"-307:input_data","SourceOutputPortId":"-300:data"},{"DestinationInputPortId":"-2136:input_data","SourceOutputPortId":"-307:data"},{"DestinationInputPortId":"-316:instruments","SourceOutputPortId":"-322:data"},{"DestinationInputPortId":"-281:instruments","SourceOutputPortId":"-322:data"},{"DestinationInputPortId":"-333:input_data","SourceOutputPortId":"-2136:data"},{"DestinationInputPortId":"-341:input_data","SourceOutputPortId":"-2140:data"},{"DestinationInputPortId":"-2431:input_2","SourceOutputPortId":"-2140:data"},{"DestinationInputPortId":"-2680:inputs","SourceOutputPortId":"-160:data"},{"DestinationInputPortId":"-3880:inputs","SourceOutputPortId":"-160:data"},{"DestinationInputPortId":"-2712:inputs","SourceOutputPortId":"-2680:data"},{"DestinationInputPortId":"-3840:inputs","SourceOutputPortId":"-2712:data"},{"DestinationInputPortId":"-3872:inputs","SourceOutputPortId":"-3840:data"},{"DestinationInputPortId":"-356:inputs","SourceOutputPortId":"-3872:data"},{"DestinationInputPortId":"-320:input_model","SourceOutputPortId":"-3880:data"},{"DestinationInputPortId":"-567:inputs","SourceOutputPortId":"-356:data"},{"DestinationInputPortId":"-412:inputs","SourceOutputPortId":"-567:data"},{"DestinationInputPortId":"-574:inputs","SourceOutputPortId":"-412:data"},{"DestinationInputPortId":"-440:inputs","SourceOutputPortId":"-574:data"},{"DestinationInputPortId":"-3880:outputs","SourceOutputPortId":"-440:data"},{"DestinationInputPortId":"-281:options_data","SourceOutputPortId":"-2431:data_1"}],"ModuleNodes":[{"Id":"-316","ModuleId":"BigQuantSpace.general_feature_extractor.general_feature_extractor-v7","ModuleParameters":[{"Name":"start_date","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"end_date","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"before_start_days","Value":90,"ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"instruments","NodeId":"-316"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"features","NodeId":"-316"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-316","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":16,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-320","ModuleId":"BigQuantSpace.dl_model_train.dl_model_train-v1","ModuleParameters":[{"Name":"optimizer","Value":"Adam","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_optimizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"loss","Value":"binary_crossentropy","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_loss","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"metrics","Value":"accuracy","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"batch_size","Value":"2048","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"epochs","Value":"100","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"n_gpus","Value":"1","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"verbose","Value":"1:输出进度条记录","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_model","NodeId":"-320"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"training_data","NodeId":"-320"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"validation_data","NodeId":"-320"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-320","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":6,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-332","ModuleId":"BigQuantSpace.dl_model_predict.dl_model_predict-v1","ModuleParameters":[{"Name":"batch_size","Value":"10240","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"n_gpus","Value":"0","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"verbose","Value":"2:每个epoch输出一行记录","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"trained_model","NodeId":"-332"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_data","NodeId":"-332"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-332","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":7,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-2295","ModuleId":"BigQuantSpace.input_features.input_features-v1","ModuleParameters":[{"Name":"features","Value":"close_0\nvolume_0\nhigh_0\nlow_0\namount_0\nopen_0\nta_ema(close_0, timeperiod=5)\nta_ema(close_0, timeperiod=13)\nta_ema(close_0, timeperiod=34)\nta_ema(close_0, timeperiod=55)\nta_cci_14_0\n\n\n","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"features_ds","NodeId":"-2295"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-2295","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":8,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-620","ModuleId":"BigQuantSpace.instruments.instruments-v2","ModuleParameters":[{"Name":"start_date","Value":"2014-01-01","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"end_date","Value":"2018-01-01","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"market","Value":"CN_STOCK_A","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"instrument_list","Value":"603336.SHA","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"max_count","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"rolling_conf","NodeId":"-620"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-620","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":24,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-692","ModuleId":"BigQuantSpace.derived_feature_extractor.derived_feature_extractor-v3","ModuleParameters":[{"Name":"date_col","Value":"date","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"instrument_col","Value":"instrument","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"drop_na","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"remove_extra_columns","Value":"False","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_functions","Value":"{}","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_data","NodeId":"-692"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"features","NodeId":"-692"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-692","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":26,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-281","ModuleId":"BigQuantSpace.trade.trade-v4","ModuleParameters":[{"Name":"start_date","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"end_date","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"initialize","Value":"# 回测引擎:初始化函数,只执行一次\ndef bigquant_run(context):\n # 加载预测数据\n context.prediction = context.options['data'].read_df()\n\n # 系统已经设置了默认的交易手续费和滑点,要修改手续费可使用如下函数\n context.set_commission(PerOrder(buy_cost=0.0003, sell_cost=0.0013, min_cost=5))","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"handle_data","Value":"# 回测引擎:每日数据处理函数,每天执行一次\ndef bigquant_run(context, data):\n # 按日期过滤得到今日的预测数据\n try:\n prediction = context.prediction[data.current_dt.strftime('%Y-%m-%d')]\n except KeyError as e:\n return\n \n instrument = context.instruments[0]\n sid = context.symbol(instrument)\n cur_position = context.portfolio.positions[sid].amount\n \n # 交易逻辑\n if prediction > 0.5 and cur_position == 0:\n context.order_target_percent(context.symbol(instrument), 1)\n print(data.current_dt, '买入!')\n \n elif prediction < 0.5 and cur_position > 0:\n context.order_target_percent(context.symbol(instrument), 0)\n print(data.current_dt, '卖出!')\n ","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"prepare","Value":"# 回测引擎:准备数据,只执行一次\ndef bigquant_run(context):\n pass\n","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"before_trading_start","Value":"# 回测引擎:每个单位时间开始前调用一次,即每日开盘前调用一次。\ndef bigquant_run(context, data):\n pass\n","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"volume_limit","Value":"0","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"order_price_field_buy","Value":"open","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"order_price_field_sell","Value":"close","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"capital_base","Value":1000000,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"auto_cancel_non_tradable_orders","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"data_frequency","Value":"daily","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"price_type","Value":"真实价格","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"product_type","Value":"股票","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"plot_charts","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"backtest_only","Value":"False","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"benchmark","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"instruments","NodeId":"-281"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"options_data","NodeId":"-281"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"history_ds","NodeId":"-281"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"benchmark_ds","NodeId":"-281"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"trading_calendar","NodeId":"-281"}],"OutputPortsInternal":[{"Name":"raw_perf","NodeId":"-281","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":1,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-333","ModuleId":"BigQuantSpace.dl_convert_to_bin.dl_convert_to_bin-v2","ModuleParameters":[{"Name":"window_size","Value":"1","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"feature_clip","Value":"11","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"flatten","Value":"False","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"window_along_col","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_data","NodeId":"-333"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"features","NodeId":"-333"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-333","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":25,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-341","ModuleId":"BigQuantSpace.dl_convert_to_bin.dl_convert_to_bin-v2","ModuleParameters":[{"Name":"window_size","Value":"1","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"feature_clip","Value":"11","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"flatten","Value":"False","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"window_along_col","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_data","NodeId":"-341"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"features","NodeId":"-341"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-341","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":27,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-300","ModuleId":"BigQuantSpace.general_feature_extractor.general_feature_extractor-v7","ModuleParameters":[{"Name":"start_date","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"end_date","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"before_start_days","Value":90,"ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"instruments","NodeId":"-300"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"features","NodeId":"-300"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-300","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":22,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-307","ModuleId":"BigQuantSpace.derived_feature_extractor.derived_feature_extractor-v3","ModuleParameters":[{"Name":"date_col","Value":"date","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"instrument_col","Value":"instrument","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"drop_na","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"remove_extra_columns","Value":"False","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_functions","Value":"{}","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_data","NodeId":"-307"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"features","NodeId":"-307"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-307","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":23,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-322","ModuleId":"BigQuantSpace.instruments.instruments-v2","ModuleParameters":[{"Name":"start_date","Value":"2018-01-01","ValueType":"Literal","LinkedGlobalParameter":"交易日期"},{"Name":"end_date","Value":"2020-09-18","ValueType":"Literal","LinkedGlobalParameter":"交易日期"},{"Name":"market","Value":"CN_STOCK_A","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"instrument_list","Value":"603336.SHA","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"max_count","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"rolling_conf","NodeId":"-322"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-322","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":28,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-2136","ModuleId":"BigQuantSpace.dropnan.dropnan-v2","ModuleParameters":[],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_data","NodeId":"-2136"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"features","NodeId":"-2136"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-2136","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":19,"IsPartOfPartialRun":null,"Comment":"去掉为nan的数据","CommentCollapsed":true},{"Id":"-2140","ModuleId":"BigQuantSpace.dropnan.dropnan-v2","ModuleParameters":[],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_data","NodeId":"-2140"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"features","NodeId":"-2140"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-2140","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":29,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-6013","ModuleId":"BigQuantSpace.hyper_rolling_train.hyper_rolling_train-v1","ModuleParameters":[{"Name":"run","Value":"def bigquant_run(\n bq_graph,\n inputs,\n trading_days_market='CN', # 使用那个市场的交易日历, TODO\n train_instruments_mid='m24', # 训练数据 证券代码列表 模块id\n test_instruments_mid='m28', # 测试数据 证券代码列表 模块id\n predict_mid='m2', # 预测 模块id\n trade_mid='m1', # 回测 模块id\n start_date='2018-01-01', # 数据开始日期\n end_date=T.live_run_param('trading_date', '2020-09-18'), # 数据结束日期\n train_update_days=5, # 更新周期,按交易日计算,每多少天更新一次\n train_update_days_for_live=None, #模拟实盘模式下的更新周期,按交易日计算,每多少天更新一次。如果需要在模拟实盘阶段使用不同的模型更新周期,可以设置这个参数\n train_data_min_days=125, # 最小数据天数,按交易日计算,所以第一个滚动的结束日期是 从开始日期到开始日期+最小数据天数\n train_data_max_days=250, # 最大数据天数,按交易日计算,0,表示没有限制,否则每一个滚动的开始日期=max(此滚动的结束日期-最大数据天数, 开始日期\n rolling_count_for_live=1, #实盘模式下滚动次数,模拟实盘模式下,取最后多少次滚动。一般在模拟实盘模式下,只用到最后一次滚动训练的模型,这里可以设置为1;如果你的滚动训练数据时间段很短,以至于期间可能没有训练数据,这里可以设置大一点。0表示没有限制\n):\n def merge_datasources(input_1):\n df_list = [ds[0].read_df().set_index('date').ix[ds[1]:].reset_index() for ds in input_1]\n df = pd.concat(df_list)\n instrument_data = {\n 'start_date': df['date'].min().strftime('%Y-%m-%d'),\n 'end_date': df['date'].max().strftime('%Y-%m-%d'),\n 'instruments': list(set(df['instrument'])),\n }\n return Outputs(data=DataSource.write_df(df), instrument_data=DataSource.write_pickle(instrument_data))\n\n def gen_rolling_dates(trading_days_market, start_date, end_date, train_update_days, train_update_days_for_live, train_data_min_days, train_data_max_days, rolling_count_for_live):\n # 是否实盘模式\n tdays = list(D.trading_days(market=trading_days_market, start_date=start_date, end_date=end_date)['date'])\n is_live_run = T.live_run_param('trading_date', None) is not None\n\n if is_live_run and train_update_days_for_live:\n train_update_days = train_update_days_for_live\n\n rollings = []\n train_end_date = train_data_min_days\n while train_end_date < len(tdays):\n if train_data_max_days is not None and train_data_max_days > 0:\n train_start_date = max(train_end_date - train_data_max_days, 0)\n else:\n train_start_date = 0\n rollings.append({\n 'train_start_date': tdays[train_start_date].strftime('%Y-%m-%d'),\n 'train_end_date': tdays[train_end_date - 1].strftime('%Y-%m-%d'),\n 'test_start_date': tdays[train_end_date].strftime('%Y-%m-%d'),\n 'test_end_date': tdays[min(train_end_date + train_update_days, len(tdays)) - 1].strftime('%Y-%m-%d'),\n })\n train_end_date += train_update_days\n\n if not rollings:\n raise Exception('没有滚动需要执行,请检查配置')\n\n if is_live_run and rolling_count_for_live:\n rollings = rollings[-rolling_count_for_live:]\n\n return rollings\n\n g = bq_graph\n\n rolling_dates = gen_rolling_dates(\n trading_days_market, start_date, end_date, train_update_days, train_update_days_for_live, train_data_min_days, train_data_max_days, rolling_count_for_live)\n\n # 训练和预测\n results = []\n for rolling in rolling_dates:\n parameters = {}\n # 先禁用回测\n parameters[trade_mid + '.__enabled__'] = False\n parameters[train_instruments_mid + '.start_date'] = rolling['train_start_date']\n parameters[train_instruments_mid + '.end_date'] = rolling['train_end_date']\n parameters[test_instruments_mid + '.start_date'] = rolling['test_start_date']\n parameters[test_instruments_mid + '.end_date'] = rolling['test_end_date']\n # print('------ rolling_train:', parameters)\n results.append(g.run(parameters))\n\n # 合并预测结果并回测\n mx = M.cached.v3(run=merge_datasources, input_1=[[result[predict_mid].data_1, result[test_instruments_mid].data.read_pickle()['start_date']] for result in results])\n parameters = {}\n parameters['*.__enabled__'] = False\n parameters[trade_mid + '.__enabled__'] = True\n parameters[trade_mid + '.instruments'] = mx.instrument_data\n parameters[trade_mid + '.options_data'] = mx.data\n\n trade = g.run(parameters)\n\n return {'rollings': results, 'trade': trade}\n","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"run_now","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bq_graph","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"bq_graph_port","NodeId":"-6013"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_1","NodeId":"-6013"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_2","NodeId":"-6013"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_3","NodeId":"-6013"}],"OutputPortsInternal":[{"Name":"result","NodeId":"-6013","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":18,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-160","ModuleId":"BigQuantSpace.dl_layer_input.dl_layer_input-v1","ModuleParameters":[{"Name":"shape","Value":"1,11","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"batch_shape","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"dtype","Value":"float32","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"sparse","Value":"False","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"name","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"inputs","NodeId":"-160"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-160","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":20,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-2680","ModuleId":"BigQuantSpace.dl_layer_conv1d.dl_layer_conv1d-v1","ModuleParameters":[{"Name":"filters","Value":"20","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_size","Value":"1","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"strides","Value":"1","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"padding","Value":"valid","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"dilation_rate","Value":1,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activation","Value":"relu","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_activation","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"use_bias","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_initializer","Value":"glorot_uniform","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_initializer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_initializer","Value":"Zeros","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_initializer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_activity_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_constraint","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_constraint","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_constraint","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_constraint","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"name","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"inputs","NodeId":"-2680"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-2680","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":30,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-2712","ModuleId":"BigQuantSpace.dl_layer_maxpooling1d.dl_layer_maxpooling1d-v1","ModuleParameters":[{"Name":"pool_size","Value":"1","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"strides","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"padding","Value":"valid","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"name","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"inputs","NodeId":"-2712"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-2712","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":31,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-3840","ModuleId":"BigQuantSpace.dl_layer_conv1d.dl_layer_conv1d-v1","ModuleParameters":[{"Name":"filters","Value":"20","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_size","Value":"1","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"strides","Value":"1","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"padding","Value":"valid","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"dilation_rate","Value":1,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activation","Value":"relu","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_activation","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"use_bias","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_initializer","Value":"glorot_uniform","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_initializer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_initializer","Value":"Zeros","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_initializer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_activity_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_constraint","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_constraint","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_constraint","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_constraint","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"name","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"inputs","NodeId":"-3840"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-3840","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":32,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-3872","ModuleId":"BigQuantSpace.dl_layer_maxpooling1d.dl_layer_maxpooling1d-v1","ModuleParameters":[{"Name":"pool_size","Value":"1","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"strides","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"padding","Value":"valid","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"name","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"inputs","NodeId":"-3872"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-3872","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":33,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-3880","ModuleId":"BigQuantSpace.dl_model_init.dl_model_init-v1","ModuleParameters":[],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"inputs","NodeId":"-3880"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"outputs","NodeId":"-3880"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-3880","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":34,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-356","ModuleId":"BigQuantSpace.dl_layer_lstm.dl_layer_lstm-v1","ModuleParameters":[{"Name":"units","Value":"7","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activation","Value":"relu","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_activation","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"recurrent_activation","Value":"hard_sigmoid","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_recurrent_activation","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"use_bias","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_initializer","Value":"glorot_uniform","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_initializer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"recurrent_initializer","Value":"Orthogonal","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_recurrent_initializer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_initializer","Value":"Zeros","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_initializer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"unit_forget_bias","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"recurrent_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"recurrent_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"recurrent_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_recurrent_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_activity_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_constraint","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_constraint","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"recurrent_constraint","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_recurrent_constraint","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_constraint","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_constraint","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"dropout","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"recurrent_dropout","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"return_sequences","Value":"False","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"implementation","Value":"2","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"name","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"inputs","NodeId":"-356"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-356","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":35,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-567","ModuleId":"BigQuantSpace.dl_layer_dropout.dl_layer_dropout-v1","ModuleParameters":[{"Name":"rate","Value":"0.2","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"noise_shape","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"seed","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"name","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"inputs","NodeId":"-567"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-567","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":36,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-412","ModuleId":"BigQuantSpace.dl_layer_dense.dl_layer_dense-v1","ModuleParameters":[{"Name":"units","Value":"4","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activation","Value":"relu","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_activation","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"use_bias","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_initializer","Value":"glorot_uniform","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_initializer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_initializer","Value":"Zeros","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_initializer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_activity_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_constraint","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_constraint","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_constraint","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_constraint","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"name","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"inputs","NodeId":"-412"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-412","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":37,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-574","ModuleId":"BigQuantSpace.dl_layer_dropout.dl_layer_dropout-v1","ModuleParameters":[{"Name":"rate","Value":"0.2","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"noise_shape","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"seed","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"name","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"inputs","NodeId":"-574"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-574","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":38,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-440","ModuleId":"BigQuantSpace.dl_layer_dense.dl_layer_dense-v1","ModuleParameters":[{"Name":"units","Value":"1","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activation","Value":"relu","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_activation","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"use_bias","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_initializer","Value":"glorot_uniform","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_initializer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_initializer","Value":"Zeros","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_initializer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_activity_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_constraint","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_constraint","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_constraint","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_constraint","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"name","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"inputs","NodeId":"-440"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-440","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":39,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-2431","ModuleId":"BigQuantSpace.cached.cached-v3","ModuleParameters":[{"Name":"run","Value":"# Python 代码入口函数,input_1/2/3 对应三个输入端,data_1/2/3 对应三个输出端\ndef bigquant_run(input_1, input_2, input_3):\n # 示例代码如下。在这里编写您的代码\n pred_label = input_1.read_pickle()\n df = input_2.read_df()\n df = pd.DataFrame({'pred_label':pred_label[:,0], 'instrument':df.instrument, 'date':df.date})\n df.sort_values(['date','pred_label'],inplace=True, ascending=[True,False])\n return Outputs(data_1=DataSource.write_df(df), data_2=None, data_3=None)\n","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"post_run","Value":"# 后处理函数,可选。输入是主函数的输出,可以在这里对数据做处理,或者返回更友好的outputs数据格式。此函数输出不会被缓存。\ndef bigquant_run(outputs):\n return outputs\n","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"input_ports","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"params","Value":"{}","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"output_ports","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_1","NodeId":"-2431"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_2","NodeId":"-2431"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_3","NodeId":"-2431"}],"OutputPortsInternal":[{"Name":"data_1","NodeId":"-2431","OutputType":null},{"Name":"data_2","NodeId":"-2431","OutputType":null},{"Name":"data_3","NodeId":"-2431","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":3,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true}],"SerializedClientData":"<?xml version='1.0' encoding='utf-16'?><DataV1 xmlns:xsd='http://www.w3.org/2001/XMLSchema' xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'><Meta /><NodePositions><NodePosition Node='-316' Position='1218,-65,200,200'/><NodePosition Node='-320' Position='817.4876708984375,299.1263122558594,200,200'/><NodePosition Node='-332' Position='864.4876708984375,370.1263122558594,200,200'/><NodePosition Node='-2295' Position='892,-167,200,200'/><NodePosition Node='-620' Position='562,-167,200,200'/><NodePosition Node='-692' Position='1219,12,200,200'/><NodePosition Node='-281' Position='861.5447998046875,520.9497375488281,200,200'/><NodePosition Node='-333' Position='852.4876708984375,151.12631225585937,200,200'/><NodePosition Node='-341' Position='1214.256103515625,168.3087158203125,200,200'/><NodePosition Node='-300' Position='732,-85,200,200'/><NodePosition Node='-307' Position='800,-8,200,200'/><NodePosition Node='-322' Position='1216,-141,200,200'/><NodePosition Node='-2136' Position='847.4876708984375,74.12631225585937,200,200'/><NodePosition Node='-2140' Position='1220.256103515625,87.3087158203125,200,200'/><NodePosition Node='-6013' Position='1185.734130859375,259.61273193359375,200,200'/><NodePosition Node='-160' Position='439,-89,200,200'/><NodePosition Node='-2680' Position='440,-24,200,200'/><NodePosition Node='-2712' Position='439,47,200,200'/><NodePosition Node='-3840' Position='439,118,200,200'/><NodePosition Node='-3872' Position='437,189,200,200'/><NodePosition Node='-3880' Position='747.4876708984375,221.12631225585937,200,200'/><NodePosition Node='-356' Position='437,257,200,200'/><NodePosition Node='-567' Position='438,332,200,200'/><NodePosition Node='-412' Position='438,403,200,200'/><NodePosition Node='-574' Position='438,475,200,200'/><NodePosition Node='-440' Position='436,547,200,200'/><NodePosition Node='-2431' Position='933.4876708984375,440.1263122558594,200,200'/></NodePositions><NodeGroups /></DataV1>"},"IsDraft":true,"ParentExperimentId":null,"WebService":{"IsWebServiceExperiment":false,"Inputs":[],"Outputs":[],"Parameters":[{"Name":"交易日期","Value":"","ParameterDefinition":{"Name":"交易日期","FriendlyName":"交易日期","DefaultValue":"","ParameterType":"String","HasDefaultValue":true,"IsOptional":true,"ParameterRules":[],"HasRules":false,"MarkupType":0,"CredentialDescriptor":null}}],"WebServiceGroupId":null,"SerializedClientData":"<?xml version='1.0' encoding='utf-16'?><DataV1 xmlns:xsd='http://www.w3.org/2001/XMLSchema' xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'><Meta /><NodePositions></NodePositions><NodeGroups /></DataV1>"},"DisableNodesUpdate":false,"Category":"user","Tags":[],"IsPartialRun":true}
    In [8]:
    # 本代码由可视化策略环境自动生成 2020年9月20日 01:02
    # 本代码单元只能在可视化模式下编辑。您也可以拷贝代码,粘贴到新建的代码单元或者策略,然后修改。
    
    
    # Python 代码入口函数,input_1/2/3 对应三个输入端,data_1/2/3 对应三个输出端
    def m3_run_bigquant_run(input_1, input_2, input_3):
        # 示例代码如下。在这里编写您的代码
        pred_label = input_1.read_pickle()
        df = input_2.read_df()
        df = pd.DataFrame({'pred_label':pred_label[:,0], 'instrument':df.instrument, 'date':df.date})
        df.sort_values(['date','pred_label'],inplace=True, ascending=[True,False])
        return Outputs(data_1=DataSource.write_df(df), data_2=None, data_3=None)
    
    # 后处理函数,可选。输入是主函数的输出,可以在这里对数据做处理,或者返回更友好的outputs数据格式。此函数输出不会被缓存。
    def m3_post_run_bigquant_run(outputs):
        return outputs
    
    # 回测引擎:初始化函数,只执行一次
    def m1_initialize_bigquant_run(context):
        # 加载预测数据
        context.prediction = context.options['data'].read_df()
    
        # 系统已经设置了默认的交易手续费和滑点,要修改手续费可使用如下函数
        context.set_commission(PerOrder(buy_cost=0.0003, sell_cost=0.0013, min_cost=5))
    # 回测引擎:每日数据处理函数,每天执行一次
    def m1_handle_data_bigquant_run(context, data):
        # 按日期过滤得到今日的预测数据
        try:
            prediction = context.prediction[data.current_dt.strftime('%Y-%m-%d')]
        except KeyError as e:
            return
        
        instrument = context.instruments[0]
        sid = context.symbol(instrument)
        cur_position = context.portfolio.positions[sid].amount
        
        # 交易逻辑
        if prediction > 0.5 and cur_position == 0:
            context.order_target_percent(context.symbol(instrument), 1)
            print(data.current_dt, '买入!')
            
        elif prediction < 0.5 and cur_position > 0:
            context.order_target_percent(context.symbol(instrument), 0)
            print(data.current_dt, '卖出!')
        
    # 回测引擎:准备数据,只执行一次
    def m1_prepare_bigquant_run(context):
        pass
    
    # 回测引擎:每个单位时间开始前调用一次,即每日开盘前调用一次。
    def m1_before_trading_start_bigquant_run(context, data):
        pass
    
    
    g = T.Graph({
    
        'm8': 'M.input_features.v1',
        'm8.features': """close_0
    volume_0
    high_0
    low_0
    amount_0
    open_0
    ta_ema(close_0, timeperiod=5)
    ta_ema(close_0, timeperiod=13)
    ta_ema(close_0, timeperiod=34)
    ta_ema(close_0, timeperiod=55)
    ta_cci_14_0
    
    
    """,
    
        'm24': 'M.instruments.v2',
        'm24.start_date': '2014-01-01',
        'm24.end_date': '2018-01-01',
        'm24.market': 'CN_STOCK_A',
        'm24.instrument_list': '603336.SHA',
        'm24.max_count': 0,
    
        'm22': 'M.general_feature_extractor.v7',
        'm22.instruments': T.Graph.OutputPort('m24.data'),
        'm22.features': T.Graph.OutputPort('m8.data'),
        'm22.start_date': '',
        'm22.end_date': '',
        'm22.before_start_days': 90,
    
        'm23': 'M.derived_feature_extractor.v3',
        'm23.input_data': T.Graph.OutputPort('m22.data'),
        'm23.features': T.Graph.OutputPort('m8.data'),
        'm23.date_col': 'date',
        'm23.instrument_col': 'instrument',
        'm23.drop_na': True,
        'm23.remove_extra_columns': False,
        'm23.user_functions': {},
        'm23.m_cached': False,
    
        'm19': 'M.dropnan.v2',
        'm19.input_data': T.Graph.OutputPort('m23.data'),
    
        'm25': 'M.dl_convert_to_bin.v2',
        'm25.input_data': T.Graph.OutputPort('m19.data'),
        'm25.features': T.Graph.OutputPort('m8.data'),
        'm25.window_size': 1,
        'm25.feature_clip': 11,
        'm25.flatten': False,
        'm25.window_along_col': '',
        'm25.m_cached': False,
    
        'm28': 'M.instruments.v2',
        'm28.start_date': T.live_run_param('trading_date', '2018-01-01'),
        'm28.end_date': T.live_run_param('trading_date', '2020-09-18'),
        'm28.market': 'CN_STOCK_A',
        'm28.instrument_list': '603336.SHA',
        'm28.max_count': 0,
    
        'm16': 'M.general_feature_extractor.v7',
        'm16.instruments': T.Graph.OutputPort('m28.data'),
        'm16.features': T.Graph.OutputPort('m8.data'),
        'm16.start_date': '',
        'm16.end_date': '',
        'm16.before_start_days': 90,
    
        'm26': 'M.derived_feature_extractor.v3',
        'm26.input_data': T.Graph.OutputPort('m16.data'),
        'm26.features': T.Graph.OutputPort('m8.data'),
        'm26.date_col': 'date',
        'm26.instrument_col': 'instrument',
        'm26.drop_na': True,
        'm26.remove_extra_columns': False,
        'm26.user_functions': {},
    
        'm29': 'M.dropnan.v2',
        'm29.input_data': T.Graph.OutputPort('m26.data'),
    
        'm27': 'M.dl_convert_to_bin.v2',
        'm27.input_data': T.Graph.OutputPort('m29.data'),
        'm27.features': T.Graph.OutputPort('m8.data'),
        'm27.window_size': 1,
        'm27.feature_clip': 11,
        'm27.flatten': False,
        'm27.window_along_col': '',
        'm27.m_cached': False,
    
        'm20': 'M.dl_layer_input.v1',
        'm20.shape': '1,11',
        'm20.batch_shape': '',
        'm20.dtype': 'float32',
        'm20.sparse': False,
        'm20.name': '',
    
        'm30': 'M.dl_layer_conv1d.v1',
        'm30.inputs': T.Graph.OutputPort('m20.data'),
        'm30.filters': 20,
        'm30.kernel_size': '1',
        'm30.strides': '1',
        'm30.padding': 'valid',
        'm30.dilation_rate': 1,
        'm30.activation': 'relu',
        'm30.use_bias': True,
        'm30.kernel_initializer': 'glorot_uniform',
        'm30.bias_initializer': 'Zeros',
        'm30.kernel_regularizer': 'None',
        'm30.kernel_regularizer_l1': 0,
        'm30.kernel_regularizer_l2': 0,
        'm30.bias_regularizer': 'None',
        'm30.bias_regularizer_l1': 0,
        'm30.bias_regularizer_l2': 0,
        'm30.activity_regularizer': 'None',
        'm30.activity_regularizer_l1': 0,
        'm30.activity_regularizer_l2': 0,
        'm30.kernel_constraint': 'None',
        'm30.bias_constraint': 'None',
        'm30.name': '',
    
        'm31': 'M.dl_layer_maxpooling1d.v1',
        'm31.inputs': T.Graph.OutputPort('m30.data'),
        'm31.pool_size': 1,
        'm31.padding': 'valid',
        'm31.name': '',
    
        'm32': 'M.dl_layer_conv1d.v1',
        'm32.inputs': T.Graph.OutputPort('m31.data'),
        'm32.filters': 20,
        'm32.kernel_size': '1',
        'm32.strides': '1',
        'm32.padding': 'valid',
        'm32.dilation_rate': 1,
        'm32.activation': 'relu',
        'm32.use_bias': True,
        'm32.kernel_initializer': 'glorot_uniform',
        'm32.bias_initializer': 'Zeros',
        'm32.kernel_regularizer': 'None',
        'm32.kernel_regularizer_l1': 0,
        'm32.kernel_regularizer_l2': 0,
        'm32.bias_regularizer': 'None',
        'm32.bias_regularizer_l1': 0,
        'm32.bias_regularizer_l2': 0,
        'm32.activity_regularizer': 'None',
        'm32.activity_regularizer_l1': 0,
        'm32.activity_regularizer_l2': 0,
        'm32.kernel_constraint': 'None',
        'm32.bias_constraint': 'None',
        'm32.name': '',
    
        'm33': 'M.dl_layer_maxpooling1d.v1',
        'm33.inputs': T.Graph.OutputPort('m32.data'),
        'm33.pool_size': 1,
        'm33.padding': 'valid',
        'm33.name': '',
    
        'm35': 'M.dl_layer_lstm.v1',
        'm35.inputs': T.Graph.OutputPort('m33.data'),
        'm35.units': 7,
        'm35.activation': 'relu',
        'm35.recurrent_activation': 'hard_sigmoid',
        'm35.use_bias': True,
        'm35.kernel_initializer': 'glorot_uniform',
        'm35.recurrent_initializer': 'Orthogonal',
        'm35.bias_initializer': 'Zeros',
        'm35.unit_forget_bias': True,
        'm35.kernel_regularizer': 'None',
        'm35.kernel_regularizer_l1': 0,
        'm35.kernel_regularizer_l2': 0,
        'm35.recurrent_regularizer': 'None',
        'm35.recurrent_regularizer_l1': 0,
        'm35.recurrent_regularizer_l2': 0,
        'm35.bias_regularizer': 'None',
        'm35.bias_regularizer_l1': 0,
        'm35.bias_regularizer_l2': 0,
        'm35.activity_regularizer': 'None',
        'm35.activity_regularizer_l1': 0,
        'm35.activity_regularizer_l2': 0,
        'm35.kernel_constraint': 'None',
        'm35.recurrent_constraint': 'None',
        'm35.bias_constraint': 'None',
        'm35.dropout': 0,
        'm35.recurrent_dropout': 0,
        'm35.return_sequences': False,
        'm35.implementation': '2',
        'm35.name': '',
    
        'm36': 'M.dl_layer_dropout.v1',
        'm36.inputs': T.Graph.OutputPort('m35.data'),
        'm36.rate': 0.2,
        'm36.noise_shape': '',
        'm36.name': '',
    
        'm37': 'M.dl_layer_dense.v1',
        'm37.inputs': T.Graph.OutputPort('m36.data'),
        'm37.units': 4,
        'm37.activation': 'relu',
        'm37.use_bias': True,
        'm37.kernel_initializer': 'glorot_uniform',
        'm37.bias_initializer': 'Zeros',
        'm37.kernel_regularizer': 'None',
        'm37.kernel_regularizer_l1': 0,
        'm37.kernel_regularizer_l2': 0,
        'm37.bias_regularizer': 'None',
        'm37.bias_regularizer_l1': 0,
        'm37.bias_regularizer_l2': 0,
        'm37.activity_regularizer': 'None',
        'm37.activity_regularizer_l1': 0,
        'm37.activity_regularizer_l2': 0,
        'm37.kernel_constraint': 'None',
        'm37.bias_constraint': 'None',
        'm37.name': '',
    
        'm38': 'M.dl_layer_dropout.v1',
        'm38.inputs': T.Graph.OutputPort('m37.data'),
        'm38.rate': 0.2,
        'm38.noise_shape': '',
        'm38.name': '',
    
        'm39': 'M.dl_layer_dense.v1',
        'm39.inputs': T.Graph.OutputPort('m38.data'),
        'm39.units': 1,
        'm39.activation': 'relu',
        'm39.use_bias': True,
        'm39.kernel_initializer': 'glorot_uniform',
        'm39.bias_initializer': 'Zeros',
        'm39.kernel_regularizer': 'None',
        'm39.kernel_regularizer_l1': 0,
        'm39.kernel_regularizer_l2': 0,
        'm39.bias_regularizer': 'None',
        'm39.bias_regularizer_l1': 0,
        'm39.bias_regularizer_l2': 0,
        'm39.activity_regularizer': 'None',
        'm39.activity_regularizer_l1': 0,
        'm39.activity_regularizer_l2': 0,
        'm39.kernel_constraint': 'None',
        'm39.bias_constraint': 'None',
        'm39.name': '',
    
        'm34': 'M.dl_model_init.v1',
        'm34.inputs': T.Graph.OutputPort('m20.data'),
        'm34.outputs': T.Graph.OutputPort('m39.data'),
    
        'm6': 'M.dl_model_train.v1',
        'm6.input_model': T.Graph.OutputPort('m34.data'),
        'm6.training_data': T.Graph.OutputPort('m25.data'),
        'm6.optimizer': 'Adam',
        'm6.loss': 'binary_crossentropy',
        'm6.metrics': 'accuracy',
        'm6.batch_size': 2048,
        'm6.epochs': 100,
        'm6.n_gpus': 1,
        'm6.verbose': '1:输出进度条记录',
    
        'm7': 'M.dl_model_predict.v1',
        'm7.trained_model': T.Graph.OutputPort('m6.data'),
        'm7.input_data': T.Graph.OutputPort('m27.data'),
        'm7.batch_size': 10240,
        'm7.n_gpus': 0,
        'm7.verbose': '2:每个epoch输出一行记录',
    
        'm3': 'M.cached.v3',
        'm3.input_1': T.Graph.OutputPort('m7.data'),
        'm3.input_2': T.Graph.OutputPort('m29.data'),
        'm3.run': m3_run_bigquant_run,
        'm3.post_run': m3_post_run_bigquant_run,
        'm3.input_ports': '',
        'm3.params': '{}',
        'm3.output_ports': '',
        'm3.m_cached': False,
    
        'm1': 'M.trade.v4',
        'm1.instruments': T.Graph.OutputPort('m28.data'),
        'm1.options_data': T.Graph.OutputPort('m3.data_1'),
        'm1.start_date': '',
        'm1.end_date': '',
        'm1.initialize': m1_initialize_bigquant_run,
        'm1.handle_data': m1_handle_data_bigquant_run,
        'm1.prepare': m1_prepare_bigquant_run,
        'm1.before_trading_start': m1_before_trading_start_bigquant_run,
        'm1.volume_limit': 0,
        'm1.order_price_field_buy': 'open',
        'm1.order_price_field_sell': 'close',
        'm1.capital_base': 1000000,
        'm1.auto_cancel_non_tradable_orders': True,
        'm1.data_frequency': 'daily',
        'm1.price_type': '真实价格',
        'm1.product_type': '股票',
        'm1.plot_charts': True,
        'm1.backtest_only': False,
        'm1.benchmark': '',
    })
    
    # g.run({})
    
    
    def m18_run_bigquant_run(
        bq_graph,
        inputs,
        trading_days_market='CN', # 使用那个市场的交易日历, TODO
        train_instruments_mid='m24', # 训练数据 证券代码列表 模块id
        test_instruments_mid='m28', # 测试数据 证券代码列表 模块id
        predict_mid='m2', # 预测 模块id
        trade_mid='m1', # 回测 模块id
        start_date='2018-01-01', # 数据开始日期
        end_date=T.live_run_param('trading_date', '2020-09-18'), # 数据结束日期
        train_update_days=5, # 更新周期,按交易日计算,每多少天更新一次
        train_update_days_for_live=None, #模拟实盘模式下的更新周期,按交易日计算,每多少天更新一次。如果需要在模拟实盘阶段使用不同的模型更新周期,可以设置这个参数
        train_data_min_days=125, # 最小数据天数,按交易日计算,所以第一个滚动的结束日期是 从开始日期到开始日期+最小数据天数
        train_data_max_days=250, # 最大数据天数,按交易日计算,0,表示没有限制,否则每一个滚动的开始日期=max(此滚动的结束日期-最大数据天数, 开始日期
        rolling_count_for_live=1, #实盘模式下滚动次数,模拟实盘模式下,取最后多少次滚动。一般在模拟实盘模式下,只用到最后一次滚动训练的模型,这里可以设置为1;如果你的滚动训练数据时间段很短,以至于期间可能没有训练数据,这里可以设置大一点。0表示没有限制
    ):
        def merge_datasources(input_1):
            df_list = [ds[0].read_df().set_index('date').ix[ds[1]:].reset_index() for ds in input_1]
            df = pd.concat(df_list)
            instrument_data = {
                'start_date': df['date'].min().strftime('%Y-%m-%d'),
                'end_date': df['date'].max().strftime('%Y-%m-%d'),
                'instruments': list(set(df['instrument'])),
            }
            return Outputs(data=DataSource.write_df(df), instrument_data=DataSource.write_pickle(instrument_data))
    
        def gen_rolling_dates(trading_days_market, start_date, end_date, train_update_days, train_update_days_for_live, train_data_min_days, train_data_max_days, rolling_count_for_live):
            # 是否实盘模式
            tdays = list(D.trading_days(market=trading_days_market, start_date=start_date, end_date=end_date)['date'])
            is_live_run = T.live_run_param('trading_date', None) is not None
    
            if is_live_run and train_update_days_for_live:
                train_update_days = train_update_days_for_live
    
            rollings = []
            train_end_date = train_data_min_days
            while train_end_date < len(tdays):
                if train_data_max_days is not None and train_data_max_days > 0:
                    train_start_date = max(train_end_date - train_data_max_days, 0)
                else:
                    train_start_date = 0
                rollings.append({
                    'train_start_date': tdays[train_start_date].strftime('%Y-%m-%d'),
                    'train_end_date': tdays[train_end_date - 1].strftime('%Y-%m-%d'),
                    'test_start_date': tdays[train_end_date].strftime('%Y-%m-%d'),
                    'test_end_date': tdays[min(train_end_date + train_update_days, len(tdays)) - 1].strftime('%Y-%m-%d'),
                })
                train_end_date += train_update_days
    
            if not rollings:
                raise Exception('没有滚动需要执行,请检查配置')
    
            if is_live_run and rolling_count_for_live:
                rollings = rollings[-rolling_count_for_live:]
    
            return rollings
    
        g = bq_graph
    
        rolling_dates = gen_rolling_dates(
            trading_days_market, start_date, end_date, train_update_days, train_update_days_for_live, train_data_min_days, train_data_max_days, rolling_count_for_live)
    
        # 训练和预测
        results = []
        for rolling in rolling_dates:
            parameters = {}
            # 先禁用回测
            parameters[trade_mid + '.__enabled__'] = False
            parameters[train_instruments_mid + '.start_date'] = rolling['train_start_date']
            parameters[train_instruments_mid + '.end_date'] = rolling['train_end_date']
            parameters[test_instruments_mid + '.start_date'] = rolling['test_start_date']
            parameters[test_instruments_mid + '.end_date'] = rolling['test_end_date']
            # print('------ rolling_train:', parameters)
            results.append(g.run(parameters))
    
        # 合并预测结果并回测
        mx = M.cached.v3(run=merge_datasources, input_1=[[result[predict_mid].data_1, result[test_instruments_mid].data.read_pickle()['start_date']] for result in results])
        parameters = {}
        parameters['*.__enabled__'] = False
        parameters[trade_mid + '.__enabled__'] = True
        parameters[trade_mid + '.instruments'] = mx.instrument_data
        parameters[trade_mid + '.options_data'] = mx.data
    
        trade = g.run(parameters)
    
        return {'rollings': results, 'trade': trade}
    
    
    m18 = M.hyper_rolling_train.v1(
        run=m18_run_bigquant_run,
        run_now=True,
        bq_graph=g
    )
    
    ---------------------------------------------------------------------------
    KeyError                                  Traceback (most recent call last)
    <ipython-input-8-363515923c58> in <module>()
        440     run=m18_run_bigquant_run,
        441     run_now=True,
    --> 442     bq_graph=g
        443 )
    
    <ipython-input-8-363515923c58> in m18_run_bigquant_run(bq_graph, inputs, trading_days_market, train_instruments_mid, test_instruments_mid, predict_mid, trade_mid, start_date, end_date, train_update_days, train_update_days_for_live, train_data_min_days, train_data_max_days, rolling_count_for_live)
        422         parameters[test_instruments_mid + '.end_date'] = rolling['test_end_date']
        423         # print('------ rolling_train:', parameters)
    --> 424         results.append(g.run(parameters))
        425 
        426     # 合并预测结果并回测
    
    KeyError: 'y'

    (adhaha111) #2

    您好,首先关于您策略的错误:训练集缺少标注 label,m25生成的数据中应该还有 “y” 的数据。然后,关于您对卷积层的疑问:Conv1D的输入是3维的数据,Conv2D的输入是4维的数据,到底要选择哪个取决于您的需求。


    (a1750271086) #3

    谢谢老师,请问我如果想让神经网络的输出预测仅仅是预测会涨还是不会涨这样的二分类,除了训练模块应该用binary_crossentropy函数,在全连接层、卷积层、LSTM层应该用什么激活函数才是对应的呢?


    (a1750271086) #4

    激活函数我已经了解了,谢谢