用CNN算法实现A股股票选股

新手专区
标签: #<Tag:0x00007fc069ea0778>

(小Q) #1

在阅读了 深度学习的简要介绍后,本文将介绍深度学习CNN模型及其在量化投资领域中的应用。

一、 深度学习在量化领域应用如何?

机器学习作为人工智能的核心,其传统算法在解决很多问题上都表现出了高效性。随着近些年数据处理技术上的进步和计算能力的提升,深度学习得以在很多问题上也大放光彩,成为近一段时间互联网、金融等领域的大热门。

在量化投资领域,机器学习尤其是由统计学延伸的各种算法一直以来都被尝试应用在选股、择时等策略的开发上,随着深度学习在其他领域上的突破,其在自动化交易甚至投资策略的自开发自学习方面的应用成为了大家探索的焦点。

二、 为什么要用深度学习?

深度学习目前最成功的场景应用是在模式识别上,即利用已知数据,对具有一定空间、时间分布信息的数据与类别标号之间的映射做一个较好的估计。之所以在结构性识别的任务中,深度学习可以表现得比传统机器学习算法更好,主要有以下三点原因:

  1. 深度学习的自动提取特征比传统机器学习的人为提取特征过程更加高效。特定的应用场景中,只需要微调结构,如神经元的激活函数,就可以得到较好的效果。

  2. 深度学习可以通过复杂的结构和多重非线性处理层更好的捕捉各类非线性关系

  3. 深度学习随着数据量的增加模型效果会不断的改善,这也是当前深度学习有逐渐取代传
    统机器学习模型趋势的最大原因。

三、如何在平台上实现深度学习算法?

我们以CNN为例子,在本文末尾可克隆策略供各位研究。关于CNN的原理介绍和相关内容大家可以参照CNN入门讲解并可搜索查看平台社区相关资料。

策略缩略图如下:

第一步:通过证券代码列表模块设置股票池
第二步:通过输入特征列表模块构建需要参与训练的因子
第三步:通过特征抽取模块获取因子数据
第四步:通过数据过滤模块将因子数据划分为训练集和预测集
第五步:设定序列窗口滚动,将数据变换为窗口大小的数据序列。例如对于一维数据[1,2,3,4,5]如果设置滚动窗口大小为3,则变换为[ [1,2,3],[2,3,4],[3,4,5] ]
第六步:构建深度学习模型
第七步:模型训练
第八步:预测
第九步:回测

接下来我们着重讲解构建模型部分

四、构建深度学习模型

深度学习模型是由多个单层模型组合而成,通常划分为输入层中间层输出层
我们可以通过可视化界面左侧的模块导航树中查看到可用的深度学习模块,如下图所示:

深度学习模型模块列表

image
各模块的说明可以参考文档
image

我们着重介绍本例中使用的模块:

全连接层(Dense)

全连接层中,所有输入层的节点一定和输出层的任一节点相连接。

  • 每条连线对应了一个权重,令这些权值构成的矩阵为kernel权值矩阵。
  • 通过使用偏置向量bias可以避免得出局部最优(只在 use_bias 为 True 时才有用)。
  • activation 是按逐个元素计算的激活函数
    Dense 将实现以下操作: output = activation(dot(input, kernel) + bias)
模块位置

image

参数列表:




参数解读:

权值初始化

多层网络初始化w的时候,一般不初始化为0,会初始化为一个非0的很小的参数。
w初始值不宜太大,若w初始过大,z值过大,dz值过小,学习会很慢(对于激活函数为sigmoid或tanh函数而言)

常量初始化(constant)

   把权值或者偏置初始化为一个常数,具体是什么常数,可以自己定义

高斯分布初始化(gaussian)

   需要给定高斯函数的均值与标准差 

positive_unitball初始化

   让每一个神经元的输入的权值和为 1,例如:一个神经元有100个输入,让这100个输入的权值和为1.  首先给这100个权值赋值为在(0,1)之间的均匀分布,然后,每一个权值再除以它们的和就可以啦。这么做,可以有助于防止权值初始化过大,从而防止激活函数(sigmoid函数)进入饱和区。所以,它应该比较适合sigmoid形的激活函数

均匀分布初始化(uniform)

   将权值与偏置进行均匀分布的初始化,用min 与 max 来控制它们的的上下限,默认为(0,1)

xavier初始化

   对于权值的分布:均值为0,方差为(1 / 输入的个数) 的 均匀分布。如果我们更注重前向传播的话,我们可以选择 fan_in,即正向传播的输入个数;如果更注重后向传播的话,我们选择 fan_out, 因为在反向传播的时候,fan_out就是神经元的输入个数;如果两者都考虑的话,就选  average = (fan_in + fan_out) /2。

msra初始化

   对于权值的分布:基于均值为0,方差为( 2/输入的个数)的高斯分布;它特别适合 ReLU激活函数,该方法主要是基于Relu函数提出的。
偏置向量

目的是更好地拟合数据,具体效果可参考神经网络中偏置的作用

激活函数的选择

用于分类器时,Sigmoid函数及其组合通常效果更好。

由于梯度消失问题,有时要避免使用sigmoid和tanh函数。

ReLU函数是一个通用的激活函数,在大多数情况下广泛使用。

如果神经网络中出现死神经元,那么PReLU函数就是最好的选择。

请记住,ReLU函数只能在隐藏层中使用。

输出空间维度

决定输出层数据的维度。过大容易造成过拟合、运算时间长。过小容易造成欠拟合

Dropout层

Dropout 包括在训练中每次更新时, 将输入单元的按比率随机设置为 0, 这有助于防止过拟合。

模块位置

image

参数列表

参数解读:

随机种子

计算机并不能产生真正的随机数,如果你不设种子,计算机会用系统时钟来作为种子,如果你要模拟什么的话,每次的随机数都是不一样的,这样就不方便你研究,如果你事先设置了种子,这样每次的随机数都是一样的,便于重现你的研究,也便于其他人检验你的分析结果。

rate

在 0 和 1 之间浮动。需要丢弃的输入比例。适当丢弃数据可以有效防止过拟合

输入层

模块位置

image

参数列表

参数解读:

dtype

输入所期望的数据类型,字符串表示 (float32, float64, int32…)

shape

一个尺寸元组(整数),不包含批量大小。A shape tuple (integer), not including the batch size. 例如,shape=(32,) 表明期望的输入是按批次的 32 维向量。本模板策略中输入特征有59个,所以此处输入的shape为59

batch_shape

一个尺寸元组(整数),包含批量大小。 例如,batch_shape=(10, 32) 表明期望的输入是 10 个 32 维向量。 batch_shape=(None, 32) 表明任意批次大小的 32 维向量。

LSTM层

长短期记忆网络层

模块位置

image

参数列表







参数解读:(激活函数等参数同上)

recurrent激活函数

用于循环时间步的激活函数(详情见激活函数)

循环核初始化方法

运用到 recurrent_kernel 权值矩阵的约束函数(详情见权值初始化)

Reshape层

将输入重新调整为特定的尺寸。

模块位置

image

参数列表

参数解读:

target_shape

目标尺寸。整数元组。 不包含表示批量的轴。

Conv2D层

该层创建了一个卷积核, 该卷积核对层输入进行卷积, 以生成输出张量,此模块是CNN模型的核心关键。

模块位置

image

参数列表






参数解读::(激活函数等参数同上)

卷积核数目

输出空间的维度 (即卷积中滤波器的输出数量)。

kernel_size

一个整数,或者 2 个整数表示的元组或列表, 指明 2D 卷积窗口的宽度和高度。 可以是一个整数,为所有空间维度指定相同的值。

步长

一个整数,或者 2 个整数表示的元组或列表, 指明卷积沿宽度和高度方向的步长。 可以是一个整数,为所有空间维度指定相同的值。

dilation_rate

一个整数或 2 个整数的元组或列表, 指定膨胀卷积的膨胀率。 可以是一个整数,为所有空间维度指定相同的值。 当前,指定任何 dilation_rate 值 != 1 与 指定 stride 值 != 1 两者不兼容。

我们将各层模型组合连接,将输入和最终输出层连接到模型构建模块完成模型构建。

image
本例中我们将输入经过Reshape层变换为卷积层识别的三维数据,然后通过Conv2D将数据卷积化降维,再通过reshape层变换为LSTM层的输入,通过加入Dropout层来防止过拟合,最终通过一个全连接层输出一维标量数据(即涨跌预测概率)。

完整的可视化案例策略如下:

克隆策略

使用深度学习技术预测股票价格

版本 v1.0

目录

  • ### 深度学习策略的交易规则

  • ### 策略构建步骤

  • ### 策略的实现

正文

一、深度学习策略的交易规则

  • 买入条件:预测的上涨概率>0.5,则买入或保持已有持仓。
  • 卖出条件 :预测的上涨概率<0.5,则卖出已有股票。

二、策略构建步骤

1、确定股票池和数据起止时间

  • 在证券代码列表m24模块中输入要回测的单只股票,以及数据的起止日期(含训练集和验证集)。

2、确定因子

  • 在输入特征列表m8模块中输入用于预测的N个因子表达式。

3、获取基础数据

  • 通过自定义模块m23获取指定股票池的基础数据,如收盘价等字段。

4、确定并计算模型标注

  • 通过自定义模块m16计算需要的标注指标,本例中首先计算未来10天收益df['return'],然后根据df['return']的正负来给每日数据标注1或0,来标识涨跌。

5、抽取因子数据

  • 通过衍生数据抽取模块m26计算因子数据。

6、合并标注与因子数据

  • 通过连接数据m17模块合并因子数据和标注数据。

7、划分训练集和预测集

  • 通过数据过滤模块m19和m20中设置的日期范围划分训练集数据和预测集数据。

8、生成序列窗口滚动数据集

  • 通过序列窗口滚动(深度学习)模块将训练集和预测集的数据生成固定窗口长度的数据序列,为后续模型训练和预测做准备。

9、构建LSTM + CNN模型构架

  • 在画布左侧模块列表中依次拖入输入层模块、Reshape层模块、Conv2D层模块、Reshape层模块、LSTM层模块、Dropout层模块和全连接层模块(两组),构成深度学习网络构架,

    最后通过“构建(深度学习)”模块组装各层。这里需要注意:

    输入层的shape参数是 窗口滚动数据集的大小 X 因子数量 , 本例为 50 行 X 5个因子

    ReShape层的参数是 窗口滚动数据集的大小 X 因子数量 X 1 ,本例为 50 行 X 5个因子 X1

    Conv2D层中的 kernel_size参数是滑动窗口的尺寸,本例中使用 3行 X 5列 的窗口, 每次滑动的步长为 1行 X 1列 , 卷积核数目为32,这里的窗口设置决定了后面ReShape层的参数

    ReShape层中的target_shape 参数,这是由 窗口滚动数据集 X 因子数量 和 Conv2D层中设置的窗口尺寸以及步长决定的。本例中 50行 X 5因子 的输入数据,使用 3行 X5列 的窗口滑动取数据,

    每次移动1行,共计可以得到48次数据(即可以通过滑动3行 X 5列的窗口48次来获取完整的数据),因此target_shape= 48 X 卷积核数32

    LSTM层的输出空间维度设置为卷积核数32,并设置激活函数

    Dropout层是防止过度拟合采用的主动裁剪数据技术,这里设置rate 为0.8

    全连接层共两层,第一层的输出空间维度与LSTM的输出维度保持一致为32,第二层将第一层的32维数据转变为1维数据输出,即获取预测的label值,此例为0到1之间的连续值,可以认为是上涨的概率。

10、训练深度学习模型

  • 在画布左侧模块列表中拖入“训练(深度学习)”模块m6,设置属性中的优化器、目标函数、评估指标、每次训练的数据量batch_size、迭代次数epochs和GPU的数量以及日志输出频率。

11、使用深度学习模型预测

  • 在画布左侧模块列表中拖入“预测(深度学习)”模块m7,并将“训练(深度学习)”模块m6的模型输出和验证集的序列窗口滚动数据集传给预测模块,通过预测模块即根据股票验证集的数据预测上涨的概率。

12、根据模型预测结果构建策略

  • 如果当日预测的上涨概率大于0.5,则保持持仓或买入

  • 如果当日预测的上涨概率小于0.5,则卖出股票或保持空仓。

13、模拟回测

  • 通过 trade 模块中的初始化函数定义交易手续费和滑点,通过 context.prediction 获取每日的上涨概率预测结果;

  • 通过 trade 模块中的主函数(handle函数)查看每日的买卖交易信号,按照买卖原则执行相应的买入/卖出操作。

三、策略的实现

可视化策略实现如下:

    {"Description":"实验创建于2017/11/15","Summary":"","Graph":{"EdgesInternal":[{"DestinationInputPortId":"-281:options_data","SourceOutputPortId":"-214:data_1"},{"DestinationInputPortId":"-316:inputs","SourceOutputPortId":"-210:data"},{"DestinationInputPortId":"-403:inputs","SourceOutputPortId":"-210:data"},{"DestinationInputPortId":"-14834:inputs","SourceOutputPortId":"-218:data"},{"DestinationInputPortId":"-320:input_model","SourceOutputPortId":"-316:data"},{"DestinationInputPortId":"-332:trained_model","SourceOutputPortId":"-320:data"},{"DestinationInputPortId":"-214:input_1","SourceOutputPortId":"-332:data"},{"DestinationInputPortId":"-692:features","SourceOutputPortId":"-2295:data"},{"DestinationInputPortId":"-285:features","SourceOutputPortId":"-2295:data"},{"DestinationInputPortId":"-293:features","SourceOutputPortId":"-2295:data"},{"DestinationInputPortId":"-316:outputs","SourceOutputPortId":"-259:data"},{"DestinationInputPortId":"-14841:inputs","SourceOutputPortId":"-14806:data"},{"DestinationInputPortId":"-14806:inputs","SourceOutputPortId":"-14834:data"},{"DestinationInputPortId":"-259:inputs","SourceOutputPortId":"-14841:data"},{"DestinationInputPortId":"-408:inputs","SourceOutputPortId":"-403:data"},{"DestinationInputPortId":"-446:inputs","SourceOutputPortId":"-408:data"},{"DestinationInputPortId":"-218:inputs","SourceOutputPortId":"-446:data"},{"DestinationInputPortId":"-2290:data1","SourceOutputPortId":"-1966:data_1"},{"DestinationInputPortId":"-2296:input_data","SourceOutputPortId":"-2290:data"},{"DestinationInputPortId":"-2300:input_data","SourceOutputPortId":"-2296:data"},{"DestinationInputPortId":"-2306:input_data","SourceOutputPortId":"-2296:data"},{"DestinationInputPortId":"-285:input_data","SourceOutputPortId":"-2300:data"},{"DestinationInputPortId":"-214:input_3","SourceOutputPortId":"-2306:data"},{"DestinationInputPortId":"-293:input_data","SourceOutputPortId":"-2306:data"},{"DestinationInputPortId":"-1966:input_1","SourceOutputPortId":"-616:data_1"},{"DestinationInputPortId":"-692:input_data","SourceOutputPortId":"-616:data_1"},{"DestinationInputPortId":"-616:input_1","SourceOutputPortId":"-620:data"},{"DestinationInputPortId":"-281:instruments","SourceOutputPortId":"-620:data"},{"DestinationInputPortId":"-2290:data2","SourceOutputPortId":"-692:data"},{"DestinationInputPortId":"-320:training_data","SourceOutputPortId":"-285:data"},{"DestinationInputPortId":"-332:input_data","SourceOutputPortId":"-293:data"},{"DestinationInputPortId":"-214:input_2","SourceOutputPortId":"-293:data"}],"ModuleNodes":[{"Id":"-214","ModuleId":"BigQuantSpace.cached.cached-v3","ModuleParameters":[{"Name":"run","Value":"# Python 代码入口函数,input_1/2/3 对应三个输入端,data_1/2/3 对应三个输出端\ndef bigquant_run(input_1, input_2, input_3):\n input_series = input_1\n input_df = input_2\n test_data = input_df.read_pickle()\n pred_label = input_series.read_pickle()\n \n pred_result = pred_label.reshape(pred_label.shape[0]) \n dt = input_3.read_df()['date'][-1*len(pred_result):]\n pred_df = pd.Series(pred_result, index=dt)\n ds = DataSource.write_df(pred_df)\n \n pred_label = np.where(pred_label>0.5,1,0)\n labels = test_data['y']\n print('准确率%s'%(np.mean(pred_label==labels)))\n \n return Outputs(data_1=ds)\n","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"post_run","Value":"# 后处理函数,可选。输入是主函数的输出,可以在这里对数据做处理,或者返回更友好的outputs数据格式。此函数输出不会被缓存。\ndef bigquant_run(outputs):\n return outputs\n","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"input_ports","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"params","Value":"{}","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"output_ports","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_1","NodeId":"-214"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_2","NodeId":"-214"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_3","NodeId":"-214"}],"OutputPortsInternal":[{"Name":"data_1","NodeId":"-214","OutputType":null},{"Name":"data_2","NodeId":"-214","OutputType":null},{"Name":"data_3","NodeId":"-214","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":2,"IsPartOfPartialRun":null,"Comment":"模型预测结果输出","CommentCollapsed":false},{"Id":"-210","ModuleId":"BigQuantSpace.dl_layer_input.dl_layer_input-v1","ModuleParameters":[{"Name":"shape","Value":"50,5","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"batch_shape","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"dtype","Value":"float32","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"sparse","Value":"False","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"name","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"inputs","NodeId":"-210"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-210","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":3,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-218","ModuleId":"BigQuantSpace.dl_layer_lstm.dl_layer_lstm-v1","ModuleParameters":[{"Name":"units","Value":"32","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activation","Value":"tanh","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_activation","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"recurrent_activation","Value":"hard_sigmoid","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_recurrent_activation","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"use_bias","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_initializer","Value":"glorot_uniform","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_initializer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"recurrent_initializer","Value":"Orthogonal","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_recurrent_initializer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_initializer","Value":"Ones","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_initializer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"unit_forget_bias","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"recurrent_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"recurrent_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"recurrent_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_recurrent_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_activity_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_constraint","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_constraint","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"recurrent_constraint","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_recurrent_constraint","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_constraint","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_constraint","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"dropout","Value":"0","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"recurrent_dropout","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"return_sequences","Value":"False","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"implementation","Value":"0","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"name","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"inputs","NodeId":"-218"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-218","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":4,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-316","ModuleId":"BigQuantSpace.dl_model_init.dl_model_init-v1","ModuleParameters":[],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"inputs","NodeId":"-316"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"outputs","NodeId":"-316"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-316","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":5,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-320","ModuleId":"BigQuantSpace.dl_model_train.dl_model_train-v1","ModuleParameters":[{"Name":"optimizer","Value":"Adam","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_optimizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"loss","Value":"binary_crossentropy","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_loss","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"metrics","Value":"accuracy","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"batch_size","Value":"2048","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"epochs","Value":"10","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"n_gpus","Value":"1","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"verbose","Value":"1:输出进度条记录","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_model","NodeId":"-320"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"training_data","NodeId":"-320"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"validation_data","NodeId":"-320"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-320","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":6,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-332","ModuleId":"BigQuantSpace.dl_model_predict.dl_model_predict-v1","ModuleParameters":[{"Name":"batch_size","Value":"10240","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"n_gpus","Value":"2","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"verbose","Value":"2:每个epoch输出一行记录","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"trained_model","NodeId":"-332"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_data","NodeId":"-332"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-332","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":7,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-2295","ModuleId":"BigQuantSpace.input_features.input_features-v1","ModuleParameters":[{"Name":"features","Value":"(close/shift(close,1)-1)*10\n(high/shift(high,1)-1)*10\n(low/shift(low,1)-1)*10\n(open/shift(open,1)-1)*10\n(volume/shift(volume,1)-1)*10","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"features_ds","NodeId":"-2295"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-2295","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":8,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-259","ModuleId":"BigQuantSpace.dl_layer_dense.dl_layer_dense-v1","ModuleParameters":[{"Name":"units","Value":"1","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activation","Value":"sigmoid","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_activation","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"use_bias","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_initializer","Value":"glorot_uniform","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_initializer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_initializer","Value":"Zeros","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_initializer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_activity_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_constraint","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_constraint","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_constraint","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_constraint","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"name","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"inputs","NodeId":"-259"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-259","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":9,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-14806","ModuleId":"BigQuantSpace.dl_layer_dense.dl_layer_dense-v1","ModuleParameters":[{"Name":"units","Value":"32","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activation","Value":"tanh","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_activation","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"use_bias","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_initializer","Value":"glorot_uniform","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_initializer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_initializer","Value":"Zeros","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_initializer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_activity_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_constraint","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_constraint","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_constraint","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_constraint","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"name","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"inputs","NodeId":"-14806"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-14806","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":10,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-14834","ModuleId":"BigQuantSpace.dl_layer_dropout.dl_layer_dropout-v1","ModuleParameters":[{"Name":"rate","Value":"0.8","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"noise_shape","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"seed","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"name","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"inputs","NodeId":"-14834"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-14834","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":11,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-14841","ModuleId":"BigQuantSpace.dl_layer_dropout.dl_layer_dropout-v1","ModuleParameters":[{"Name":"rate","Value":"0.8","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"noise_shape","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"seed","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"name","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"inputs","NodeId":"-14841"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-14841","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":12,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-403","ModuleId":"BigQuantSpace.dl_layer_reshape.dl_layer_reshape-v1","ModuleParameters":[{"Name":"target_shape","Value":"50,5,1","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"name","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"inputs","NodeId":"-403"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-403","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":13,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-408","ModuleId":"BigQuantSpace.dl_layer_conv2d.dl_layer_conv2d-v1","ModuleParameters":[{"Name":"filters","Value":"32","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_size","Value":"3,5","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"strides","Value":"1,1","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"padding","Value":"valid","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"data_format","Value":"channels_last","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"dilation_rate","Value":"1,1","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activation","Value":"relu","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_activation","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"use_bias","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_initializer","Value":"glorot_uniform","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_initializer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_initializer","Value":"Zeros","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_initializer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer_l1","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"activity_regularizer_l2","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_activity_regularizer","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"kernel_constraint","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_kernel_constraint","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"bias_constraint","Value":"None","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_bias_constraint","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"name","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"inputs","NodeId":"-408"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-408","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":14,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-446","ModuleId":"BigQuantSpace.dl_layer_reshape.dl_layer_reshape-v1","ModuleParameters":[{"Name":"target_shape","Value":"48,32","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"name","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"inputs","NodeId":"-446"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-446","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":15,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-1966","ModuleId":"BigQuantSpace.cached.cached-v3","ModuleParameters":[{"Name":"run","Value":"# Python 代码入口函数,input_1/2/3 对应三个输入端,data_1/2/3 对应三个输出端\ndef bigquant_run(input_1, input_2, input_3):\n input_ds = input_1\n df = input_ds.read_df()\n df['return'] = (df.close.shift(-10)/df.close - 1)\n df['label'] = np.where(df['return'] > 0, 1, 0)\n ds = DataSource.write_df(df[['date','instrument','label']])\n return Outputs(data_1=ds)\n\n","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"post_run","Value":"# 后处理函数,可选。输入是主函数的输出,可以在这里对数据做处理,或者返回更友好的outputs数据格式。此函数输出不会被缓存。\ndef bigquant_run(outputs):\n return outputs\n","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"input_ports","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"params","Value":"{}","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"output_ports","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_1","NodeId":"-1966"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_2","NodeId":"-1966"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_3","NodeId":"-1966"}],"OutputPortsInternal":[{"Name":"data_1","NodeId":"-1966","OutputType":null},{"Name":"data_2","NodeId":"-1966","OutputType":null},{"Name":"data_3","NodeId":"-1966","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":16,"IsPartOfPartialRun":null,"Comment":"数据标注","CommentCollapsed":false},{"Id":"-2290","ModuleId":"BigQuantSpace.join.join-v3","ModuleParameters":[{"Name":"on","Value":"date,instrument","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"how","Value":"inner","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"sort","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"data1","NodeId":"-2290"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"data2","NodeId":"-2290"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-2290","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":17,"IsPartOfPartialRun":null,"Comment":"标注特征连接","CommentCollapsed":false},{"Id":"-2296","ModuleId":"BigQuantSpace.dropnan.dropnan-v1","ModuleParameters":[],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_data","NodeId":"-2296"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-2296","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":18,"IsPartOfPartialRun":null,"Comment":"去掉为nan的数据","CommentCollapsed":true},{"Id":"-2300","ModuleId":"BigQuantSpace.filter.filter-v3","ModuleParameters":[{"Name":"expr","Value":"date<'2017-03-01'","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"output_left_data","Value":"False","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_data","NodeId":"-2300"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-2300","OutputType":null},{"Name":"left_data","NodeId":"-2300","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":19,"IsPartOfPartialRun":null,"Comment":"训练数据","CommentCollapsed":false},{"Id":"-2306","ModuleId":"BigQuantSpace.filter.filter-v3","ModuleParameters":[{"Name":"expr","Value":"date>'2017-03-01'","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"output_left_data","Value":"False","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_data","NodeId":"-2306"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-2306","OutputType":null},{"Name":"left_data","NodeId":"-2306","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":20,"IsPartOfPartialRun":null,"Comment":"测试数据","CommentCollapsed":false},{"Id":"-616","ModuleId":"BigQuantSpace.cached.cached-v3","ModuleParameters":[{"Name":"run","Value":"# Python 代码入口函数,input_1/2/3 对应三个输入端,data_1/2/3 对应三个输出端\ndef bigquant_run(input_1, input_2, input_3):\n fields = ['open','high','low','close','volume']\n input_1_df = input_1.read_pickle()\n ins = input_1_df['instruments']\n start_date = input_1_df['start_date']\n end_date = input_1_df['end_date']\n df = D.history_data(ins, start_date, end_date, fields) \n data_1 = DataSource.write_df(df)\n return Outputs(data_1=data_1, data_2=None, data_3=None)\n","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"post_run","Value":"# 后处理函数,可选。输入是主函数的输出,可以在这里对数据做处理,或者返回更友好的outputs数据格式。此函数输出不会被缓存。\ndef bigquant_run(outputs):\n return outputs\n","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"input_ports","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"params","Value":"{}","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"output_ports","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_1","NodeId":"-616"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_2","NodeId":"-616"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_3","NodeId":"-616"}],"OutputPortsInternal":[{"Name":"data_1","NodeId":"-616","OutputType":null},{"Name":"data_2","NodeId":"-616","OutputType":null},{"Name":"data_3","NodeId":"-616","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":23,"IsPartOfPartialRun":null,"Comment":"获取基本数据","CommentCollapsed":false},{"Id":"-620","ModuleId":"BigQuantSpace.instruments.instruments-v2","ModuleParameters":[{"Name":"start_date","Value":"2015-01-01","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"end_date","Value":"2018-10-30","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"market","Value":"CN_STOCK_A","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"instrument_list","Value":"600009.SHA","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"max_count","Value":0,"ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"rolling_conf","NodeId":"-620"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-620","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":24,"IsPartOfPartialRun":null,"Comment":"证券标的及起始截止时间","CommentCollapsed":false},{"Id":"-692","ModuleId":"BigQuantSpace.derived_feature_extractor.derived_feature_extractor-v3","ModuleParameters":[{"Name":"date_col","Value":"date","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"instrument_col","Value":"instrument","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"drop_na","Value":"False","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"remove_extra_columns","Value":"False","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_functions","Value":"{}","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_data","NodeId":"-692"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"features","NodeId":"-692"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-692","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":26,"IsPartOfPartialRun":null,"Comment":"计算需要使用的特征","CommentCollapsed":false},{"Id":"-281","ModuleId":"BigQuantSpace.trade.trade-v4","ModuleParameters":[{"Name":"start_date","Value":"2017-04-01","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"end_date","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"handle_data","Value":"# 回测引擎:每日数据处理函数,每天执行一次\ndef bigquant_run(context, data):\n # 按日期过滤得到今日的预测数据\n try:\n prediction = context.prediction[data.current_dt.strftime('%Y-%m-%d')]\n except KeyError as e:\n return\n \n instrument = context.instruments[0]\n sid = context.symbol(instrument)\n cur_position = context.portfolio.positions[sid].amount\n \n # 交易逻辑\n if prediction > 0.5 and cur_position == 0:\n context.order_target_percent(context.symbol(instrument), 1)\n print(data.current_dt, '买入!')\n \n elif prediction < 0.5 and cur_position > 0:\n context.order_target_percent(context.symbol(instrument), 0)\n print(data.current_dt, '卖出!')\n ","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"prepare","Value":"# 回测引擎:准备数据,只执行一次\ndef bigquant_run(context):\n pass\n","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"initialize","Value":"# 回测引擎:初始化函数,只执行一次\ndef bigquant_run(context):\n # 加载预测数据\n context.prediction = context.options['data'].read_df()\n\n # 系统已经设置了默认的交易手续费和滑点,要修改手续费可使用如下函数\n context.set_commission(PerOrder(buy_cost=0.0003, sell_cost=0.0013, min_cost=5))","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"before_trading_start","Value":"# 回测引擎:每个单位时间开始前调用一次,即每日开盘前调用一次。\ndef bigquant_run(context, data):\n pass\n","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"volume_limit","Value":0.025,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"order_price_field_buy","Value":"open","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"order_price_field_sell","Value":"close","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"capital_base","Value":1000000,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"auto_cancel_non_tradable_orders","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"data_frequency","Value":"daily","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"price_type","Value":"真实价格","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"product_type","Value":"股票","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"plot_charts","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"backtest_only","Value":"False","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"benchmark","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"instruments","NodeId":"-281"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"options_data","NodeId":"-281"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"history_ds","NodeId":"-281"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"benchmark_ds","NodeId":"-281"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"trading_calendar","NodeId":"-281"}],"OutputPortsInternal":[{"Name":"raw_perf","NodeId":"-281","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":1,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-285","ModuleId":"BigQuantSpace.dl_convert_to_bin.dl_convert_to_bin-v2","ModuleParameters":[{"Name":"window_size","Value":"50","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"feature_clip","Value":5,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"flatten","Value":"False","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"window_along_col","Value":"instrument","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_data","NodeId":"-285"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"features","NodeId":"-285"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-285","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":25,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-293","ModuleId":"BigQuantSpace.dl_convert_to_bin.dl_convert_to_bin-v2","ModuleParameters":[{"Name":"window_size","Value":"50","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"feature_clip","Value":5,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"flatten","Value":"False","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"window_along_col","Value":"instrument","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_data","NodeId":"-293"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"features","NodeId":"-293"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-293","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":27,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true}],"SerializedClientData":"<?xml version='1.0' encoding='utf-16'?><DataV1 xmlns:xsd='http://www.w3.org/2001/XMLSchema' xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'><Meta /><NodePositions><NodePosition Node='-214' Position='791,762,200,200'/><NodePosition Node='-210' Position='-20,-256,200,200'/><NodePosition Node='-218' Position='280,67,200,200'/><NodePosition Node='-316' Position='89,485,200,200'/><NodePosition Node='-320' Position='332,582,200,200'/><NodePosition Node='-332' Position='550,695,200,200'/><NodePosition Node='-2295' Position='1054,-339,200,200'/><NodePosition Node='-259' Position='281,387,200,200'/><NodePosition Node='-14806' Position='279,211,200,200'/><NodePosition Node='-14834' Position='279,146,200,200'/><NodePosition Node='-14841' Position='280,301,200,200'/><NodePosition Node='-403' Position='279,-195,200,200'/><NodePosition Node='-408' Position='280,-107,200,200'/><NodePosition Node='-446' Position='278,-23,200,200'/><NodePosition Node='-1966' Position='715,-224,200,200'/><NodePosition Node='-2290' Position='710,-72,200,200'/><NodePosition Node='-2296' Position='710,52,200,200'/><NodePosition Node='-2300' Position='707,211,200,200'/><NodePosition Node='-2306' Position='1057,212,200,200'/><NodePosition Node='-616' Position='717,-346,200,200'/><NodePosition Node='-620' Position='423,-457,200,200'/><NodePosition Node='-692' Position='1045,-227,200,200'/><NodePosition Node='-281' Position='976,893,200,200'/><NodePosition Node='-285' Position='710,373,200,200'/><NodePosition Node='-293' Position='1060,372,200,200'/></NodePositions><NodeGroups /></DataV1>"},"IsDraft":true,"ParentExperimentId":null,"WebService":{"IsWebServiceExperiment":false,"Inputs":[],"Outputs":[],"Parameters":[{"Name":"交易日期","Value":"","ParameterDefinition":{"Name":"交易日期","FriendlyName":"交易日期","DefaultValue":"","ParameterType":"String","HasDefaultValue":true,"IsOptional":true,"ParameterRules":[],"HasRules":false,"MarkupType":0,"CredentialDescriptor":null}}],"WebServiceGroupId":null,"SerializedClientData":"<?xml version='1.0' encoding='utf-16'?><DataV1 xmlns:xsd='http://www.w3.org/2001/XMLSchema' xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'><Meta /><NodePositions></NodePositions><NodeGroups /></DataV1>"},"DisableNodesUpdate":false,"Category":"user","Tags":[],"IsPartialRun":true}
    In [6]:
    # 本代码由可视化策略环境自动生成 2019年4月3日 15:45
    # 本代码单元只能在可视化模式下编辑。您也可以拷贝代码,粘贴到新建的代码单元或者策略,然后修改。
    
    
    # Python 代码入口函数,input_1/2/3 对应三个输入端,data_1/2/3 对应三个输出端
    def m23_run_bigquant_run(input_1, input_2, input_3):
        fields = ['open','high','low','close','volume']
        input_1_df = input_1.read_pickle()
        ins = input_1_df['instruments']
        start_date = input_1_df['start_date']
        end_date = input_1_df['end_date']
        df = D.history_data(ins, start_date, end_date, fields)     
        data_1 = DataSource.write_df(df)
        return Outputs(data_1=data_1, data_2=None, data_3=None)
    
    # 后处理函数,可选。输入是主函数的输出,可以在这里对数据做处理,或者返回更友好的outputs数据格式。此函数输出不会被缓存。
    def m23_post_run_bigquant_run(outputs):
        return outputs
    
    # Python 代码入口函数,input_1/2/3 对应三个输入端,data_1/2/3 对应三个输出端
    def m16_run_bigquant_run(input_1, input_2, input_3):
        input_ds = input_1
        df = input_ds.read_df()
        df['return'] = (df.close.shift(-10)/df.close - 1)
        df['label'] = np.where(df['return'] > 0, 1, 0)
        ds = DataSource.write_df(df[['date','instrument','label']])
        return Outputs(data_1=ds)
    
    
    # 后处理函数,可选。输入是主函数的输出,可以在这里对数据做处理,或者返回更友好的outputs数据格式。此函数输出不会被缓存。
    def m16_post_run_bigquant_run(outputs):
        return outputs
    
    # Python 代码入口函数,input_1/2/3 对应三个输入端,data_1/2/3 对应三个输出端
    def m2_run_bigquant_run(input_1, input_2, input_3):
        input_series = input_1
        input_df = input_2
        test_data = input_df.read_pickle()
        pred_label = input_series.read_pickle()
      
        pred_result = pred_label.reshape(pred_label.shape[0]) 
        dt = input_3.read_df()['date'][-1*len(pred_result):]
        pred_df = pd.Series(pred_result, index=dt)
        ds = DataSource.write_df(pred_df)
        
        pred_label = np.where(pred_label>0.5,1,0)
        labels = test_data['y']
        print('准确率%s'%(np.mean(pred_label==labels)))
        
        return Outputs(data_1=ds)
    
    # 后处理函数,可选。输入是主函数的输出,可以在这里对数据做处理,或者返回更友好的outputs数据格式。此函数输出不会被缓存。
    def m2_post_run_bigquant_run(outputs):
        return outputs
    
    # 回测引擎:每日数据处理函数,每天执行一次
    def m1_handle_data_bigquant_run(context, data):
        # 按日期过滤得到今日的预测数据
        try:
            prediction = context.prediction[data.current_dt.strftime('%Y-%m-%d')]
        except KeyError as e:
            return
        
        instrument = context.instruments[0]
        sid = context.symbol(instrument)
        cur_position = context.portfolio.positions[sid].amount
        
        # 交易逻辑
        if prediction > 0.5 and cur_position == 0:
            context.order_target_percent(context.symbol(instrument), 1)
            print(data.current_dt, '买入!')
            
        elif prediction < 0.5 and cur_position > 0:
            context.order_target_percent(context.symbol(instrument), 0)
            print(data.current_dt, '卖出!')
        
    # 回测引擎:准备数据,只执行一次
    def m1_prepare_bigquant_run(context):
        pass
    
    # 回测引擎:初始化函数,只执行一次
    def m1_initialize_bigquant_run(context):
        # 加载预测数据
        context.prediction = context.options['data'].read_df()
    
        # 系统已经设置了默认的交易手续费和滑点,要修改手续费可使用如下函数
        context.set_commission(PerOrder(buy_cost=0.0003, sell_cost=0.0013, min_cost=5))
    # 回测引擎:每个单位时间开始前调用一次,即每日开盘前调用一次。
    def m1_before_trading_start_bigquant_run(context, data):
        pass
    
    
    m3 = M.dl_layer_input.v1(
        shape='50,5',
        batch_shape='',
        dtype='float32',
        sparse=False,
        name=''
    )
    
    m13 = M.dl_layer_reshape.v1(
        inputs=m3.data,
        target_shape='50,5,1',
        name=''
    )
    
    m14 = M.dl_layer_conv2d.v1(
        inputs=m13.data,
        filters=32,
        kernel_size='3,5',
        strides='1,1',
        padding='valid',
        data_format='channels_last',
        dilation_rate='1,1',
        activation='relu',
        use_bias=True,
        kernel_initializer='glorot_uniform',
        bias_initializer='Zeros',
        kernel_regularizer='None',
        kernel_regularizer_l1=0,
        kernel_regularizer_l2=0,
        bias_regularizer='None',
        bias_regularizer_l1=0,
        bias_regularizer_l2=0,
        activity_regularizer='None',
        activity_regularizer_l1=0,
        activity_regularizer_l2=0,
        kernel_constraint='None',
        bias_constraint='None',
        name=''
    )
    
    m15 = M.dl_layer_reshape.v1(
        inputs=m14.data,
        target_shape='48,32',
        name=''
    )
    
    m4 = M.dl_layer_lstm.v1(
        inputs=m15.data,
        units=32,
        activation='tanh',
        recurrent_activation='hard_sigmoid',
        use_bias=True,
        kernel_initializer='glorot_uniform',
        recurrent_initializer='Orthogonal',
        bias_initializer='Ones',
        unit_forget_bias=True,
        kernel_regularizer='None',
        kernel_regularizer_l1=0,
        kernel_regularizer_l2=0,
        recurrent_regularizer='None',
        recurrent_regularizer_l1=0,
        recurrent_regularizer_l2=0,
        bias_regularizer='None',
        bias_regularizer_l1=0,
        bias_regularizer_l2=0,
        activity_regularizer='None',
        activity_regularizer_l1=0,
        activity_regularizer_l2=0,
        kernel_constraint='None',
        recurrent_constraint='None',
        bias_constraint='None',
        dropout=0,
        recurrent_dropout=0,
        return_sequences=False,
        implementation='0',
        name=''
    )
    
    m11 = M.dl_layer_dropout.v1(
        inputs=m4.data,
        rate=0.8,
        noise_shape='',
        name=''
    )
    
    m10 = M.dl_layer_dense.v1(
        inputs=m11.data,
        units=32,
        activation='tanh',
        use_bias=True,
        kernel_initializer='glorot_uniform',
        bias_initializer='Zeros',
        kernel_regularizer='None',
        kernel_regularizer_l1=0,
        kernel_regularizer_l2=0,
        bias_regularizer='None',
        bias_regularizer_l1=0,
        bias_regularizer_l2=0,
        activity_regularizer='None',
        activity_regularizer_l1=0,
        activity_regularizer_l2=0,
        kernel_constraint='None',
        bias_constraint='None',
        name=''
    )
    
    m12 = M.dl_layer_dropout.v1(
        inputs=m10.data,
        rate=0.8,
        noise_shape='',
        name=''
    )
    
    m9 = M.dl_layer_dense.v1(
        inputs=m12.data,
        units=1,
        activation='sigmoid',
        use_bias=True,
        kernel_initializer='glorot_uniform',
        bias_initializer='Zeros',
        kernel_regularizer='None',
        kernel_regularizer_l1=0,
        kernel_regularizer_l2=0,
        bias_regularizer='None',
        bias_regularizer_l1=0,
        bias_regularizer_l2=0,
        activity_regularizer='None',
        activity_regularizer_l1=0,
        activity_regularizer_l2=0,
        kernel_constraint='None',
        bias_constraint='None',
        name=''
    )
    
    m5 = M.dl_model_init.v1(
        inputs=m3.data,
        outputs=m9.data
    )
    
    m8 = M.input_features.v1(
        features="""(close/shift(close,1)-1)*10
    (high/shift(high,1)-1)*10
    (low/shift(low,1)-1)*10
    (open/shift(open,1)-1)*10
    (volume/shift(volume,1)-1)*10"""
    )
    
    m24 = M.instruments.v2(
        start_date='2015-01-01',
        end_date='2018-10-30',
        market='CN_STOCK_A',
        instrument_list='600009.SHA',
        max_count=0
    )
    
    m23 = M.cached.v3(
        input_1=m24.data,
        run=m23_run_bigquant_run,
        post_run=m23_post_run_bigquant_run,
        input_ports='',
        params='{}',
        output_ports=''
    )
    
    m16 = M.cached.v3(
        input_1=m23.data_1,
        run=m16_run_bigquant_run,
        post_run=m16_post_run_bigquant_run,
        input_ports='',
        params='{}',
        output_ports=''
    )
    
    m26 = M.derived_feature_extractor.v3(
        input_data=m23.data_1,
        features=m8.data,
        date_col='date',
        instrument_col='instrument',
        drop_na=False,
        remove_extra_columns=False,
        user_functions={}
    )
    
    m17 = M.join.v3(
        data1=m16.data_1,
        data2=m26.data,
        on='date,instrument',
        how='inner',
        sort=True
    )
    
    m18 = M.dropnan.v1(
        input_data=m17.data
    )
    
    m19 = M.filter.v3(
        input_data=m18.data,
        expr='date<\'2017-03-01\'',
        output_left_data=False
    )
    
    m25 = M.dl_convert_to_bin.v2(
        input_data=m19.data,
        features=m8.data,
        window_size=50,
        feature_clip=5,
        flatten=False,
        window_along_col='instrument'
    )
    
    m6 = M.dl_model_train.v1(
        input_model=m5.data,
        training_data=m25.data,
        optimizer='Adam',
        loss='binary_crossentropy',
        metrics='accuracy',
        batch_size=2048,
        epochs=10,
        n_gpus=1,
        verbose='1:输出进度条记录'
    )
    
    m20 = M.filter.v3(
        input_data=m18.data,
        expr='date>\'2017-03-01\'',
        output_left_data=False
    )
    
    m27 = M.dl_convert_to_bin.v2(
        input_data=m20.data,
        features=m8.data,
        window_size=50,
        feature_clip=5,
        flatten=False,
        window_along_col='instrument'
    )
    
    m7 = M.dl_model_predict.v1(
        trained_model=m6.data,
        input_data=m27.data,
        batch_size=10240,
        n_gpus=2,
        verbose='2:每个epoch输出一行记录'
    )
    
    m2 = M.cached.v3(
        input_1=m7.data,
        input_2=m27.data,
        input_3=m20.data,
        run=m2_run_bigquant_run,
        post_run=m2_post_run_bigquant_run,
        input_ports='',
        params='{}',
        output_ports=''
    )
    
    m1 = M.trade.v4(
        instruments=m24.data,
        options_data=m2.data_1,
        start_date='2017-04-01',
        end_date='',
        handle_data=m1_handle_data_bigquant_run,
        prepare=m1_prepare_bigquant_run,
        initialize=m1_initialize_bigquant_run,
        before_trading_start=m1_before_trading_start_bigquant_run,
        volume_limit=0.025,
        order_price_field_buy='open',
        order_price_field_sell='close',
        capital_base=1000000,
        auto_cancel_non_tradable_orders=True,
        data_frequency='daily',
        price_type='真实价格',
        product_type='股票',
        plot_charts=True,
        backtest_only=False,
        benchmark=''
    )
    
    Epoch 1/10
    523/523 [==============================] - 2s 3ms/step - loss: 1.0792 - acc: 0.4818
    Epoch 2/10
    523/523 [==============================] - 0s 574us/step - loss: 1.0034 - acc: 0.4933
    Epoch 3/10
    523/523 [==============================] - 0s 588us/step - loss: 1.0716 - acc: 0.4914
    Epoch 4/10
    523/523 [==============================] - 0s 586us/step - loss: 1.0058 - acc: 0.5124
    Epoch 5/10
    523/523 [==============================] - 0s 563us/step - loss: 1.0928 - acc: 0.4818
    Epoch 6/10
    523/523 [==============================] - 0s 584us/step - loss: 0.9313 - acc: 0.5201
    Epoch 7/10
    523/523 [==============================] - 0s 724us/step - loss: 0.9981 - acc: 0.5124
    Epoch 8/10
    523/523 [==============================] - 0s 563us/step - loss: 1.0129 - acc: 0.4704
    Epoch 9/10
    523/523 [==============================] - 0s 561us/step - loss: 0.8987 - acc: 0.5392
    Epoch 10/10
    523/523 [==============================] - 0s 452us/step - loss: 1.0076 - acc: 0.5048
    
    DataSource(96b48e995b284a02b368395acdc7c448, v3)
    
    准确率0.5872235872235873
    
    2017-04-05 15:00:00+00:00 买入!
    
    • 收益率61.27%
    • 年化收益率36.73%
    • 基准收益率-10.01%
    • 阿尔法0.42
    • 贝塔0.81
    • 夏普比率0.96
    • 胜率1.0
    • 盈亏比0.0
    • 收益波动率36.51%
    • 信息比率0.08
    • 最大回撤29.6%

    【宽客学院】深度学习简介
    (oversky2003) #2

    这个例子好像限定了特征只能从’open’,‘high’,‘low’,‘close’,'volume’产生,如果我要加入其它特征,这些特征不在D.history_data里面,怎么弄?自定义python模块多了,就看不太明白了。


    (iQuant) #3

    收到您的提问,已提交至策略工程师,会尽快给您回复。


    (达达) #4

    这个特征就是列操作,您可以从DataSource拿任意表的列抽取出来作为基础因子,或者通过基础特征抽取模块从平台因子库数据抽取因子数据,前者DataSource拿到的数据您需要自己填充清洗日期对齐,因子库的因子数据是日频对齐过的,这是数据的来源/版本差异。

    无论用哪个方法拿到数据后,您可以根据数据表的列名作为基础因子,然后利用衍生特征抽取模块来计算复杂的自定义因子,比如你的数据表中有close_0, 你就可以用表达式引擎构建mean(close_0,5)这种因子,如果你的数据表中有自己计算的一个列pct,那你可以用表达式引擎构建ts_min(pct,10)这种因子,总之只要是数据表中的列名,都是可以在特征因子列表构建表达式因子,然后通过衍生特征抽取模块把这个因子计算出来。

    本文的例子无非是通过第一种方式从DataSource的一张表取了几个价格数据作为因子而已。您可以根据需要自己构建需要的因子。


    (Crimes777) #5

    深度学习预测那里m7提示错误,运行不出来啦