期货分钟LSTM例子报错

developer · 2020-04-20 02:51:09 UTC

克隆策略

深度学习在期货高频上的应用¶

策略思想：¶

使用最近50分钟的价格、成交量、持仓量数据预测未来50分钟的涨跌幅。

交易标的：¶

股指期货 IF1906

模型算法：¶

LSTM深度学习算法

模型标注：¶

未来50分钟收益率

模型因子：¶

mean(open_intl,5)/amount
mean(open_intl,10)/amount
mean(open_intl,20)/amount
mean(open_intl,30)/amount
mean(open_intl,50)/amount
close/shift(close,30)
close/shift(close,20)
close/shift(close,10)
close/shift(close,5)
mean(amount,30)
max(high,10)/close 
open_intl
sum(open_intl,10)/amount 
amount/open_intl
mean(amount/open_intl,5)
mean(amount/open_intl,10)
mean(open_intl,5)

交易信号：¶

当预测值大于0.2时并且没有多仓，进场做多。（如果有空单，需要先平掉空单）
当预测值小于0.2时并且没有空仓，进场做空。（如果有多单，需要先平掉多单）

In [10]:

# 本代码由可视化策略环境自动生成 2020年4月20日 10:51
# 本代码单元只能在可视化模式下编辑。您也可以拷贝代码，粘贴到新建的代码单元或者策略，然后修改。


# Python 代码入口函数，input_1/2/3 对应三个输入端，data_1/2/3 对应三个输出端
def m20_run_bigquant_run(input_1, input_2, input_3):
     
    start_date=input_1.read_pickle()['start_date']
    end_date=input_1.read_pickle()['end_date']
    ins=input_1.read_pickle()['instruments']
    df = DataSource('bar1m_IF1906.CFE').read(instruments=ins,start_date=start_date,end_date=end_date)
    df['adjust_factor']=1.0
    data_1 = DataSource.write_df(df)
    return Outputs(data_1=data_1, data_2=None, data_3=None)

 
# 后处理函数，可选。输入是主函数的输出，可以在这里对数据做处理，或者返回更友好的outputs数据格式。此函数输出不会被缓存。
def m20_post_run_bigquant_run(outputs):
    return outputs

# Python 代码入口函数，input_1/2/3 对应三个输入端，data_1/2/3 对应三个输出端
def m2_run_bigquant_run(input_1, input_2, input_3):
    predictions = input_1.read_pickle()
    pred_result = predictions.reshape(predictions.shape[0]) 
    dt = input_2.read_df()['date']
    pred_df = pd.Series(pred_result, index=dt)
    ds = DataSource.write_df(pred_df)
    return Outputs(data_1=ds)

# 后处理函数，可选。输入是主函数的输出，可以在这里对数据做处理，或者返回更友好的outputs数据格式。此函数输出不会被缓存。
def m2_post_run_bigquant_run(outputs):
    return outputs

# 回测引擎：初始化函数，只执行一次
def m1_initialize_bigquant_run(context):
    # 加载预测数据
    context.prediction = context.options['data'].read_df()

     
# 回测引擎：每日数据处理函数，每天执行一次
def m1_handle_data_bigquant_run(context, data):
    # 按日期过滤得到今日的预测数据
    try:
        prediction = context.prediction[data.current_dt]
    except KeyError as e:
        return
    instrument = context.instruments[0]
    sid = context.symbol(instrument)
    cur_position = context.portfolio.positions[sid].amount
    
    
    # 交易逻辑
    if prediction > 0.2 and cur_position == 0:
        context.order_target(context.future_symbol(instrument), 1)
        print(data.current_dt, '买入！')
        
    elif prediction < -0.2 and cur_position > 0:
        context.order_target(context.future_symbol(instrument), 0)
        print(data.current_dt, '卖出！')
    
# 回测引擎：准备数据，只执行一次
def m1_prepare_bigquant_run(context):
    pass

# 回测引擎：每个单位时间开始前调用一次，即每日开盘前调用一次。
def m1_before_trading_start_bigquant_run(context, data):
    pass


m3 = M.dl_layer_input.v1(
    shape='50,17',
    batch_shape='',
    dtype='float32',
    sparse=False,
    name=''
)

m4 = M.dl_layer_lstm.v1(
    inputs=m3.data,
    units=32,
    activation='linear',
    recurrent_activation='hard_sigmoid',
    use_bias=True,
    kernel_initializer='glorot_uniform',
    recurrent_initializer='Orthogonal',
    bias_initializer='Ones',
    unit_forget_bias=True,
    kernel_regularizer='None',
    kernel_regularizer_l1=0,
    kernel_regularizer_l2=0,
    recurrent_regularizer='None',
    recurrent_regularizer_l1=0,
    recurrent_regularizer_l2=0,
    bias_regularizer='None',
    bias_regularizer_l1=0,
    bias_regularizer_l2=0,
    activity_regularizer='None',
    activity_regularizer_l1=0,
    activity_regularizer_l2=0,
    kernel_constraint='None',
    recurrent_constraint='None',
    bias_constraint='None',
    dropout=0,
    recurrent_dropout=0,
    return_sequences=True,
    implementation='0',
    name=''
)

m10 = M.dl_layer_dropout.v1(
    inputs=m4.data,
    rate=0.2,
    noise_shape='',
    seed=0,
    name=''
)

m25 = M.dl_layer_lstm.v1(
    inputs=m10.data,
    units=32,
    activation='sigmoid',
    recurrent_activation='hard_sigmoid',
    use_bias=True,
    kernel_initializer='glorot_uniform',
    recurrent_initializer='Orthogonal',
    bias_initializer='Zeros',
    unit_forget_bias=True,
    kernel_regularizer='None',
    kernel_regularizer_l1=0,
    kernel_regularizer_l2=0,
    recurrent_regularizer='None',
    recurrent_regularizer_l1=0,
    recurrent_regularizer_l2=0,
    bias_regularizer='None',
    bias_regularizer_l1=0,
    bias_regularizer_l2=0,
    activity_regularizer='None',
    activity_regularizer_l1=0,
    activity_regularizer_l2=0,
    kernel_constraint='None',
    recurrent_constraint='None',
    bias_constraint='None',
    dropout=0,
    recurrent_dropout=0,
    return_sequences=False,
    implementation='0',
    name=''
)

m11 = M.dl_layer_dropout.v1(
    inputs=m25.data,
    rate=0.1,
    noise_shape='',
    seed=0,
    name=''
)

m9 = M.dl_layer_dense.v1(
    inputs=m11.data,
    units=1,
    activation='linear',
    use_bias=True,
    kernel_initializer='glorot_uniform',
    bias_initializer='Zeros',
    kernel_regularizer='None',
    kernel_regularizer_l1=0,
    kernel_regularizer_l2=0,
    bias_regularizer='None',
    bias_regularizer_l1=0,
    bias_regularizer_l2=0,
    activity_regularizer='None',
    activity_regularizer_l1=0,
    activity_regularizer_l2=0,
    kernel_constraint='None',
    bias_constraint='None',
    name=''
)

m5 = M.dl_model_init.v1(
    inputs=m3.data,
    outputs=m9.data
)

m8 = M.input_features.v1(
    features="""mean(open_intl,5)/amount
mean(open_intl,10)/amount
mean(open_intl,20)/amount
mean(open_intl,30)/amount
mean(open_intl,50)/amount
close/shift(close,30)
close/shift(close,20)
close/shift(close,10)
close/shift(close,5)
mean(amount,30)
max(high,10)/close 
open_intl
sum(open_intl,10)/amount 
amount/open_intl
mean(amount/open_intl,5)
mean(amount/open_intl,10)
mean(open_intl,5)
label= shift(close,-50)/close-1


"""
)

m24 = M.instruments.v2(
    start_date='2018-01-01',
    end_date='2019-01-01',
    market='CN_FUTURE',
    instrument_list="""RU8888.SHF
 """,
    max_count=0
)

m20 = M.cached.v3(
    input_1=m24.data,
    run=m20_run_bigquant_run,
    post_run=m20_post_run_bigquant_run,
    input_ports='',
    params='{}',
    output_ports=''
)

m12 = M.derived_feature_extractor.v3(
    input_data=m20.data_1,
    features=m8.data,
    date_col='date',
    instrument_col='instrument',
    drop_na=False,
    remove_extra_columns=False,
    user_functions={}
)

m21 = M.filter.v3(
    input_data=m12.data,
    expr='date<\'2020-06-01\'',
    output_left_data=False
)

m13 = M.dropnan.v1(
    input_data=m21.data
)

m16 = M.filter.v3(
    input_data=m12.data,
    expr='date<\'2020-06-01\'',
    output_left_data=False
)

m28 = M.dropnan.v1(
    input_data=m16.data
)

m15 = M.input_features.v1(
    features="""mean(open_intl,5)/amount
mean(open_intl,10)/amount
mean(open_intl,20)/amount
mean(open_intl,30)/amount
mean(open_intl,50)/amount
close/shift(close,30)
close/shift(close,20)
close/shift(close,10)
close/shift(close,5)
mean(amount,30)
max(high,10)/close 
open_intl
sum(open_intl,10)/amount 
amount/open_intl
mean(amount/open_intl,5)
mean(amount/open_intl,10)
mean(open_intl,5)"""
)

m17 = M.dl_convert_to_bin.v2(
    input_data=m13.data,
    features=m15.data,
    window_size=50,
    feature_clip=5,
    flatten=False,
    window_along_col='instrument'
)

m14 = M.dl_convert_to_bin.v2(
    input_data=m28.data,
    features=m15.data,
    window_size=50,
    feature_clip=5,
    flatten=False,
    window_along_col='instrument'
)

m6 = M.dl_model_train.v1(
    input_model=m5.data,
    training_data=m14.data,
    optimizer='RMSprop',
    loss='mean_squared_error',
    metrics='mse',
    batch_size=256,
    epochs=5,
    n_gpus=0,
    verbose='1:输出进度条记录'
)

m7 = M.dl_model_predict.v1(
    trained_model=m6.data,
    input_data=m17.data,
    batch_size=128,
    n_gpus=0,
    verbose='0:不显示'
)

m2 = M.cached.v3(
    input_1=m7.data,
    input_2=m13.data,
    run=m2_run_bigquant_run,
    post_run=m2_post_run_bigquant_run,
    input_ports='',
    params='{}',
    output_ports=''
)

m18 = M.instruments.v2(
    start_date='2019-01-01',
    end_date='2020-04-20',
    market='CN_FUTURE',
    instrument_list="""RU8888.SHF
 """,
    max_count=0
)

m1 = M.trade.v4(
    instruments=m18.data,
    options_data=m2.data_1,
    start_date='',
    end_date='',
    initialize=m1_initialize_bigquant_run,
    handle_data=m1_handle_data_bigquant_run,
    prepare=m1_prepare_bigquant_run,
    before_trading_start=m1_before_trading_start_bigquant_run,
    volume_limit=0,
    order_price_field_buy='open',
    order_price_field_sell='open',
    capital_base=200000,
    auto_cancel_non_tradable_orders=True,
    data_frequency='minute',
    price_type='真实价格',
    product_type='期货',
    plot_charts=True,
    backtest_only=False,
    benchmark=''
)

[2020-04-20 10:49:32.998445] INFO: moduleinvoker: dl_layer_input.v1 运行完成[0.002618s].

[2020-04-20 10:49:33.179227] INFO: moduleinvoker: dl_layer_lstm.v1 运行完成[0.179064s].

[2020-04-20 10:49:33.204005] INFO: moduleinvoker: dl_layer_dropout.v1 运行完成[0.02297s].

[2020-04-20 10:49:33.626300] INFO: moduleinvoker: dl_layer_lstm.v1 运行完成[0.420677s].

[2020-04-20 10:49:33.653124] INFO: moduleinvoker: dl_layer_dropout.v1 运行完成[0.024371s].

[2020-04-20 10:49:33.673971] INFO: moduleinvoker: dl_layer_dense.v1 运行完成[0.018129s].

[2020-04-20 10:49:33.696085] INFO: moduleinvoker: cached.v3 开始运行..

[2020-04-20 10:49:34.105818] INFO: moduleinvoker: cached.v3 运行完成[0.409721s].

[2020-04-20 10:49:34.107160] INFO: moduleinvoker: dl_model_init.v1 运行完成[0.429197s].

[2020-04-20 10:49:34.108948] INFO: moduleinvoker: input_features.v1 开始运行..

[2020-04-20 10:49:34.117178] INFO: moduleinvoker: 命中缓存

[2020-04-20 10:49:34.118287] INFO: moduleinvoker: input_features.v1 运行完成[0.009323s].

[2020-04-20 10:49:34.120474] INFO: moduleinvoker: instruments.v2 开始运行..

[2020-04-20 10:49:34.125393] INFO: moduleinvoker: 命中缓存

[2020-04-20 10:49:34.127190] INFO: moduleinvoker: instruments.v2 运行完成[0.006696s].

[2020-04-20 10:49:34.130712] INFO: moduleinvoker: cached.v3 开始运行..

[2020-04-20 10:49:34.135659] INFO: moduleinvoker: 命中缓存

[2020-04-20 10:49:34.137182] INFO: moduleinvoker: cached.v3 运行完成[0.006469s].

[2020-04-20 10:49:34.140137] INFO: moduleinvoker: derived_feature_extractor.v3 开始运行..

[2020-04-20 10:49:34.144815] INFO: moduleinvoker: 命中缓存

[2020-04-20 10:49:34.145859] INFO: moduleinvoker: derived_feature_extractor.v3 运行完成[0.00573s].

[2020-04-20 10:49:34.148649] INFO: moduleinvoker: filter.v3 开始运行..

[2020-04-20 10:49:34.153098] INFO: moduleinvoker: 命中缓存

[2020-04-20 10:49:34.154825] INFO: moduleinvoker: filter.v3 运行完成[0.006167s].

[2020-04-20 10:49:34.157304] INFO: moduleinvoker: dropnan.v1 开始运行..

[2020-04-20 10:49:34.283070] ERROR: moduleinvoker: module name: dropnan, module version: v1, trackeback: Traceback (most recent call last):
Exception: no data left after dropnan

[2020-04-20 10:49:33.000614] WARNING tensorflow: `implementation=0` has been deprecated, and now defaults to `implementation=1`.Please update your layer call.

[2020-04-20 10:49:33.206084] WARNING tensorflow: `implementation=0` has been deprecated, and now defaults to `implementation=1`.Please update your layer call.

---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
<ipython-input-10-8080892f71eb> in <module>()
    241 
    242 m13 = M.dropnan.v1(
--> 243     input_data=m21.data
    244 )
    245 

Exception: no data left after dropnan

bigrzz · 2020-04-20 10:35:43 UTC

特征列表中表达式有错误，因子计算详见
https://bigquant.com/docs/data.html#A股预计算因子

developer · 2020-08-26 06:34:20 UTC

你倒是说说哪里不对啊