可视化策略—LSTM大盘择时+Stockranker选股

策略分享
标签: #<Tag:0x00007fcc1150a048>

(达达) #1
克隆策略

    {"Description":"实验创建于2017/8/26","Summary":"","Graph":{"EdgesInternal":[{"DestinationInputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-15:instruments","SourceOutputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-8:data"},{"DestinationInputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-29:instruments","SourceOutputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-8:data"},{"DestinationInputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-53:data1","SourceOutputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-15:data"},{"DestinationInputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-76:features","SourceOutputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-24:data"},{"DestinationInputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-43:features","SourceOutputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-24:data"},{"DestinationInputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-70:features","SourceOutputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-24:data"},{"DestinationInputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-29:features","SourceOutputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-24:data"},{"DestinationInputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-35:features","SourceOutputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-24:data"},{"DestinationInputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-35:input_data","SourceOutputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-29:data"},{"DestinationInputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-53:data2","SourceOutputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-35:data"},{"DestinationInputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-60:model","SourceOutputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-43:model"},{"DestinationInputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-84:input_data","SourceOutputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-53:data"},{"DestinationInputPortId":"-293:options_data","SourceOutputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-60:predictions"},{"DestinationInputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-70:instruments","SourceOutputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-62:data"},{"DestinationInputPortId":"-293:instruments","SourceOutputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-62:data"},{"DestinationInputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-76:input_data","SourceOutputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-70:data"},{"DestinationInputPortId":"-86:input_data","SourceOutputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-76:data"},{"DestinationInputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-43:training_ds","SourceOutputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-84:data"},{"DestinationInputPortId":"287d2cb0-f53c-4101-bdf8-104b137c8601-60:data","SourceOutputPortId":"-86:data"}],"ModuleNodes":[{"Id":"287d2cb0-f53c-4101-bdf8-104b137c8601-8","ModuleId":"BigQuantSpace.instruments.instruments-v2","ModuleParameters":[{"Name":"start_date","Value":"2017-06-01","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"end_date","Value":"2017-07-01","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"market","Value":"CN_STOCK_A","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"instrument_list","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"max_count","Value":"0","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"rolling_conf","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-8"}],"OutputPortsInternal":[{"Name":"data","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-8","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":1,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"287d2cb0-f53c-4101-bdf8-104b137c8601-15","ModuleId":"BigQuantSpace.advanced_auto_labeler.advanced_auto_labeler-v2","ModuleParameters":[{"Name":"label_expr","Value":"# #号开始的表示注释\n# 0. 每行一个,顺序执行,从第二个开始,可以使用label字段\n# 1. 可用数据字段见 https://bigquant.com/docs/data_history_data.html\n# 添加benchmark_前缀,可使用对应的benchmark数据\n# 2. 可用操作符和函数见 `表达式引擎 <https://bigquant.com/docs/big_expr.html>`_\n\n# 计算收益:5日收盘价(作为卖出价格)除以明日开盘价(作为买入价格)\nshift(close, -5) / shift(open, -1)\n\n# 极值处理:用1%和99%分位的值做clip\nclip(label, all_quantile(label, 0.01), all_quantile(label, 0.99))\n\n# 将分数映射到分类,这里使用20个分类\nall_wbins(label, 20)\n\n# 过滤掉一字涨停的情况 (设置label为NaN,在后续处理和训练中会忽略NaN的label)\nwhere(shift(high, -1) == shift(low, -1), NaN, label)\n","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"start_date","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"end_date","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"benchmark","Value":"000300.SHA","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"drop_na_label","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"cast_label_int","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_functions","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"instruments","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-15"}],"OutputPortsInternal":[{"Name":"data","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-15","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":2,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"287d2cb0-f53c-4101-bdf8-104b137c8601-24","ModuleId":"BigQuantSpace.input_features.input_features-v1","ModuleParameters":[{"Name":"features","Value":"# #号开始的表示注释\n# 多个特征,每行一个,可以包含基础特征和衍生特征\nrank_avg_amount_5\nrank_avg_turn_5\nrank_volatility_5_0\nrank_swing_volatility_5_0\nrank_avg_mf_net_amount_5","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"features_ds","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-24"}],"OutputPortsInternal":[{"Name":"data","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-24","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":3,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"287d2cb0-f53c-4101-bdf8-104b137c8601-29","ModuleId":"BigQuantSpace.general_feature_extractor.general_feature_extractor-v6","ModuleParameters":[{"Name":"start_date","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"end_date","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"before_start_days","Value":"300","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"instruments","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-29"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"features","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-29"}],"OutputPortsInternal":[{"Name":"data","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-29","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":4,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"287d2cb0-f53c-4101-bdf8-104b137c8601-35","ModuleId":"BigQuantSpace.derived_feature_extractor.derived_feature_extractor-v2","ModuleParameters":[{"Name":"date_col","Value":"date","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"instrument_col","Value":"instrument","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_functions","Value":"{}","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_data","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-35"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"features","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-35"}],"OutputPortsInternal":[{"Name":"data","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-35","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":5,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"287d2cb0-f53c-4101-bdf8-104b137c8601-43","ModuleId":"BigQuantSpace.stock_ranker_train.stock_ranker_train-v5","ModuleParameters":[{"Name":"learning_algorithm","Value":"排序","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"number_of_leaves","Value":30,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"minimum_docs_per_leaf","Value":1000,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"number_of_trees","Value":20,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"learning_rate","Value":0.1,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"max_bins","Value":1023,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"feature_fraction","Value":1,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"m_lazy_run","Value":"False","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"training_ds","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-43"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"features","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-43"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"test_ds","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-43"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"base_model","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-43"}],"OutputPortsInternal":[{"Name":"model","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-43","OutputType":null},{"Name":"feature_gains","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-43","OutputType":null},{"Name":"m_lazy_run","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-43","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":6,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"287d2cb0-f53c-4101-bdf8-104b137c8601-53","ModuleId":"BigQuantSpace.join.join-v3","ModuleParameters":[{"Name":"on","Value":"date,instrument","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"how","Value":"inner","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"sort","Value":"False","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"data1","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-53"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"data2","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-53"}],"OutputPortsInternal":[{"Name":"data","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-53","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":7,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"287d2cb0-f53c-4101-bdf8-104b137c8601-60","ModuleId":"BigQuantSpace.stock_ranker_predict.stock_ranker_predict-v5","ModuleParameters":[{"Name":"m_lazy_run","Value":"False","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"model","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-60"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"data","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-60"}],"OutputPortsInternal":[{"Name":"predictions","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-60","OutputType":null},{"Name":"m_lazy_run","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-60","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":8,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"287d2cb0-f53c-4101-bdf8-104b137c8601-62","ModuleId":"BigQuantSpace.instruments.instruments-v2","ModuleParameters":[{"Name":"start_date","Value":"2017-07-01","ValueType":"Literal","LinkedGlobalParameter":"交易日期"},{"Name":"end_date","Value":"2018-04-30","ValueType":"Literal","LinkedGlobalParameter":"交易日期"},{"Name":"market","Value":"CN_STOCK_A","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"instrument_list","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"max_count","Value":"0","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"rolling_conf","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-62"}],"OutputPortsInternal":[{"Name":"data","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-62","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":9,"IsPartOfPartialRun":null,"Comment":"预测数据,用于回测和模拟","CommentCollapsed":false},{"Id":"287d2cb0-f53c-4101-bdf8-104b137c8601-70","ModuleId":"BigQuantSpace.general_feature_extractor.general_feature_extractor-v6","ModuleParameters":[{"Name":"start_date","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"end_date","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"before_start_days","Value":"300","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"instruments","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-70"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"features","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-70"}],"OutputPortsInternal":[{"Name":"data","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-70","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":10,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"287d2cb0-f53c-4101-bdf8-104b137c8601-76","ModuleId":"BigQuantSpace.derived_feature_extractor.derived_feature_extractor-v2","ModuleParameters":[{"Name":"date_col","Value":"date","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"instrument_col","Value":"instrument","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"user_functions","Value":"","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_data","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-76"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"features","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-76"}],"OutputPortsInternal":[{"Name":"data","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-76","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":11,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"287d2cb0-f53c-4101-bdf8-104b137c8601-84","ModuleId":"BigQuantSpace.dropnan.dropnan-v1","ModuleParameters":[],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_data","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-84"}],"OutputPortsInternal":[{"Name":"data","NodeId":"287d2cb0-f53c-4101-bdf8-104b137c8601-84","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":12,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-86","ModuleId":"BigQuantSpace.dropnan.dropnan-v1","ModuleParameters":[],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"input_data","NodeId":"-86"}],"OutputPortsInternal":[{"Name":"data","NodeId":"-86","OutputType":null}],"UsePreviousResults":true,"moduleIdForCode":13,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true},{"Id":"-293","ModuleId":"BigQuantSpace.trade.trade-v3","ModuleParameters":[{"Name":"start_date","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"end_date","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"handle_data","Value":"# 回测引擎:每日数据处理函数,每天执行一次\ndef bigquant_run(context, data):\n # 按日期过滤得到今日的预测数据\n ranker_prediction = context.ranker_prediction[\n context.ranker_prediction.date == data.current_dt.strftime('%Y-%m-%d')]\n\n # 1. 资金分配\n # 平均持仓时间是hold_days,每日都将买入股票,每日预期使用 1/hold_days 的资金\n # 实际操作中,会存在一定的买入误差,所以在前hold_days天,等量使用资金;之后,尽量使用剩余资金(这里设置最多用等量的1.5倍)\n is_staging = context.trading_day_index < context.hold_days # 是否在建仓期间(前 hold_days 天)\n cash_avg = context.portfolio.portfolio_value / context.hold_days\n cash_for_buy = min(context.portfolio.cash, (1 if is_staging else 1.5) * cash_avg)\n cash_for_sell = cash_avg - (context.portfolio.cash - cash_for_buy)\n positions = {e.symbol: p.amount * p.last_sale_price\n for e, p in context.perf_tracker.position_tracker.positions.items()}\n\n # 2. 生成卖出订单:hold_days天之后才开始卖出;对持仓的股票,按StockRanker预测的排序末位淘汰\n if not is_staging and cash_for_sell > 0:\n equities = {e.symbol: e for e, p in context.perf_tracker.position_tracker.positions.items()}\n instruments = list(reversed(list(ranker_prediction.instrument[ranker_prediction.instrument.apply(\n lambda x: x in equities and not context.has_unfinished_sell_order(equities[x]))])))\n # print('rank order for sell %s' % instruments)\n for instrument in instruments:\n context.order_target(context.symbol(instrument), 0)\n cash_for_sell -= positions[instrument]\n if cash_for_sell <= 0:\n break\n #---------------------START:大盘风控(含建仓期)--------------------------\n today_date = data.current_dt.strftime('%Y-%m-%d')\n positions_all = [equity.symbol for equity in context.portfolio.positions]\n #ds_id=context.predataid\n #today_prediction = DataSource(id=ds_id).read_pickle()\n dataprediction=context.dataprediction\n today_prediction=dataprediction[dataprediction.date==today_date].direction.values[0]\n #print(today_date,'预测仓位',today_prediction)\n # 满足空仓条件\n if today_prediction<0:\t\n if len(positions_all)>0:\n # 全部卖出后返回\n for i in positions_all:\n if data.can_trade(context.symbol(i)):\n context.order_target_percent(context.symbol(i), 0)\n print('风控执行',today_date)\n return\n #运行风控后当日结束,不再执行后续的买卖订单\n #------------------------END:大盘风控(含建仓期)---------------------------\n \n # 3. 生成买入订单:按StockRanker预测的排序,买入前面的stock_count只股票\n buy_cash_weights = context.stock_weights\n buy_instruments = list(ranker_prediction.instrument[:len(buy_cash_weights)])\n max_cash_per_instrument = context.portfolio.portfolio_value * context.max_cash_per_instrument\n for i, instrument in enumerate(buy_instruments):\n cash = cash_for_buy * buy_cash_weights[i]\n if cash > max_cash_per_instrument - positions.get(instrument, 0):\n # 确保股票持仓量不会超过每次股票最大的占用资金量\n cash = max_cash_per_instrument - positions.get(instrument, 0)\n if cash > 0:\n context.order_value(context.symbol(instrument), cash)\n","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"prepare","Value":"# 回测引擎:准备数据,只执行一次\ndef bigquant_run(context):\n seq_len=5 #每个input的长度\n # 导入包\n from keras.layers.core import Dense, Activation, Dropout\n from keras.layers.recurrent import LSTM\n from keras.models import Sequential\n from keras import optimizers\n import tensorflow as tf \n from sklearn.preprocessing import scale\n from keras.layers import Input, Dense, LSTM, merge\n from keras.models import Model\n # 基础参数配置\n instrument = '000300.SHA' #股票代码\n #设置用于训练和回测的开始/结束日期\n train_length=seq_len*10\n start_date_temp= (pd.to_datetime(context.start_date) - datetime.timedelta(days=2*train_length)).strftime('%Y-%m-%d') # 多取几天的数据,这里取5倍\n len1=len(D.trading_days(start_date=start_date_temp, end_date=context.end_date)) \n len2=len(D.trading_days(start_date=context.start_date, end_date=context.end_date))\n distance=len1-len2\n trade_day=D.trading_days(start_date=start_date_temp, end_date=context.end_date)\n start_date = trade_day.iloc[distance-train_length][0].strftime('%Y-%m-%d')\n split_date = trade_day.iloc[distance-1][0].strftime('%Y-%m-%d')\n #print('start_date',start_date,'split_date',split_date)\n fields = ['close', 'open', 'high', 'low', 'amount', 'volume'] # features因子\n batch = 100#整数,指定进行梯度下降时每个batch包含的样本数,训练时一个batch的样本会被计算一次梯度下降,使目标函数优化一步\n \n # 数据导入以及初步处理\n data1 = D.history_data(instrument, start_date, context.end_date, fields)\n data1['return'] = data1['close'].shift(-5) / data1['open'].shift(-1) - 1 #计算未来5日收益率(未来第五日的收盘价/明日的开盘价)\n data1=data1[data1.amount>0]\n datatime = data1['date'][data1.date>split_date] #记录predictions的时间,回测要用\n data1['return'] = data1['return']#.apply(lambda x:np.where(x>=0.2,0.2,np.where(x>-0.2,x,-0.2))) #去极值\n data1['return'] = data1['return']*10 # 适当增大return范围,利于LSTM模型训练\n data1.reset_index(drop=True, inplace=True)\n scaledata = data1[fields]\n traindata = data1[data1.date<=split_date]\n \n # 数据处理:设定每个input(series×6features)以及数据标准化\n train_input = []\n train_output = []\n test_input = []\n for i in range(seq_len-1, len(traindata)):\n a = scale(scaledata[i+1-seq_len:i+1])\n train_input.append(a)\n c = data1['return'][i]\n train_output.append(c)\n for j in range(len(traindata), len(data1)):\n b = scale(scaledata[j+1-seq_len:j+1])\n test_input.append(b)\n\n\n # LSTM接受数组类型的输入\n train_x = np.array(train_input)\n train_y = np.array(train_output)\n test_x = np.array(test_input) \n\n # 自定义激活函数\n import tensorflow as tf\n def atan(x): \n return tf.atan(x)\n # 构建神经网络层 1层LSTM层+3层Dense层\n # 用于1个输入情况\n lstm_input = Input(shape=(seq_len,len(fields)), name='lstm_input')\n lstm_output = LSTM(32, activation=atan, dropout_W=0.2, dropout_U=0.1)(lstm_input)\n Dense_output_1 = Dense(16, activation='linear')(lstm_output)\n Dense_output_2 = Dense(4, activation='linear')(Dense_output_1)\n predictions = Dense(1, activation=atan)(Dense_output_2)\n model = Model(input=lstm_input, output=predictions)\n model.compile(optimizer='adam', loss='mse', metrics=['mse'])\n model.fit(train_x, train_y, batch_size=batch, nb_epoch=5, verbose=0)\n # 预测\n predictions = model.predict(test_x)\n # 如果预测值>0,取为1;如果预测值<=0,取为-1.为回测做准备\n for i in range(len(predictions)):\n if predictions[i]>0:\n predictions[i]=1\n elif predictions[i]<=0:\n predictions[i]=-1\n \n # 将预测值与时间整合作为回测数据\n cc = np.reshape(predictions,len(predictions), 1)\n dataprediction = pd.DataFrame()\n dataprediction['date'] = datatime\n dataprediction['direction']=np.round(cc)\n context.dataprediction=dataprediction\n","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"initialize","Value":"# 回测引擎:初始化函数,只执行一次\ndef bigquant_run(context):\n # 加载预测数据\n context.ranker_prediction = context.options['data'].read_df()\n\n # 系统已经设置了默认的交易手续费和滑点,要修改手续费可使用如下函数\n context.set_commission(PerOrder(buy_cost=0.0003, sell_cost=0.0013, min_cost=5))\n # 预测数据,通过options传入进来,使用 read_df 函数,加载到内存 (DataFrame)\n # 设置买入的股票数量,这里买入预测股票列表排名靠前的5只\n stock_count = 5\n # 每只的股票的权重,如下的权重分配会使得靠前的股票分配多一点的资金,[0.339160, 0.213986, 0.169580, ..]\n context.stock_weights = T.norm([1 / math.log(i + 2) for i in range(0, stock_count)])\n # 设置每只股票占用的最大资金比例\n context.max_cash_per_instrument = 0.2\n context.hold_days = 5\n","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"before_trading_start","Value":"","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"volume_limit","Value":0.025,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"order_price_field_buy","Value":"open","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"order_price_field_sell","Value":"close","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"capital_base","Value":1000000,"ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"benchmark","Value":"000300.SHA","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"auto_cancel_non_tradable_orders","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"data_frequency","Value":"daily","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"price_type","Value":"后复权","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"plot_charts","Value":"True","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"backtest_only","Value":"False","ValueType":"Literal","LinkedGlobalParameter":null},{"Name":"amount_integer","Value":"False","ValueType":"Literal","LinkedGlobalParameter":null}],"InputPortsInternal":[{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"instruments","NodeId":"-293"},{"DataSourceId":null,"TrainedModelId":null,"TransformModuleId":null,"Name":"options_data","NodeId":"-293"}],"OutputPortsInternal":[{"Name":"raw_perf","NodeId":"-293","OutputType":null}],"UsePreviousResults":false,"moduleIdForCode":14,"IsPartOfPartialRun":null,"Comment":"","CommentCollapsed":true}],"SerializedClientData":"<?xml version='1.0' encoding='utf-16'?><DataV1 xmlns:xsd='http://www.w3.org/2001/XMLSchema' xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'><Meta /><NodePositions><NodePosition Node='287d2cb0-f53c-4101-bdf8-104b137c8601-8' Position='211,64,200,200'/><NodePosition Node='287d2cb0-f53c-4101-bdf8-104b137c8601-15' Position='70,183,200,200'/><NodePosition Node='287d2cb0-f53c-4101-bdf8-104b137c8601-24' Position='728,-14,200,200'/><NodePosition Node='287d2cb0-f53c-4101-bdf8-104b137c8601-29' Position='381,188,200,200'/><NodePosition Node='287d2cb0-f53c-4101-bdf8-104b137c8601-35' Position='381,283,200,200'/><NodePosition Node='287d2cb0-f53c-4101-bdf8-104b137c8601-43' Position='638,561,200,200'/><NodePosition Node='287d2cb0-f53c-4101-bdf8-104b137c8601-53' Position='236,363,200,200'/><NodePosition Node='287d2cb0-f53c-4101-bdf8-104b137c8601-60' Position='904,647,200,200'/><NodePosition Node='287d2cb0-f53c-4101-bdf8-104b137c8601-62' Position='1074,124,200,200'/><NodePosition Node='287d2cb0-f53c-4101-bdf8-104b137c8601-70' Position='1078,233,200,200'/><NodePosition Node='287d2cb0-f53c-4101-bdf8-104b137c8601-76' Position='1081,324,200,200'/><NodePosition Node='287d2cb0-f53c-4101-bdf8-104b137c8601-84' Position='406,496,200,200'/><NodePosition Node='-86' Position='1078,418,200,200'/><NodePosition Node='-293' Position='1131,770,200,200'/></NodePositions><NodeGroups /></DataV1>"},"IsDraft":true,"ParentExperimentId":null,"WebService":{"IsWebServiceExperiment":false,"Inputs":[],"Outputs":[],"Parameters":[{"Name":"交易日期","Value":"","ParameterDefinition":{"Name":"交易日期","FriendlyName":"交易日期","DefaultValue":"","ParameterType":"String","HasDefaultValue":true,"IsOptional":true,"ParameterRules":[],"HasRules":false,"MarkupType":0,"CredentialDescriptor":null}}],"WebServiceGroupId":null,"SerializedClientData":"<?xml version='1.0' encoding='utf-16'?><DataV1 xmlns:xsd='http://www.w3.org/2001/XMLSchema' xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'><Meta /><NodePositions></NodePositions><NodeGroups /></DataV1>"},"DisableNodesUpdate":false,"Category":"user","Tags":[],"IsPartialRun":true}
    In [38]:
    # 本代码由可视化策略环境自动生成 2018年5月14日 21:10
    # 本代码单元只能在可视化模式下编辑。您也可以拷贝代码,粘贴到新建的代码单元或者策略,然后修改。
    
    
    m1 = M.instruments.v2(
        start_date='2017-06-01',
        end_date='2017-07-01',
        market='CN_STOCK_A',
        instrument_list='',
        max_count=0
    )
    
    m2 = M.advanced_auto_labeler.v2(
        instruments=m1.data,
        label_expr="""# #号开始的表示注释
    # 0. 每行一个,顺序执行,从第二个开始,可以使用label字段
    # 1. 可用数据字段见 https://bigquant.com/docs/data_history_data.html
    #   添加benchmark_前缀,可使用对应的benchmark数据
    # 2. 可用操作符和函数见 `表达式引擎 <https://bigquant.com/docs/big_expr.html>`_
    
    # 计算收益:5日收盘价(作为卖出价格)除以明日开盘价(作为买入价格)
    shift(close, -5) / shift(open, -1)
    
    # 极值处理:用1%和99%分位的值做clip
    clip(label, all_quantile(label, 0.01), all_quantile(label, 0.99))
    
    # 将分数映射到分类,这里使用20个分类
    all_wbins(label, 20)
    
    # 过滤掉一字涨停的情况 (设置label为NaN,在后续处理和训练中会忽略NaN的label)
    where(shift(high, -1) == shift(low, -1), NaN, label)
    """,
        start_date='',
        end_date='',
        benchmark='000300.SHA',
        drop_na_label=True,
        cast_label_int=True
    )
    
    m3 = M.input_features.v1(
        features="""# #号开始的表示注释
    # 多个特征,每行一个,可以包含基础特征和衍生特征
    rank_avg_amount_5
    rank_avg_turn_5
    rank_volatility_5_0
    rank_swing_volatility_5_0
    rank_avg_mf_net_amount_5"""
    )
    
    m4 = M.general_feature_extractor.v6(
        instruments=m1.data,
        features=m3.data,
        start_date='',
        end_date='',
        before_start_days=300
    )
    
    m5 = M.derived_feature_extractor.v2(
        input_data=m4.data,
        features=m3.data,
        date_col='date',
        instrument_col='instrument',
        user_functions={}
    )
    
    m7 = M.join.v3(
        data1=m2.data,
        data2=m5.data,
        on='date,instrument',
        how='inner',
        sort=False
    )
    
    m12 = M.dropnan.v1(
        input_data=m7.data
    )
    
    m6 = M.stock_ranker_train.v5(
        training_ds=m12.data,
        features=m3.data,
        learning_algorithm='排序',
        number_of_leaves=30,
        minimum_docs_per_leaf=1000,
        number_of_trees=20,
        learning_rate=0.1,
        max_bins=1023,
        feature_fraction=1,
        m_lazy_run=False
    )
    
    m9 = M.instruments.v2(
        start_date=T.live_run_param('trading_date', '2017-07-01'),
        end_date=T.live_run_param('trading_date', '2018-04-30'),
        market='CN_STOCK_A',
        instrument_list='',
        max_count=0
    )
    
    m10 = M.general_feature_extractor.v6(
        instruments=m9.data,
        features=m3.data,
        start_date='',
        end_date='',
        before_start_days=300
    )
    
    m11 = M.derived_feature_extractor.v2(
        input_data=m10.data,
        features=m3.data,
        date_col='date',
        instrument_col='instrument'
    )
    
    m13 = M.dropnan.v1(
        input_data=m11.data
    )
    
    m8 = M.stock_ranker_predict.v5(
        model=m6.model,
        data=m13.data,
        m_lazy_run=False
    )
    
    # 回测引擎:每日数据处理函数,每天执行一次
    def m14_handle_data_bigquant_run(context, data):
        # 按日期过滤得到今日的预测数据
        ranker_prediction = context.ranker_prediction[
            context.ranker_prediction.date == data.current_dt.strftime('%Y-%m-%d')]
    
        # 1. 资金分配
        # 平均持仓时间是hold_days,每日都将买入股票,每日预期使用 1/hold_days 的资金
        # 实际操作中,会存在一定的买入误差,所以在前hold_days天,等量使用资金;之后,尽量使用剩余资金(这里设置最多用等量的1.5倍)
        is_staging = context.trading_day_index < context.hold_days # 是否在建仓期间(前 hold_days 天)
        cash_avg = context.portfolio.portfolio_value / context.hold_days
        cash_for_buy = min(context.portfolio.cash, (1 if is_staging else 1.5) * cash_avg)
        cash_for_sell = cash_avg - (context.portfolio.cash - cash_for_buy)
        positions = {e.symbol: p.amount * p.last_sale_price
                     for e, p in context.perf_tracker.position_tracker.positions.items()}
    
        # 2. 生成卖出订单:hold_days天之后才开始卖出;对持仓的股票,按StockRanker预测的排序末位淘汰
        if not is_staging and cash_for_sell > 0:
            equities = {e.symbol: e for e, p in context.perf_tracker.position_tracker.positions.items()}
            instruments = list(reversed(list(ranker_prediction.instrument[ranker_prediction.instrument.apply(
                    lambda x: x in equities and not context.has_unfinished_sell_order(equities[x]))])))
            # print('rank order for sell %s' % instruments)
            for instrument in instruments:
                context.order_target(context.symbol(instrument), 0)
                cash_for_sell -= positions[instrument]
                if cash_for_sell <= 0:
                    break
        #---------------------START:大盘风控(含建仓期)--------------------------
        today_date = data.current_dt.strftime('%Y-%m-%d')
        positions_all = [equity.symbol for equity in context.portfolio.positions]
        #ds_id=context.predataid
        #today_prediction = DataSource(id=ds_id).read_pickle()
        dataprediction=context.dataprediction
        today_prediction=dataprediction[dataprediction.date==today_date].direction.values[0]
        #print(today_date,'预测仓位',today_prediction)
        # 满足空仓条件
        if today_prediction<0:	
            if len(positions_all)>0:
                # 全部卖出后返回
                for i in positions_all:
                    if data.can_trade(context.symbol(i)):
                        context.order_target_percent(context.symbol(i), 0)
                        print('风控执行',today_date)
                        return
                    #运行风控后当日结束,不再执行后续的买卖订单
        #------------------------END:大盘风控(含建仓期)---------------------------
        
        # 3. 生成买入订单:按StockRanker预测的排序,买入前面的stock_count只股票
        buy_cash_weights = context.stock_weights
        buy_instruments = list(ranker_prediction.instrument[:len(buy_cash_weights)])
        max_cash_per_instrument = context.portfolio.portfolio_value * context.max_cash_per_instrument
        for i, instrument in enumerate(buy_instruments):
            cash = cash_for_buy * buy_cash_weights[i]
            if cash > max_cash_per_instrument - positions.get(instrument, 0):
                # 确保股票持仓量不会超过每次股票最大的占用资金量
                cash = max_cash_per_instrument - positions.get(instrument, 0)
            if cash > 0:
                context.order_value(context.symbol(instrument), cash)
    
    # 回测引擎:准备数据,只执行一次
    def m14_prepare_bigquant_run(context):
        seq_len=5    #每个input的长度
        # 导入包
        from keras.layers.core import Dense, Activation, Dropout
        from keras.layers.recurrent import LSTM
        from keras.models import Sequential
        from keras import optimizers
        import tensorflow as tf   
        from sklearn.preprocessing import scale
        from keras.layers import Input, Dense, LSTM, merge
        from keras.models import Model
        # 基础参数配置
        instrument = '000300.SHA'  #股票代码
        #设置用于训练和回测的开始/结束日期
        train_length=seq_len*10
        start_date_temp= (pd.to_datetime(context.start_date) - datetime.timedelta(days=2*train_length)).strftime('%Y-%m-%d') # 多取几天的数据,这里取5倍
        len1=len(D.trading_days(start_date=start_date_temp, end_date=context.end_date)) 
        len2=len(D.trading_days(start_date=context.start_date, end_date=context.end_date))
        distance=len1-len2
        trade_day=D.trading_days(start_date=start_date_temp, end_date=context.end_date)
        start_date = trade_day.iloc[distance-train_length][0].strftime('%Y-%m-%d')
        split_date = trade_day.iloc[distance-1][0].strftime('%Y-%m-%d')
        #print('start_date',start_date,'split_date',split_date)
        fields = ['close', 'open', 'high', 'low', 'amount', 'volume']  # features因子
        batch = 100#整数,指定进行梯度下降时每个batch包含的样本数,训练时一个batch的样本会被计算一次梯度下降,使目标函数优化一步
        
        # 数据导入以及初步处理
        data1 = D.history_data(instrument, start_date, context.end_date, fields)
        data1['return'] = data1['close'].shift(-5) / data1['open'].shift(-1) - 1 #计算未来5日收益率(未来第五日的收盘价/明日的开盘价)
        data1=data1[data1.amount>0]
        datatime = data1['date'][data1.date>split_date]  #记录predictions的时间,回测要用
        data1['return'] = data1['return']#.apply(lambda x:np.where(x>=0.2,0.2,np.where(x>-0.2,x,-0.2)))  #去极值
        data1['return'] = data1['return']*10  # 适当增大return范围,利于LSTM模型训练
        data1.reset_index(drop=True, inplace=True)
        scaledata = data1[fields]
        traindata = data1[data1.date<=split_date]
        
        # 数据处理:设定每个input(series×6features)以及数据标准化
        train_input = []
        train_output = []
        test_input = []
        for i in range(seq_len-1, len(traindata)):
            a = scale(scaledata[i+1-seq_len:i+1])
            train_input.append(a)
            c = data1['return'][i]
            train_output.append(c)
        for j in range(len(traindata), len(data1)):
            b = scale(scaledata[j+1-seq_len:j+1])
            test_input.append(b)
    
    
        # LSTM接受数组类型的输入
        train_x = np.array(train_input)
        train_y = np.array(train_output)
        test_x = np.array(test_input) 
    
        # 自定义激活函数
        import tensorflow as tf
        def atan(x): 
            return tf.atan(x)
        # 构建神经网络层 1层LSTM层+3层Dense层
        # 用于1个输入情况
        lstm_input = Input(shape=(seq_len,len(fields)), name='lstm_input')
        lstm_output = LSTM(32, activation=atan, dropout_W=0.2, dropout_U=0.1)(lstm_input)
        Dense_output_1 = Dense(16, activation='linear')(lstm_output)
        Dense_output_2 = Dense(4, activation='linear')(Dense_output_1)
        predictions = Dense(1, activation=atan)(Dense_output_2)
        model = Model(input=lstm_input, output=predictions)
        model.compile(optimizer='adam', loss='mse', metrics=['mse'])
        model.fit(train_x, train_y, batch_size=batch, nb_epoch=5, verbose=0)
        # 预测
        predictions = model.predict(test_x)
        # 如果预测值>0,取为1;如果预测值<=0,取为-1.为回测做准备
        for i in range(len(predictions)):
            if predictions[i]>0:
                predictions[i]=1
            elif predictions[i]<=0:
                predictions[i]=-1
                
        # 将预测值与时间整合作为回测数据
        cc = np.reshape(predictions,len(predictions), 1)
        dataprediction = pd.DataFrame()
        dataprediction['date'] = datatime
        dataprediction['direction']=np.round(cc)
        context.dataprediction=dataprediction
    
    # 回测引擎:初始化函数,只执行一次
    def m14_initialize_bigquant_run(context):
        # 加载预测数据
        context.ranker_prediction = context.options['data'].read_df()
    
        # 系统已经设置了默认的交易手续费和滑点,要修改手续费可使用如下函数
        context.set_commission(PerOrder(buy_cost=0.0003, sell_cost=0.0013, min_cost=5))
        # 预测数据,通过options传入进来,使用 read_df 函数,加载到内存 (DataFrame)
        # 设置买入的股票数量,这里买入预测股票列表排名靠前的5只
        stock_count = 5
        # 每只的股票的权重,如下的权重分配会使得靠前的股票分配多一点的资金,[0.339160, 0.213986, 0.169580, ..]
        context.stock_weights = T.norm([1 / math.log(i + 2) for i in range(0, stock_count)])
        # 设置每只股票占用的最大资金比例
        context.max_cash_per_instrument = 0.2
        context.hold_days = 5
    
    m14 = M.trade.v3(
        instruments=m9.data,
        options_data=m8.predictions,
        start_date='',
        end_date='',
        handle_data=m14_handle_data_bigquant_run,
        prepare=m14_prepare_bigquant_run,
        initialize=m14_initialize_bigquant_run,
        volume_limit=0.025,
        order_price_field_buy='open',
        order_price_field_sell='close',
        capital_base=1000000,
        benchmark='000300.SHA',
        auto_cancel_non_tradable_orders=True,
        data_frequency='daily',
        price_type='后复权',
        plot_charts=True,
        backtest_only=False,
        amount_integer=False
    )
    
    [2018-05-14 11:30:40.295471] INFO: bigquant: instruments.v2 开始运行..
    [2018-05-14 11:30:40.299067] INFO: bigquant: 命中缓存
    [2018-05-14 11:30:40.300291] INFO: bigquant: instruments.v2 运行完成[0.004858s].
    [2018-05-14 11:30:40.310936] INFO: bigquant: advanced_auto_labeler.v2 开始运行..
    [2018-05-14 11:30:40.314435] INFO: bigquant: 命中缓存
    [2018-05-14 11:30:40.315599] INFO: bigquant: advanced_auto_labeler.v2 运行完成[0.004688s].
    [2018-05-14 11:30:40.321521] INFO: bigquant: input_features.v1 开始运行..
    [2018-05-14 11:30:40.324884] INFO: bigquant: 命中缓存
    [2018-05-14 11:30:40.325884] INFO: bigquant: input_features.v1 运行完成[0.004384s].
    [2018-05-14 11:30:40.335499] INFO: bigquant: general_feature_extractor.v6 开始运行..
    [2018-05-14 11:30:40.338661] INFO: bigquant: 命中缓存
    [2018-05-14 11:30:40.340147] INFO: bigquant: general_feature_extractor.v6 运行完成[0.004654s].
    [2018-05-14 11:30:40.348528] INFO: bigquant: derived_feature_extractor.v2 开始运行..
    [2018-05-14 11:30:40.352010] INFO: bigquant: 命中缓存
    [2018-05-14 11:30:40.353078] INFO: bigquant: derived_feature_extractor.v2 运行完成[0.00459s].
    [2018-05-14 11:30:40.360846] INFO: bigquant: join.v3 开始运行..
    [2018-05-14 11:30:40.363821] INFO: bigquant: 命中缓存
    [2018-05-14 11:30:40.364803] INFO: bigquant: join.v3 运行完成[0.003969s].
    [2018-05-14 11:30:40.371971] INFO: bigquant: dropnan.v1 开始运行..
    [2018-05-14 11:30:40.375029] INFO: bigquant: 命中缓存
    [2018-05-14 11:30:40.376205] INFO: bigquant: dropnan.v1 运行完成[0.004237s].
    [2018-05-14 11:30:40.385533] INFO: bigquant: stock_ranker_train.v5 开始运行..
    [2018-05-14 11:30:40.388979] INFO: bigquant: 命中缓存
    [2018-05-14 11:30:40.389876] INFO: bigquant: stock_ranker_train.v5 运行完成[0.004365s].
    [2018-05-14 11:30:40.395393] INFO: bigquant: instruments.v2 开始运行..
    [2018-05-14 11:30:40.397878] INFO: bigquant: 命中缓存
    [2018-05-14 11:30:40.398725] INFO: bigquant: instruments.v2 运行完成[0.003346s].
    [2018-05-14 11:30:40.408261] INFO: bigquant: general_feature_extractor.v6 开始运行..
    [2018-05-14 11:30:40.410756] INFO: bigquant: 命中缓存
    [2018-05-14 11:30:40.411677] INFO: bigquant: general_feature_extractor.v6 运行完成[0.003436s].
    [2018-05-14 11:30:40.418692] INFO: bigquant: derived_feature_extractor.v2 开始运行..
    [2018-05-14 11:30:40.421185] INFO: bigquant: 命中缓存
    [2018-05-14 11:30:40.421969] INFO: bigquant: derived_feature_extractor.v2 运行完成[0.003284s].
    [2018-05-14 11:30:40.428724] INFO: bigquant: dropnan.v1 开始运行..
    [2018-05-14 11:30:40.431914] INFO: bigquant: 命中缓存
    [2018-05-14 11:30:40.432964] INFO: bigquant: dropnan.v1 运行完成[0.004248s].
    [2018-05-14 11:30:40.442477] INFO: bigquant: stock_ranker_predict.v5 开始运行..
    [2018-05-14 11:30:40.450106] INFO: bigquant: 命中缓存
    [2018-05-14 11:30:40.451247] INFO: bigquant: stock_ranker_predict.v5 运行完成[0.008773s].
    [2018-05-14 11:30:40.489990] INFO: bigquant: backtest.v7 开始运行..
    [2018-05-14 11:30:48.643585] INFO: algo: set price type:backward_adjusted
    风控执行 2017-07-04
    风控执行 2017-07-05
    风控执行 2017-07-06
    风控执行 2017-07-07
    风控执行 2017-07-10
    风控执行 2017-07-12
    风控执行 2017-07-13
    风控执行 2017-07-19
    风控执行 2017-07-20
    风控执行 2017-07-26
    风控执行 2017-07-27
    风控执行 2017-08-01
    风控执行 2017-08-11
    风控执行 2017-08-25
    风控执行 2017-08-28
    风控执行 2017-09-05
    风控执行 2017-09-12
    风控执行 2017-09-15
    风控执行 2017-09-18
    风控执行 2017-09-19
    风控执行 2017-09-29
    风控执行 2017-10-09
    风控执行 2017-10-25
    风控执行 2017-10-26
    风控执行 2017-11-08
    风控执行 2017-11-21
    风控执行 2017-11-22
    风控执行 2017-12-05
    风控执行 2017-12-15
    风控执行 2017-12-21
    风控执行 2017-12-25
    风控执行 2017-12-27
    风控执行 2017-12-28
    风控执行 2017-12-29
    风控执行 2018-01-02
    风控执行 2018-01-03
    风控执行 2018-01-09
    风控执行 2018-01-18
    风控执行 2018-01-19
    风控执行 2018-01-22
    风控执行 2018-02-01
    风控执行 2018-02-06
    风控执行 2018-02-07
    风控执行 2018-02-09
    风控执行 2018-03-07
    风控执行 2018-03-21
    风控执行 2018-03-22
    风控执行 2018-03-23
    风控执行 2018-03-26
    风控执行 2018-04-11
    风控执行 2018-04-16
    风控执行 2018-04-17
    风控执行 2018-04-18
    风控执行 2018-04-19
    风控执行 2018-04-24
    风控执行 2018-04-25
    [2018-05-14 11:31:14.660438] INFO: Performance: Simulated 202 trading days out of 202.
    [2018-05-14 11:31:14.661407] INFO: Performance: first open: 2017-07-03 01:30:00+00:00
    [2018-05-14 11:31:14.662371] INFO: Performance: last close: 2018-04-27 07:00:00+00:00
    
    • 收益率30.21%
    • 年化收益率39.0%
    • 基准收益率2.46%
    • 阿尔法0.36
    • 贝塔0.74
    • 夏普比率2.29
    • 胜率0.597
    • 盈亏比1.392
    • 收益波动率15.11%
    • 信息比率3.21
    • 最大回撤8.13%
    [2018-05-14 11:31:20.050883] INFO: bigquant: backtest.v7 运行完成[39.560856s].
    

    LSTM择时+StockRanker选股的可视化策略实现
    (NaturalAI) #2

    感觉这个如果加上对冲,alpha的结果还是可以的。增加个完全对冲的结果看看。


    (fsm) #3

    从下面代码,可以看出split_date 为end_date的日期

    split_date = trade_day.iloc[distance-1][0].strftime('%Y-%m-%d')
    
    

    但是以下代码说明,已经去除了最后5天的数据,也就是说预测值(或者说test_input为空)是空数据。不知道有没有理解错误

    data1['return'] = data1['close'].shift(-5) / data1['open'].shift(-1) - 1
    data1.reset_index(drop=True, inplace=True)
    traindata = data1[data1.date<=split_date]
    

    (达达) #4
    split_date = trade_day.iloc[distance-1][0].strftime('%Y-%m-%d')
    

    这段代码是说自然日期和交易日历不一样,D.history接口调用时中间可能有空值无法保证中间的训练数据长度是准确的train_length天,通过交易日期的查询精确控制一下 trade_day.iloc[distance-1][0]这个位置的数据到现在的天数来分割时间,以精确保证训练集是train_length天


    (fsm) #5

    但是还是要去除最后(最新)一次未来5天数据,你的split_date为离end_date
    只有1天,在drop后,明显所有的日期都小于split_date,也就是说

    traindata = data1[data1.date<=split_date]
    

    不起作用了len(traindata)==len(data1)


    (boris54) #6

    我觉得没错


    (qci133) #7

    有点奇怪,本策略复制后不做任何修改,重新跑一遍发现年华收益是-1%. 是否发表的程序中有什么参数写错了,跟贴出来的回测结果对不上呀


    (达达) #8

    跑了一下如下图所示,这个帖子比较久了,最近有些模块更新可能会导致数据有些变化


    (qci133) #10

    感谢回复。
    但我尚有疑问:即使是你的两次回测的结果,差别也挺大的。年化收益从39.0%变成了12.38%. 如果理解成是模块升级导致的,那反而令人不安:毕竟有些人已经在用策略指导实盘交易了,如果策略对模块升级这么敏感,那就很难放心使用了;另外,模块升级按说应该有回归测试,不应该会导致数据与效果层面的大的diff.(除非是 bugfix——而会带来效果显著变化的bugfix,我想是不是应该给出changelog,并列出可能影响的上层模块,这样用户可以针对性地重新测试自己的策略。)

    我想是不是还有一种可能性:多次回测时,模型训练中的一些随机性(比如 dropout 什么的)带来了结果的显著变化?若是这样,倒是能安心一些。小建议:在这种情况下,给出的样例程序中固定好随机种子,使得结果能复现,可能比较好。


    (小Q) #11

    两次回测差异的原因有很多,比如模型训练的随机性、样本数据不一样、个别模块细节改动、模型参数、回测参数设置等。
    样本数据不一样,是因为数据在不断的优化,之前的数据有问题的会进行修复。
    模型训练随机性您给的建议很好,以后我们的教程贴可以设置一个随机种子,便于复现。
    平台上的每个模块都有自己的版本号,表示为v1、v2等,模块升级应该给出changelog,这样体验会更优化。
    参数也会带来结果的不一致。如果之前的一个模型再次复现时,效果差异很大,那么也能一定程度反映模型的稳健性。因此模型的评估应该考虑模型的稳健性。
    回测参数类似于模型的参数,比如手续费设置,这些参数的微小变动就会导致回测结果有不小差异,这类参数也是需要仔细检查和稳定性评估的。

    总上,本贴中的策略重新运行很难重现截图的结果,再次运行后模型和回测都会有差异。差异的大小不能简单地拿之前的结果和现在结果直接衡量。


    (qci133) #12

    了解。赞认真仔细的回复^_^


    (ahxdct) #13

    代码中LTSM 和Dense 分别从两个模块中导入两次,是什么目的,请教大神!