复制链接
克隆策略

探究 训练时间对 模型收益影响？

训练集长时间 （5-10年） VS 训练集 短时间 （6个月-1，2年） 会对模型的数据造成怎么样的影响？

长时间： 从05年开始往后 到21年 增加数据 首先是标注的训练数据发生了改变， 其次是 模型中 因子的特征权重发生改变 再然后是 ndcg的评价也发生了改变 最后 整个模型的预测结果也会产生偏差

简单评价：长时间训练 搜索出来的回测结果  回测时间更长，得出结论更稳定  也更容易 过拟合

短时间 从21年开始往前 到20年 增加数据 变化同上 但数据量更少，有可能 NDCG反而提高 收益也更高

简单评价：短时间训练 搜索出来的回测结果  在未来回测更陡峭 收益有可能更高  也更容易欠拟合

探究 有没有 一些较好的训练时间作为参考？

05年-16年，13-19年，10-20年都可以试试。

探究 是策略思路重要还是因子重要？

1.好的策略设计思路,能节省你挖掘因子的时间，同时,聚焦某一个领域去钻研，

也更容易找到适配的组合，挖掘出好的因子模型。 （比如追涨的逻辑，就要配合动量因子，做追龙头的逻辑，就要配合波动率因子和收益率因子）

2.因子和策略的表现跟市场是强相关的。

有的年份 小市值股票厉害，风格暴露在小市值上的模型，超额收益更多，有的年份则相反 一类因子其实也只是一种套路或者说一种市场偏好选股风格的集成， 如何去高效利用好这种因子， 才是我们应该去研究的问题。 平常做模型的因子如果想不到在哪来， 可以在因子看板里面去找找。

    {"description":"实验创建于2017/8/26","graph":{"edges":[{"to_node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-15:instruments","from_node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-8:data"},{"to_node_id":"-215:instruments","from_node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-8:data"},{"to_node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-53:data1","from_node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-15:data"},{"to_node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-43:features","from_node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-24:data"},{"to_node_id":"-5940:features_ds","from_node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-24:data"},{"to_node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-60:model","from_node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-43:model"},{"to_node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-84:input_data","from_node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-53:data"},{"to_node_id":"-250:options_data","from_node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-60:predictions"},{"to_node_id":"-231:instruments","from_node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-62:data"},{"to_node_id":"-250:instruments","from_node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-62:data"},{"to_node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-43:training_ds","from_node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-84:data"},{"to_node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-43:test_ds","from_node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-84:data"},{"to_node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-60:data","from_node_id":"-86:data"},{"to_node_id":"-222:input_data","from_node_id":"-215:data"},{"to_node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-53:data2","from_node_id":"-222:data"},{"to_node_id":"-238:input_data","from_node_id":"-231:data"},{"to_node_id":"-86:input_data","from_node_id":"-238:data"},{"to_node_id":"-215:features","from_node_id":"-5940:data"},{"to_node_id":"-222:features","from_node_id":"-5940:data"},{"to_node_id":"-231:features","from_node_id":"-5940:data"},{"to_node_id":"-238:features","from_node_id":"-5940:data"}],"nodes":[{"node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-8","module_id":"BigQuantSpace.instruments.instruments-v2","parameters":[{"name":"start_date","value":"2005-01-01","type":"Literal","bound_global_parameter":null},{"name":"end_date","value":"2016-01-01","type":"Literal","bound_global_parameter":null},{"name":"market","value":"CN_STOCK_A","type":"Literal","bound_global_parameter":null},{"name":"instrument_list","value":"","type":"Literal","bound_global_parameter":null},{"name":"max_count","value":"0","type":"Literal","bound_global_parameter":null}],"input_ports":[{"name":"rolling_conf","node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-8"}],"output_ports":[{"name":"data","node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-8"}],"cacheable":true,"seq_num":1,"comment":"","comment_collapsed":true},{"node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-15","module_id":"BigQuantSpace.advanced_auto_labeler.advanced_auto_labeler-v2","parameters":[{"name":"label_expr","value":"# #号开始的表示注释\n# 0. 每行一个，顺序执行，从第二个开始，可以使用label字段\n# 1. 可用数据字段见 https://bigquant.com/docs/develop/datasource/deprecated/history_data.html\n# 添加benchmark_前缀，可使用对应的benchmark数据\n# 2. 可用操作符和函数见 `表达式引擎 <https://bigquant.com/docs/develop/bigexpr/usage.html>`_\n\n# 计算收益：5日收盘价(作为卖出价格)除以明日开盘价(作为买入价格)\nshift(close, -5) / shift(open, -1)\n\n# 极值处理：用1%和99%分位的值做clip\nclip(label, all_quantile(label, 0.01), all_quantile(label, 0.99))\n\n# 将分数映射到分类，这里使用20个分类\nall_wbins(label, 20)\n\n# 过滤掉一字涨停的情况 (设置label为NaN，在后续处理和训练中会忽略NaN的label)\nwhere(shift(high, -1) == shift(low, -1), NaN, label)\n","type":"Literal","bound_global_parameter":null},{"name":"start_date","value":"","type":"Literal","bound_global_parameter":null},{"name":"end_date","value":"","type":"Literal","bound_global_parameter":null},{"name":"benchmark","value":"000300.HIX","type":"Literal","bound_global_parameter":null},{"name":"drop_na_label","value":"True","type":"Literal","bound_global_parameter":null},{"name":"cast_label_int","value":"True","type":"Literal","bound_global_parameter":null},{"name":"user_functions","value":"","type":"Literal","bound_global_parameter":null}],"input_ports":[{"name":"instruments","node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-15"}],"output_ports":[{"name":"data","node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-15"}],"cacheable":true,"seq_num":2,"comment":"","comment_collapsed":true},{"node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-24","module_id":"BigQuantSpace.input_features.input_features-v1","parameters":[{"name":"features","value":"# #号开始的表示注释\n# 多个特征，每行一个，可以包含基础特征和衍生特征\nreturn_5\nreturn_10\nreturn_20\navg_amount_0/avg_amount_5\navg_amount_5/avg_amount_20\nrank_avg_amount_0/rank_avg_amount_5\nrank_avg_amount_5/rank_avg_amount_10\nrank_return_0\nrank_return_5\nrank_return_10\nrank_return_0/rank_return_5\nrank_return_5/rank_return_10\npe_ttm_0\n","type":"Literal","bound_global_parameter":null}],"input_ports":[{"name":"features_ds","node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-24"}],"output_ports":[{"name":"data","node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-24"}],"cacheable":true,"seq_num":3,"comment":"模板因子","comment_collapsed":false},{"node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-43","module_id":"BigQuantSpace.stock_ranker_train.stock_ranker_train-v5","parameters":[{"name":"learning_algorithm","value":"排序","type":"Literal","bound_global_parameter":null},{"name":"number_of_leaves","value":30,"type":"Literal","bound_global_parameter":null},{"name":"minimum_docs_per_leaf","value":1000,"type":"Literal","bound_global_parameter":null},{"name":"number_of_trees","value":20,"type":"Literal","bound_global_parameter":null},{"name":"learning_rate","value":0.1,"type":"Literal","bound_global_parameter":null},{"name":"max_bins","value":1023,"type":"Literal","bound_global_parameter":null},{"name":"feature_fraction","value":1,"type":"Literal","bound_global_parameter":null},{"name":"m_lazy_run","value":"False","type":"Literal","bound_global_parameter":null}],"input_ports":[{"name":"training_ds","node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-43"},{"name":"features","node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-43"},{"name":"test_ds","node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-43"},{"name":"base_model","node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-43"}],"output_ports":[{"name":"model","node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-43"},{"name":"feature_gains","node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-43"},{"name":"m_lazy_run","node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-43"}],"cacheable":true,"seq_num":6,"comment":"","comment_collapsed":true},{"node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-53","module_id":"BigQuantSpace.join.join-v3","parameters":[{"name":"on","value":"date,instrument","type":"Literal","bound_global_parameter":null},{"name":"how","value":"inner","type":"Literal","bound_global_parameter":null},{"name":"sort","value":"False","type":"Literal","bound_global_parameter":null}],"input_ports":[{"name":"data1","node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-53"},{"name":"data2","node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-53"}],"output_ports":[{"name":"data","node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-53"}],"cacheable":true,"seq_num":7,"comment":"","comment_collapsed":true},{"node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-60","module_id":"BigQuantSpace.stock_ranker_predict.stock_ranker_predict-v5","parameters":[{"name":"m_lazy_run","value":"False","type":"Literal","bound_global_parameter":null}],"input_ports":[{"name":"model","node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-60"},{"name":"data","node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-60"}],"output_ports":[{"name":"predictions","node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-60"},{"name":"m_lazy_run","node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-60"}],"cacheable":true,"seq_num":8,"comment":"","comment_collapsed":true},{"node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-62","module_id":"BigQuantSpace.instruments.instruments-v2","parameters":[{"name":"start_date","value":"2021-01-01","type":"Literal","bound_global_parameter":"交易日期"},{"name":"end_date","value":"2021-12-31","type":"Literal","bound_global_parameter":"交易日期"},{"name":"market","value":"CN_STOCK_A","type":"Literal","bound_global_parameter":null},{"name":"instrument_list","value":"","type":"Literal","bound_global_parameter":null},{"name":"max_count","value":"0","type":"Literal","bound_global_parameter":null}],"input_ports":[{"name":"rolling_conf","node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-62"}],"output_ports":[{"name":"data","node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-62"}],"cacheable":true,"seq_num":9,"comment":"预测数据，用于回测和模拟","comment_collapsed":false},{"node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-84","module_id":"BigQuantSpace.dropnan.dropnan-v1","parameters":[],"input_ports":[{"name":"input_data","node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-84"}],"output_ports":[{"name":"data","node_id":"287d2cb0-f53c-4101-bdf8-104b137c8601-84"}],"cacheable":true,"seq_num":13,"comment":"","comment_collapsed":true},{"node_id":"-86","module_id":"BigQuantSpace.dropnan.dropnan-v1","parameters":[],"input_ports":[{"name":"input_data","node_id":"-86"}],"output_ports":[{"name":"data","node_id":"-86"}],"cacheable":true,"seq_num":14,"comment":"","comment_collapsed":true},{"node_id":"-215","module_id":"BigQuantSpace.general_feature_extractor.general_feature_extractor-v7","parameters":[{"name":"start_date","value":"","type":"Literal","bound_global_parameter":null},{"name":"end_date","value":"","type":"Literal","bound_global_parameter":null},{"name":"before_start_days","value":90,"type":"Literal","bound_global_parameter":null}],"input_ports":[{"name":"instruments","node_id":"-215"},{"name":"features","node_id":"-215"}],"output_ports":[{"name":"data","node_id":"-215"}],"cacheable":true,"seq_num":15,"comment":"","comment_collapsed":true},{"node_id":"-222","module_id":"BigQuantSpace.derived_feature_extractor.derived_feature_extractor-v3","parameters":[{"name":"date_col","value":"date","type":"Literal","bound_global_parameter":null},{"name":"instrument_col","value":"instrument","type":"Literal","bound_global_parameter":null},{"name":"drop_na","value":"False","type":"Literal","bound_global_parameter":null},{"name":"remove_extra_columns","value":"False","type":"Literal","bound_global_parameter":null},{"name":"user_functions","value":"","type":"Literal","bound_global_parameter":null}],"input_ports":[{"name":"input_data","node_id":"-222"},{"name":"features","node_id":"-222"}],"output_ports":[{"name":"data","node_id":"-222"}],"cacheable":true,"seq_num":16,"comment":"","comment_collapsed":true},{"node_id":"-231","module_id":"BigQuantSpace.general_feature_extractor.general_feature_extractor-v7","parameters":[{"name":"start_date","value":"","type":"Literal","bound_global_parameter":null},{"name":"end_date","value":"","type":"Literal","bound_global_parameter":null},{"name":"before_start_days","value":90,"type":"Literal","bound_global_parameter":null}],"input_ports":[{"name":"instruments","node_id":"-231"},{"name":"features","node_id":"-231"}],"output_ports":[{"name":"data","node_id":"-231"}],"cacheable":true,"seq_num":17,"comment":"","comment_collapsed":true},{"node_id":"-238","module_id":"BigQuantSpace.derived_feature_extractor.derived_feature_extractor-v3","parameters":[{"name":"date_col","value":"date","type":"Literal","bound_global_parameter":null},{"name":"instrument_col","value":"instrument","type":"Literal","bound_global_parameter":null},{"name":"drop_na","value":"False","type":"Literal","bound_global_parameter":null},{"name":"remove_extra_columns","value":"False","type":"Literal","bound_global_parameter":null},{"name":"user_functions","value":"","type":"Literal","bound_global_parameter":null}],"input_ports":[{"name":"input_data","node_id":"-238"},{"name":"features","node_id":"-238"}],"output_ports":[{"name":"data","node_id":"-238"}],"cacheable":true,"seq_num":18,"comment":"","comment_collapsed":true},{"node_id":"-250","module_id":"BigQuantSpace.trade.trade-v4","parameters":[{"name":"start_date","value":"","type":"Literal","bound_global_parameter":null},{"name":"end_date","value":"","type":"Literal","bound_global_parameter":null},{"name":"initialize","value":"# 回测引擎：初始化函数，只执行一次\ndef bigquant_run(context):\n # 加载预测数据\n context.ranker_prediction = context.options['data'].read_df()\n\n # 系统已经设置了默认的交易手续费和滑点，要修改手续费可使用如下函数\n context.set_commission(PerOrder(buy_cost=0.0003, sell_cost=0.0013, min_cost=5))\n # 预测数据，通过options传入进来，使用 read_df 函数，加载到内存 (DataFrame)\n # 设置买入的股票数量，这里买入预测股票列表排名靠前的5只\n stock_count = 5\n # 每只的股票的权重，如下的权重分配会使得靠前的股票分配多一点的资金，[0.339160, 0.213986, 0.169580, ..]\n context.stock_weights = T.norm([1 / math.log(i + 2) for i in range(0, stock_count)])\n # 设置每只股票占用的最大资金比例\n context.max_cash_per_instrument = 0.2\n context.options['hold_days'] = 5\n","type":"Literal","bound_global_parameter":null},{"name":"handle_data","value":"# 回测引擎：每日数据处理函数，每天执行一次\ndef bigquant_run(context, data):\n # 按日期过滤得到今日的预测数据\n ranker_prediction = context.ranker_prediction[\n context.ranker_prediction.date == data.current_dt.strftime('%Y-%m-%d')]\n\n # 1. 资金分配\n # 平均持仓时间是hold_days，每日都将买入股票，每日预期使用 1/hold_days 的资金\n # 实际操作中，会存在一定的买入误差，所以在前hold_days天，等量使用资金；之后，尽量使用剩余资金（这里设置最多用等量的1.5倍）\n is_staging = context.trading_day_index < context.options['hold_days'] # 是否在建仓期间（前 hold_days 天）\n cash_avg = context.portfolio.portfolio_value / context.options['hold_days']\n cash_for_buy = min(context.portfolio.cash, (1 if is_staging else 1.5) * cash_avg)\n cash_for_sell = cash_avg - (context.portfolio.cash - cash_for_buy)\n positions = {e.symbol: p.amount * p.last_sale_price\n for e, p in context.portfolio.positions.items()}\n\n # 2. 生成卖出订单：hold_days天之后才开始卖出；对持仓的股票，按机器学习算法预测的排序末位淘汰\n if not is_staging and cash_for_sell > 0:\n equities = {e.symbol: e for e, p in context.portfolio.positions.items()}\n instruments = list(reversed(list(ranker_prediction.instrument[ranker_prediction.instrument.apply(\n lambda x: x in equities)])))\n\n for instrument in instruments:\n context.order_target(context.symbol(instrument), 0)\n cash_for_sell -= positions[instrument]\n if cash_for_sell <= 0:\n break\n\n # 3. 生成买入订单：按机器学习算法预测的排序，买入前面的stock_count只股票\n buy_cash_weights = context.stock_weights\n buy_instruments = list(ranker_prediction.instrument[:len(buy_cash_weights)])\n max_cash_per_instrument = context.portfolio.portfolio_value * context.max_cash_per_instrument\n for i, instrument in enumerate(buy_instruments):\n cash = cash_for_buy * buy_cash_weights[i]\n if cash > max_cash_per_instrument - positions.get(instrument, 0):\n # 确保股票持仓量不会超过每次股票最大的占用资金量\n cash = max_cash_per_instrument - positions.get(instrument, 0)\n if cash > 0:\n context.order_value(context.symbol(instrument), cash)\n","type":"Literal","bound_global_parameter":null},{"name":"prepare","value":"# 回测引擎：准备数据，只执行一次\ndef bigquant_run(context):\n pass\n","type":"Literal","bound_global_parameter":null},{"name":"before_trading_start","value":"","type":"Literal","bound_global_parameter":null},{"name":"volume_limit","value":0.025,"type":"Literal","bound_global_parameter":null},{"name":"order_price_field_buy","value":"open","type":"Literal","bound_global_parameter":null},{"name":"order_price_field_sell","value":"close","type":"Literal","bound_global_parameter":null},{"name":"capital_base","value":1000000,"type":"Literal","bound_global_parameter":null},{"name":"auto_cancel_non_tradable_orders","value":"True","type":"Literal","bound_global_parameter":null},{"name":"data_frequency","value":"daily","type":"Literal","bound_global_parameter":null},{"name":"price_type","value":"真实价格","type":"Literal","bound_global_parameter":null},{"name":"product_type","value":"股票","type":"Literal","bound_global_parameter":null},{"name":"plot_charts","value":"True","type":"Literal","bound_global_parameter":null},{"name":"backtest_only","value":"False","type":"Literal","bound_global_parameter":null},{"name":"benchmark","value":"000300.HIX","type":"Literal","bound_global_parameter":null}],"input_ports":[{"name":"instruments","node_id":"-250"},{"name":"options_data","node_id":"-250"},{"name":"history_ds","node_id":"-250"},{"name":"benchmark_ds","node_id":"-250"},{"name":"trading_calendar","node_id":"-250"}],"output_ports":[{"name":"raw_perf","node_id":"-250"}],"cacheable":false,"seq_num":19,"comment":"","comment_collapsed":true},{"node_id":"-5940","module_id":"BigQuantSpace.input_features.input_features-v1","parameters":[{"name":"features","value":"# #号开始的表示注释\n# 多个特征，每行一个，可以包含基础特征和衍生特征\n\n# 未来2天的 收益\nday_return_2=(shift(close_0, -2)-shift(open_0, -1))/shift(open_0, -1)\n# 未来3天的 收益\nday_return_3=(shift(close_0, -3)-shift(open_0, -1))/shift(open_0, -1)\n# 未来5天的 收益\nday_return_5=(shift(close_0, -5)-shift(open_0, -1))/shift(open_0, -1)","type":"Literal","bound_global_parameter":null}],"input_ports":[{"name":"features_ds","node_id":"-5940"}],"output_ports":[{"name":"data","node_id":"-5940"}],"cacheable":true,"seq_num":4,"comment":"用来检测时间长短带来的收益变化","comment_collapsed":false},{"node_id":"-2484","module_id":"BigQuantSpace.hyper_parameter_search.hyper_parameter_search-v1","parameters":[{"name":"param_grid_builder","value":"def bigquant_run():\n param_grid = {}\n #'m1.start_date': '2005-01-01',\n #'m1.end_date': '2010-01-01',\n \n# year_list=['2020-01-01','2021-01-01']\n# for i in range(10):\n# year_list.append('201{}-01-01'.format(i))\n# param_grid['m1.end_date']=year_list\n year_last=['2018-08-01','2019-10-01',\n '2020-02-01',\n '2020-06-01',\n '2020-10-01',\n '2020-12-01']\n# for i in range(12,-1):\n# year_list.append('2020-{}-01'.format(i))\n param_grid['m1.start_date']=year_last\n \n return param_grid\n","type":"Literal","bound_global_parameter":null},{"name":"scoring","value":"def bigquant_run(result):\n score = result.get('m19').read_raw_perf()['sharpe'].tail(1)[0]\n\n return {'score': score}\n","type":"Literal","bound_global_parameter":null},{"name":"search_algorithm","value":"网格搜索","type":"Literal","bound_global_parameter":null},{"name":"search_iterations","value":10,"type":"Literal","bound_global_parameter":null},{"name":"random_state","value":"","type":"Literal","bound_global_parameter":null},{"name":"workers","value":"3","type":"Literal","bound_global_parameter":null},{"name":"worker_distributed_run","value":"True","type":"Literal","bound_global_parameter":null},{"name":"worker_silent","value":"False","type":"Literal","bound_global_parameter":null},{"name":"run_now","value":"True","type":"Literal","bound_global_parameter":null},{"name":"bq_graph","value":"True","type":"Literal","bound_global_parameter":null}],"input_ports":[{"name":"bq_graph_port","node_id":"-2484"},{"name":"input_1","node_id":"-2484"},{"name":"input_2","node_id":"-2484"},{"name":"input_3","node_id":"-2484"}],"output_ports":[{"name":"result","node_id":"-2484"}],"cacheable":false,"seq_num":5,"comment":"","comment_collapsed":true}],"node_layout":"<node_postions><node_position Node='287d2cb0-f53c-4101-bdf8-104b137c8601-8' Position='211,64,200,200'/><node_position Node='287d2cb0-f53c-4101-bdf8-104b137c8601-15' Position='79,187,200,200'/><node_position Node='287d2cb0-f53c-4101-bdf8-104b137c8601-24' Position='713,13,200,200'/><node_position Node='287d2cb0-f53c-4101-bdf8-104b137c8601-43' Position='657,557,200,200'/><node_position Node='287d2cb0-f53c-4101-bdf8-104b137c8601-53' Position='236,370,200,200'/><node_position Node='287d2cb0-f53c-4101-bdf8-104b137c8601-60' Position='906,647,200,200'/><node_position Node='287d2cb0-f53c-4101-bdf8-104b137c8601-62' Position='1074,127,200,200'/><node_position Node='287d2cb0-f53c-4101-bdf8-104b137c8601-84' Position='391,443,200,200'/><node_position Node='-86' Position='1078,418,200,200'/><node_position Node='-215' Position='365,194,200,200'/><node_position Node='-222' Position='374,290,200,200'/><node_position Node='-231' Position='1078,236,200,200'/><node_position Node='-238' Position='1081,327,200,200'/><node_position Node='-250' Position='1037,751,200,200'/><node_position Node='-5940' Position='715,173,200,200'/><node_position Node='-2484' Position='396,623,200,200'/></node_postions>"},"nodes_readonly":false,"studio_version":"v2"}
    In [1]:
    # 本代码由可视化策略环境自动生成 2023年2月28日 11:33
# 本代码单元只能在可视化模式下编辑。您也可以拷贝代码，粘贴到新建的代码单元或者策略，然后修改。


# 回测引擎：初始化函数，只执行一次
def m19_initialize_bigquant_run(context):
    # 加载预测数据
    context.ranker_prediction = context.options['data'].read_df()

    # 系统已经设置了默认的交易手续费和滑点，要修改手续费可使用如下函数
    context.set_commission(PerOrder(buy_cost=0.0003, sell_cost=0.0013, min_cost=5))
    # 预测数据，通过options传入进来，使用 read_df 函数，加载到内存 (DataFrame)
    # 设置买入的股票数量，这里买入预测股票列表排名靠前的5只
    stock_count = 5
    # 每只的股票的权重，如下的权重分配会使得靠前的股票分配多一点的资金，[0.339160, 0.213986, 0.169580, ..]
    context.stock_weights = T.norm([1 / math.log(i + 2) for i in range(0, stock_count)])
    # 设置每只股票占用的最大资金比例
    context.max_cash_per_instrument = 0.2
    context.options['hold_days'] = 5

# 回测引擎：每日数据处理函数，每天执行一次
def m19_handle_data_bigquant_run(context, data):
    # 按日期过滤得到今日的预测数据
    ranker_prediction = context.ranker_prediction[
        context.ranker_prediction.date == data.current_dt.strftime('%Y-%m-%d')]

    # 1. 资金分配
    # 平均持仓时间是hold_days，每日都将买入股票，每日预期使用 1/hold_days 的资金
    # 实际操作中，会存在一定的买入误差，所以在前hold_days天，等量使用资金；之后，尽量使用剩余资金（这里设置最多用等量的1.5倍）
    is_staging = context.trading_day_index < context.options['hold_days'] # 是否在建仓期间（前 hold_days 天）
    cash_avg = context.portfolio.portfolio_value / context.options['hold_days']
    cash_for_buy = min(context.portfolio.cash, (1 if is_staging else 1.5) * cash_avg)
    cash_for_sell = cash_avg - (context.portfolio.cash - cash_for_buy)
    positions = {e.symbol: p.amount * p.last_sale_price
                 for e, p in context.portfolio.positions.items()}

    # 2. 生成卖出订单：hold_days天之后才开始卖出；对持仓的股票，按机器学习算法预测的排序末位淘汰
    if not is_staging and cash_for_sell > 0:
        equities = {e.symbol: e for e, p in context.portfolio.positions.items()}
        instruments = list(reversed(list(ranker_prediction.instrument[ranker_prediction.instrument.apply(
                lambda x: x in equities)])))

        for instrument in instruments:
            context.order_target(context.symbol(instrument), 0)
            cash_for_sell -= positions[instrument]
            if cash_for_sell <= 0:
                break

    # 3. 生成买入订单：按机器学习算法预测的排序，买入前面的stock_count只股票
    buy_cash_weights = context.stock_weights
    buy_instruments = list(ranker_prediction.instrument[:len(buy_cash_weights)])
    max_cash_per_instrument = context.portfolio.portfolio_value * context.max_cash_per_instrument
    for i, instrument in enumerate(buy_instruments):
        cash = cash_for_buy * buy_cash_weights[i]
        if cash > max_cash_per_instrument - positions.get(instrument, 0):
            # 确保股票持仓量不会超过每次股票最大的占用资金量
            cash = max_cash_per_instrument - positions.get(instrument, 0)
        if cash > 0:
            context.order_value(context.symbol(instrument), cash)

# 回测引擎：准备数据，只执行一次
def m19_prepare_bigquant_run(context):
    pass


g = T.Graph({

    'm1': 'M.instruments.v2',
    'm1.start_date': '2005-01-01',
    'm1.end_date': '2016-01-01',
    'm1.market': 'CN_STOCK_A',
    'm1.instrument_list': '',
    'm1.max_count': 0,

    'm2': 'M.advanced_auto_labeler.v2',
    'm2.instruments': T.Graph.OutputPort('m1.data'),
    'm2.label_expr': """# #号开始的表示注释
# 0. 每行一个，顺序执行，从第二个开始，可以使用label字段
# 1. 可用数据字段见 https://bigquant.com/docs/develop/datasource/deprecated/history_data.html
#   添加benchmark_前缀，可使用对应的benchmark数据
# 2. 可用操作符和函数见 `表达式引擎 <https://bigquant.com/docs/develop/bigexpr/usage.html>`_

# 计算收益：5日收盘价(作为卖出价格)除以明日开盘价(作为买入价格)
shift(close, -5) / shift(open, -1)

# 极值处理：用1%和99%分位的值做clip
clip(label, all_quantile(label, 0.01), all_quantile(label, 0.99))

# 将分数映射到分类，这里使用20个分类
all_wbins(label, 20)

# 过滤掉一字涨停的情况 (设置label为NaN，在后续处理和训练中会忽略NaN的label)
where(shift(high, -1) == shift(low, -1), NaN, label)
""",
    'm2.start_date': '',
    'm2.end_date': '',
    'm2.benchmark': '000300.HIX',
    'm2.drop_na_label': True,
    'm2.cast_label_int': True,

    'm3': 'M.input_features.v1',
    'm3.features': """# #号开始的表示注释
# 多个特征，每行一个，可以包含基础特征和衍生特征
return_5
return_10
return_20
avg_amount_0/avg_amount_5
avg_amount_5/avg_amount_20
rank_avg_amount_0/rank_avg_amount_5
rank_avg_amount_5/rank_avg_amount_10
rank_return_0
rank_return_5
rank_return_10
rank_return_0/rank_return_5
rank_return_5/rank_return_10
pe_ttm_0
""",

    'm4': 'M.input_features.v1',
    'm4.features_ds': T.Graph.OutputPort('m3.data'),
    'm4.features': """# #号开始的表示注释
# 多个特征，每行一个，可以包含基础特征和衍生特征

# 未来2天的 收益
day_return_2=(shift(close_0, -2)-shift(open_0, -1))/shift(open_0, -1)
# 未来3天的 收益
day_return_3=(shift(close_0, -3)-shift(open_0, -1))/shift(open_0, -1)
# 未来5天的 收益
day_return_5=(shift(close_0, -5)-shift(open_0, -1))/shift(open_0, -1)""",

    'm15': 'M.general_feature_extractor.v7',
    'm15.instruments': T.Graph.OutputPort('m1.data'),
    'm15.features': T.Graph.OutputPort('m4.data'),
    'm15.start_date': '',
    'm15.end_date': '',
    'm15.before_start_days': 90,

    'm16': 'M.derived_feature_extractor.v3',
    'm16.input_data': T.Graph.OutputPort('m15.data'),
    'm16.features': T.Graph.OutputPort('m4.data'),
    'm16.date_col': 'date',
    'm16.instrument_col': 'instrument',
    'm16.drop_na': False,
    'm16.remove_extra_columns': False,

    'm7': 'M.join.v3',
    'm7.data1': T.Graph.OutputPort('m2.data'),
    'm7.data2': T.Graph.OutputPort('m16.data'),
    'm7.on': 'date,instrument',
    'm7.how': 'inner',
    'm7.sort': False,

    'm13': 'M.dropnan.v1',
    'm13.input_data': T.Graph.OutputPort('m7.data'),

    'm6': 'M.stock_ranker_train.v5',
    'm6.training_ds': T.Graph.OutputPort('m13.data'),
    'm6.features': T.Graph.OutputPort('m3.data'),
    'm6.test_ds': T.Graph.OutputPort('m13.data'),
    'm6.learning_algorithm': '排序',
    'm6.number_of_leaves': 30,
    'm6.minimum_docs_per_leaf': 1000,
    'm6.number_of_trees': 20,
    'm6.learning_rate': 0.1,
    'm6.max_bins': 1023,
    'm6.feature_fraction': 1,
    'm6.m_lazy_run': False,

    'm9': 'M.instruments.v2',
    'm9.start_date': T.live_run_param('trading_date', '2021-01-01'),
    'm9.end_date': T.live_run_param('trading_date', '2021-12-31'),
    'm9.market': 'CN_STOCK_A',
    'm9.instrument_list': '',
    'm9.max_count': 0,

    'm17': 'M.general_feature_extractor.v7',
    'm17.instruments': T.Graph.OutputPort('m9.data'),
    'm17.features': T.Graph.OutputPort('m4.data'),
    'm17.start_date': '',
    'm17.end_date': '',
    'm17.before_start_days': 90,

    'm18': 'M.derived_feature_extractor.v3',
    'm18.input_data': T.Graph.OutputPort('m17.data'),
    'm18.features': T.Graph.OutputPort('m4.data'),
    'm18.date_col': 'date',
    'm18.instrument_col': 'instrument',
    'm18.drop_na': False,
    'm18.remove_extra_columns': False,

    'm14': 'M.dropnan.v1',
    'm14.input_data': T.Graph.OutputPort('m18.data'),

    'm8': 'M.stock_ranker_predict.v5',
    'm8.model': T.Graph.OutputPort('m6.model'),
    'm8.data': T.Graph.OutputPort('m14.data'),
    'm8.m_lazy_run': False,

    'm19': 'M.trade.v4',
    'm19.instruments': T.Graph.OutputPort('m9.data'),
    'm19.options_data': T.Graph.OutputPort('m8.predictions'),
    'm19.start_date': '',
    'm19.end_date': '',
    'm19.initialize': m19_initialize_bigquant_run,
    'm19.handle_data': m19_handle_data_bigquant_run,
    'm19.prepare': m19_prepare_bigquant_run,
    'm19.volume_limit': 0.025,
    'm19.order_price_field_buy': 'open',
    'm19.order_price_field_sell': 'close',
    'm19.capital_base': 1000000,
    'm19.auto_cancel_non_tradable_orders': True,
    'm19.data_frequency': 'daily',
    'm19.price_type': '真实价格',
    'm19.product_type': '股票',
    'm19.plot_charts': True,
    'm19.backtest_only': False,
    'm19.benchmark': '000300.HIX',
})

# g.run({})


def m5_param_grid_builder_bigquant_run():
    param_grid = {}
    #'m1.start_date': '2005-01-01',
    #'m1.end_date': '2010-01-01',
    
#     year_list=['2020-01-01','2021-01-01']
#     for i in range(10):
#         year_list.append('201{}-01-01'.format(i))
#     param_grid['m1.end_date']=year_list
    year_last=['2018-08-01','2019-10-01',
 '2020-02-01',
 '2020-06-01',
 '2020-10-01',
 '2020-12-01']
#     for i in range(12,-1):
#         year_list.append('2020-{}-01'.format(i))
    param_grid['m1.start_date']=year_last
 
    return param_grid

def m5_scoring_bigquant_run(result):
    score = result.get('m19').read_raw_perf()['sharpe'].tail(1)[0]

    return {'score': score}


m5 = M.hyper_parameter_search.v1(
    param_grid_builder=m5_param_grid_builder_bigquant_run,
    scoring=m5_scoring_bigquant_run,
    search_algorithm='网格搜索',
    search_iterations=10,
    workers=3,
    worker_distributed_run=True,
    worker_silent=False,
    run_now=True,
    bq_graph=g
)

    Fitting 1 folds for each of 6 candidates, totalling 6 fits
[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.
[CV 1/1; 1/6] START m1.start_date=2018-08-01....................................

    [CV 1/1; 1/6] END ..................m1.start_date=2018-08-01; total time=  33.8s
[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:   33.8s remaining:    0.0s
[CV 1/1; 2/6] START m1.start_date=2019-10-01....................................

    [CV 1/1; 2/6] END ..................m1.start_date=2019-10-01; total time=  40.6s
[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:  1.2min remaining:    0.0s
[CV 1/1; 3/6] START m1.start_date=2020-02-01....................................

    [CV 1/1; 3/6] END ..................m1.start_date=2020-02-01; total time=  40.6s
[Parallel(n_jobs=1)]: Done   3 out of   3 | elapsed:  1.9min remaining:    0.0s
[CV 1/1; 4/6] START m1.start_date=2020-06-01....................................

    [CV 1/1; 4/6] END ..................m1.start_date=2020-06-01; total time=  40.7s
[Parallel(n_jobs=1)]: Done   4 out of   4 | elapsed:  2.6min remaining:    0.0s
[CV 1/1; 5/6] START m1.start_date=2020-10-01....................................

    [CV 1/1; 5/6] END ..................m1.start_date=2020-10-01; total time=  30.6s
[Parallel(n_jobs=1)]: Done   5 out of   5 | elapsed:  3.1min remaining:    0.0s
[CV 1/1; 6/6] START m1.start_date=2020-12-01....................................

    [CV 1/1; 6/6] END ..................m1.start_date=2020-12-01; total time=  40.6s
[Parallel(n_jobs=1)]: Done   6 out of   6 | elapsed:  3.8min remaining:    0.0s
[Parallel(n_jobs=1)]: Done   6 out of   6 | elapsed:  3.8min finished
    In [4]:
    m6.plot_test_data_ndcg('Series')

    ---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-4-b3d5a30e577f> in <module>
----> 1 m6.plot_test_data_ndcg('Series')

NameError: name 'm6' is not defined
    In [3]:
    #data1=eval(m6.plot_test_data_ndcg('Series'))#[2]#["name"]#=="NDCG@1"
#print(tyabspe(data1))
data1=m6.plot_test_data_ndcg('Series')
#eval()
data1

    ---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-3-3572e94b72c7> in <module>
      1 #data1=eval(m6.plot_test_data_ndcg('Series'))#[2]#["name"]#=="NDCG@1"
      2 #print(tyabspe(data1))
----> 3 data1=m6.plot_test_data_ndcg('Series')
      4 #eval()
      5 data1

NameError: name 'm6' is not defined
    In [27]:
    def index_of_str(s1, s2):
    res = []
    index = 0
    if s1 == "" or s2 == "":
        return -1
    split_list = s1.split(s2)
    for i in range(len(split_list) - 1):
        index += len(split_list[i])
        res.append(index)
        index += len(s2)
    return res if res else -1

str1 =data1#"cdembccdefacddelhpzmrtcdeqpjcde"
str2 = "data"
print("字符串 {}  中出现的位置：{} ".format(str2,  index_of_str(str1, str2)))

    字符串 data  中出现的位置：[96, 138, 360, 619, 887, 1155, 1423, 1690, 1950, 2216, 2484, 2752, 3043]
    In [ ]:
    In [ ]:
    In [26]:
    date_windows =['2005-01,2010-01','2005-01,2011-01','2005-01,2012-01','2005-01,2013-01','2005-01,2014-01','2005-01,2015-01','2005-01,2016-01','2005-01,2017-01','2005-01,2018-01','2005-01,2019-01','2005-01,2020-01','2005-01,2021-01',]
date_df_1 = pd.DataFrame( columns=['训练集数据时间','训练集NDCG_max','预测集收益率','预测集夏普',] )

ndg_max=[0.527,0.524,0.527,0.518,0.523,0.512,0.508,0.502,0.493,0.495,0.493,0.492]
shape=[1.724,1.499,0.993,1.857,0.825,0.60,3.278,0.453,1.348,2.440,2.381,2.154]
return_pre=[0.509,0.374,0.275,0.56,0.21,0.16,1.02,0.101,0.325,0.547,0.51,0.168]
date_df_1['训练集数据时间']=date_windows
date_df_1['训练集NDCG_max']=ndg_max
date_df_1['预测集收益率']=return_pre
date_df_1['预测集夏普']=shape
date_df_1
#date_df_1.to_csv('long_term_predit.csv')
    Out[26]:
    训练集数据时间 训练集NDCG_max 预测集收益率 预测集夏普
    0 2005-01,2010-01 0.527 0.509 1.724
    1 2005-01,2011-01 0.524 0.374 1.499
    2 2005-01,2012-01 0.527 0.275 0.993
    3 2005-01,2013-01 0.518 0.560 1.857
    4 2005-01,2014-01 0.523 0.210 0.825
    5 2005-01,2015-01 0.512 0.160 0.600
    6 2005-01,2016-01 0.508 1.020 3.278
    7 2005-01,2017-01 0.502 0.101 0.453
    8 2005-01,2018-01 0.493 0.325 1.348
    9 2005-01,2019-01 0.495 0.547 2.440
    10 2005-01,2020-01 0.493 0.510 2.381
    11 2005-01,2021-01 0.492 0.168 2.154
    In [28]:
    import matplotlib.pyplot as plt
import numpy as np
df =pd.DataFrame(date_df_1, columns=['训练集数据时间','训练集NDCG_max','预测集收益率','预测集夏普']) 
plt.plot(date_df_1['训练集数据时间'],date_df_1['训练集NDCG_max'],) 
plt.plot(date_df_1['训练集数据时间'],date_df_1['预测集收益率'],) 
plt.plot(date_df_1['训练集数据时间'],date_df_1['预测集夏普'],)
plt.figure(); df.plot(); plt.legend(loc='best')
    Out[28]:
    <matplotlib.legend.Legend at 0x7fe8a4abf250>
    <Figure size 432x288 with 0 Axes>
    In [22]:
    date_windows1 =['2018-08-01,2021-01-01','2019-10-01,2021-01-01',
 '2020-02-01,2021-01-01',
 '2020-06-01,2021-01-01',
 '2020-10-01,2021-01-01',
 '2020-12-01,2021-01-01']
date_df_2 = pd.DataFrame( columns=['训练集数据时间','训练集NDCG_max','预测集收益率','预测集夏普',] )
#date_df.index.name = 'CompanyName'
ndg_max1=[0.496,0.486,0.491,0.534,0.550,0.383]
shape1=[1.618,0.802,1.613,2.496,0.146,0.903]
return_pre1=[0.39,0.19,0.32,0.678,0.031,0.35,]
date_df_2['训练集数据时间']=date_windows1
date_df_2['训练集NDCG_max']=ndg_max1
date_df_2['预测集收益率']=return_pre1
date_df_2['预测集夏普']=shape1
#date_df_2.to_csv('short_term_predit.csv')
date_df_2
    Out[22]:
    训练集数据时间 训练集NDCG_max 预测集收益率 预测集夏普
    0 2018-08-01,2021-01-01 0.496 0.390 1.618
    1 2019-10-01,2021-01-01 0.486 0.190 0.802
    2 2020-02-01,2021-01-01 0.491 0.320 1.613
    3 2020-06-01,2021-01-01 0.534 0.678 2.496
    4 2020-10-01,2021-01-01 0.550 0.031 0.146
    5 2020-12-01,2021-01-01 0.383 0.350 0.903
    In [19]:
    import matplotlib.pyplot as plt
import numpy as np
df =pd.DataFrame(date_df_2, columns=['训练集数据时间','训练集NDCG_max','预测集收益率','预测集夏普']) 
plt.plot(date_df_2['训练集数据时间'],date_df_2['训练集NDCG_max'],) 
plt.plot(date_df_2['训练集数据时间'],date_df_2['预测集收益率'],) 
plt.plot(date_df_2['训练集数据时间'],date_df_2['预测集夏普'],)
plt.figure(); df.plot(); plt.legend(loc='best')
    Out[19]:
    <matplotlib.legend.Legend at 0x7fe879a83550>
    <Figure size 432x288 with 0 Axes>
    In [ ]:
    # import matplotlib.pyplot as plt
# import numpy as np
# plt.figure(figsize=(36,12))
# for i in range(1,3):
       
        
#     date_df_1
#     #abnormalreturns_df =  cal_abnormol(day1=day1,day2=x1,day_total_end=y1)
#     plt.subplot(4,4,i)
#     date_df_1[date_df_1.columns[i-1]].plot()
#     plt.xlabel('Event Window{}'.format(date_windows[i-1]),fontsize=13, fontdict={"family": 'SimSun', "size": 15, "color": "r"})
#     plt.ylabel('Return',fontsize=13, fontdict={"family": 'SimSun', "size": 15, "color": "r"})
#     #plt.axhline(y=(np.sqrt((abnormalreturns_df.iloc[:,i-1].std()**2/21))*1.96),color='red',linestyle='--')
#     #plt.axhline(y=(np.sqrt((abnormalreturns_df.iloc[:,i-1].std()**2/21))*-1.96),color='red',linestyle='--')
#     plt.title(date_df_1.columns[i-1])