{"description":"实验创建于2020/2/14","graph":{"edges":[{"to_node_id":"-67:features","from_node_id":"-70:data"},{"to_node_id":"-60:features","from_node_id":"-70:data"},{"to_node_id":"-2650:factors_info","from_node_id":"-8309:save_data"},{"to_node_id":"-67:input_data","from_node_id":"-60:data"},{"to_node_id":"-43:feature_data1","from_node_id":"-67:data"},{"to_node_id":"-8309:user_factor_data","from_node_id":"-67:data"},{"to_node_id":"-60:instruments","from_node_id":"-75:data"},{"to_node_id":"-8309:features","from_node_id":"-83:data"}],"nodes":[{"node_id":"-70","module_id":"BigQuantSpace.input_features.input_features-v1","parameters":[{"name":"features","value":"# var1=(close_0*2+high_0+low_0)/4\n# var2=ta_ema(var1,13)-ta_ema(var1,34)\n# var3=ta_ema(var2,5)\n# var4=var2-var3\n# var5=covariance(var1, var4, 30)\n# var6=var4-shift(var4,1)\nAA=ta_ma((2*close_0+high_0+low_0)/4,5) \ntd1=AA*102/100\ntd2=AA*(200-102)/100\nCC=abs((2*close_0+high_0+low_0)/4-ta_ma(close_0,20))/ta_ma(close_0,20)\nDD=ta_sma2(close_0,30,CC) #加权平均\ntd3=(1+7/100)*DD; \ntd4=(1-7/100)*DD;\n\n\n\n","type":"Literal","bound_global_parameter":null}],"input_ports":[{"name":"features_ds","node_id":"-70"}],"output_ports":[{"name":"data","node_id":"-70"}],"cacheable":true,"seq_num":1,"comment":"","comment_collapsed":true},{"node_id":"-8309","module_id":"BigQuantSpace.factorlens.factorlens-v2","parameters":[{"name":"title","value":"因子分析: {factor_name}","type":"Literal","bound_global_parameter":null},{"name":"start_date","value":"2022-01-01","type":"Literal","bound_global_parameter":null},{"name":"end_date","value":"2022-07-30","type":"Literal","bound_global_parameter":null},{"name":"rebalance_period","value":"1","type":"Literal","bound_global_parameter":null},{"name":"delay_rebalance_days","value":0,"type":"Literal","bound_global_parameter":null},{"name":"rebalance_price","value":"close_0","type":"Literal","bound_global_parameter":null},{"name":"stock_pool","value":"全市场","type":"Literal","bound_global_parameter":null},{"name":"quantile_count","value":"20","type":"Literal","bound_global_parameter":null},{"name":"commission_rate","value":0.0016,"type":"Literal","bound_global_parameter":null},{"name":"returns_calculation_method","value":"累乘","type":"Literal","bound_global_parameter":null},{"name":"benchmark","value":"无","type":"Literal","bound_global_parameter":null},{"name":"drop_new_stocks","value":60,"type":"Literal","bound_global_parameter":null},{"name":"drop_price_limit_stocks","value":"True","type":"Literal","bound_global_parameter":null},{"name":"drop_st_stocks","value":"True","type":"Literal","bound_global_parameter":null},{"name":"drop_suspended_stocks","value":"True","type":"Literal","bound_global_parameter":null},{"name":"cutoutliers","value":"True","type":"Literal","bound_global_parameter":null},{"name":"normalization","value":"True","type":"Literal","bound_global_parameter":null},{"name":"neutralization","value":"%7B%22enumItems%22%3A%5B%7B%22value%22%3A%22%E8%A1%8C%E4%B8%9A%22%2C%22displayValue%22%3A%22%E8%A1%8C%E4%B8%9A%22%2C%22selected%22%3Atrue%7D%2C%7B%22value%22%3A%22%E5%B8%82%E5%80%BC%22%2C%22displayValue%22%3A%22%E5%B8%82%E5%80%BC%22%2C%22selected%22%3Atrue%7D%5D%7D","type":"Literal","bound_global_parameter":null},{"name":"metrics","value":"%7B%22enumItems%22%3A%5B%7B%22value%22%3A%22%E5%9B%A0%E5%AD%90%E8%A1%A8%E7%8E%B0%E6%A6%82%E8%A7%88%22%2C%22displayValue%22%3A%22%E5%9B%A0%E5%AD%90%E8%A1%A8%E7%8E%B0%E6%A6%82%E8%A7%88%22%2C%22selected%22%3Atrue%7D%2C%7B%22value%22%3A%22%E5%9B%A0%E5%AD%90%E5%88%86%E5%B8%83%22%2C%22displayValue%22%3A%22%E5%9B%A0%E5%AD%90%E5%88%86%E5%B8%83%22%2C%22selected%22%3Atrue%7D%2C%7B%22value%22%3A%22%E5%9B%A0%E5%AD%90%E8%A1%8C%E4%B8%9A%E5%88%86%E5%B8%83%22%2C%22displayValue%22%3A%22%E5%9B%A0%E5%AD%90%E8%A1%8C%E4%B8%9A%E5%88%86%E5%B8%83%22%2C%22selected%22%3Atrue%7D%2C%7B%22value%22%3A%22%E5%9B%A0%E5%AD%90%E5%B8%82%E5%80%BC%E5%88%86%E5%B8%83%22%2C%22displayValue%22%3A%22%E5%9B%A0%E5%AD%90%E5%B8%82%E5%80%BC%E5%88%86%E5%B8%83%22%2C%22selected%22%3Atrue%7D%2C%7B%22value%22%3A%22IC%E5%88%86%E6%9E%90%22%2C%22displayValue%22%3A%22IC%E5%88%86%E6%9E%90%22%2C%22selected%22%3Atrue%7D%2C%7B%22value%22%3A%22%E4%B9%B0%E5%85%A5%E4%BF%A1%E5%8F%B7%E9%87%8D%E5%90%88%E5%88%86%E6%9E%90%22%2C%22displayValue%22%3A%22%E4%B9%B0%E5%85%A5%E4%BF%A1%E5%8F%B7%E9%87%8D%E5%90%88%E5%88%86%E6%9E%90%22%2C%22selected%22%3Atrue%7D%2C%7B%22value%22%3A%22%E5%9B%A0%E5%AD%90%E4%BC%B0%E5%80%BC%E5%88%86%E6%9E%90%22%2C%22displayValue%22%3A%22%E5%9B%A0%E5%AD%90%E4%BC%B0%E5%80%BC%E5%88%86%E6%9E%90%22%2C%22selected%22%3Atrue%7D%2C%7B%22value%22%3A%22%E5%9B%A0%E5%AD%90%E6%8B%A5%E6%8C%A4%E5%BA%A6%E5%88%86%E6%9E%90%22%2C%22displayValue%22%3A%22%E5%9B%A0%E5%AD%90%E6%8B%A5%E6%8C%A4%E5%BA%A6%E5%88%86%E6%9E%90%22%2C%22selected%22%3Atrue%7D%2C%7B%22value%22%3A%22%E5%9B%A0%E5%AD%90%E5%80%BC%E6%9C%80%E5%A4%A7%2F%E6%9C%80%E5%B0%8F%E8%82%A1%E7%A5%A8%22%2C%22displayValue%22%3A%22%E5%9B%A0%E5%AD%90%E5%80%BC%E6%9C%80%E5%A4%A7%2F%E6%9C%80%E5%B0%8F%E8%82%A1%E7%A5%A8%22%2C%22selected%22%3Atrue%7D%2C%7B%22value%22%3A%22%E8%A1%A8%E8%BE%BE%E5%BC%8F%E5%9B%A0%E5%AD%90%E5%80%BC%22%2C%22displayValue%22%3A%22%E8%A1%A8%E8%BE%BE%E5%BC%8F%E5%9B%A0%E5%AD%90%E5%80%BC%22%2C%22selected%22%3Atrue%7D%2C%7B%22value%22%3A%22%E5%A4%9A%E5%9B%A0%E5%AD%90%E7%9B%B8%E5%85%B3%E6%80%A7%E5%88%86%E6%9E%90%22%2C%22displayValue%22%3A%22%E5%A4%9A%E5%9B%A0%E5%AD%90%E7%9B%B8%E5%85%B3%E6%80%A7%E5%88%86%E6%9E%90%22%2C%22selected%22%3Atrue%7D%5D%7D","type":"Literal","bound_global_parameter":null},{"name":"factor_coverage","value":0.5,"type":"Literal","bound_global_parameter":null},{"name":"user_data_merge","value":"left","type":"Literal","bound_global_parameter":null}],"input_ports":[{"name":"features","node_id":"-8309"},{"name":"user_factor_data","node_id":"-8309"}],"output_ports":[{"name":"data","node_id":"-8309"},{"name":"save_data","node_id":"-8309"}],"cacheable":true,"seq_num":2,"comment":"","comment_collapsed":true},{"node_id":"-2650","module_id":"BigQuantSpace.factorlens_preservation.factorlens_preservation-v2","parameters":[{"name":"factor_fields","value":"# 定义因子名称\n# {\n# \"列名\": {'name': \"因子名\", 'desc': \"因子描述\"},\n# \"列名\": {'name': \"因子名\", 'desc': \"因子描述\"},\n# ... \n# }\n{}\n","type":"Literal","bound_global_parameter":null},{"name":"table","value":"","type":"Literal","bound_global_parameter":null}],"input_ports":[{"name":"factors_info","node_id":"-2650"}],"output_ports":[{"name":"data","node_id":"-2650"}],"cacheable":false,"seq_num":3,"comment":"","comment_collapsed":true},{"node_id":"-43","module_id":"BigQuantSpace.dataclean.dataclean-v53","parameters":[{"name":"nan_method","value":"中位数填充","type":"Literal","bound_global_parameter":null},{"name":"outlier_method","value":"MAD法","type":"Literal","bound_global_parameter":null},{"name":"nan_fillvalue ","value":"0.0","type":"Literal","bound_global_parameter":null},{"name":"n_components","value":"1","type":"Literal","bound_global_parameter":null},{"name":"grp_len","value":"0","type":"Literal","bound_global_parameter":null},{"name":"if_inf","value":"True","type":"Literal","bound_global_parameter":null},{"name":"if_nan","value":"True","type":"Literal","bound_global_parameter":null},{"name":"if_outlier","value":"True","type":"Literal","bound_global_parameter":null},{"name":"if_standardize","value":"True","type":"Literal","bound_global_parameter":null},{"name":"if_neutralize","value":"True","type":"Literal","bound_global_parameter":null},{"name":"neutral_feature_remained","value":"True","type":"Literal","bound_global_parameter":null},{"name":"if_pca","value":"False","type":"Literal","bound_global_parameter":null},{"name":"whiten","value":"False","type":"Literal","bound_global_parameter":null}],"input_ports":[{"name":"feature_data1","node_id":"-43"},{"name":"feature_data2","node_id":"-43"},{"name":"cleaning_feature_list","node_id":"-43"},{"name":"neutral_feature_list","node_id":"-43"}],"output_ports":[{"name":"cleaned_data1","node_id":"-43"},{"name":"cleaned_data2","node_id":"-43"},{"name":"cleaned_factors","node_id":"-43"}],"cacheable":true,"seq_num":4,"comment":"","comment_collapsed":true},{"node_id":"-60","module_id":"BigQuantSpace.general_feature_extractor.general_feature_extractor-v7","parameters":[{"name":"start_date","value":"","type":"Literal","bound_global_parameter":null},{"name":"end_date","value":"","type":"Literal","bound_global_parameter":null},{"name":"before_start_days","value":"540","type":"Literal","bound_global_parameter":null}],"input_ports":[{"name":"instruments","node_id":"-60"},{"name":"features","node_id":"-60"}],"output_ports":[{"name":"data","node_id":"-60"}],"cacheable":true,"seq_num":5,"comment":"","comment_collapsed":true},{"node_id":"-67","module_id":"BigQuantSpace.derived_feature_extractor.derived_feature_extractor-v3","parameters":[{"name":"date_col","value":"date","type":"Literal","bound_global_parameter":null},{"name":"instrument_col","value":"instrument","type":"Literal","bound_global_parameter":null},{"name":"drop_na","value":"True","type":"Literal","bound_global_parameter":null},{"name":"remove_extra_columns","value":"True","type":"Literal","bound_global_parameter":null},{"name":"user_functions","value":"{}","type":"Literal","bound_global_parameter":null}],"input_ports":[{"name":"input_data","node_id":"-67"},{"name":"features","node_id":"-67"}],"output_ports":[{"name":"data","node_id":"-67"}],"cacheable":true,"seq_num":6,"comment":"","comment_collapsed":true},{"node_id":"-75","module_id":"BigQuantSpace.instruments.instruments-v2","parameters":[{"name":"start_date","value":"2022-01-01","type":"Literal","bound_global_parameter":null},{"name":"end_date","value":"2022-07-30","type":"Literal","bound_global_parameter":null},{"name":"market","value":"CN_STOCK_A","type":"Literal","bound_global_parameter":null},{"name":"instrument_list","value":"","type":"Literal","bound_global_parameter":null},{"name":"max_count","value":0,"type":"Literal","bound_global_parameter":null}],"input_ports":[{"name":"rolling_conf","node_id":"-75"}],"output_ports":[{"name":"data","node_id":"-75"}],"cacheable":true,"seq_num":7,"comment":"","comment_collapsed":true},{"node_id":"-83","module_id":"BigQuantSpace.input_features.input_features-v1","parameters":[{"name":"features","value":"td1\ntd2\ntd3\ntd4\nDD\n","type":"Literal","bound_global_parameter":null}],"input_ports":[{"name":"features_ds","node_id":"-83"}],"output_ports":[{"name":"data","node_id":"-83"}],"cacheable":true,"seq_num":8,"comment":"","comment_collapsed":true}],"node_layout":"<node_postions><node_position Node='-70' Position='631,-170,200,200'/><node_position Node='-8309' Position='296,307,200,200'/><node_position Node='-2650' Position='243,457,200,200'/><node_position Node='-43' Position='777,392,200,200'/><node_position Node='-60' Position='546,62,200,200'/><node_position Node='-67' Position='725,181,200,200'/><node_position Node='-75' Position='225,-43,200,200'/><node_position Node='-83' Position='299,160,200,200'/></node_postions>"},"nodes_readonly":false,"studio_version":"v2"}

# 本代码由可视化策略环境自动生成 2022年8月2日 08:17
# 本代码单元只能在可视化模式下编辑。您也可以拷贝代码，粘贴到新建的代码单元或者策略，然后修改。


m1 = M.input_features.v1(
    features="""# var1=(close_0*2+high_0+low_0)/4
# var2=ta_ema(var1,13)-ta_ema(var1,34)
# var3=ta_ema(var2,5)
# var4=var2-var3
# var5=covariance(var1, var4, 30)
# var6=var4-shift(var4,1)
AA=ta_ma((2*close_0+high_0+low_0)/4,5) 
td1=AA*102/100
td2=AA*(200-102)/100
CC=abs((2*close_0+high_0+low_0)/4-ta_ma(close_0,20))/ta_ma(close_0,20)
DD=ta_sma2(close_0,30,CC) #加权平均
td3=(1+7/100)*DD; 
td4=(1-7/100)*DD;



"""
)

m7 = M.instruments.v2(
    start_date='2022-01-01',
    end_date='2022-07-30',
    market='CN_STOCK_A',
    instrument_list='',
    max_count=0
)

m5 = M.general_feature_extractor.v7(
    instruments=m7.data,
    features=m1.data,
    start_date='',
    end_date='',
    before_start_days=540
)

m6 = M.derived_feature_extractor.v3(
    input_data=m5.data,
    features=m1.data,
    date_col='date',
    instrument_col='instrument',
    drop_na=True,
    remove_extra_columns=True,
    user_functions={}
)

m4 = M.dataclean.v53(
    feature_data1=m6.data,
    nan_method='中位数填充',
    outlier_method='MAD法',
    nan_fillvalue =0.0,
    n_components=1,
    grp_len=0,
    if_inf=True,
    if_nan=True,
    if_outlier=True,
    if_standardize=True,
    if_neutralize=True,
    neutral_feature_remained=True,
    if_pca=False,
    whiten=False
)

m8 = M.input_features.v1(
    features="""td1
td2
td3
td4
DD
"""
)

m2 = M.factorlens.v2(
    features=m8.data,
    user_factor_data=m6.data,
    title='因子分析: {factor_name}',
    start_date='2022-01-01',
    end_date='2022-07-30',
    rebalance_period=1,
    delay_rebalance_days=0,
    rebalance_price='close_0',
    stock_pool='全市场',
    quantile_count=20,
    commission_rate=0.0016,
    returns_calculation_method='累乘',
    benchmark='无',
    drop_new_stocks=60,
    drop_price_limit_stocks=True,
    drop_st_stocks=True,
    drop_suspended_stocks=True,
    cutoutliers=True,
    normalization=True,
    neutralization=['行业', '市值'],
    metrics=['因子表现概览', '因子分布', '因子行业分布', '因子市值分布', 'IC分析', '买入信号重合分析', '因子估值分析', '因子拥挤度分析', '因子值最大/最小股票', '表达式因子值', '多因子相关性分析'],
    factor_coverage=0.5,
    user_data_merge='left'
)

m3 = M.factorlens_preservation.v2(
    factors_info=m2.save_data,
    factor_fields=# 定义因子名称
# {
#     "列名": {'name': "因子名", 'desc': "因子描述"},
#     "列名": {'name': "因子名", 'desc': "因子描述"},
#     ... 
# }
{}
,
    table=''
)

[2022-08-02 07:58:55.609591] INFO: moduleinvoker: input_features.v1 开始运行..

[2022-08-02 07:58:55.618177] INFO: moduleinvoker: 命中缓存

[2022-08-02 07:58:55.620224] INFO: moduleinvoker: input_features.v1 运行完成[0.010662s].

[2022-08-02 07:58:55.626115] INFO: moduleinvoker: instruments.v2 开始运行..

[2022-08-02 07:58:55.633175] INFO: moduleinvoker: 命中缓存

[2022-08-02 07:58:55.635118] INFO: moduleinvoker: instruments.v2 运行完成[0.009003s].

[2022-08-02 07:58:55.675359] INFO: moduleinvoker: general_feature_extractor.v7 开始运行..

[2022-08-02 07:58:55.685438] INFO: moduleinvoker: 命中缓存

[2022-08-02 07:58:55.687403] INFO: moduleinvoker: general_feature_extractor.v7 运行完成[0.012069s].

[2022-08-02 07:58:55.695202] INFO: moduleinvoker: derived_feature_extractor.v3 开始运行..

[2022-08-02 07:58:55.714346] INFO: moduleinvoker: 命中缓存

[2022-08-02 07:58:55.716293] INFO: moduleinvoker: derived_feature_extractor.v3 运行完成[0.021082s].

[2022-08-02 07:58:55.727652] INFO: moduleinvoker: dataclean.v53 开始运行..

[2022-08-02 07:59:06.573295] INFO: moduleinvoker: dataclean.v53 运行完成[10.845617s].

[2022-08-02 07:59:06.581600] INFO: moduleinvoker: input_features.v1 开始运行..

[2022-08-02 07:59:06.589274] INFO: moduleinvoker: 命中缓存

[2022-08-02 07:59:06.591389] INFO: moduleinvoker: input_features.v1 运行完成[0.009818s].

[2022-08-02 07:59:06.600041] INFO: moduleinvoker: factorlens.v2 开始运行..

[2022-08-02 07:59:08.034810] INFO: 因子分析: batch_process start

[2022-08-02 07:59:08.037290] INFO: 因子分析: load_instruments 2022-01-01, 2022-07-30

[2022-08-02 07:59:08.372528] INFO: 因子分析: load_instruments, 4890 rows.

[2022-08-02 07:59:08.374897] INFO: 因子分析: load_benchmark_data 2022-01-01, 2022-07-30

[2022-08-02 07:59:08.428301] INFO: 因子分析: load_benchmark_data, 414 rows.

[2022-08-02 07:59:08.430203] INFO: 因子分析: StockPool.before_load_general_feature_data

[2022-08-02 07:59:08.432616] INFO: 因子分析: UserDataMerge.before_load_general_feature_data

[2022-08-02 07:59:08.435282] INFO: 因子分析: DropSTStocks.before_load_general_feature_data

[2022-08-02 07:59:08.437216] INFO: 因子分析: DropNewStocks.before_load_general_feature_data

[2022-08-02 07:59:08.438644] INFO: 因子分析: Neutralization.before_load_general_feature_data

[2022-08-02 07:59:08.440229] INFO: 因子分析: DelayRebalanceDays.before_load_general_feature_data

[2022-08-02 07:59:08.441818] INFO: 因子分析: RebalancePeriod.before_load_general_feature_data

[2022-08-02 07:59:08.443083] INFO: 因子分析: RebalancePrice.before_load_general_feature_data

[2022-08-02 07:59:08.444296] INFO: 因子分析: FactorCoverage.before_load_general_feature_data

[2022-08-02 07:59:08.445513] INFO: 因子分析: Industry.before_load_general_feature_data

[2022-08-02 07:59:08.446725] INFO: 因子分析: PBRatio.before_load_general_feature_data

[2022-08-02 07:59:08.447957] INFO: 因子分析: Turnover.before_load_general_feature_data

[2022-08-02 07:59:08.449296] INFO: 因子分析: MarketCap.before_load_general_feature_data

[2022-08-02 07:59:08.450572] INFO: 因子分析: load_general_feature_data, load data

[2022-08-02 07:59:56.650527] INFO: 因子分析: RebalancePeriod.after_load_general_feature_data

[2022-08-02 07:59:56.713721] INFO: 因子分析: RebalancePeriodsReturns.after_load_general_feature_data

[2022-08-02 08:00:25.834879] INFO: 因子分析: RebalancePrice.after_load_general_feature_data

[2022-08-02 08:00:25.855464] INFO: 因子分析: load_general_feature_data, 1717181 rows.

[2022-08-02 08:00:25.858793] INFO: 因子分析: load_derived_feature_data, 1717181 rows, 30 columns.

[2022-08-02 08:00:25.860624] INFO: 因子分析: process, td1

[2022-08-02 08:00:25.862216] INFO: 因子分析: calculate_factor, td1

[2022-08-02 08:00:26.160535] INFO: 因子分析: calculate_factor, done

[2022-08-02 08:00:26.238601] ERROR: 因子分析: 原始因子值覆盖率为 0.23654995730145176，小于 50.0%

[2022-08-02 08:00:26.256439] ERROR: moduleinvoker: module name: factorlens, module version: v2, trackeback: Exception: 原始因子值覆盖率小于 50.0%，无法进行后续指标计算

数据清洗（预处理）模块开始。。。 Tue Aug  2 07:58:55 2022
获取待清洗数据。。。 Tue Aug  2 07:58:55 2022
获取待清洗数据 OK！ Tue Aug  2 07:58:57 2022
去重。。。 Tue Aug  2 07:58:57 2022
去重 OK！ Tue Aug  2 07:58:57 2022
无穷值处理。。。 Tue Aug  2 07:58:57 2022
缺失值处理。。。中位数填充法 Tue Aug  2 07:58:58 2022
   ----残留缺失值nan、无穷值inf处理。。。
异常值处理。。。MAD法 Tue Aug  2 07:58:59 2022
   ----残留缺失值nan、无穷值inf处理。。。
标准化。。。 Tue Aug  2 07:59:01 2022
   ----残留缺失值nan、无穷值inf处理。。。

---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
<ipython-input-11-47710336f671> in <module>
     75 )
     76 
---> 77 m2 = M.factorlens.v2(
     78     features=m8.data,
     79     user_factor_data=m6.data,

Exception: 原始因子值覆盖率小于 50.0%，无法进行后续指标计算