将IsolationForest决策分数转换为概率算法

我期待创建一个通用功能的输出转换decision_scores的sklearn's IsolationForest成真概率[0.0, 1.0]。

我知道并阅读了原始论文，并且我在数学上理解该函数的输出不是概率，而是每个基估计器构建的路径长度的平均值，以隔离异常。

问题

我想将该输出转换为tuple (x,y)wherex=P(anomaly)和形式的概率y=1-x。

当前方法

def convert_probabilities(predictions, scores):
    from sklearn.preprocessing import MinMaxScaler

    new_scores = [(1,1) for _ in range(len(scores))]

    anomalous_idxs = [i for i in (range(len(predictions))) if predictions[i] == -1]
    regular_idxs = [i for i in (range(len(predictions))) if predictions[i] == 1]

    anomalous_scores = np.asarray(np.abs([scores[i] for i in anomalous_idxs]))
    regular_scores = np.asarray(np.abs([scores[i] for i in regular_idxs]))

    scaler = MinMaxScaler()

    anomalous_scores_scaled = scaler.fit_transform(anomalous_scores.reshape(-1,1))
    regular_scores_scaled = scaler.fit_transform(regular_scores.reshape(-1,1))

    for i, j in zip(anomalous_idxs, range(len(anomalous_scores_scaled))):
        new_scores[i] = (anomalous_scores_scaled[j][0], 1-anomalous_scores_scaled[j][0])
    
    for i, j in zip(regular_idxs, range(len(regular_scores_scaled))):
        new_scores[i] = (1-regular_scores_scaled[j][0], regular_scores_scaled[j][0])

    return new_scores

modified_scores = convert_probabilities(model_predictions, model_decisions)

最小的、可重现的示例

import pandas as pd
from sklearn.datasets import make_classification, load_iris
from sklearn.ensemble import IsolationForest
from sklearn.decomposition import PCA
from sklearn.model_selection import train_test_split

# Get data
X, y = load_iris(return_X_y=True, as_frame=True)
anomalies, anomalies_classes = make_classification(n_samples=int(X.shape[0]*0.05), n_features=X.shape[1], hypercube=False, random_state=60, shuffle=True)
anomalies_df = pd.DataFrame(data=anomalies, columns=X.columns)

# Split into train/test
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.15, random_state=60)

# Combine testing data
X_test['anomaly'] = 1
anomalies_df['anomaly'] = -1
X_test = X_test.append(anomalies_df, ignore_index=True)
y_test = X_test['anomaly']
X_test.drop('anomaly', inplace=True, axis=1)

# Build a model
model = IsolationForest(n_jobs=1, bootstrap=False, random_state=60)

# Fit it
model.fit(X_train)

# Test it
model_predictions = model.predict(X_test)
model_decisions = model.decision_function(X_test)

# Print results
for a,b,c in zip(y_test, model_predictions, model_decisions):
    print_str = """
    Class: {} | Model Prediction: {} | Model Decision Score: {}
    """.format(a,b,c)

    print(print_str)

问题

modified_scores = convert_probabilities(model_predictions, model_decisions)

# Print results
for a,b in zip(model_predictions, modified_scores):
    ans = False
    if a==-1:
        if b[0] > b[1]:
            ans = True
        else:
            ans = False
    elif a==1:
        if b[1] > b[0]:
            ans=True
        else:
            ans=False
    print_str = """
    Model Prediction: {} | Model Decision Score: {} | Correct: {}
    """.format(a,b, str(ans))

    print(print_str)

显示一些奇怪的结果，例如：

Model Prediction: 1 | Model Decision Score: (0.17604259932311161, 0.8239574006768884) | Correct: True
Model Prediction: 1 | Model Decision Score: (0.7120367886017022, 0.28796321139829784) | Correct: False
Model Prediction: 1 | Model Decision Score: (0.7251531538304419, 0.27484684616955807) | Correct: False
Model Prediction: -1 | Model Decision Score: (0.16776449326185877, 0.8322355067381413) | Correct: False
Model Prediction: 1 | Model Decision Score: (0.8395087028516501, 0.1604912971483499) | Correct: False

模型预测：1 | 模型决策分数：(0.0, 1.0) | 正确：正确

怎么可能预测是-1 (anomaly)，但概率只有 37%？或者预测为1 (normal)，但概率为 26%？

请注意，玩具数据集已标记，但无监督异常检测算法显然不假设任何标签。