{ "cells": [ { "cell_type": "markdown", "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 1000 }, "executionInfo": { "elapsed": 66330, "status": "ok", "timestamp": 1707202682814, "user": { "displayName": "Alice H", "userId": "13901604971984976961" }, "user_tz": -60 }, "id": "R1WJwi7C9Ng5", "outputId": "ec9c5093-9e95-40c2-d729-00d2bc4c01ad" }, "source": [ "Today we will need to use two different python env\n", "\n", "```\n", "python3.10 -m venv env_fairness\n", "source env_fairness/bin/activate\n", "pip install --upgrade pip\n", "pip install numpy==1.25 fairlearn==0.9.0 plotly==5.24.1 nbformat==5.10.4 aif360['inFairness']==0.6.1 ipykernel==6.29.5 BlackBoxAuditing==0.1.54 cvxpy==1.6.0\n", "cd env_fairness/lib/python3.9/site-packages/aif360/data/raw/meps\n", "Rscipt generate_data.R\n", "```\n", "\n", "```\n", "python3.10 -m venv env_adv\n", "source env_fairness/bin/activate\n", "pip install --upgrade pip\n", "pip install numpy==1.26 fairlearn==0.9.0 plotly==5.24.1 nbformat==5.10.4 aif360['AdversarialDebiasing']==0.6.1 aif360['inFairness']==0.6.1 ipykernel==6.29.5 BlackBoxAuditing==0.1.54 cvxpy==1.6.0\n", "cd env_adv/lib/python3.9/site-packages/aif360/data/raw/meps\n", "Rscipt generate_data.R\n", "```\n", "\n", "The part of the TD on Adversarial Devbiasing can run only with the second env, the rest of the document can run on the first env.\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "id": "OepcePtF1hIS" }, "source": [ "!!! Attention sur Colab!!!, après avoir executé la cellule ci-dessus, il faudra redémarrer la session (onglet \"Execution\") afin de charger l'environnement installé" ] }, { "cell_type": "markdown", "metadata": { "id": "HRe6S30b9Ng7" }, "source": [ "# TD 4: Mitigation des biais avec une méthode de in-processsing Prejudice Remover\n", "\n", "The aim of this notebook is to use the Prejudice Remover in-processing approach and analyse its impact on the model output.\n", "In terms of Machine Learning we will go a bit further in the train/valid/test paradigm.\n", "\n", "The model has to be learn on the train dataset, then the model parameters has to be optimized on the valid dataset, and finally the model performance is evaluated on the test dataset.\n", "No choice/decision etc can be taken depending on the test dataset. This could result on an overfitting on the test dataset.\n", "\n", "Here you will manipulate:\n", "- Prejudice Remover approach as a black box\n", "- Training of the prejudice remover using the train/valid paradigm. to choice the 'best' threshold\n", "- Combine Prejudice Remover with Reweighing\n", "\n", "As a reminder of pre-processing approach we encourage you to :\n", "- analyse the impact of the Reweighing on different model (Logistic Regression, Decision Tree, Random Forest, etc.)\n" ] }, { "cell_type": "markdown", "metadata": { "id": "9c9ITNS89NhA" }, "source": [ "## 1. Import and load the dataset" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "executionInfo": { "elapsed": 9, "status": "ok", "timestamp": 1707202697233, "user": { "displayName": "Alice H", "userId": "13901604971984976961" }, "user_tz": -60 }, "id": "88mMZIic9NhB", "outputId": "f0be967d-0531-4dd0-8d13-39accc70509e" }, "outputs": [], "source": [ "# imports\n", "import numpy as np\n", "import pandas as pd\n", "import plotly.express as px\n", "import warnings\n", "\n", "warnings.simplefilter(action=\"ignore\", category=FutureWarning)\n", "warnings.simplefilter(action=\"ignore\", append=True, category=UserWarning)\n", "# Datasets\n", "from aif360.datasets import MEPSDataset19\n", "\n", "# Fairness metrics\n", "from sklearn.metrics import accuracy_score, balanced_accuracy_score\n", "from sklearn.preprocessing import StandardScaler\n", "\n", "MEPSDataset19_data = MEPSDataset19()\n", "(dataset_orig_panel19_train, dataset_orig_panel19_val, dataset_orig_panel19_test) = (\n", " MEPSDataset19().split([0.5, 0.8], shuffle=True)\n", ")" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "executionInfo": { "elapsed": 20083, "status": "ok", "timestamp": 1707202719711, "user": { "displayName": "Alice H", "userId": "13901604971984976961" }, "user_tz": -60 }, "id": "LYbdfIPs9NhE" }, "outputs": [ { "data": { "text/plain": [ "(7915, 4749, 3166)" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(dataset_orig_panel19_train.instance_weights), len(\n", " dataset_orig_panel19_val.instance_weights\n", "), len(dataset_orig_panel19_test.instance_weights)" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([21854.981705, 18169.604822, 17191.832515, ..., 3896.116219,\n", " 4883.851005, 6630.588948])" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "instance_weights = MEPSDataset19_data.instance_weights\n", "instance_weights" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'Taille du dataset 15830, poids total du dataset 141367240.546316.'" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "f\"Taille du dataset {len(instance_weights)}, poids total du dataset {instance_weights.sum()}.\"" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2025-02-21 00:05:58.902977: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.\n", "2025-02-21 00:05:58.914642: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:477] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\n", "WARNING: All log messages before absl::InitializeLog() is called are written to STDERR\n", "E0000 00:00:1740092758.925168 1925928 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\n", "E0000 00:00:1740092758.928153 1925928 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n", "2025-02-21 00:05:58.939068: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.\n", "To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.\n" ] } ], "source": [ "from aif360.sklearn.metrics import *\n", "from sklearn.metrics import balanced_accuracy_score\n", "\n", " \n", "# This method takes lists\n", "def get_metrics(\n", " y_true, # list or np.array of truth values\n", " y_pred=None, # list or np.array of predictions\n", " prot_attr=None, # list or np.array of protected/sensitive attribute values\n", " priv_group=1, # value taken by the privileged group\n", " pos_label=1, # value taken by the positive truth/prediction\n", " sample_weight=None # list or np.array of weights value,\n", "):\n", " group_metrics = {}\n", " group_metrics[\"base_rate_truth\"] = base_rate(\n", " y_true=y_true, pos_label=pos_label, sample_weight=sample_weight\n", " )\n", " group_metrics[\"statistical_parity_difference\"] = statistical_parity_difference(\n", " y_true=y_true, y_pred=y_pred, prot_attr=prot_attr, priv_group=priv_group, pos_label=pos_label, sample_weight=sample_weight\n", " )\n", " group_metrics[\"disparate_impact_ratio\"] = disparate_impact_ratio(\n", " y_true=y_true, y_pred=y_pred, prot_attr=prot_attr, priv_group=priv_group, pos_label=pos_label, sample_weight=sample_weight\n", " )\n", " if not y_pred is None:\n", " group_metrics[\"base_rate_preds\"] = base_rate(\n", " y_true=y_pred, pos_label=pos_label, sample_weight=sample_weight\n", " )\n", " group_metrics[\"equal_opportunity_difference\"] = equal_opportunity_difference(\n", " y_true=y_true, y_pred=y_pred, prot_attr=prot_attr, priv_group=priv_group, pos_label=pos_label, sample_weight=sample_weight\n", " )\n", " group_metrics[\"average_odds_difference\"] = average_odds_difference(\n", " y_true=y_true, y_pred=y_pred, prot_attr=prot_attr, priv_group=priv_group, pos_label=pos_label, sample_weight=sample_weight\n", " )\n", " if len(set(y_pred))>1:\n", " group_metrics[\"conditional_demographic_disparity\"] = conditional_demographic_disparity(\n", " y_true=y_true, y_pred=y_pred, prot_attr=prot_attr, pos_label=pos_label, sample_weight=sample_weight\n", " )\n", " else:\n", " group_metrics[\"conditional_demographic_disparity\"] =None\n", " group_metrics[\"smoothed_edf\"] = smoothed_edf(\n", " y_true=y_true, y_pred=y_pred, prot_attr=prot_attr, pos_label=pos_label, sample_weight=sample_weight\n", " )\n", " group_metrics[\"df_bias_amplification\"] = df_bias_amplification(\n", " y_true=y_true, y_pred=y_pred, prot_attr=prot_attr, pos_label=pos_label, sample_weight=sample_weight\n", " )\n", " group_metrics[\"balanced_accuracy_score\"] = balanced_accuracy_score(\n", " y_true=y_true, y_pred=y_pred, sample_weight=sample_weight\n", " )\n", " return group_metrics" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Learning a Prejudice Remover model on the training dataset, and choose the best parameters with the validation dataset" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "# Bias mitigation techniques\n", "from aif360.algorithms.preprocessing import Reweighing\n", "from aif360.algorithms.inprocessing import PrejudiceRemover" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Question1 : Learn a Standard Scaler on the training dataset features, its output will be used as input of the model learned" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "pr_orig_scaler = StandardScaler()\n", "train_dataset_scaled = dataset_orig_panel19_train.copy()\n", "train_dataset_scaled.features = pr_orig_scaler.fit_transform(train_dataset_scaled.features)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Question2: Create a method to learn a Prejudice Remover on the train dataset and retrieve the model learned\n", "Execute the method with the parameter eta arbitrarily set at 25.0\n", "\n" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "def train_pr_model(eta=25.0, train_dataset=train_dataset_scaled):\n", " model = PrejudiceRemover(sensitive_attr='RACE', eta=eta)\n", " pr_orig_panel19 = model.fit(train_dataset)\n", " return pr_orig_panel19\n", "\n", "pr_orig_panel19 = train_pr_model()" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "((7915, 1),\n", " array([[0.24968875],\n", " [0.13644747],\n", " [0.17826004],\n", " [0.23045409],\n", " [0.43822148],\n", " [0.0735638 ],\n", " [0.03395558],\n", " [0.07592396],\n", " [0.15190029],\n", " [0.01911056]]))" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "train_scores = pr_orig_panel19.predict(train_dataset_scaled).scores\n", "train_scores.shape, train_scores[:10]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Le score du Prejudice Remover donne un sortie pour chaque instance une seule valeur, c'est un seuil, arbritrairement fixé à 0.5 par défault, qui permet à partir de ce score de décider la prédiction 1 ou 0.\n", "Si le score est supérieur au seuil la prédiction est 1, sinon c'est 0." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Validating: Choose the best parameters\n", "\n", "Here there are two parameters :\n", "- eta: fairness penalty parameter of the PR model\n", "- thershold: the threshold of the binary classification\n", "\n", "The threshold is used to obtains predictions from the model output.\n", "The eta is used during the training\n", "\n", "Question3: Create a method that will loop over 50 threshold ]0:0.5( and 5 values of ETA [1.0: 100.0], and outputs the metrics" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'base_rate_truth': 0.22072904528726825,\n", " 'statistical_parity_difference': -0.14003550662578024,\n", " 'disparate_impact_ratio': 0.4948564702571672}" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "get_metrics(\n", " y_true = dataset_orig_panel19_val.labels[:,0],\n", " prot_attr= dataset_orig_panel19_val.protected_attributes[:,0],\n", " sample_weight= dataset_orig_panel19_val.instance_weights\n", ")" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "def validation_loop(thereshold_list, eta_list, train_dataset=train_dataset_scaled, val_dataset=dataset_orig_panel19_val):\n", " dataset = val_dataset.copy()\n", " dataset.features = pr_orig_scaler.transform(dataset.features)\n", " y_true = val_dataset.labels[:,0]\n", " prot_attr = val_dataset.protected_attributes[:,0]\n", " sample_weight = val_dataset.instance_weights\n", " metrics_list=[]\n", "\n", " for eta in eta_list:\n", " eta_model = train_pr_model(eta=eta, train_dataset=train_dataset)\n", " y_val_pred_prob = eta_model.predict(dataset).scores\n", " for thr in thereshold_list:\n", " y_val_pred = (y_val_pred_prob[:, -1] > thr).astype(np.float64)\n", " metrics = get_metrics(y_true=y_true, y_pred=y_val_pred, prot_attr=prot_attr, sample_weight=sample_weight)\n", " metrics['threshold'] = thr\n", " metrics['eta']=eta\n", " metrics_list.append(metrics)\n", " return metrics_list\n" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
base_rate_truthstatistical_parity_differencedisparate_impact_ratiobase_rate_predsequal_opportunity_differenceaverage_odds_differenceconditional_demographic_disparitysmoothed_edfdf_bias_amplificationbalanced_accuracy_scorethresholdeta
00.220729-0.0348000.9650930.9829160.000000-0.019825-0.0963642.5199411.8164540.5109610.010.0
10.220729-0.0348000.9650930.9829160.000000-0.019825-0.0963642.5199411.8164540.5109610.010.0
20.220729-0.1242000.8736190.932640-0.010704-0.074537-0.0919262.1037431.4002560.5414980.020.0
30.220729-0.2041080.7868690.875326-0.010715-0.119249-0.0869651.7614791.0579920.5713150.030.0
40.220729-0.2481570.7269850.808846-0.031087-0.148392-0.0746291.3152230.6117350.6077330.040.0
.......................................
2950.220729-0.1263890.4361140.1731550.013595-0.070038-0.0410470.8298510.1263630.5205410.45100.0
2960.220729-0.1256170.4280690.1689630.014538-0.069146-0.0415970.8484710.1449830.5207660.46100.0
2970.220729-0.1232710.4247740.1645730.014929-0.067455-0.0416890.8561990.1527110.5211780.47100.0
2980.220729-0.1224070.4188600.1612540.012728-0.067504-0.0420820.8702180.1667300.5227490.48100.0
2990.220729-0.1187620.4189150.1564720.004199-0.068461-0.0418380.8700860.1665990.5233920.49100.0
\n", "

300 rows × 12 columns

\n", "
" ], "text/plain": [ " base_rate_truth statistical_parity_difference disparate_impact_ratio \\\n", "0 0.220729 -0.034800 0.965093 \n", "1 0.220729 -0.034800 0.965093 \n", "2 0.220729 -0.124200 0.873619 \n", "3 0.220729 -0.204108 0.786869 \n", "4 0.220729 -0.248157 0.726985 \n", ".. ... ... ... \n", "295 0.220729 -0.126389 0.436114 \n", "296 0.220729 -0.125617 0.428069 \n", "297 0.220729 -0.123271 0.424774 \n", "298 0.220729 -0.122407 0.418860 \n", "299 0.220729 -0.118762 0.418915 \n", "\n", " base_rate_preds equal_opportunity_difference average_odds_difference \\\n", "0 0.982916 0.000000 -0.019825 \n", "1 0.982916 0.000000 -0.019825 \n", "2 0.932640 -0.010704 -0.074537 \n", "3 0.875326 -0.010715 -0.119249 \n", "4 0.808846 -0.031087 -0.148392 \n", ".. ... ... ... \n", "295 0.173155 0.013595 -0.070038 \n", "296 0.168963 0.014538 -0.069146 \n", "297 0.164573 0.014929 -0.067455 \n", "298 0.161254 0.012728 -0.067504 \n", "299 0.156472 0.004199 -0.068461 \n", "\n", " conditional_demographic_disparity smoothed_edf df_bias_amplification \\\n", "0 -0.096364 2.519941 1.816454 \n", "1 -0.096364 2.519941 1.816454 \n", "2 -0.091926 2.103743 1.400256 \n", "3 -0.086965 1.761479 1.057992 \n", "4 -0.074629 1.315223 0.611735 \n", ".. ... ... ... \n", "295 -0.041047 0.829851 0.126363 \n", "296 -0.041597 0.848471 0.144983 \n", "297 -0.041689 0.856199 0.152711 \n", "298 -0.042082 0.870218 0.166730 \n", "299 -0.041838 0.870086 0.166599 \n", "\n", " balanced_accuracy_score threshold eta \n", "0 0.510961 0.01 0.0 \n", "1 0.510961 0.01 0.0 \n", "2 0.541498 0.02 0.0 \n", "3 0.571315 0.03 0.0 \n", "4 0.607733 0.04 0.0 \n", ".. ... ... ... \n", "295 0.520541 0.45 100.0 \n", "296 0.520766 0.46 100.0 \n", "297 0.521178 0.47 100.0 \n", "298 0.522749 0.48 100.0 \n", "299 0.523392 0.49 100.0 \n", "\n", "[300 rows x 12 columns]" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "metrics_list = validation_loop([float(x/100) if x>0 else 0.01 for x in range(50) ], [ float(x*20) for x in range(6)])\n", "df_metrics = pd.DataFrame.from_records(metrics_list)\n", "df_metrics" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Question4 : Make plot to choose the best set of parameters" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "data": { "text/html": [ " \n", " " ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "dimensions": [ { "label": "eta", "values": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100, 100 ] }, { "label": "threshold", "values": [ 0.01, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.2, 0.21, 0.22, 0.23, 0.24, 0.25, 0.26, 0.27, 0.28, 0.29, 0.3, 0.31, 0.32, 0.33, 0.34, 0.35, 0.36, 0.37, 0.38, 0.39, 0.4, 0.41, 0.42, 0.43, 0.44, 0.45, 0.46, 0.47, 0.48, 0.49, 0.01, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.2, 0.21, 0.22, 0.23, 0.24, 0.25, 0.26, 0.27, 0.28, 0.29, 0.3, 0.31, 0.32, 0.33, 0.34, 0.35, 0.36, 0.37, 0.38, 0.39, 0.4, 0.41, 0.42, 0.43, 0.44, 0.45, 0.46, 0.47, 0.48, 0.49, 0.01, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.2, 0.21, 0.22, 0.23, 0.24, 0.25, 0.26, 0.27, 0.28, 0.29, 0.3, 0.31, 0.32, 0.33, 0.34, 0.35, 0.36, 0.37, 0.38, 0.39, 0.4, 0.41, 0.42, 0.43, 0.44, 0.45, 0.46, 0.47, 0.48, 0.49, 0.01, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.2, 0.21, 0.22, 0.23, 0.24, 0.25, 0.26, 0.27, 0.28, 0.29, 0.3, 0.31, 0.32, 0.33, 0.34, 0.35, 0.36, 0.37, 0.38, 0.39, 0.4, 0.41, 0.42, 0.43, 0.44, 0.45, 0.46, 0.47, 0.48, 0.49, 0.01, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.2, 0.21, 0.22, 0.23, 0.24, 0.25, 0.26, 0.27, 0.28, 0.29, 0.3, 0.31, 0.32, 0.33, 0.34, 0.35, 0.36, 0.37, 0.38, 0.39, 0.4, 0.41, 0.42, 0.43, 0.44, 0.45, 0.46, 0.47, 0.48, 0.49, 0.01, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.2, 0.21, 0.22, 0.23, 0.24, 0.25, 0.26, 0.27, 0.28, 0.29, 0.3, 0.31, 0.32, 0.33, 0.34, 0.35, 0.36, 0.37, 0.38, 0.39, 0.4, 0.41, 0.42, 0.43, 0.44, 0.45, 0.46, 0.47, 0.48, 0.49 ] }, { "label": "disparate_impact_ratio", "values": [ 0.9650933673240849, 0.9650933673240849, 0.8736186118170117, 0.7868689601016461, 0.726985472406239, 0.6643861650549473, 0.619054675044755, 0.5836034526708127, 0.5562282888572373, 0.5429960605688472, 0.5180583276333358, 0.5026421611048213, 0.47581110707400687, 0.4602974332652371, 0.44231808584776444, 0.4315357298705358, 0.41725445554766716, 0.40749181539203333, 0.392600993313623, 0.3901948488593432, 0.38058914710875863, 0.3827970089729716, 0.3753984617516786, 0.3677696972213884, 0.3656698303744842, 0.3558696517582266, 0.34821855542910946, 0.3479140451779799, 0.3463670866292369, 0.34333265095753035, 0.335396357734376, 0.32563264519202745, 0.30764245746559443, 0.30273984705223117, 0.2890012869904753, 0.27797808159449855, 0.2820348567525622, 0.2751242294941177, 0.277737207688373, 0.2637733390301788, 0.2611401792693285, 0.2637213623673389, 0.2610504747278131, 0.2530168219392297, 0.2550628060682903, 0.2526311441596078, 0.25428799955091147, 0.24087310144709956, 0.23779833535299616, 0.22967589864907073, 0.963314181471242, 0.963314181471242, 0.8807131707355261, 0.8411481260947196, 0.8194489122162136, 0.7799764640724375, 0.7511721389708301, 0.7291453660461874, 0.7288967731781038, 0.725569795757268, 0.707033785193194, 0.7123510777366497, 0.7395570026144208, 0.7399684539516608, 0.7225728781911076, 0.7368361028596564, 0.7126268956228481, 0.7110902597760814, 0.6866296192661444, 0.6836832625688659, 0.6800404080644112, 0.6866719269569938, 0.682794532378674, 0.702281146370338, 0.7063262591019529, 0.6911175913082453, 0.7130891807029798, 0.7203833446786981, 0.7136206769116691, 0.7091364084122266, 0.6988519482950569, 0.7096051749311225, 0.7188695125491821, 0.7141560660406046, 0.7224734771970732, 0.7087907827964245, 0.6914602825593985, 0.6805658760440624, 0.681470822572435, 0.6461713208521921, 0.6750907572113202, 0.6778921939246015, 0.6821472037660717, 0.6856435869795626, 0.6640601412974245, 0.6666071714116432, 0.6796082613395651, 0.6741716390432214, 0.6723522442912154, 0.6530797770141087, 0.8973830150421825, 0.8973830150421825, 0.8893082211537583, 0.9077512505661129, 0.9093999571214672, 0.9114200987378945, 0.9263986499360489, 0.9319833718202538, 0.9253360705991622, 0.9615691638015581, 0.9827056422049681, 1.0007424789319401, 1.0117318079304696, 0.9959284565401977, 0.9769627677051677, 0.966514924977367, 0.9809523900133243, 0.9950234408170374, 1.0305052263287187, 1.0206231297249595, 1.0098135883940225, 1.015473141952556, 0.9835133287586665, 0.9683265941976198, 0.9596771254276361, 0.974828173560442, 0.9433558284949821, 0.9381445746261989, 0.9388135231715095, 0.9256330206336196, 0.9258372606675406, 0.9163805949392881, 0.9164869505186075, 0.8978930992464901, 0.9066033648477941, 0.9119912628993024, 0.92434832700193, 0.9757891835005937, 1.005654516359029, 0.9977226626216316, 0.9970795585926706, 1.0198235485825378, 1.0019104734307813, 0.998047634899678, 0.9927119490043712, 0.9732173613869628, 0.9986399611053889, 0.9988010744231925, 0.9493227954549315, 0.94588712114835, 0.8816165542663743, 0.8816165542663743, 0.8680454500406405, 0.883240666636796, 0.8952612856990633, 0.9038971693357876, 0.9166265773556613, 0.9353400878394497, 0.9571071916253527, 0.9991253052299575, 1.003641737120282, 1.0065882985019203, 1.012234218590537, 1.0242871228076842, 1.0067960035202286, 1.0168902228171322, 1.0188342967762265, 1.0230689343699548, 1.0159170322192015, 0.998408484678902, 0.9916348409376852, 0.9719378031761577, 1.0209366471645702, 0.9948763091150228, 0.9696121885725438, 0.9737764586764689, 0.9884690200104471, 0.9802534702242749, 0.974104934817171, 0.96441051237234, 0.9543474697156412, 0.966649602152849, 0.9807824204295365, 0.9595563090848613, 0.9548637341680961, 0.9440178156955563, 0.9679981640789487, 0.9734598292100191, 1.0113921056047315, 1.0122027678507155, 1.0171678788795435, 1.0085943876256325, 1.0421353694476119, 1.033747572149649, 1.0321474070544723, 1.0401003061115957, 0.9953985500837865, 1.0125368010569384, 1.001277820501402, 0.994914179220583, 0.9524783709626349, 0.9524783709626349, 0.9617961869386813, 0.9556532741172704, 0.9514194809467734, 0.9820375089286361, 0.9941060990580717, 1.0163592540484268, 1.0136946816514847, 1.0123593315268002, 0.9998716998436612, 0.9849481067965148, 0.9810667413408432, 0.9568836148347759, 0.9393926705239393, 0.9288508996041105, 0.9351849586516416, 0.8852408512192036, 0.8695160071914747, 0.8455030328213646, 0.8066423345978126, 0.7847190708282261, 0.7537855856031581, 0.7279758197322921, 0.7048190855773415, 0.6814759461589325, 0.6592075716704164, 0.645659516944468, 0.6260366190341587, 0.6108924309613357, 0.5983744310672011, 0.5885854822612764, 0.576782422243781, 0.5656607719217658, 0.5355988130963429, 0.5180227033797546, 0.5110847513286874, 0.5038654001435389, 0.49695827998271414, 0.4953731707550673, 0.4766558069802402, 0.4697863377128181, 0.446032196851839, 0.43455718794316817, 0.43374010814925, 0.42740194198840387, 0.42165468244020576, 0.42999872308561377, 0.4347300159637954, 0.4273008738765101, 0.9616567125507279, 0.9616567125507279, 0.9651058642496508, 0.9755715168217017, 0.9807082825684995, 1.0165655028441114, 1.0375112176332775, 1.0323012136589789, 1.050224485550277, 1.0521874558685076, 1.0380457483564414, 1.0237478827340214, 1.0248331050160095, 1.0035716866068263, 1.0085756067419482, 1.0117289210918006, 0.9945351141668962, 0.9754027482360388, 0.9336032206668309, 0.885739193303206, 0.8561340603464314, 0.8193433493424315, 0.8055040575682382, 0.7705310087903976, 0.7479843184223732, 0.7209286233022053, 0.6971464966220862, 0.6808528385041172, 0.667013978768577, 0.653960828905449, 0.6435774603504585, 0.6310638278809134, 0.6160951786951875, 0.5923655966044981, 0.5796168821527131, 0.5645059229376111, 0.5474645703116824, 0.5332477928275347, 0.5145242560141252, 0.5026854608975677, 0.4945103105640736, 0.49516247368598854, 0.4882028576851781, 0.47066358664742586, 0.45306920136987155, 0.4361142278820672, 0.4280690058018673, 0.42477357307408437, 0.4188603273839654, 0.4189152331975446 ] }, { "label": "df_bias_amplification", "values": [ 1.81645382221327, 1.81645382221327, 1.4002559518068471, 1.0579917225715805, 0.6117354090699063, 0.48617873159950564, 0.33720032094776586, 0.2464991090407712, 0.15042954625061133, 0.03468009212578993, -0.038979209883453825, -0.015610663999989782, 0.03924689374768642, 0.07239495377058924, 0.11223854679464218, 0.1369175035300847, 0.17057156597967704, 0.1942469520698269, 0.23147397904993605, 0.23762155812835906, 0.26254733676467124, 0.25676292489306496, 0.2762797359063134, 0.2968108354694008, 0.30253692581121117, 0.3297032222477023, 0.35143741281467245, 0.35231227118753505, 0.3567685604254467, 0.3655679034029822, 0.38895471049562214, 0.41849779736129356, 0.47532941063544754, 0.49139381095178325, 0.5378364950005399, 0.5767253499173324, 0.5622369410532162, 0.5870448536914246, 0.5775922180519846, 0.6291773924687998, 0.6392101972268684, 0.6293744425614578, 0.639553752908063, 0.670811538165796, 0.6627576957848431, 0.6723369965541559, 0.6658000037959424, 0.7199972048086598, 0.7328444530050475, 0.7675982211442376, 1.2620048397770605, 1.2620048397770605, 0.5191488629310479, 0.05613204036097752, -0.13776762152582123, -0.18538976164380672, -0.2498049325843028, -0.29117217058307754, -0.36681312047298364, -0.3826894047000684, -0.3568105893300755, -0.3643030178381119, -0.40178350481909586, -0.40233970180009093, -0.3785504355938083, -0.39809763729222114, -0.3646901561920799, -0.362531534507853, -0.32752719200166713, -0.32322692086387295, -0.31788440204307467, -0.3275888210094846, -0.3219261747551856, -0.3500660092799297, -0.35580945022798716, -0.33404218719391343, -0.3653386965107266, -0.3755157026508207, -0.3660837709010558, -0.35978012398864956, -0.3451711420017214, -0.3604409518951457, -0.3734120896208162, -0.36683375722150613, -0.3784129490430441, -0.3592926454445735, -0.33453797356532267, -0.3186568936953522, -0.31998571270914455, -0.2667969684053637, -0.31057943557087375, -0.31472057410226606, -0.3209777856984162, -0.3260902530401548, -0.2941050573607984, -0.2979332680007496, -0.3172488915094842, -0.3092170919061057, -0.3065147452904613, -0.27743167894374166, 0.8899636366758958, 0.8899636366758958, -0.13531619388395844, -0.4270069401914731, -0.5100833135230554, -0.5573400666231043, -0.605448512169324, -0.630327294874838, -0.625889110460961, -0.6642986148891612, -0.6860417467382441, -0.7027451832518199, -0.6918238623069664, -0.6994075481816908, -0.6801806695896688, -0.6694288701410256, -0.6842560560418696, -0.6984984254460618, -0.6734381854340958, -0.6830740233899439, -0.6937216250129241, -0.6881327182305506, -0.686863330901017, -0.6713015651076493, -0.6623290478338133, -0.6779933704856058, -0.6451757015408672, -0.6396362236882867, -0.6403490269544394, -0.6262100126463477, -0.6264306395543842, -0.6161639439450399, -0.6162800011946188, -0.5957831971543754, -0.605437229542537, -0.6113625901551343, -0.6248211589494956, -0.6789787368804308, -0.6978487466854579, -0.7012075197722103, -0.7005627420010323, -0.6838577103352788, -0.7015786735831577, -0.7015331917190148, -0.6961727306764045, -0.6763396488218816, -0.7021265120322719, -0.7022878332993518, -0.6514811003606065, -0.6478554617854702, 1.0483491488929104, 1.0483491488929104, -0.006872483775803628, -0.34249792191246886, -0.47645179254193093, -0.5435313472649541, -0.5915319446365056, -0.6329206291338549, -0.6596475107777116, -0.7026123184730025, -0.6998522669477966, -0.6969206970915085, -0.6913274010849642, -0.6794905050711535, -0.6967143681535302, -0.6867382129061206, -0.6848282536165067, -0.6806805107236309, -0.6876956933001597, -0.7018946278383629, -0.6950870698633136, -0.6750239520134025, -0.6827668848592874, -0.6983505563202788, -0.6726283308573768, -0.6769139150778185, -0.6918894480604163, -0.6835433307637087, -0.6772511861680961, -0.6672492023353525, -0.6567599882321118, -0.669568233954744, -0.6840828015226619, -0.6622031656993168, -0.6573008138766652, -0.645877212790058, -0.6709623624316405, -0.6765887322917363, -0.692159634596559, -0.6913584231979626, -0.6864651591586779, -0.6949296655466615, -0.6622154872007191, -0.6702967123642685, -0.6718458346072538, -0.6641701707298369, -0.6988754034319009, -0.6910284482560507, -0.7022103035521794, -0.6983886858711097, 0.11573071455447037, 0.11573071455447037, -0.5285240927933361, -0.5887955286918329, -0.6157273037928295, -0.67796817555324, -0.6965332042110978, -0.6867936045159485, -0.6898856361221201, -0.6912038132304764, -0.7033590878141719, -0.6883210755477932, -0.6843726114829177, -0.6594138923102941, -0.6409656965585768, -0.629680357507735, -0.6364764547672117, -0.5815918903211046, -0.5636688800583369, -0.5356638958799467, -0.48861251238159653, -0.46105793221089386, -0.4208401137590585, -0.3859999939930392, -0.3536733203592475, -0.31999312704447824, -0.2867706421132119, -0.2660044832930659, -0.2351410542235879, -0.21065308262419102, -0.18994889430417627, -0.173454372259076, -0.1531973174855119, -0.13372676800263905, -0.07911762256747346, -0.04575130416479678, -0.03226767042212941, -0.018041415907404046, -0.004238332039779635, -0.0010436149949741846, 0.03747307984038084, 0.05198973183236122, 0.10387656468309103, 0.12994013628563406, 0.13182215767315775, 0.1465427854454011, 0.1600809725640624, 0.1404854242435043, 0.12954247339182223, 0.14677925919364077, -0.15005527843175281, -0.15005527843175281, -0.5603438859141037, -0.6471895951203992, -0.671573197673573, -0.6819038621341708, -0.6627163514078925, -0.6716968963057631, -0.6544834573602633, -0.6526161051013896, -0.6661475346479804, -0.6800171020874937, -0.6789576144731817, -0.6999220639038044, -0.6949483413470998, -0.6918267153065778, -0.6980075296621087, -0.6785825868544304, -0.6347836613888991, -0.5821546793237256, -0.5481591168105009, -0.5042353716940051, -0.48720038854455106, -0.4428120533830102, -0.4131141718155569, -0.3762723000134687, -0.34272774034363107, -0.31907836100158793, -0.29854318229119947, -0.2787796378116889, -0.26277457999085985, -0.24313920232598396, -0.21913365881216507, -0.17985621030259913, -0.15809954640830148, -0.1316830904737527, -0.10102997039911243, -0.07471844700714314, -0.038974937444129054, -0.01569689540516883, 0.0006997426429686815, -0.000618198767752709, 0.013536721937538854, 0.05012413601868326, 0.08822283107486162, 0.12636349190293128, 0.14498327021165536, 0.15271141551767853, 0.16673014770898553, 0.16659906516198397 ] }, { "label": "balanced_accuracy_score", "values": [ 0.5109612848476206, 0.5109612848476206, 0.5414980403675151, 0.5713152611497219, 0.6077327484607126, 0.6302069654208775, 0.6554754912618395, 0.6745039201264618, 0.6945410655666613, 0.7122819786586643, 0.7276071114725813, 0.7408166330735442, 0.7462868347936277, 0.7484058258209678, 0.7546824757503479, 0.7589299313141137, 0.7654051566579012, 0.7685909189057216, 0.7656697283258168, 0.7641215702641719, 0.7644582632156895, 0.7644128654631557, 0.7645963167586065, 0.7640349324003542, 0.7634299645376372, 0.763874778125602, 0.7649867975341536, 0.7628723956621075, 0.7620287218184678, 0.7607340752773838, 0.7598688495300403, 0.7581641235312324, 0.752064453017862, 0.7523485802387999, 0.7492375777823028, 0.7465586088887175, 0.7409890259290601, 0.7355735236849783, 0.7328135296071252, 0.7308807007445438, 0.7246745564388668, 0.7221866661437648, 0.7211041965563163, 0.7195842453999919, 0.7143761847891007, 0.7093141207427606, 0.7060480192898155, 0.7006881598255246, 0.6982816246961777, 0.6944818967855964, 0.5126705370674992, 0.5126705370674992, 0.5493875371908447, 0.591281787154237, 0.6206649782419081, 0.6476553447261496, 0.6633225287819831, 0.6719880994874016, 0.6810735811222104, 0.6899789509797821, 0.6932549876153992, 0.7016947402587042, 0.6926806974808929, 0.6878662916227524, 0.6880466229086158, 0.6972122229495723, 0.6927570089212414, 0.6907016154989256, 0.6863580610736686, 0.684790189541366, 0.6821169260979657, 0.6715641774040277, 0.6698580348714673, 0.6675196546767965, 0.6632999473166097, 0.6628150784723333, 0.6558657588062996, 0.6512763344962544, 0.6471552679599704, 0.6412765842703892, 0.6392158231381957, 0.6379990495784312, 0.6319347231837107, 0.6334364102105626, 0.6351538143485723, 0.6349392690640366, 0.6322297882658493, 0.6274863669663393, 0.6253773398373993, 0.6211365146595915, 0.6177272943602008, 0.6149608705613974, 0.6131997577081179, 0.6115112089547314, 0.6077028854165943, 0.6046897947104902, 0.6017097162012759, 0.5999666741223565, 0.5950215885454772, 0.5939348195266626, 0.5344143556982293, 0.5344143556982293, 0.5786617276969358, 0.6063574244208711, 0.6165983819936012, 0.6299284777355438, 0.6325380132372375, 0.6391826709573607, 0.6400387720968757, 0.6436083677785807, 0.6438904254829957, 0.6371617414500982, 0.638405062463147, 0.6381117369995997, 0.6385177406154925, 0.6342717417528732, 0.6370767852720836, 0.6403077948554274, 0.6373054309334272, 0.6386428056597853, 0.6355835278472929, 0.634735260421625, 0.6331097654717294, 0.6308867459064713, 0.6261268128222707, 0.627916384432293, 0.6275176074034999, 0.6268211175082008, 0.6217671780702397, 0.6187687706171652, 0.6179423496150632, 0.6136346242753352, 0.6152654551764044, 0.6138722548659016, 0.6109582468788554, 0.6113083157048934, 0.6082905572260283, 0.6083692212450277, 0.6047903393309497, 0.6033937767534855, 0.6031379093410626, 0.6016325835117555, 0.5978313372919, 0.5966588735063345, 0.5964337241014482, 0.5938192047370462, 0.5909567340702249, 0.5889608348971433, 0.5886236566936239, 0.5876859652874807, 0.5375852131043547, 0.5375852131043547, 0.5759465197987601, 0.6170743528773298, 0.6246474535885453, 0.6375850550598052, 0.6496390169617441, 0.6544963140978595, 0.6572024019523344, 0.6578825726385547, 0.6527007259683517, 0.6505059496538688, 0.6479433948024194, 0.652997799175308, 0.6558266426956025, 0.6521325955996295, 0.6487455787884058, 0.6484406092962859, 0.6483160076550225, 0.6466517421105656, 0.6460032646525091, 0.6471304617477004, 0.6399254668947846, 0.6376966998664171, 0.6366362444344578, 0.6330574747544541, 0.6336623105573074, 0.6343812611836358, 0.6320433515989059, 0.627151555982793, 0.6240470145861636, 0.6217951844348495, 0.621179616399871, 0.6196801156741383, 0.6200493754176573, 0.6179567435961176, 0.6157599704249092, 0.6135472530088901, 0.6125112321378356, 0.6103799315980579, 0.6053829891556127, 0.6038093005888412, 0.5995023647905098, 0.5983180061377673, 0.5993840645124938, 0.5986727918141388, 0.5976928640523376, 0.5932399011023737, 0.5914756955730279, 0.5894433681580621, 0.5294043099348525, 0.5294043099348525, 0.5870562565954851, 0.6134617061648169, 0.6233411891362031, 0.6264215886535833, 0.6230158946120922, 0.6075640166236542, 0.6071790402348902, 0.600348744224781, 0.5937804349006589, 0.5893250379362429, 0.5819776598189736, 0.581259621406339, 0.5786046075922222, 0.5701330264106896, 0.5564610600382216, 0.5581498790771527, 0.5599564127233174, 0.5546617125617017, 0.5514016274967056, 0.5490038501001832, 0.5455835845009352, 0.544353777839586, 0.5440415506839139, 0.5423433355449078, 0.5432463066579941, 0.5417407470282458, 0.539990584758715, 0.53977328767059, 0.5390418995933569, 0.5365696289097346, 0.5374811942040083, 0.5331758386541696, 0.5294053374064596, 0.5282864103094083, 0.5287939464457109, 0.530322122378893, 0.5309250151349905, 0.5291515786034502, 0.5283737307823646, 0.5279508920020282, 0.5295875596594358, 0.5297111721517156, 0.5295889137881509, 0.52991340482779, 0.5300264298969698, 0.5292720585952391, 0.5288455239794005, 0.5307673941726737, 0.5333913928246267, 0.5333913928246267, 0.5911254881481989, 0.6175560449562696, 0.6251695292046912, 0.6235962584427117, 0.6090183120620544, 0.6036708118546279, 0.599564773684074, 0.5877032408233256, 0.5848203852073115, 0.5769630094477575, 0.575916940735451, 0.5709092144200417, 0.5559633831647647, 0.5525326854104131, 0.545728299266838, 0.5476282762020166, 0.5455141035821116, 0.54603060902421, 0.5406530973919239, 0.5377968826680176, 0.5378978381200284, 0.5364499637108333, 0.5355011989530811, 0.5344763917573115, 0.5322124367077071, 0.5352005055193234, 0.5349784115855506, 0.5319636451447178, 0.5283341229748559, 0.5298786916880929, 0.5260727997787753, 0.5251659773676381, 0.5224821007622513, 0.5218688080280524, 0.5197261592968138, 0.5190416602237282, 0.520500400946504, 0.5177999215055209, 0.5215442065989206, 0.5194179714833178, 0.518916980193778, 0.5197961022450263, 0.5201119605278067, 0.5205409686121052, 0.5207664113890408, 0.5211779308333035, 0.5227491167875202, 0.5233922548283434 ] } ], "domain": { "x": [ 0, 1 ], "y": [ 0, 1 ] }, "line": { "color": [ 0.5109612848476206, 0.5109612848476206, 0.5414980403675151, 0.5713152611497219, 0.6077327484607126, 0.6302069654208775, 0.6554754912618395, 0.6745039201264618, 0.6945410655666613, 0.7122819786586643, 0.7276071114725813, 0.7408166330735442, 0.7462868347936277, 0.7484058258209678, 0.7546824757503479, 0.7589299313141137, 0.7654051566579012, 0.7685909189057216, 0.7656697283258168, 0.7641215702641719, 0.7644582632156895, 0.7644128654631557, 0.7645963167586065, 0.7640349324003542, 0.7634299645376372, 0.763874778125602, 0.7649867975341536, 0.7628723956621075, 0.7620287218184678, 0.7607340752773838, 0.7598688495300403, 0.7581641235312324, 0.752064453017862, 0.7523485802387999, 0.7492375777823028, 0.7465586088887175, 0.7409890259290601, 0.7355735236849783, 0.7328135296071252, 0.7308807007445438, 0.7246745564388668, 0.7221866661437648, 0.7211041965563163, 0.7195842453999919, 0.7143761847891007, 0.7093141207427606, 0.7060480192898155, 0.7006881598255246, 0.6982816246961777, 0.6944818967855964, 0.5126705370674992, 0.5126705370674992, 0.5493875371908447, 0.591281787154237, 0.6206649782419081, 0.6476553447261496, 0.6633225287819831, 0.6719880994874016, 0.6810735811222104, 0.6899789509797821, 0.6932549876153992, 0.7016947402587042, 0.6926806974808929, 0.6878662916227524, 0.6880466229086158, 0.6972122229495723, 0.6927570089212414, 0.6907016154989256, 0.6863580610736686, 0.684790189541366, 0.6821169260979657, 0.6715641774040277, 0.6698580348714673, 0.6675196546767965, 0.6632999473166097, 0.6628150784723333, 0.6558657588062996, 0.6512763344962544, 0.6471552679599704, 0.6412765842703892, 0.6392158231381957, 0.6379990495784312, 0.6319347231837107, 0.6334364102105626, 0.6351538143485723, 0.6349392690640366, 0.6322297882658493, 0.6274863669663393, 0.6253773398373993, 0.6211365146595915, 0.6177272943602008, 0.6149608705613974, 0.6131997577081179, 0.6115112089547314, 0.6077028854165943, 0.6046897947104902, 0.6017097162012759, 0.5999666741223565, 0.5950215885454772, 0.5939348195266626, 0.5344143556982293, 0.5344143556982293, 0.5786617276969358, 0.6063574244208711, 0.6165983819936012, 0.6299284777355438, 0.6325380132372375, 0.6391826709573607, 0.6400387720968757, 0.6436083677785807, 0.6438904254829957, 0.6371617414500982, 0.638405062463147, 0.6381117369995997, 0.6385177406154925, 0.6342717417528732, 0.6370767852720836, 0.6403077948554274, 0.6373054309334272, 0.6386428056597853, 0.6355835278472929, 0.634735260421625, 0.6331097654717294, 0.6308867459064713, 0.6261268128222707, 0.627916384432293, 0.6275176074034999, 0.6268211175082008, 0.6217671780702397, 0.6187687706171652, 0.6179423496150632, 0.6136346242753352, 0.6152654551764044, 0.6138722548659016, 0.6109582468788554, 0.6113083157048934, 0.6082905572260283, 0.6083692212450277, 0.6047903393309497, 0.6033937767534855, 0.6031379093410626, 0.6016325835117555, 0.5978313372919, 0.5966588735063345, 0.5964337241014482, 0.5938192047370462, 0.5909567340702249, 0.5889608348971433, 0.5886236566936239, 0.5876859652874807, 0.5375852131043547, 0.5375852131043547, 0.5759465197987601, 0.6170743528773298, 0.6246474535885453, 0.6375850550598052, 0.6496390169617441, 0.6544963140978595, 0.6572024019523344, 0.6578825726385547, 0.6527007259683517, 0.6505059496538688, 0.6479433948024194, 0.652997799175308, 0.6558266426956025, 0.6521325955996295, 0.6487455787884058, 0.6484406092962859, 0.6483160076550225, 0.6466517421105656, 0.6460032646525091, 0.6471304617477004, 0.6399254668947846, 0.6376966998664171, 0.6366362444344578, 0.6330574747544541, 0.6336623105573074, 0.6343812611836358, 0.6320433515989059, 0.627151555982793, 0.6240470145861636, 0.6217951844348495, 0.621179616399871, 0.6196801156741383, 0.6200493754176573, 0.6179567435961176, 0.6157599704249092, 0.6135472530088901, 0.6125112321378356, 0.6103799315980579, 0.6053829891556127, 0.6038093005888412, 0.5995023647905098, 0.5983180061377673, 0.5993840645124938, 0.5986727918141388, 0.5976928640523376, 0.5932399011023737, 0.5914756955730279, 0.5894433681580621, 0.5294043099348525, 0.5294043099348525, 0.5870562565954851, 0.6134617061648169, 0.6233411891362031, 0.6264215886535833, 0.6230158946120922, 0.6075640166236542, 0.6071790402348902, 0.600348744224781, 0.5937804349006589, 0.5893250379362429, 0.5819776598189736, 0.581259621406339, 0.5786046075922222, 0.5701330264106896, 0.5564610600382216, 0.5581498790771527, 0.5599564127233174, 0.5546617125617017, 0.5514016274967056, 0.5490038501001832, 0.5455835845009352, 0.544353777839586, 0.5440415506839139, 0.5423433355449078, 0.5432463066579941, 0.5417407470282458, 0.539990584758715, 0.53977328767059, 0.5390418995933569, 0.5365696289097346, 0.5374811942040083, 0.5331758386541696, 0.5294053374064596, 0.5282864103094083, 0.5287939464457109, 0.530322122378893, 0.5309250151349905, 0.5291515786034502, 0.5283737307823646, 0.5279508920020282, 0.5295875596594358, 0.5297111721517156, 0.5295889137881509, 0.52991340482779, 0.5300264298969698, 0.5292720585952391, 0.5288455239794005, 0.5307673941726737, 0.5333913928246267, 0.5333913928246267, 0.5911254881481989, 0.6175560449562696, 0.6251695292046912, 0.6235962584427117, 0.6090183120620544, 0.6036708118546279, 0.599564773684074, 0.5877032408233256, 0.5848203852073115, 0.5769630094477575, 0.575916940735451, 0.5709092144200417, 0.5559633831647647, 0.5525326854104131, 0.545728299266838, 0.5476282762020166, 0.5455141035821116, 0.54603060902421, 0.5406530973919239, 0.5377968826680176, 0.5378978381200284, 0.5364499637108333, 0.5355011989530811, 0.5344763917573115, 0.5322124367077071, 0.5352005055193234, 0.5349784115855506, 0.5319636451447178, 0.5283341229748559, 0.5298786916880929, 0.5260727997787753, 0.5251659773676381, 0.5224821007622513, 0.5218688080280524, 0.5197261592968138, 0.5190416602237282, 0.520500400946504, 0.5177999215055209, 0.5215442065989206, 0.5194179714833178, 0.518916980193778, 0.5197961022450263, 0.5201119605278067, 0.5205409686121052, 0.5207664113890408, 0.5211779308333035, 0.5227491167875202, 0.5233922548283434 ], "coloraxis": "coloraxis" }, "name": "", "type": "parcoords" } ], "layout": { "coloraxis": { "colorbar": { "title": { "text": "balanced_accuracy_score" } }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "legend": { "tracegroupgap": 0 }, "margin": { "t": 60 }, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } } } }, "text/html": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fig = px.parallel_coordinates(\n", " df_metrics, \n", " color=\"balanced_accuracy_score\", \n", " dimensions=[\"eta\", 'threshold', 'disparate_impact_ratio','df_bias_amplification', 'balanced_accuracy_score'])\n", "fig.show()\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "L'ETA et le threshold ont de l'impact sur l'accuracy et le disparate impact.\n", "On remarque que le compromis est difficile à trouver, comme le montre le graphique ci-dessous, peut etre plus lisible." ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "customdata": [ [ 0.01, 0 ], [ 0.01, 0 ], [ 0.02, 0 ], [ 0.03, 0 ], [ 0.04, 0 ], [ 0.05, 0 ], [ 0.06, 0 ], [ 0.07, 0 ], [ 0.08, 0 ], [ 0.09, 0 ], [ 0.1, 0 ], [ 0.11, 0 ], [ 0.12, 0 ], [ 0.13, 0 ], [ 0.14, 0 ], [ 0.15, 0 ], [ 0.16, 0 ], [ 0.17, 0 ], [ 0.18, 0 ], [ 0.19, 0 ], [ 0.2, 0 ], [ 0.21, 0 ], [ 0.22, 0 ], [ 0.23, 0 ], [ 0.24, 0 ], [ 0.25, 0 ], [ 0.26, 0 ], [ 0.27, 0 ], [ 0.28, 0 ], [ 0.29, 0 ], [ 0.3, 0 ], [ 0.31, 0 ], [ 0.32, 0 ], [ 0.33, 0 ], [ 0.34, 0 ], [ 0.35, 0 ], [ 0.36, 0 ], [ 0.37, 0 ], [ 0.38, 0 ], [ 0.39, 0 ], [ 0.4, 0 ], [ 0.41, 0 ], [ 0.42, 0 ], [ 0.43, 0 ], [ 0.44, 0 ], [ 0.45, 0 ], [ 0.46, 0 ], [ 0.47, 0 ], [ 0.48, 0 ], [ 0.49, 0 ] ], "hovertemplate": "eta=%{customdata[1]}
balanced_accuracy_score=%{x}
disparate_impact_ratio=%{y}
threshold=%{marker.color}", "legendgroup": "", "marker": { "color": [ 0.01, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.2, 0.21, 0.22, 0.23, 0.24, 0.25, 0.26, 0.27, 0.28, 0.29, 0.3, 0.31, 0.32, 0.33, 0.34, 0.35, 0.36, 0.37, 0.38, 0.39, 0.4, 0.41, 0.42, 0.43, 0.44, 0.45, 0.46, 0.47, 0.48, 0.49 ], "coloraxis": "coloraxis", "symbol": "circle" }, "mode": "markers", "name": "", "orientation": "v", "showlegend": false, "type": "scatter", "x": [ 0.5109612848476206, 0.5109612848476206, 0.5414980403675151, 0.5713152611497219, 0.6077327484607126, 0.6302069654208775, 0.6554754912618395, 0.6745039201264618, 0.6945410655666613, 0.7122819786586643, 0.7276071114725813, 0.7408166330735442, 0.7462868347936277, 0.7484058258209678, 0.7546824757503479, 0.7589299313141137, 0.7654051566579012, 0.7685909189057216, 0.7656697283258168, 0.7641215702641719, 0.7644582632156895, 0.7644128654631557, 0.7645963167586065, 0.7640349324003542, 0.7634299645376372, 0.763874778125602, 0.7649867975341536, 0.7628723956621075, 0.7620287218184678, 0.7607340752773838, 0.7598688495300403, 0.7581641235312324, 0.752064453017862, 0.7523485802387999, 0.7492375777823028, 0.7465586088887175, 0.7409890259290601, 0.7355735236849783, 0.7328135296071252, 0.7308807007445438, 0.7246745564388668, 0.7221866661437648, 0.7211041965563163, 0.7195842453999919, 0.7143761847891007, 0.7093141207427606, 0.7060480192898155, 0.7006881598255246, 0.6982816246961777, 0.6944818967855964 ], "xaxis": "x", "y": [ 0.9650933673240849, 0.9650933673240849, 0.8736186118170117, 0.7868689601016461, 0.726985472406239, 0.6643861650549473, 0.619054675044755, 0.5836034526708127, 0.5562282888572373, 0.5429960605688472, 0.5180583276333358, 0.5026421611048213, 0.47581110707400687, 0.4602974332652371, 0.44231808584776444, 0.4315357298705358, 0.41725445554766716, 0.40749181539203333, 0.392600993313623, 0.3901948488593432, 0.38058914710875863, 0.3827970089729716, 0.3753984617516786, 0.3677696972213884, 0.3656698303744842, 0.3558696517582266, 0.34821855542910946, 0.3479140451779799, 0.3463670866292369, 0.34333265095753035, 0.335396357734376, 0.32563264519202745, 0.30764245746559443, 0.30273984705223117, 0.2890012869904753, 0.27797808159449855, 0.2820348567525622, 0.2751242294941177, 0.277737207688373, 0.2637733390301788, 0.2611401792693285, 0.2637213623673389, 0.2610504747278131, 0.2530168219392297, 0.2550628060682903, 0.2526311441596078, 0.25428799955091147, 0.24087310144709956, 0.23779833535299616, 0.22967589864907073 ], "yaxis": "y" }, { "customdata": [ [ 0.01, 20 ], [ 0.01, 20 ], [ 0.02, 20 ], [ 0.03, 20 ], [ 0.04, 20 ], [ 0.05, 20 ], [ 0.06, 20 ], [ 0.07, 20 ], [ 0.08, 20 ], [ 0.09, 20 ], [ 0.1, 20 ], [ 0.11, 20 ], [ 0.12, 20 ], [ 0.13, 20 ], [ 0.14, 20 ], [ 0.15, 20 ], [ 0.16, 20 ], [ 0.17, 20 ], [ 0.18, 20 ], [ 0.19, 20 ], [ 0.2, 20 ], [ 0.21, 20 ], [ 0.22, 20 ], [ 0.23, 20 ], [ 0.24, 20 ], [ 0.25, 20 ], [ 0.26, 20 ], [ 0.27, 20 ], [ 0.28, 20 ], [ 0.29, 20 ], [ 0.3, 20 ], [ 0.31, 20 ], [ 0.32, 20 ], [ 0.33, 20 ], [ 0.34, 20 ], [ 0.35, 20 ], [ 0.36, 20 ], [ 0.37, 20 ], [ 0.38, 20 ], [ 0.39, 20 ], [ 0.4, 20 ], [ 0.41, 20 ], [ 0.42, 20 ], [ 0.43, 20 ], [ 0.44, 20 ], [ 0.45, 20 ], [ 0.46, 20 ], [ 0.47, 20 ], [ 0.48, 20 ], [ 0.49, 20 ] ], "hovertemplate": "eta=%{customdata[1]}
balanced_accuracy_score=%{x}
disparate_impact_ratio=%{y}
threshold=%{marker.color}", "legendgroup": "", "marker": { "color": [ 0.01, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.2, 0.21, 0.22, 0.23, 0.24, 0.25, 0.26, 0.27, 0.28, 0.29, 0.3, 0.31, 0.32, 0.33, 0.34, 0.35, 0.36, 0.37, 0.38, 0.39, 0.4, 0.41, 0.42, 0.43, 0.44, 0.45, 0.46, 0.47, 0.48, 0.49 ], "coloraxis": "coloraxis", "symbol": "circle" }, "mode": "markers", "name": "", "orientation": "v", "showlegend": false, "type": "scatter", "x": [ 0.5126705370674992, 0.5126705370674992, 0.5493875371908447, 0.591281787154237, 0.6206649782419081, 0.6476553447261496, 0.6633225287819831, 0.6719880994874016, 0.6810735811222104, 0.6899789509797821, 0.6932549876153992, 0.7016947402587042, 0.6926806974808929, 0.6878662916227524, 0.6880466229086158, 0.6972122229495723, 0.6927570089212414, 0.6907016154989256, 0.6863580610736686, 0.684790189541366, 0.6821169260979657, 0.6715641774040277, 0.6698580348714673, 0.6675196546767965, 0.6632999473166097, 0.6628150784723333, 0.6558657588062996, 0.6512763344962544, 0.6471552679599704, 0.6412765842703892, 0.6392158231381957, 0.6379990495784312, 0.6319347231837107, 0.6334364102105626, 0.6351538143485723, 0.6349392690640366, 0.6322297882658493, 0.6274863669663393, 0.6253773398373993, 0.6211365146595915, 0.6177272943602008, 0.6149608705613974, 0.6131997577081179, 0.6115112089547314, 0.6077028854165943, 0.6046897947104902, 0.6017097162012759, 0.5999666741223565, 0.5950215885454772, 0.5939348195266626 ], "xaxis": "x2", "y": [ 0.963314181471242, 0.963314181471242, 0.8807131707355261, 0.8411481260947196, 0.8194489122162136, 0.7799764640724375, 0.7511721389708301, 0.7291453660461874, 0.7288967731781038, 0.725569795757268, 0.707033785193194, 0.7123510777366497, 0.7395570026144208, 0.7399684539516608, 0.7225728781911076, 0.7368361028596564, 0.7126268956228481, 0.7110902597760814, 0.6866296192661444, 0.6836832625688659, 0.6800404080644112, 0.6866719269569938, 0.682794532378674, 0.702281146370338, 0.7063262591019529, 0.6911175913082453, 0.7130891807029798, 0.7203833446786981, 0.7136206769116691, 0.7091364084122266, 0.6988519482950569, 0.7096051749311225, 0.7188695125491821, 0.7141560660406046, 0.7224734771970732, 0.7087907827964245, 0.6914602825593985, 0.6805658760440624, 0.681470822572435, 0.6461713208521921, 0.6750907572113202, 0.6778921939246015, 0.6821472037660717, 0.6856435869795626, 0.6640601412974245, 0.6666071714116432, 0.6796082613395651, 0.6741716390432214, 0.6723522442912154, 0.6530797770141087 ], "yaxis": "y2" }, { "customdata": [ [ 0.01, 40 ], [ 0.01, 40 ], [ 0.02, 40 ], [ 0.03, 40 ], [ 0.04, 40 ], [ 0.05, 40 ], [ 0.06, 40 ], [ 0.07, 40 ], [ 0.08, 40 ], [ 0.09, 40 ], [ 0.1, 40 ], [ 0.11, 40 ], [ 0.12, 40 ], [ 0.13, 40 ], [ 0.14, 40 ], [ 0.15, 40 ], [ 0.16, 40 ], [ 0.17, 40 ], [ 0.18, 40 ], [ 0.19, 40 ], [ 0.2, 40 ], [ 0.21, 40 ], [ 0.22, 40 ], [ 0.23, 40 ], [ 0.24, 40 ], [ 0.25, 40 ], [ 0.26, 40 ], [ 0.27, 40 ], [ 0.28, 40 ], [ 0.29, 40 ], [ 0.3, 40 ], [ 0.31, 40 ], [ 0.32, 40 ], [ 0.33, 40 ], [ 0.34, 40 ], [ 0.35, 40 ], [ 0.36, 40 ], [ 0.37, 40 ], [ 0.38, 40 ], [ 0.39, 40 ], [ 0.4, 40 ], [ 0.41, 40 ], [ 0.42, 40 ], [ 0.43, 40 ], [ 0.44, 40 ], [ 0.45, 40 ], [ 0.46, 40 ], [ 0.47, 40 ], [ 0.48, 40 ], [ 0.49, 40 ] ], "hovertemplate": "eta=%{customdata[1]}
balanced_accuracy_score=%{x}
disparate_impact_ratio=%{y}
threshold=%{marker.color}", "legendgroup": "", "marker": { "color": [ 0.01, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.2, 0.21, 0.22, 0.23, 0.24, 0.25, 0.26, 0.27, 0.28, 0.29, 0.3, 0.31, 0.32, 0.33, 0.34, 0.35, 0.36, 0.37, 0.38, 0.39, 0.4, 0.41, 0.42, 0.43, 0.44, 0.45, 0.46, 0.47, 0.48, 0.49 ], "coloraxis": "coloraxis", "symbol": "circle" }, "mode": "markers", "name": "", "orientation": "v", "showlegend": false, "type": "scatter", "x": [ 0.5344143556982293, 0.5344143556982293, 0.5786617276969358, 0.6063574244208711, 0.6165983819936012, 0.6299284777355438, 0.6325380132372375, 0.6391826709573607, 0.6400387720968757, 0.6436083677785807, 0.6438904254829957, 0.6371617414500982, 0.638405062463147, 0.6381117369995997, 0.6385177406154925, 0.6342717417528732, 0.6370767852720836, 0.6403077948554274, 0.6373054309334272, 0.6386428056597853, 0.6355835278472929, 0.634735260421625, 0.6331097654717294, 0.6308867459064713, 0.6261268128222707, 0.627916384432293, 0.6275176074034999, 0.6268211175082008, 0.6217671780702397, 0.6187687706171652, 0.6179423496150632, 0.6136346242753352, 0.6152654551764044, 0.6138722548659016, 0.6109582468788554, 0.6113083157048934, 0.6082905572260283, 0.6083692212450277, 0.6047903393309497, 0.6033937767534855, 0.6031379093410626, 0.6016325835117555, 0.5978313372919, 0.5966588735063345, 0.5964337241014482, 0.5938192047370462, 0.5909567340702249, 0.5889608348971433, 0.5886236566936239, 0.5876859652874807 ], "xaxis": "x3", "y": [ 0.8973830150421825, 0.8973830150421825, 0.8893082211537583, 0.9077512505661129, 0.9093999571214672, 0.9114200987378945, 0.9263986499360489, 0.9319833718202538, 0.9253360705991622, 0.9615691638015581, 0.9827056422049681, 1.0007424789319401, 1.0117318079304696, 0.9959284565401977, 0.9769627677051677, 0.966514924977367, 0.9809523900133243, 0.9950234408170374, 1.0305052263287187, 1.0206231297249595, 1.0098135883940225, 1.015473141952556, 0.9835133287586665, 0.9683265941976198, 0.9596771254276361, 0.974828173560442, 0.9433558284949821, 0.9381445746261989, 0.9388135231715095, 0.9256330206336196, 0.9258372606675406, 0.9163805949392881, 0.9164869505186075, 0.8978930992464901, 0.9066033648477941, 0.9119912628993024, 0.92434832700193, 0.9757891835005937, 1.005654516359029, 0.9977226626216316, 0.9970795585926706, 1.0198235485825378, 1.0019104734307813, 0.998047634899678, 0.9927119490043712, 0.9732173613869628, 0.9986399611053889, 0.9988010744231925, 0.9493227954549315, 0.94588712114835 ], "yaxis": "y3" }, { "customdata": [ [ 0.01, 60 ], [ 0.01, 60 ], [ 0.02, 60 ], [ 0.03, 60 ], [ 0.04, 60 ], [ 0.05, 60 ], [ 0.06, 60 ], [ 0.07, 60 ], [ 0.08, 60 ], [ 0.09, 60 ], [ 0.1, 60 ], [ 0.11, 60 ], [ 0.12, 60 ], [ 0.13, 60 ], [ 0.14, 60 ], [ 0.15, 60 ], [ 0.16, 60 ], [ 0.17, 60 ], [ 0.18, 60 ], [ 0.19, 60 ], [ 0.2, 60 ], [ 0.21, 60 ], [ 0.22, 60 ], [ 0.23, 60 ], [ 0.24, 60 ], [ 0.25, 60 ], [ 0.26, 60 ], [ 0.27, 60 ], [ 0.28, 60 ], [ 0.29, 60 ], [ 0.3, 60 ], [ 0.31, 60 ], [ 0.32, 60 ], [ 0.33, 60 ], [ 0.34, 60 ], [ 0.35, 60 ], [ 0.36, 60 ], [ 0.37, 60 ], [ 0.38, 60 ], [ 0.39, 60 ], [ 0.4, 60 ], [ 0.41, 60 ], [ 0.42, 60 ], [ 0.43, 60 ], [ 0.44, 60 ], [ 0.45, 60 ], [ 0.46, 60 ], [ 0.47, 60 ], [ 0.48, 60 ], [ 0.49, 60 ] ], "hovertemplate": "eta=%{customdata[1]}
balanced_accuracy_score=%{x}
disparate_impact_ratio=%{y}
threshold=%{marker.color}", "legendgroup": "", "marker": { "color": [ 0.01, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.2, 0.21, 0.22, 0.23, 0.24, 0.25, 0.26, 0.27, 0.28, 0.29, 0.3, 0.31, 0.32, 0.33, 0.34, 0.35, 0.36, 0.37, 0.38, 0.39, 0.4, 0.41, 0.42, 0.43, 0.44, 0.45, 0.46, 0.47, 0.48, 0.49 ], "coloraxis": "coloraxis", "symbol": "circle" }, "mode": "markers", "name": "", "orientation": "v", "showlegend": false, "type": "scatter", "x": [ 0.5375852131043547, 0.5375852131043547, 0.5759465197987601, 0.6170743528773298, 0.6246474535885453, 0.6375850550598052, 0.6496390169617441, 0.6544963140978595, 0.6572024019523344, 0.6578825726385547, 0.6527007259683517, 0.6505059496538688, 0.6479433948024194, 0.652997799175308, 0.6558266426956025, 0.6521325955996295, 0.6487455787884058, 0.6484406092962859, 0.6483160076550225, 0.6466517421105656, 0.6460032646525091, 0.6471304617477004, 0.6399254668947846, 0.6376966998664171, 0.6366362444344578, 0.6330574747544541, 0.6336623105573074, 0.6343812611836358, 0.6320433515989059, 0.627151555982793, 0.6240470145861636, 0.6217951844348495, 0.621179616399871, 0.6196801156741383, 0.6200493754176573, 0.6179567435961176, 0.6157599704249092, 0.6135472530088901, 0.6125112321378356, 0.6103799315980579, 0.6053829891556127, 0.6038093005888412, 0.5995023647905098, 0.5983180061377673, 0.5993840645124938, 0.5986727918141388, 0.5976928640523376, 0.5932399011023737, 0.5914756955730279, 0.5894433681580621 ], "xaxis": "x4", "y": [ 0.8816165542663743, 0.8816165542663743, 0.8680454500406405, 0.883240666636796, 0.8952612856990633, 0.9038971693357876, 0.9166265773556613, 0.9353400878394497, 0.9571071916253527, 0.9991253052299575, 1.003641737120282, 1.0065882985019203, 1.012234218590537, 1.0242871228076842, 1.0067960035202286, 1.0168902228171322, 1.0188342967762265, 1.0230689343699548, 1.0159170322192015, 0.998408484678902, 0.9916348409376852, 0.9719378031761577, 1.0209366471645702, 0.9948763091150228, 0.9696121885725438, 0.9737764586764689, 0.9884690200104471, 0.9802534702242749, 0.974104934817171, 0.96441051237234, 0.9543474697156412, 0.966649602152849, 0.9807824204295365, 0.9595563090848613, 0.9548637341680961, 0.9440178156955563, 0.9679981640789487, 0.9734598292100191, 1.0113921056047315, 1.0122027678507155, 1.0171678788795435, 1.0085943876256325, 1.0421353694476119, 1.033747572149649, 1.0321474070544723, 1.0401003061115957, 0.9953985500837865, 1.0125368010569384, 1.001277820501402, 0.994914179220583 ], "yaxis": "y4" }, { "customdata": [ [ 0.01, 80 ], [ 0.01, 80 ], [ 0.02, 80 ], [ 0.03, 80 ], [ 0.04, 80 ], [ 0.05, 80 ], [ 0.06, 80 ], [ 0.07, 80 ], [ 0.08, 80 ], [ 0.09, 80 ], [ 0.1, 80 ], [ 0.11, 80 ], [ 0.12, 80 ], [ 0.13, 80 ], [ 0.14, 80 ], [ 0.15, 80 ], [ 0.16, 80 ], [ 0.17, 80 ], [ 0.18, 80 ], [ 0.19, 80 ], [ 0.2, 80 ], [ 0.21, 80 ], [ 0.22, 80 ], [ 0.23, 80 ], [ 0.24, 80 ], [ 0.25, 80 ], [ 0.26, 80 ], [ 0.27, 80 ], [ 0.28, 80 ], [ 0.29, 80 ], [ 0.3, 80 ], [ 0.31, 80 ], [ 0.32, 80 ], [ 0.33, 80 ], [ 0.34, 80 ], [ 0.35, 80 ], [ 0.36, 80 ], [ 0.37, 80 ], [ 0.38, 80 ], [ 0.39, 80 ], [ 0.4, 80 ], [ 0.41, 80 ], [ 0.42, 80 ], [ 0.43, 80 ], [ 0.44, 80 ], [ 0.45, 80 ], [ 0.46, 80 ], [ 0.47, 80 ], [ 0.48, 80 ], [ 0.49, 80 ] ], "hovertemplate": "eta=%{customdata[1]}
balanced_accuracy_score=%{x}
disparate_impact_ratio=%{y}
threshold=%{marker.color}", "legendgroup": "", "marker": { "color": [ 0.01, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.2, 0.21, 0.22, 0.23, 0.24, 0.25, 0.26, 0.27, 0.28, 0.29, 0.3, 0.31, 0.32, 0.33, 0.34, 0.35, 0.36, 0.37, 0.38, 0.39, 0.4, 0.41, 0.42, 0.43, 0.44, 0.45, 0.46, 0.47, 0.48, 0.49 ], "coloraxis": "coloraxis", "symbol": "circle" }, "mode": "markers", "name": "", "orientation": "v", "showlegend": false, "type": "scatter", "x": [ 0.5294043099348525, 0.5294043099348525, 0.5870562565954851, 0.6134617061648169, 0.6233411891362031, 0.6264215886535833, 0.6230158946120922, 0.6075640166236542, 0.6071790402348902, 0.600348744224781, 0.5937804349006589, 0.5893250379362429, 0.5819776598189736, 0.581259621406339, 0.5786046075922222, 0.5701330264106896, 0.5564610600382216, 0.5581498790771527, 0.5599564127233174, 0.5546617125617017, 0.5514016274967056, 0.5490038501001832, 0.5455835845009352, 0.544353777839586, 0.5440415506839139, 0.5423433355449078, 0.5432463066579941, 0.5417407470282458, 0.539990584758715, 0.53977328767059, 0.5390418995933569, 0.5365696289097346, 0.5374811942040083, 0.5331758386541696, 0.5294053374064596, 0.5282864103094083, 0.5287939464457109, 0.530322122378893, 0.5309250151349905, 0.5291515786034502, 0.5283737307823646, 0.5279508920020282, 0.5295875596594358, 0.5297111721517156, 0.5295889137881509, 0.52991340482779, 0.5300264298969698, 0.5292720585952391, 0.5288455239794005, 0.5307673941726737 ], "xaxis": "x5", "y": [ 0.9524783709626349, 0.9524783709626349, 0.9617961869386813, 0.9556532741172704, 0.9514194809467734, 0.9820375089286361, 0.9941060990580717, 1.0163592540484268, 1.0136946816514847, 1.0123593315268002, 0.9998716998436612, 0.9849481067965148, 0.9810667413408432, 0.9568836148347759, 0.9393926705239393, 0.9288508996041105, 0.9351849586516416, 0.8852408512192036, 0.8695160071914747, 0.8455030328213646, 0.8066423345978126, 0.7847190708282261, 0.7537855856031581, 0.7279758197322921, 0.7048190855773415, 0.6814759461589325, 0.6592075716704164, 0.645659516944468, 0.6260366190341587, 0.6108924309613357, 0.5983744310672011, 0.5885854822612764, 0.576782422243781, 0.5656607719217658, 0.5355988130963429, 0.5180227033797546, 0.5110847513286874, 0.5038654001435389, 0.49695827998271414, 0.4953731707550673, 0.4766558069802402, 0.4697863377128181, 0.446032196851839, 0.43455718794316817, 0.43374010814925, 0.42740194198840387, 0.42165468244020576, 0.42999872308561377, 0.4347300159637954, 0.4273008738765101 ], "yaxis": "y5" }, { "customdata": [ [ 0.01, 100 ], [ 0.01, 100 ], [ 0.02, 100 ], [ 0.03, 100 ], [ 0.04, 100 ], [ 0.05, 100 ], [ 0.06, 100 ], [ 0.07, 100 ], [ 0.08, 100 ], [ 0.09, 100 ], [ 0.1, 100 ], [ 0.11, 100 ], [ 0.12, 100 ], [ 0.13, 100 ], [ 0.14, 100 ], [ 0.15, 100 ], [ 0.16, 100 ], [ 0.17, 100 ], [ 0.18, 100 ], [ 0.19, 100 ], [ 0.2, 100 ], [ 0.21, 100 ], [ 0.22, 100 ], [ 0.23, 100 ], [ 0.24, 100 ], [ 0.25, 100 ], [ 0.26, 100 ], [ 0.27, 100 ], [ 0.28, 100 ], [ 0.29, 100 ], [ 0.3, 100 ], [ 0.31, 100 ], [ 0.32, 100 ], [ 0.33, 100 ], [ 0.34, 100 ], [ 0.35, 100 ], [ 0.36, 100 ], [ 0.37, 100 ], [ 0.38, 100 ], [ 0.39, 100 ], [ 0.4, 100 ], [ 0.41, 100 ], [ 0.42, 100 ], [ 0.43, 100 ], [ 0.44, 100 ], [ 0.45, 100 ], [ 0.46, 100 ], [ 0.47, 100 ], [ 0.48, 100 ], [ 0.49, 100 ] ], "hovertemplate": "eta=%{customdata[1]}
balanced_accuracy_score=%{x}
disparate_impact_ratio=%{y}
threshold=%{marker.color}", "legendgroup": "", "marker": { "color": [ 0.01, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.2, 0.21, 0.22, 0.23, 0.24, 0.25, 0.26, 0.27, 0.28, 0.29, 0.3, 0.31, 0.32, 0.33, 0.34, 0.35, 0.36, 0.37, 0.38, 0.39, 0.4, 0.41, 0.42, 0.43, 0.44, 0.45, 0.46, 0.47, 0.48, 0.49 ], "coloraxis": "coloraxis", "symbol": "circle" }, "mode": "markers", "name": "", "orientation": "v", "showlegend": false, "type": "scatter", "x": [ 0.5333913928246267, 0.5333913928246267, 0.5911254881481989, 0.6175560449562696, 0.6251695292046912, 0.6235962584427117, 0.6090183120620544, 0.6036708118546279, 0.599564773684074, 0.5877032408233256, 0.5848203852073115, 0.5769630094477575, 0.575916940735451, 0.5709092144200417, 0.5559633831647647, 0.5525326854104131, 0.545728299266838, 0.5476282762020166, 0.5455141035821116, 0.54603060902421, 0.5406530973919239, 0.5377968826680176, 0.5378978381200284, 0.5364499637108333, 0.5355011989530811, 0.5344763917573115, 0.5322124367077071, 0.5352005055193234, 0.5349784115855506, 0.5319636451447178, 0.5283341229748559, 0.5298786916880929, 0.5260727997787753, 0.5251659773676381, 0.5224821007622513, 0.5218688080280524, 0.5197261592968138, 0.5190416602237282, 0.520500400946504, 0.5177999215055209, 0.5215442065989206, 0.5194179714833178, 0.518916980193778, 0.5197961022450263, 0.5201119605278067, 0.5205409686121052, 0.5207664113890408, 0.5211779308333035, 0.5227491167875202, 0.5233922548283434 ], "xaxis": "x6", "y": [ 0.9616567125507279, 0.9616567125507279, 0.9651058642496508, 0.9755715168217017, 0.9807082825684995, 1.0165655028441114, 1.0375112176332775, 1.0323012136589789, 1.050224485550277, 1.0521874558685076, 1.0380457483564414, 1.0237478827340214, 1.0248331050160095, 1.0035716866068263, 1.0085756067419482, 1.0117289210918006, 0.9945351141668962, 0.9754027482360388, 0.9336032206668309, 0.885739193303206, 0.8561340603464314, 0.8193433493424315, 0.8055040575682382, 0.7705310087903976, 0.7479843184223732, 0.7209286233022053, 0.6971464966220862, 0.6808528385041172, 0.667013978768577, 0.653960828905449, 0.6435774603504585, 0.6310638278809134, 0.6160951786951875, 0.5923655966044981, 0.5796168821527131, 0.5645059229376111, 0.5474645703116824, 0.5332477928275347, 0.5145242560141252, 0.5026854608975677, 0.4945103105640736, 0.49516247368598854, 0.4882028576851781, 0.47066358664742586, 0.45306920136987155, 0.4361142278820672, 0.4280690058018673, 0.42477357307408437, 0.4188603273839654, 0.4189152331975446 ], "yaxis": "y6" } ], "layout": { "annotations": [ { "font": {}, "showarrow": false, "text": "eta=0.0", "x": 0.075, "xanchor": "center", "xref": "paper", "y": 1, "yanchor": "bottom", "yref": "paper" }, { "font": {}, "showarrow": false, "text": "eta=20.0", "x": 0.24499999999999997, "xanchor": "center", "xref": "paper", "y": 1, "yanchor": "bottom", "yref": "paper" }, { "font": {}, "showarrow": false, "text": "eta=40.0", "x": 0.415, "xanchor": "center", "xref": "paper", "y": 1, "yanchor": "bottom", "yref": "paper" }, { "font": {}, "showarrow": false, "text": "eta=60.0", "x": 0.585, "xanchor": "center", "xref": "paper", "y": 1, "yanchor": "bottom", "yref": "paper" }, { "font": {}, "showarrow": false, "text": "eta=80.0", "x": 0.7549999999999999, "xanchor": "center", "xref": "paper", "y": 1, "yanchor": "bottom", "yref": "paper" }, { "font": {}, "showarrow": false, "text": "eta=100.0", "x": 0.925, "xanchor": "center", "xref": "paper", "y": 1, "yanchor": "bottom", "yref": "paper" } ], "coloraxis": { "colorbar": { "title": { "text": "threshold" } }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "legend": { "tracegroupgap": 0 }, "margin": { "t": 60 }, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "xaxis": { "anchor": "y", "domain": [ 0, 0.15 ], "title": { "text": "balanced_accuracy_score" } }, "xaxis2": { "anchor": "y2", "domain": [ 0.16999999999999998, 0.31999999999999995 ], "matches": "x", "title": { "text": "balanced_accuracy_score" } }, "xaxis3": { "anchor": "y3", "domain": [ 0.33999999999999997, 0.49 ], "matches": "x", "title": { "text": "balanced_accuracy_score" } }, "xaxis4": { "anchor": "y4", "domain": [ 0.51, 0.66 ], "matches": "x", "title": { "text": "balanced_accuracy_score" } }, "xaxis5": { "anchor": "y5", "domain": [ 0.6799999999999999, 0.83 ], "matches": "x", "title": { "text": "balanced_accuracy_score" } }, "xaxis6": { "anchor": "y6", "domain": [ 0.85, 1 ], "matches": "x", "title": { "text": "balanced_accuracy_score" } }, "yaxis": { "anchor": "x", "domain": [ 0, 1 ], "title": { "text": "disparate_impact_ratio" } }, "yaxis2": { "anchor": "x2", "domain": [ 0, 1 ], "matches": "y", "showticklabels": false }, "yaxis3": { "anchor": "x3", "domain": [ 0, 1 ], "matches": "y", "showticklabels": false }, "yaxis4": { "anchor": "x4", "domain": [ 0, 1 ], "matches": "y", "showticklabels": false }, "yaxis5": { "anchor": "x5", "domain": [ 0, 1 ], "matches": "y", "showticklabels": false }, "yaxis6": { "anchor": "x6", "domain": [ 0, 1 ], "matches": "y", "showticklabels": false } } }, "text/html": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fig = px.scatter(df_metrics, x='balanced_accuracy_score', y='disparate_impact_ratio', color='threshold', hover_data=[\"threshold\", \"eta\"], facet_col=\"eta\")\n", "fig.show()\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Plus l'eta est grand, plus la balanced_accuracy est faible et le disparate impact élevé, allant meme largement dépasser 1.\n", "Aussi l'eta 50.0 semble etre un bon compris avec un threshold à 0.14" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [], "source": [ "eta = 50.0\n", "thr = 0.14\n", "pr_orig_panel19 = train_pr_model(eta=eta)" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [], "source": [ "val_dataset_scaled = dataset_orig_panel19_val.copy()\n", "val_dataset_scaled.features = pr_orig_scaler.transform(val_dataset_scaled.features)\n", "val_scores = pr_orig_panel19.predict(val_dataset_scaled).scores\n", "val_preds = (val_scores[:, -1] > thr).astype(np.float64)\n", "\n" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'base_rate_truth': 0.22072904528726825,\n", " 'statistical_parity_difference': -0.015236638043276352,\n", " 'disparate_impact_ratio': 0.9540549388774919,\n", " 'base_rate_preds': 0.3254808860506231,\n", " 'equal_opportunity_difference': 0.10584863869161054,\n", " 'average_odds_difference': 0.06220191885464482,\n", " 'conditional_demographic_disparity': -0.0032270032590193246,\n", " 'smoothed_edf': 0.04703400723670348,\n", " 'df_bias_amplification': -0.6564533858858657,\n", " 'balanced_accuracy_score': 0.6725374394435751}" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "get_metrics(\n", " y_true = dataset_orig_panel19_val.labels[:,0],\n", " y_pred= val_preds,\n", " prot_attr= dataset_orig_panel19_val.protected_attributes[:,0],\n", " sample_weight= dataset_orig_panel19_val.instance_weights\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Question 5: Evaluate : compute the metrics on the test dataset using the model learnt with the selected parameters \n" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [], "source": [ "test_dataset = dataset_orig_panel19_test.copy()\n", "test_dataset.features = pr_orig_scaler.transform(test_dataset.features)\n", "test_scores = pr_orig_panel19.predict(test_dataset).scores\n", "test_preds = (test_scores[:, -1] > thr).astype(np.float64)" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'base_rate_truth': 0.20798000643857775,\n", " 'statistical_parity_difference': 0.0030802864679043696,\n", " 'disparate_impact_ratio': 1.0099440229406393,\n", " 'base_rate_preds': 0.31101048725128383,\n", " 'equal_opportunity_difference': 0.12829083368876892,\n", " 'average_odds_difference': 0.0848705674191425,\n", " 'conditional_demographic_disparity': 0.0006573978858304182,\n", " 'smoothed_edf': 0.00989492246891599,\n", " 'df_bias_amplification': -0.698304266044081,\n", " 'balanced_accuracy_score': 0.6934228599384545}" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "get_metrics(\n", " y_true = dataset_orig_panel19_test.labels[:,0],\n", " y_pred= test_preds,\n", " prot_attr= dataset_orig_panel19_test.protected_attributes[:,0],\n", " sample_weight= dataset_orig_panel19_test.instance_weights\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Combine pre-processing and in-processing\n", "### Question6: Redo the Prejudice Remover approach using first the Reweighing pre-processing" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [], "source": [ "from aif360.algorithms.preprocessing import Reweighing\n", "\n", "RW = Reweighing(\n", " unprivileged_groups=[{'RACE': 0.0}], privileged_groups=[{'RACE': 1.0}]\n", ")\n", "RW.fit(dataset_orig_panel19_train)\n", "dataset_rw_train = RW.transform(dataset_orig_panel19_train)\n", "dataset_rw_val = RW.transform(dataset_orig_panel19_val)\n", "dataset_rw_test = RW.transform(dataset_orig_panel19_test)\n" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [], "source": [ "# Standard Scaler\n", "pr_rw_scaler = StandardScaler()\n", "dataset_rw_train_scaled = dataset_rw_train.copy()\n", "dataset_rw_train_scaled.features = pr_rw_scaler.fit_transform(dataset_rw_train_scaled.features)" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [], "source": [ "# Boucle d'exploration des paramètres\n", "metrics_rw_list = validation_loop(\n", " thereshold_list=[float(x/100) if x>0 else 0.01 for x in range(50) ], \n", " eta_list=[ float(x*20) for x in range(5)], \n", " train_dataset=dataset_rw_train_scaled,\n", " val_dataset=dataset_rw_val)" ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [], "source": [ "df_metrics_rw = pd.DataFrame.from_records(metrics_rw_list)" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "dimensions": [ { "label": "eta", "values": [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 60, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, 80 ] }, { "label": "threshold", "values": [ 0.01, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.2, 0.21, 0.22, 0.23, 0.24, 0.25, 0.26, 0.27, 0.28, 0.29, 0.3, 0.31, 0.32, 0.33, 0.34, 0.35, 0.36, 0.37, 0.38, 0.39, 0.4, 0.41, 0.42, 0.43, 0.44, 0.45, 0.46, 0.47, 0.48, 0.49, 0.01, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.2, 0.21, 0.22, 0.23, 0.24, 0.25, 0.26, 0.27, 0.28, 0.29, 0.3, 0.31, 0.32, 0.33, 0.34, 0.35, 0.36, 0.37, 0.38, 0.39, 0.4, 0.41, 0.42, 0.43, 0.44, 0.45, 0.46, 0.47, 0.48, 0.49, 0.01, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.2, 0.21, 0.22, 0.23, 0.24, 0.25, 0.26, 0.27, 0.28, 0.29, 0.3, 0.31, 0.32, 0.33, 0.34, 0.35, 0.36, 0.37, 0.38, 0.39, 0.4, 0.41, 0.42, 0.43, 0.44, 0.45, 0.46, 0.47, 0.48, 0.49, 0.01, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.2, 0.21, 0.22, 0.23, 0.24, 0.25, 0.26, 0.27, 0.28, 0.29, 0.3, 0.31, 0.32, 0.33, 0.34, 0.35, 0.36, 0.37, 0.38, 0.39, 0.4, 0.41, 0.42, 0.43, 0.44, 0.45, 0.46, 0.47, 0.48, 0.49, 0.01, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.2, 0.21, 0.22, 0.23, 0.24, 0.25, 0.26, 0.27, 0.28, 0.29, 0.3, 0.31, 0.32, 0.33, 0.34, 0.35, 0.36, 0.37, 0.38, 0.39, 0.4, 0.41, 0.42, 0.43, 0.44, 0.45, 0.46, 0.47, 0.48, 0.49 ] }, { "label": "disparate_impact_ratio", "values": [ 0.9688954691291617, 0.9688954691291617, 0.8873357112207394, 0.8110172773651341, 0.7621076797594153, 0.706749787651074, 0.6680487207972623, 0.6401992754506641, 0.6213606995449826, 0.6164297995807039, 0.5999710287924372, 0.5922302007379822, 0.5690614137553458, 0.5562235932373445, 0.5423332803538656, 0.5368370268550045, 0.5269890762028218, 0.5220350210725772, 0.5064326676083765, 0.507583487553912, 0.5006765501807043, 0.5096500774291315, 0.5053420903336956, 0.4987904730307501, 0.49842949345341636, 0.4886560113693399, 0.4853176679374404, 0.48766163005829194, 0.4899686342265356, 0.4903513858548104, 0.4823799123140933, 0.4681254507969624, 0.4430192255839038, 0.44113396354497836, 0.4222786028989714, 0.4095828180229269, 0.416914396971667, 0.4065477374330675, 0.41220129888390167, 0.39545737510870205, 0.3929466918153599, 0.3975680785991909, 0.39791390583253206, 0.38790801536397473, 0.3911098433031685, 0.38711889255940723, 0.3901664610540577, 0.37013470367752493, 0.36506710800794695, 0.35202957027155385, 0.9674292435817748, 0.9674292435817748, 0.8970348718572944, 0.8720747328037255, 0.8621161912479318, 0.8344145388699385, 0.8141030150435918, 0.8007956164427751, 0.8120832443580133, 0.8194532688475739, 0.8064085273884303, 0.8263037873212007, 0.8668079135330319, 0.873360740333157, 0.8638187789809917, 0.9008564276171582, 0.8774524432962336, 0.885974070996392, 0.8561611915429095, 0.8619354302826988, 0.864217010519137, 0.880347569570955, 0.8802505478906777, 0.9103011081785898, 0.9193823788587844, 0.9069049317241792, 0.9471240529586484, 0.9634288440495172, 0.9553345351264209, 0.9490554839122496, 0.9417123461548879, 0.9614937807637732, 0.9774952106563704, 0.9819385162850498, 1.0015293332848187, 0.9942536376668438, 0.9745295680359843, 0.9635866497360265, 0.9727818606135813, 0.9180888854879304, 0.9713512767823332, 0.97818294022909, 0.988184652125508, 0.9974548102408158, 0.966795414560364, 0.9687266437575486, 0.9932077304645387, 0.9879829493510183, 0.9845151217796839, 0.954837351204516, 0.908480382049405, 0.908480382049405, 0.915378900883649, 0.9481134179841031, 0.9614667427905772, 0.9757943138524989, 1.0000675114737716, 1.020133669120301, 1.0183229680753865, 1.0684546958937982, 1.1017557430497051, 1.1268863713715218, 1.1457610343996703, 1.1377492110640757, 1.126078035817363, 1.1170986232323405, 1.1433576294635506, 1.1688889682043435, 1.2139699059918394, 1.216858641952215, 1.2100711577664067, 1.2254137537987542, 1.1927450775523, 1.1800806711644074, 1.1753056612594694, 1.2022832178732787, 1.1699418866825715, 1.1719021958613005, 1.1764029083223277, 1.1597459327367317, 1.1633258664329242, 1.1508466975230294, 1.168003431097993, 1.1522296524158697, 1.1663696348284591, 1.1794027379508938, 1.1997537359572066, 1.2755205703348984, 1.3169683688018785, 1.3123519576453841, 1.3198035815819678, 1.3572614444669828, 1.3328015551075258, 1.3364679961960328, 1.340039177291924, 1.3230608577049265, 1.3633356183706935, 1.3638103619852397, 1.305863540984835, 1.3065822583879216, 0.8936277294248832, 0.8936277294248832, 0.8931106016546037, 0.9249433153371039, 0.9473288561002408, 0.9692327484530635, 0.9948661666778812, 1.0268414419487164, 1.0604412739131819, 1.1153042567550195, 1.128033373673074, 1.1385886733333164, 1.1510204974967033, 1.1753740483336605, 1.1677562956568213, 1.1874230998382806, 1.1931415957136902, 1.2059016045667428, 1.2010979574823941, 1.1855002506649333, 1.187711494625417, 1.1741217623622477, 1.239357588011743, 1.2128303163255085, 1.1914854604389067, 1.2017737079656499, 1.2263951989479767, 1.2289443895588792, 1.2290852675051127, 1.21904827381769, 1.204992658657032, 1.2241058619581373, 1.2544706178237146, 1.2320839793785445, 1.2403575143523151, 1.2289087671129744, 1.264626905049271, 1.2726972781539438, 1.332365100447379, 1.3363436164410363, 1.347212488048771, 1.3391050145495047, 1.3878812302289658, 1.3789220521971237, 1.3918480923318244, 1.4112017761322027, 1.3615431216662586, 1.385439478068816, 1.370921932806776, 1.3709490001415805, 0.9621848972136837, 0.9621848972136837, 0.9908503416369152, 0.9984221693436495, 1.003874400720913, 1.0420088265086773, 1.0598834366055063, 1.0819394608020798, 1.0846917538143837, 1.080435033517835, 1.0670924799098738, 1.0514997632796024, 1.0463616795720176, 1.0232008737965348, 1.0043039858088127, 0.9906329436319288, 0.9952324213624218, 0.9495212059386908, 0.9373805244052266, 0.9110970839894246, 0.8662748076746031, 0.84601208078959, 0.8096203490403612, 0.7830013839825727, 0.7584605610677618, 0.7326580129015846, 0.7114923364547191, 0.6979256032516018, 0.6752583503000541, 0.6616002586597175, 0.6505227647646147, 0.6402963259433453, 0.6303490016206612, 0.6149753102021063, 0.5805769948143321, 0.5643938741935286, 0.5593257017774904, 0.5545211023337384, 0.5498022358122776, 0.5484060891369106, 0.5306877763032635, 0.5249660042748281, 0.5046902107390748, 0.49176953098991594, 0.4929827468702903, 0.4831345025191001, 0.47612119169253386, 0.48623625693548733, 0.4920392460611732, 0.4865010471834858 ] }, { "label": "df_bias_amplification", "values": [ 2.3285275959924876, 2.3285275959924876, 1.920177278478242, 1.5971699639422496, 1.1499168027108548, 1.0350623824260548, 0.8901543062471592, 0.8011273505070404, 0.7053815510484632, 0.5923582465152611, 0.5219173851556842, 0.5038820442698758, 0.5437890892487771, 0.56660708585033, 0.5918967199327936, 0.602082876391409, 0.6205976118997543, 0.6300427519587787, 0.6603860434469981, 0.6581162131961662, 0.6718171283432155, 0.6540530428199007, 0.6625417988737299, 0.6755912869428957, 0.6763152552391456, 0.6961186019710499, 0.7029737228722903, 0.6981555983321075, 0.6934360021164876, 0.6926551267795753, 0.7090453668130197, 0.7390410435812542, 0.7941641777192248, 0.7984287392936704, 0.8421120333691645, 0.8726381894585751, 0.8548963923231245, 0.8800759403735234, 0.8662654765241504, 0.9077342732408726, 0.9141033109630619, 0.9024110818322377, 0.9015415987792068, 0.9270090159229436, 0.9187888007394318, 0.9290453777287544, 0.9212037631636567, 0.9739102067059373, 0.98769600665973, 1.0240619948480163, 1.7828966912925854, 1.7828966912925854, 1.0663500535300423, 0.602961772285411, 0.41605315407477317, 0.37106372758698813, 0.31738696187380067, 0.2814025952242687, 0.21085661125190902, 0.17914010181881146, 0.1951869992553389, 0.17081498255580385, 0.12296007048228008, 0.11542877818176067, 0.12641446478928442, 0.08443156832790377, 0.11075470259682407, 0.10108977413804765, 0.13531878667247432, 0.12859709049426948, 0.12595354206371767, 0.107460652218752, 0.10757086452890063, 0.07400201347639479, 0.06407532737969546, 0.07773981277170905, 0.034347362321237807, 0.01727880894589995, 0.025715861492204395, 0.032310172975250406, 0.04007756930659001, 0.0192893369113063, 0.0027840404265806473, -0.0017512644527915011, -0.018449575357972314, -0.014214913963964904, 0.005822562346776916, 0.017115001340225255, 0.007617552639590741, 0.06548318908729689, 0.009089235884332414, 0.0020806969433060374, -0.008092170983160063, -0.01742944003704361, 0.013790482697430217, 0.011794914608177276, -0.013162447818053202, -0.007888055897152135, -0.004371880410672269, 0.026236350445571155, 1.444750640448683, 1.444750640448683, 0.42036421351958886, 0.1367377326964796, 0.06300282269539803, 0.020144030996223306, -0.019887738244187103, 0.0016605040818336025, -0.0018206680204502224, 0.046235600777472574, 0.07692724259068806, 0.09948061106085482, 0.11609127991505008, 0.10907414083593903, 0.09876303894612004, 0.09075701865039298, 0.1139914312055954, 0.1360759054900218, 0.17391810872989222, 0.17629486054984334, 0.17070137344752334, 0.1833007521589356, 0.15627964910033554, 0.1456050140102152, 0.1415504651085222, 0.16424464275561523, 0.1369762944483084, 0.1386504539191844, 0.1424836174442894, 0.12822317813354478, 0.1313052501866807, 0.12052015415876172, 0.13531804526840463, 0.1217211202624573, 0.1339182744140599, 0.145030380730109, 0.16213853880327767, 0.22337659949581745, 0.2553546135344287, 0.25184312500223793, 0.25750513310487344, 0.28549123060505543, 0.2673053683385369, 0.270052519341371, 0.2727210593394509, 0.2599700963916307, 0.2899565641567299, 0.2903047258235656, 0.24688675767404034, 0.24743698433745065, 1.601508398963949, 1.601508398963949, 0.5532920014493197, 0.212964315635646, 0.09458162219177146, 0.0312754755660728, -0.013117916683770714, 0.009056708764877741, 0.03870732237552588, 0.08914944702058647, 0.10049794286800107, 0.10981169368833132, 0.12067114235670506, 0.14160863983498884, 0.13510641687438008, 0.15180770170354174, 0.15661202994231238, 0.16724971171052183, 0.16325830914321515, 0.15018704573615516, 0.15205055017010038, 0.14054264223121948, 0.1946153770387844, 0.1729789413601419, 0.15522302668380794, 0.16382076727815953, 0.18410134411447499, 0.18617779186040173, 0.18629241935376695, 0.1780926642552738, 0.16649569033333478, 0.18223288360390888, 0.20673587724494857, 0.18872924361551613, 0.19542187199378658, 0.18614881252482585, 0.21479935595728028, 0.22116070292298207, 0.26697784027581495, 0.2699594460385515, 0.27805983929343014, 0.27202369799664394, 0.30780048964406137, 0.3013242743132749, 0.31065462731600046, 0.324463862475719, 0.28864091100662126, 0.3060396046576259, 0.295505662926304, 0.2955254071934048, 0.6518059255149953, 0.6518059255149953, 0.022087451902794264, -0.015917438641301906, -0.013048129438716582, 0.03876854591702128, 0.049551464801173783, 0.06255111393004997, 0.06131805216372321, 0.057385973465492235, 0.044959847499626227, 0.030239698886618482, 0.025341288599989298, 0.0029580362134440374, -0.0156830366775198, -0.010566597869925642, -0.015198822141618828, 0.031819608604323646, 0.04468816133177733, 0.07312800687385645, 0.123575273478991, 0.14724381906704287, 0.19121202020241257, 0.22464298465066723, 0.2564866415411917, 0.29109840314637614, 0.32041278703165954, 0.33966491816483946, 0.37268206505208457, 0.39311588526459107, 0.410001123507832, 0.42584633456825194, 0.44150377250616124, 0.4661952824088924, 0.5237549640306443, 0.5520250182739666, 0.561045425710869, 0.5696725126120201, 0.5782187283111906, 0.5807613172663566, 0.6136035024929802, 0.6244438452626757, 0.6638325421638345, 0.6897671530356384, 0.6873031450657952, 0.7074822228561672, 0.7221048768728884, 0.701082672574243, 0.689218818663969, 0.7005382376215514 ] }, { "label": "balanced_accuracy_score", "values": [ 0.5101527447172517, 0.5101527447172517, 0.5378790357466372, 0.5658721131247575, 0.5999989129212435, 0.619810140629466, 0.6415390849293939, 0.6601033306926302, 0.6800129141537656, 0.6974525720780628, 0.7124083607207825, 0.7257372088192566, 0.7299945402498915, 0.731113279166935, 0.7369278170082718, 0.7414087295394418, 0.7473160641723016, 0.7506870055100614, 0.7467676863754708, 0.7455801728777256, 0.7455474709837779, 0.7467302945451311, 0.746764627160918, 0.745810338588464, 0.7448503592615416, 0.744176296760539, 0.7453475122166527, 0.7437703510317849, 0.7433197781366894, 0.7422109746406618, 0.74118476811349, 0.7376266090978195, 0.7299048776288908, 0.7304879006969554, 0.7258468516342487, 0.7227263975219709, 0.7180531762704747, 0.7121718654857072, 0.7101477283409641, 0.70758487410409, 0.701855847627076, 0.6999064397045067, 0.6993823870462978, 0.6976206998440799, 0.6931927710880683, 0.6882114263355854, 0.6853181122186084, 0.6793568616526064, 0.6767965077382592, 0.6721622855473894, 0.51157881696686, 0.51157881696686, 0.546582668991831, 0.5886989005043334, 0.6180354586116428, 0.6442779665616746, 0.6586136778679105, 0.668391969362572, 0.6790816063269062, 0.6875946594045605, 0.6898435462952628, 0.6993591206574766, 0.6937451589241344, 0.6893563299729808, 0.6904291226267587, 0.7010602491184637, 0.6956105105713837, 0.6942976746816614, 0.6871062592471924, 0.6865774458585325, 0.6837545189114449, 0.6749806890885138, 0.6730096546517004, 0.6709558013027529, 0.6664567886629638, 0.6659089071804902, 0.6604441162523182, 0.6566123500742214, 0.652091280950556, 0.6459197039906054, 0.6439699098298873, 0.6430363688857124, 0.6371740741977047, 0.6384282606877606, 0.6401284821250796, 0.6394503513236905, 0.6364814092983538, 0.6319839034029758, 0.6304232105981571, 0.624239915168371, 0.6215741479967227, 0.6186479907600815, 0.6167117200730733, 0.6153346316882257, 0.6098756321988558, 0.6065582303257013, 0.6042321705184002, 0.6021856907284405, 0.5972257836411323, 0.5951986193752999, 0.5310950592126688, 0.5310950592126688, 0.5767974858875248, 0.607810388874328, 0.6219021010284282, 0.6371128346492003, 0.6416723124356212, 0.6508562785387265, 0.6502319045731717, 0.6552968179637835, 0.6566678644180983, 0.6507186295015888, 0.6516772289143236, 0.6515490263796483, 0.6520709357415031, 0.6476842070026996, 0.6505226851051997, 0.6535701007686854, 0.6496778290164663, 0.6516732693906202, 0.6485745567524321, 0.6480507648740433, 0.6453460277619001, 0.6433697176035391, 0.6388007686499166, 0.6405623319930795, 0.6391635531704687, 0.6390316941667258, 0.6343275361888527, 0.6301771967287272, 0.6285626289466587, 0.6236269916824825, 0.6257301487384594, 0.6239269591243468, 0.6215691968645989, 0.6219300420585068, 0.6191245594061824, 0.6193304593727076, 0.6163154675580284, 0.6144362271397941, 0.6143536860843412, 0.6123534794250612, 0.6081294670413789, 0.6072898346121255, 0.6070100149628218, 0.6047783181197242, 0.601442977977854, 0.5996110629776095, 0.5982985482485814, 0.5967553284579196, 0.5334687467512373, 0.5334687467512373, 0.5730655580929543, 0.6160868407264533, 0.6272545283687392, 0.643081854349349, 0.656520194840543, 0.663500574368048, 0.66699805541077, 0.6686013438390599, 0.6652434131685133, 0.6631902081363013, 0.660577795243797, 0.664714618406135, 0.667529590374758, 0.6646201095090998, 0.6614915724833814, 0.6610291135999611, 0.659091467742526, 0.6575310001818921, 0.6576155279624244, 0.6580600717655118, 0.6533410995120468, 0.6504358640923018, 0.648292891128871, 0.6451509876432338, 0.6450603477466073, 0.6460611355533483, 0.6440595610951709, 0.6394452367865121, 0.63516580112615, 0.6324004190309924, 0.6323962173999347, 0.6300397123773547, 0.6302773559539211, 0.627885573424549, 0.6263243831450636, 0.6238583687141142, 0.6231421510899411, 0.6212668141742306, 0.6167953838884811, 0.6149991253820576, 0.6111953315887803, 0.6091451744731858, 0.6098302275006798, 0.6093072913776251, 0.6079644401870321, 0.6038252463273233, 0.6012524753898476, 0.5993528985784129, 0.5290894521365719, 0.5290894521365719, 0.5875294617280878, 0.6141988112344821, 0.6245718409738805, 0.6286017956192502, 0.6269190339340851, 0.6137723325242606, 0.6146326808745675, 0.6057553265543452, 0.5992687498698256, 0.5943225177583671, 0.5878249307151105, 0.5866998946669525, 0.5829493676668869, 0.5748540438634749, 0.5637126337336046, 0.5654992593990527, 0.5674138045892558, 0.5619680211518923, 0.5563569901704157, 0.554677989506518, 0.5493214842276386, 0.5473350267685193, 0.5458419567693843, 0.5431523031638009, 0.5436925420551108, 0.542130189982804, 0.5392575462363642, 0.5392222541783446, 0.538854752315832, 0.5365560670632803, 0.5375118393311019, 0.5325148764779346, 0.5275951972525362, 0.5267474784276367, 0.5274494301025384, 0.5290666634011578, 0.5299112031749197, 0.5284979576474007, 0.5277386325479096, 0.5274283694599304, 0.5290704324139703, 0.528404110043077, 0.5286473326636235, 0.5277873965097291, 0.5274104692960092, 0.5272427070381294, 0.5270668023846345, 0.5287562515151876 ] } ], "domain": { "x": [ 0, 1 ], "y": [ 0, 1 ] }, "line": { "color": [ 0.5101527447172517, 0.5101527447172517, 0.5378790357466372, 0.5658721131247575, 0.5999989129212435, 0.619810140629466, 0.6415390849293939, 0.6601033306926302, 0.6800129141537656, 0.6974525720780628, 0.7124083607207825, 0.7257372088192566, 0.7299945402498915, 0.731113279166935, 0.7369278170082718, 0.7414087295394418, 0.7473160641723016, 0.7506870055100614, 0.7467676863754708, 0.7455801728777256, 0.7455474709837779, 0.7467302945451311, 0.746764627160918, 0.745810338588464, 0.7448503592615416, 0.744176296760539, 0.7453475122166527, 0.7437703510317849, 0.7433197781366894, 0.7422109746406618, 0.74118476811349, 0.7376266090978195, 0.7299048776288908, 0.7304879006969554, 0.7258468516342487, 0.7227263975219709, 0.7180531762704747, 0.7121718654857072, 0.7101477283409641, 0.70758487410409, 0.701855847627076, 0.6999064397045067, 0.6993823870462978, 0.6976206998440799, 0.6931927710880683, 0.6882114263355854, 0.6853181122186084, 0.6793568616526064, 0.6767965077382592, 0.6721622855473894, 0.51157881696686, 0.51157881696686, 0.546582668991831, 0.5886989005043334, 0.6180354586116428, 0.6442779665616746, 0.6586136778679105, 0.668391969362572, 0.6790816063269062, 0.6875946594045605, 0.6898435462952628, 0.6993591206574766, 0.6937451589241344, 0.6893563299729808, 0.6904291226267587, 0.7010602491184637, 0.6956105105713837, 0.6942976746816614, 0.6871062592471924, 0.6865774458585325, 0.6837545189114449, 0.6749806890885138, 0.6730096546517004, 0.6709558013027529, 0.6664567886629638, 0.6659089071804902, 0.6604441162523182, 0.6566123500742214, 0.652091280950556, 0.6459197039906054, 0.6439699098298873, 0.6430363688857124, 0.6371740741977047, 0.6384282606877606, 0.6401284821250796, 0.6394503513236905, 0.6364814092983538, 0.6319839034029758, 0.6304232105981571, 0.624239915168371, 0.6215741479967227, 0.6186479907600815, 0.6167117200730733, 0.6153346316882257, 0.6098756321988558, 0.6065582303257013, 0.6042321705184002, 0.6021856907284405, 0.5972257836411323, 0.5951986193752999, 0.5310950592126688, 0.5310950592126688, 0.5767974858875248, 0.607810388874328, 0.6219021010284282, 0.6371128346492003, 0.6416723124356212, 0.6508562785387265, 0.6502319045731717, 0.6552968179637835, 0.6566678644180983, 0.6507186295015888, 0.6516772289143236, 0.6515490263796483, 0.6520709357415031, 0.6476842070026996, 0.6505226851051997, 0.6535701007686854, 0.6496778290164663, 0.6516732693906202, 0.6485745567524321, 0.6480507648740433, 0.6453460277619001, 0.6433697176035391, 0.6388007686499166, 0.6405623319930795, 0.6391635531704687, 0.6390316941667258, 0.6343275361888527, 0.6301771967287272, 0.6285626289466587, 0.6236269916824825, 0.6257301487384594, 0.6239269591243468, 0.6215691968645989, 0.6219300420585068, 0.6191245594061824, 0.6193304593727076, 0.6163154675580284, 0.6144362271397941, 0.6143536860843412, 0.6123534794250612, 0.6081294670413789, 0.6072898346121255, 0.6070100149628218, 0.6047783181197242, 0.601442977977854, 0.5996110629776095, 0.5982985482485814, 0.5967553284579196, 0.5334687467512373, 0.5334687467512373, 0.5730655580929543, 0.6160868407264533, 0.6272545283687392, 0.643081854349349, 0.656520194840543, 0.663500574368048, 0.66699805541077, 0.6686013438390599, 0.6652434131685133, 0.6631902081363013, 0.660577795243797, 0.664714618406135, 0.667529590374758, 0.6646201095090998, 0.6614915724833814, 0.6610291135999611, 0.659091467742526, 0.6575310001818921, 0.6576155279624244, 0.6580600717655118, 0.6533410995120468, 0.6504358640923018, 0.648292891128871, 0.6451509876432338, 0.6450603477466073, 0.6460611355533483, 0.6440595610951709, 0.6394452367865121, 0.63516580112615, 0.6324004190309924, 0.6323962173999347, 0.6300397123773547, 0.6302773559539211, 0.627885573424549, 0.6263243831450636, 0.6238583687141142, 0.6231421510899411, 0.6212668141742306, 0.6167953838884811, 0.6149991253820576, 0.6111953315887803, 0.6091451744731858, 0.6098302275006798, 0.6093072913776251, 0.6079644401870321, 0.6038252463273233, 0.6012524753898476, 0.5993528985784129, 0.5290894521365719, 0.5290894521365719, 0.5875294617280878, 0.6141988112344821, 0.6245718409738805, 0.6286017956192502, 0.6269190339340851, 0.6137723325242606, 0.6146326808745675, 0.6057553265543452, 0.5992687498698256, 0.5943225177583671, 0.5878249307151105, 0.5866998946669525, 0.5829493676668869, 0.5748540438634749, 0.5637126337336046, 0.5654992593990527, 0.5674138045892558, 0.5619680211518923, 0.5563569901704157, 0.554677989506518, 0.5493214842276386, 0.5473350267685193, 0.5458419567693843, 0.5431523031638009, 0.5436925420551108, 0.542130189982804, 0.5392575462363642, 0.5392222541783446, 0.538854752315832, 0.5365560670632803, 0.5375118393311019, 0.5325148764779346, 0.5275951972525362, 0.5267474784276367, 0.5274494301025384, 0.5290666634011578, 0.5299112031749197, 0.5284979576474007, 0.5277386325479096, 0.5274283694599304, 0.5290704324139703, 0.528404110043077, 0.5286473326636235, 0.5277873965097291, 0.5274104692960092, 0.5272427070381294, 0.5270668023846345, 0.5287562515151876 ], "coloraxis": "coloraxis" }, "name": "", "type": "parcoords" } ], "layout": { "coloraxis": { "colorbar": { "title": { "text": "balanced_accuracy_score" } }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "legend": { "tracegroupgap": 0 }, "margin": { "t": 60 }, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } } } }, "text/html": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Visualisation\n", "fig = px.parallel_coordinates(\n", " df_metrics_rw, \n", " color=\"balanced_accuracy_score\", \n", " dimensions=[\"eta\", 'threshold', 'disparate_impact_ratio','df_bias_amplification', 'balanced_accuracy_score'])\n", "fig.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "On note ici que le df_bias_amplification est quasiment toujours positif, car le dataset préprocessé avec le RW est déjà presque parfait en terme de biais mesurables." ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "customdata": [ [ 0.01, 0 ], [ 0.01, 0 ], [ 0.02, 0 ], [ 0.03, 0 ], [ 0.04, 0 ], [ 0.05, 0 ], [ 0.06, 0 ], [ 0.07, 0 ], [ 0.08, 0 ], [ 0.09, 0 ], [ 0.1, 0 ], [ 0.11, 0 ], [ 0.12, 0 ], [ 0.13, 0 ], [ 0.14, 0 ], [ 0.15, 0 ], [ 0.16, 0 ], [ 0.17, 0 ], [ 0.18, 0 ], [ 0.19, 0 ], [ 0.2, 0 ], [ 0.21, 0 ], [ 0.22, 0 ], [ 0.23, 0 ], [ 0.24, 0 ], [ 0.25, 0 ], [ 0.26, 0 ], [ 0.27, 0 ], [ 0.28, 0 ], [ 0.29, 0 ], [ 0.3, 0 ], [ 0.31, 0 ], [ 0.32, 0 ], [ 0.33, 0 ], [ 0.34, 0 ], [ 0.35, 0 ], [ 0.36, 0 ], [ 0.37, 0 ], [ 0.38, 0 ], [ 0.39, 0 ], [ 0.4, 0 ], [ 0.41, 0 ], [ 0.42, 0 ], [ 0.43, 0 ], [ 0.44, 0 ], [ 0.45, 0 ], [ 0.46, 0 ], [ 0.47, 0 ], [ 0.48, 0 ], [ 0.49, 0 ] ], "hovertemplate": "eta=%{customdata[1]}
balanced_accuracy_score=%{x}
disparate_impact_ratio=%{y}
threshold=%{marker.color}", "legendgroup": "", "marker": { "color": [ 0.01, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.2, 0.21, 0.22, 0.23, 0.24, 0.25, 0.26, 0.27, 0.28, 0.29, 0.3, 0.31, 0.32, 0.33, 0.34, 0.35, 0.36, 0.37, 0.38, 0.39, 0.4, 0.41, 0.42, 0.43, 0.44, 0.45, 0.46, 0.47, 0.48, 0.49 ], "coloraxis": "coloraxis", "symbol": "circle" }, "mode": "markers", "name": "", "orientation": "v", "showlegend": false, "type": "scatter", "x": [ 0.5101527447172517, 0.5101527447172517, 0.5378790357466372, 0.5658721131247575, 0.5999989129212435, 0.619810140629466, 0.6415390849293939, 0.6601033306926302, 0.6800129141537656, 0.6974525720780628, 0.7124083607207825, 0.7257372088192566, 0.7299945402498915, 0.731113279166935, 0.7369278170082718, 0.7414087295394418, 0.7473160641723016, 0.7506870055100614, 0.7467676863754708, 0.7455801728777256, 0.7455474709837779, 0.7467302945451311, 0.746764627160918, 0.745810338588464, 0.7448503592615416, 0.744176296760539, 0.7453475122166527, 0.7437703510317849, 0.7433197781366894, 0.7422109746406618, 0.74118476811349, 0.7376266090978195, 0.7299048776288908, 0.7304879006969554, 0.7258468516342487, 0.7227263975219709, 0.7180531762704747, 0.7121718654857072, 0.7101477283409641, 0.70758487410409, 0.701855847627076, 0.6999064397045067, 0.6993823870462978, 0.6976206998440799, 0.6931927710880683, 0.6882114263355854, 0.6853181122186084, 0.6793568616526064, 0.6767965077382592, 0.6721622855473894 ], "xaxis": "x", "y": [ 0.9688954691291617, 0.9688954691291617, 0.8873357112207394, 0.8110172773651341, 0.7621076797594153, 0.706749787651074, 0.6680487207972623, 0.6401992754506641, 0.6213606995449826, 0.6164297995807039, 0.5999710287924372, 0.5922302007379822, 0.5690614137553458, 0.5562235932373445, 0.5423332803538656, 0.5368370268550045, 0.5269890762028218, 0.5220350210725772, 0.5064326676083765, 0.507583487553912, 0.5006765501807043, 0.5096500774291315, 0.5053420903336956, 0.4987904730307501, 0.49842949345341636, 0.4886560113693399, 0.4853176679374404, 0.48766163005829194, 0.4899686342265356, 0.4903513858548104, 0.4823799123140933, 0.4681254507969624, 0.4430192255839038, 0.44113396354497836, 0.4222786028989714, 0.4095828180229269, 0.416914396971667, 0.4065477374330675, 0.41220129888390167, 0.39545737510870205, 0.3929466918153599, 0.3975680785991909, 0.39791390583253206, 0.38790801536397473, 0.3911098433031685, 0.38711889255940723, 0.3901664610540577, 0.37013470367752493, 0.36506710800794695, 0.35202957027155385 ], "yaxis": "y" }, { "customdata": [ [ 0.01, 20 ], [ 0.01, 20 ], [ 0.02, 20 ], [ 0.03, 20 ], [ 0.04, 20 ], [ 0.05, 20 ], [ 0.06, 20 ], [ 0.07, 20 ], [ 0.08, 20 ], [ 0.09, 20 ], [ 0.1, 20 ], [ 0.11, 20 ], [ 0.12, 20 ], [ 0.13, 20 ], [ 0.14, 20 ], [ 0.15, 20 ], [ 0.16, 20 ], [ 0.17, 20 ], [ 0.18, 20 ], [ 0.19, 20 ], [ 0.2, 20 ], [ 0.21, 20 ], [ 0.22, 20 ], [ 0.23, 20 ], [ 0.24, 20 ], [ 0.25, 20 ], [ 0.26, 20 ], [ 0.27, 20 ], [ 0.28, 20 ], [ 0.29, 20 ], [ 0.3, 20 ], [ 0.31, 20 ], [ 0.32, 20 ], [ 0.33, 20 ], [ 0.34, 20 ], [ 0.35, 20 ], [ 0.36, 20 ], [ 0.37, 20 ], [ 0.38, 20 ], [ 0.39, 20 ], [ 0.4, 20 ], [ 0.41, 20 ], [ 0.42, 20 ], [ 0.43, 20 ], [ 0.44, 20 ], [ 0.45, 20 ], [ 0.46, 20 ], [ 0.47, 20 ], [ 0.48, 20 ], [ 0.49, 20 ] ], "hovertemplate": "eta=%{customdata[1]}
balanced_accuracy_score=%{x}
disparate_impact_ratio=%{y}
threshold=%{marker.color}", "legendgroup": "", "marker": { "color": [ 0.01, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.2, 0.21, 0.22, 0.23, 0.24, 0.25, 0.26, 0.27, 0.28, 0.29, 0.3, 0.31, 0.32, 0.33, 0.34, 0.35, 0.36, 0.37, 0.38, 0.39, 0.4, 0.41, 0.42, 0.43, 0.44, 0.45, 0.46, 0.47, 0.48, 0.49 ], "coloraxis": "coloraxis", "symbol": "circle" }, "mode": "markers", "name": "", "orientation": "v", "showlegend": false, "type": "scatter", "x": [ 0.51157881696686, 0.51157881696686, 0.546582668991831, 0.5886989005043334, 0.6180354586116428, 0.6442779665616746, 0.6586136778679105, 0.668391969362572, 0.6790816063269062, 0.6875946594045605, 0.6898435462952628, 0.6993591206574766, 0.6937451589241344, 0.6893563299729808, 0.6904291226267587, 0.7010602491184637, 0.6956105105713837, 0.6942976746816614, 0.6871062592471924, 0.6865774458585325, 0.6837545189114449, 0.6749806890885138, 0.6730096546517004, 0.6709558013027529, 0.6664567886629638, 0.6659089071804902, 0.6604441162523182, 0.6566123500742214, 0.652091280950556, 0.6459197039906054, 0.6439699098298873, 0.6430363688857124, 0.6371740741977047, 0.6384282606877606, 0.6401284821250796, 0.6394503513236905, 0.6364814092983538, 0.6319839034029758, 0.6304232105981571, 0.624239915168371, 0.6215741479967227, 0.6186479907600815, 0.6167117200730733, 0.6153346316882257, 0.6098756321988558, 0.6065582303257013, 0.6042321705184002, 0.6021856907284405, 0.5972257836411323, 0.5951986193752999 ], "xaxis": "x2", "y": [ 0.9674292435817748, 0.9674292435817748, 0.8970348718572944, 0.8720747328037255, 0.8621161912479318, 0.8344145388699385, 0.8141030150435918, 0.8007956164427751, 0.8120832443580133, 0.8194532688475739, 0.8064085273884303, 0.8263037873212007, 0.8668079135330319, 0.873360740333157, 0.8638187789809917, 0.9008564276171582, 0.8774524432962336, 0.885974070996392, 0.8561611915429095, 0.8619354302826988, 0.864217010519137, 0.880347569570955, 0.8802505478906777, 0.9103011081785898, 0.9193823788587844, 0.9069049317241792, 0.9471240529586484, 0.9634288440495172, 0.9553345351264209, 0.9490554839122496, 0.9417123461548879, 0.9614937807637732, 0.9774952106563704, 0.9819385162850498, 1.0015293332848187, 0.9942536376668438, 0.9745295680359843, 0.9635866497360265, 0.9727818606135813, 0.9180888854879304, 0.9713512767823332, 0.97818294022909, 0.988184652125508, 0.9974548102408158, 0.966795414560364, 0.9687266437575486, 0.9932077304645387, 0.9879829493510183, 0.9845151217796839, 0.954837351204516 ], "yaxis": "y2" }, { "customdata": [ [ 0.01, 40 ], [ 0.01, 40 ], [ 0.02, 40 ], [ 0.03, 40 ], [ 0.04, 40 ], [ 0.05, 40 ], [ 0.06, 40 ], [ 0.07, 40 ], [ 0.08, 40 ], [ 0.09, 40 ], [ 0.1, 40 ], [ 0.11, 40 ], [ 0.12, 40 ], [ 0.13, 40 ], [ 0.14, 40 ], [ 0.15, 40 ], [ 0.16, 40 ], [ 0.17, 40 ], [ 0.18, 40 ], [ 0.19, 40 ], [ 0.2, 40 ], [ 0.21, 40 ], [ 0.22, 40 ], [ 0.23, 40 ], [ 0.24, 40 ], [ 0.25, 40 ], [ 0.26, 40 ], [ 0.27, 40 ], [ 0.28, 40 ], [ 0.29, 40 ], [ 0.3, 40 ], [ 0.31, 40 ], [ 0.32, 40 ], [ 0.33, 40 ], [ 0.34, 40 ], [ 0.35, 40 ], [ 0.36, 40 ], [ 0.37, 40 ], [ 0.38, 40 ], [ 0.39, 40 ], [ 0.4, 40 ], [ 0.41, 40 ], [ 0.42, 40 ], [ 0.43, 40 ], [ 0.44, 40 ], [ 0.45, 40 ], [ 0.46, 40 ], [ 0.47, 40 ], [ 0.48, 40 ], [ 0.49, 40 ] ], "hovertemplate": "eta=%{customdata[1]}
balanced_accuracy_score=%{x}
disparate_impact_ratio=%{y}
threshold=%{marker.color}", "legendgroup": "", "marker": { "color": [ 0.01, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.2, 0.21, 0.22, 0.23, 0.24, 0.25, 0.26, 0.27, 0.28, 0.29, 0.3, 0.31, 0.32, 0.33, 0.34, 0.35, 0.36, 0.37, 0.38, 0.39, 0.4, 0.41, 0.42, 0.43, 0.44, 0.45, 0.46, 0.47, 0.48, 0.49 ], "coloraxis": "coloraxis", "symbol": "circle" }, "mode": "markers", "name": "", "orientation": "v", "showlegend": false, "type": "scatter", "x": [ 0.5310950592126688, 0.5310950592126688, 0.5767974858875248, 0.607810388874328, 0.6219021010284282, 0.6371128346492003, 0.6416723124356212, 0.6508562785387265, 0.6502319045731717, 0.6552968179637835, 0.6566678644180983, 0.6507186295015888, 0.6516772289143236, 0.6515490263796483, 0.6520709357415031, 0.6476842070026996, 0.6505226851051997, 0.6535701007686854, 0.6496778290164663, 0.6516732693906202, 0.6485745567524321, 0.6480507648740433, 0.6453460277619001, 0.6433697176035391, 0.6388007686499166, 0.6405623319930795, 0.6391635531704687, 0.6390316941667258, 0.6343275361888527, 0.6301771967287272, 0.6285626289466587, 0.6236269916824825, 0.6257301487384594, 0.6239269591243468, 0.6215691968645989, 0.6219300420585068, 0.6191245594061824, 0.6193304593727076, 0.6163154675580284, 0.6144362271397941, 0.6143536860843412, 0.6123534794250612, 0.6081294670413789, 0.6072898346121255, 0.6070100149628218, 0.6047783181197242, 0.601442977977854, 0.5996110629776095, 0.5982985482485814, 0.5967553284579196 ], "xaxis": "x3", "y": [ 0.908480382049405, 0.908480382049405, 0.915378900883649, 0.9481134179841031, 0.9614667427905772, 0.9757943138524989, 1.0000675114737716, 1.020133669120301, 1.0183229680753865, 1.0684546958937982, 1.1017557430497051, 1.1268863713715218, 1.1457610343996703, 1.1377492110640757, 1.126078035817363, 1.1170986232323405, 1.1433576294635506, 1.1688889682043435, 1.2139699059918394, 1.216858641952215, 1.2100711577664067, 1.2254137537987542, 1.1927450775523, 1.1800806711644074, 1.1753056612594694, 1.2022832178732787, 1.1699418866825715, 1.1719021958613005, 1.1764029083223277, 1.1597459327367317, 1.1633258664329242, 1.1508466975230294, 1.168003431097993, 1.1522296524158697, 1.1663696348284591, 1.1794027379508938, 1.1997537359572066, 1.2755205703348984, 1.3169683688018785, 1.3123519576453841, 1.3198035815819678, 1.3572614444669828, 1.3328015551075258, 1.3364679961960328, 1.340039177291924, 1.3230608577049265, 1.3633356183706935, 1.3638103619852397, 1.305863540984835, 1.3065822583879216 ], "yaxis": "y3" }, { "customdata": [ [ 0.01, 60 ], [ 0.01, 60 ], [ 0.02, 60 ], [ 0.03, 60 ], [ 0.04, 60 ], [ 0.05, 60 ], [ 0.06, 60 ], [ 0.07, 60 ], [ 0.08, 60 ], [ 0.09, 60 ], [ 0.1, 60 ], [ 0.11, 60 ], [ 0.12, 60 ], [ 0.13, 60 ], [ 0.14, 60 ], [ 0.15, 60 ], [ 0.16, 60 ], [ 0.17, 60 ], [ 0.18, 60 ], [ 0.19, 60 ], [ 0.2, 60 ], [ 0.21, 60 ], [ 0.22, 60 ], [ 0.23, 60 ], [ 0.24, 60 ], [ 0.25, 60 ], [ 0.26, 60 ], [ 0.27, 60 ], [ 0.28, 60 ], [ 0.29, 60 ], [ 0.3, 60 ], [ 0.31, 60 ], [ 0.32, 60 ], [ 0.33, 60 ], [ 0.34, 60 ], [ 0.35, 60 ], [ 0.36, 60 ], [ 0.37, 60 ], [ 0.38, 60 ], [ 0.39, 60 ], [ 0.4, 60 ], [ 0.41, 60 ], [ 0.42, 60 ], [ 0.43, 60 ], [ 0.44, 60 ], [ 0.45, 60 ], [ 0.46, 60 ], [ 0.47, 60 ], [ 0.48, 60 ], [ 0.49, 60 ] ], "hovertemplate": "eta=%{customdata[1]}
balanced_accuracy_score=%{x}
disparate_impact_ratio=%{y}
threshold=%{marker.color}", "legendgroup": "", "marker": { "color": [ 0.01, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.2, 0.21, 0.22, 0.23, 0.24, 0.25, 0.26, 0.27, 0.28, 0.29, 0.3, 0.31, 0.32, 0.33, 0.34, 0.35, 0.36, 0.37, 0.38, 0.39, 0.4, 0.41, 0.42, 0.43, 0.44, 0.45, 0.46, 0.47, 0.48, 0.49 ], "coloraxis": "coloraxis", "symbol": "circle" }, "mode": "markers", "name": "", "orientation": "v", "showlegend": false, "type": "scatter", "x": [ 0.5334687467512373, 0.5334687467512373, 0.5730655580929543, 0.6160868407264533, 0.6272545283687392, 0.643081854349349, 0.656520194840543, 0.663500574368048, 0.66699805541077, 0.6686013438390599, 0.6652434131685133, 0.6631902081363013, 0.660577795243797, 0.664714618406135, 0.667529590374758, 0.6646201095090998, 0.6614915724833814, 0.6610291135999611, 0.659091467742526, 0.6575310001818921, 0.6576155279624244, 0.6580600717655118, 0.6533410995120468, 0.6504358640923018, 0.648292891128871, 0.6451509876432338, 0.6450603477466073, 0.6460611355533483, 0.6440595610951709, 0.6394452367865121, 0.63516580112615, 0.6324004190309924, 0.6323962173999347, 0.6300397123773547, 0.6302773559539211, 0.627885573424549, 0.6263243831450636, 0.6238583687141142, 0.6231421510899411, 0.6212668141742306, 0.6167953838884811, 0.6149991253820576, 0.6111953315887803, 0.6091451744731858, 0.6098302275006798, 0.6093072913776251, 0.6079644401870321, 0.6038252463273233, 0.6012524753898476, 0.5993528985784129 ], "xaxis": "x4", "y": [ 0.8936277294248832, 0.8936277294248832, 0.8931106016546037, 0.9249433153371039, 0.9473288561002408, 0.9692327484530635, 0.9948661666778812, 1.0268414419487164, 1.0604412739131819, 1.1153042567550195, 1.128033373673074, 1.1385886733333164, 1.1510204974967033, 1.1753740483336605, 1.1677562956568213, 1.1874230998382806, 1.1931415957136902, 1.2059016045667428, 1.2010979574823941, 1.1855002506649333, 1.187711494625417, 1.1741217623622477, 1.239357588011743, 1.2128303163255085, 1.1914854604389067, 1.2017737079656499, 1.2263951989479767, 1.2289443895588792, 1.2290852675051127, 1.21904827381769, 1.204992658657032, 1.2241058619581373, 1.2544706178237146, 1.2320839793785445, 1.2403575143523151, 1.2289087671129744, 1.264626905049271, 1.2726972781539438, 1.332365100447379, 1.3363436164410363, 1.347212488048771, 1.3391050145495047, 1.3878812302289658, 1.3789220521971237, 1.3918480923318244, 1.4112017761322027, 1.3615431216662586, 1.385439478068816, 1.370921932806776, 1.3709490001415805 ], "yaxis": "y4" }, { "customdata": [ [ 0.01, 80 ], [ 0.01, 80 ], [ 0.02, 80 ], [ 0.03, 80 ], [ 0.04, 80 ], [ 0.05, 80 ], [ 0.06, 80 ], [ 0.07, 80 ], [ 0.08, 80 ], [ 0.09, 80 ], [ 0.1, 80 ], [ 0.11, 80 ], [ 0.12, 80 ], [ 0.13, 80 ], [ 0.14, 80 ], [ 0.15, 80 ], [ 0.16, 80 ], [ 0.17, 80 ], [ 0.18, 80 ], [ 0.19, 80 ], [ 0.2, 80 ], [ 0.21, 80 ], [ 0.22, 80 ], [ 0.23, 80 ], [ 0.24, 80 ], [ 0.25, 80 ], [ 0.26, 80 ], [ 0.27, 80 ], [ 0.28, 80 ], [ 0.29, 80 ], [ 0.3, 80 ], [ 0.31, 80 ], [ 0.32, 80 ], [ 0.33, 80 ], [ 0.34, 80 ], [ 0.35, 80 ], [ 0.36, 80 ], [ 0.37, 80 ], [ 0.38, 80 ], [ 0.39, 80 ], [ 0.4, 80 ], [ 0.41, 80 ], [ 0.42, 80 ], [ 0.43, 80 ], [ 0.44, 80 ], [ 0.45, 80 ], [ 0.46, 80 ], [ 0.47, 80 ], [ 0.48, 80 ], [ 0.49, 80 ] ], "hovertemplate": "eta=%{customdata[1]}
balanced_accuracy_score=%{x}
disparate_impact_ratio=%{y}
threshold=%{marker.color}", "legendgroup": "", "marker": { "color": [ 0.01, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.2, 0.21, 0.22, 0.23, 0.24, 0.25, 0.26, 0.27, 0.28, 0.29, 0.3, 0.31, 0.32, 0.33, 0.34, 0.35, 0.36, 0.37, 0.38, 0.39, 0.4, 0.41, 0.42, 0.43, 0.44, 0.45, 0.46, 0.47, 0.48, 0.49 ], "coloraxis": "coloraxis", "symbol": "circle" }, "mode": "markers", "name": "", "orientation": "v", "showlegend": false, "type": "scatter", "x": [ 0.5290894521365719, 0.5290894521365719, 0.5875294617280878, 0.6141988112344821, 0.6245718409738805, 0.6286017956192502, 0.6269190339340851, 0.6137723325242606, 0.6146326808745675, 0.6057553265543452, 0.5992687498698256, 0.5943225177583671, 0.5878249307151105, 0.5866998946669525, 0.5829493676668869, 0.5748540438634749, 0.5637126337336046, 0.5654992593990527, 0.5674138045892558, 0.5619680211518923, 0.5563569901704157, 0.554677989506518, 0.5493214842276386, 0.5473350267685193, 0.5458419567693843, 0.5431523031638009, 0.5436925420551108, 0.542130189982804, 0.5392575462363642, 0.5392222541783446, 0.538854752315832, 0.5365560670632803, 0.5375118393311019, 0.5325148764779346, 0.5275951972525362, 0.5267474784276367, 0.5274494301025384, 0.5290666634011578, 0.5299112031749197, 0.5284979576474007, 0.5277386325479096, 0.5274283694599304, 0.5290704324139703, 0.528404110043077, 0.5286473326636235, 0.5277873965097291, 0.5274104692960092, 0.5272427070381294, 0.5270668023846345, 0.5287562515151876 ], "xaxis": "x5", "y": [ 0.9621848972136837, 0.9621848972136837, 0.9908503416369152, 0.9984221693436495, 1.003874400720913, 1.0420088265086773, 1.0598834366055063, 1.0819394608020798, 1.0846917538143837, 1.080435033517835, 1.0670924799098738, 1.0514997632796024, 1.0463616795720176, 1.0232008737965348, 1.0043039858088127, 0.9906329436319288, 0.9952324213624218, 0.9495212059386908, 0.9373805244052266, 0.9110970839894246, 0.8662748076746031, 0.84601208078959, 0.8096203490403612, 0.7830013839825727, 0.7584605610677618, 0.7326580129015846, 0.7114923364547191, 0.6979256032516018, 0.6752583503000541, 0.6616002586597175, 0.6505227647646147, 0.6402963259433453, 0.6303490016206612, 0.6149753102021063, 0.5805769948143321, 0.5643938741935286, 0.5593257017774904, 0.5545211023337384, 0.5498022358122776, 0.5484060891369106, 0.5306877763032635, 0.5249660042748281, 0.5046902107390748, 0.49176953098991594, 0.4929827468702903, 0.4831345025191001, 0.47612119169253386, 0.48623625693548733, 0.4920392460611732, 0.4865010471834858 ], "yaxis": "y5" } ], "layout": { "annotations": [ { "font": {}, "showarrow": false, "text": "eta=0.0", "x": 0.09200000000000001, "xanchor": "center", "xref": "paper", "y": 1, "yanchor": "bottom", "yref": "paper" }, { "font": {}, "showarrow": false, "text": "eta=20.0", "x": 0.29600000000000004, "xanchor": "center", "xref": "paper", "y": 1, "yanchor": "bottom", "yref": "paper" }, { "font": {}, "showarrow": false, "text": "eta=40.0", "x": 0.5, "xanchor": "center", "xref": "paper", "y": 1, "yanchor": "bottom", "yref": "paper" }, { "font": {}, "showarrow": false, "text": "eta=60.0", "x": 0.7040000000000002, "xanchor": "center", "xref": "paper", "y": 1, "yanchor": "bottom", "yref": "paper" }, { "font": {}, "showarrow": false, "text": "eta=80.0", "x": 0.908, "xanchor": "center", "xref": "paper", "y": 1, "yanchor": "bottom", "yref": "paper" } ], "coloraxis": { "colorbar": { "title": { "text": "threshold" } }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "legend": { "tracegroupgap": 0 }, "margin": { "t": 60 }, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 }, "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "pattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "fillpattern": { "fillmode": "overlay", "size": 10, "solidity": 0.2 }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "autotypenumbers": "strict", "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } }, "xaxis": { "anchor": "y", "domain": [ 0, 0.18400000000000002 ], "title": { "text": "balanced_accuracy_score" } }, "xaxis2": { "anchor": "y2", "domain": [ 0.20400000000000001, 0.388 ], "matches": "x", "title": { "text": "balanced_accuracy_score" } }, "xaxis3": { "anchor": "y3", "domain": [ 0.40800000000000003, 0.5920000000000001 ], "matches": "x", "title": { "text": "balanced_accuracy_score" } }, "xaxis4": { "anchor": "y4", "domain": [ 0.6120000000000001, 0.7960000000000002 ], "matches": "x", "title": { "text": "balanced_accuracy_score" } }, "xaxis5": { "anchor": "y5", "domain": [ 0.8160000000000001, 1 ], "matches": "x", "title": { "text": "balanced_accuracy_score" } }, "yaxis": { "anchor": "x", "domain": [ 0, 1 ], "title": { "text": "disparate_impact_ratio" } }, "yaxis2": { "anchor": "x2", "domain": [ 0, 1 ], "matches": "y", "showticklabels": false }, "yaxis3": { "anchor": "x3", "domain": [ 0, 1 ], "matches": "y", "showticklabels": false }, "yaxis4": { "anchor": "x4", "domain": [ 0, 1 ], "matches": "y", "showticklabels": false }, "yaxis5": { "anchor": "x5", "domain": [ 0, 1 ], "matches": "y", "showticklabels": false } } }, "text/html": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fig = px.scatter(df_metrics_rw, x='balanced_accuracy_score', y='disparate_impact_ratio', color='threshold', hover_data=[\"threshold\", \"eta\"], facet_col=\"eta\")\n", "fig.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Ces deux méthodes semblent bien complémentaires et compatibles entre elles, le compromis entre accuracy et fairness est plus facile à trouver. Avec les ETA 25 et 50, avec un threshold autour de 0.13 on obtient une balanced accuracy de 73% et un disparate impact de 1." ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [], "source": [ "eta = 50.0\n", "thr = 0.13\n", "pr_rw_panel19 = train_pr_model(eta=eta, train_dataset=dataset_rw_train_scaled)" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'base_rate_truth': 0.22117821483019162,\n", " 'statistical_parity_difference': 0.03832036232413294,\n", " 'disparate_impact_ratio': 1.1153052332563305,\n", " 'base_rate_preds': 0.347839000877081,\n", " 'equal_opportunity_difference': 0.0847068571534847,\n", " 'average_odds_difference': 0.05601406034261591,\n", " 'conditional_demographic_disparity': 0.007772020127861992,\n", " 'smoothed_edf': 0.10912811961880509,\n", " 'df_bias_amplification': 0.0891503265936765,\n", " 'balanced_accuracy_score': 0.686282474833064}" ] }, "execution_count": 28, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Evaluation (calcul des métriques) sur valid et test dataset\n", "\n", "val_dataset_rw_scaled = dataset_rw_val.copy()\n", "val_dataset_rw_scaled.features = pr_rw_scaler.transform(val_dataset_rw_scaled.features)\n", "val_scores = pr_rw_panel19.predict(val_dataset_rw_scaled).scores\n", "val_preds = (val_scores[:,-1] > thr).astype(np.float64)\n", "get_metrics(\n", " y_true = val_dataset_rw_scaled.labels[:,0],\n", " y_pred= val_preds,\n", " prot_attr= val_dataset_rw_scaled.protected_attributes[:,0],\n", " sample_weight= val_dataset_rw_scaled.instance_weights\n", ")\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Les métriques de fairness sont excellentes et le balanced_accuracy_score est proche de 70%.\n", "Nous sommes parvenus à améliorer la fairness sans impacter la performance pure du modèle.\n", "Le df_bias_amplification est quasi nul, car le dataset avec Reweighing a très de biais, les métriques de fairness laissent peu de marge de manoeuvre, ce score ne peut pas donc pas être aussi négatif que sans le Reweighing." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Adversarial Debiasing\n", "\n", "Adversarial debiasing [1] is an in-processing technique that learns a classifier to maximize prediction accuracy and simultaneously reduce an adversary's ability to determine the protected attribute from the predictions.\n", "\n", "See [AIF360 tuto](https://github.com/Trusted-AI/AIF360/blob/main/examples/demo_adversarial_debiasing.ipynb)\n", "\n", "Here we show how to learn and Adversarial Debiasing with the argumetn debias set to False" ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "WARNING:tensorflow:From /home/alice/Documents/Fairness/td/env_adv/lib/python3.10/site-packages/tensorflow/python/util/dispatch.py:1260: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.\n", "Instructions for updating:\n", "Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-02-21 00:09:47.579644: E external/local_xla/xla/stream_executor/cuda/cuda_driver.cc:152] failed call to cuInit: INTERNAL: CUDA error: Failed call to cuInit: UNKNOWN ERROR (303)\n", "WARNING: All log messages before absl::InitializeLog() is called are written to STDERR\n", "I0000 00:00:1740092987.686078 1925928 mlir_graph_optimization_pass.cc:401] MLIR V1 optimization pass is not enabled\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "epoch 0; iter: 0; batch classifier loss: 5.374690\n", "epoch 1; iter: 0; batch classifier loss: 0.612059\n", "epoch 2; iter: 0; batch classifier loss: 0.550926\n", "epoch 3; iter: 0; batch classifier loss: 0.543599\n", "epoch 4; iter: 0; batch classifier loss: 0.374120\n", "epoch 5; iter: 0; batch classifier loss: 0.472071\n", "epoch 6; iter: 0; batch classifier loss: 0.545163\n", "epoch 7; iter: 0; batch classifier loss: 0.336630\n", "epoch 8; iter: 0; batch classifier loss: 0.421914\n", "epoch 9; iter: 0; batch classifier loss: 0.395803\n", "epoch 10; iter: 0; batch classifier loss: 0.512769\n", "epoch 11; iter: 0; batch classifier loss: 0.330521\n", "epoch 12; iter: 0; batch classifier loss: 0.315490\n", "epoch 13; iter: 0; batch classifier loss: 0.345536\n", "epoch 14; iter: 0; batch classifier loss: 0.239229\n", "epoch 15; iter: 0; batch classifier loss: 0.387996\n", "epoch 16; iter: 0; batch classifier loss: 0.289149\n", "epoch 17; iter: 0; batch classifier loss: 0.356012\n", "epoch 18; iter: 0; batch classifier loss: 0.248340\n", "epoch 19; iter: 0; batch classifier loss: 0.265588\n", "epoch 20; iter: 0; batch classifier loss: 0.325461\n", "epoch 21; iter: 0; batch classifier loss: 0.453441\n", "epoch 22; iter: 0; batch classifier loss: 0.294618\n", "epoch 23; iter: 0; batch classifier loss: 0.226426\n", "epoch 24; iter: 0; batch classifier loss: 0.291230\n", "epoch 25; iter: 0; batch classifier loss: 0.285099\n", "epoch 26; iter: 0; batch classifier loss: 0.251792\n", "epoch 27; iter: 0; batch classifier loss: 0.315021\n", "epoch 28; iter: 0; batch classifier loss: 0.290551\n", "epoch 29; iter: 0; batch classifier loss: 0.255226\n", "epoch 30; iter: 0; batch classifier loss: 0.353877\n", "epoch 31; iter: 0; batch classifier loss: 0.325062\n", "epoch 32; iter: 0; batch classifier loss: 0.272300\n", "epoch 33; iter: 0; batch classifier loss: 0.332646\n", "epoch 34; iter: 0; batch classifier loss: 0.345016\n", "epoch 35; iter: 0; batch classifier loss: 0.291186\n", "epoch 36; iter: 0; batch classifier loss: 0.284835\n", "epoch 37; iter: 0; batch classifier loss: 0.333033\n", "epoch 38; iter: 0; batch classifier loss: 0.333749\n", "epoch 39; iter: 0; batch classifier loss: 0.304978\n", "epoch 40; iter: 0; batch classifier loss: 0.290110\n", "epoch 41; iter: 0; batch classifier loss: 0.222259\n", "epoch 42; iter: 0; batch classifier loss: 0.320647\n", "epoch 43; iter: 0; batch classifier loss: 0.343352\n", "epoch 44; iter: 0; batch classifier loss: 0.364100\n", "epoch 45; iter: 0; batch classifier loss: 0.307913\n", "epoch 46; iter: 0; batch classifier loss: 0.277407\n", "epoch 47; iter: 0; batch classifier loss: 0.274056\n", "epoch 48; iter: 0; batch classifier loss: 0.298007\n", "epoch 49; iter: 0; batch classifier loss: 0.179530\n" ] }, { "data": { "text/plain": [ "" ] }, "execution_count": 29, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import tensorflow.compat.v1 as tf\n", "tf.disable_eager_execution()\n", "from aif360.algorithms.inprocessing.adversarial_debiasing import AdversarialDebiasing\n", "\n", "sess = tf.Session()\n", "\n", "plain_model = AdversarialDebiasing(\n", " unprivileged_groups=[{'RACE': 0.0}], \n", " privileged_groups=[{'RACE': 1.0}],\n", " scope_name='plain_classifier',\n", " debias=False, \n", " sess=sess)\n", "\n", "plain_model.fit(dataset_orig_panel19_train)" ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [], "source": [ "# Apply the plain model to train and val data\n", "dataset_nodebiasing_train = plain_model.predict(dataset_orig_panel19_train)\n", "dataset_nodebiasing_val = plain_model.predict(dataset_orig_panel19_val)" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'base_rate_truth': 0.21451724306616177,\n", " 'statistical_parity_difference': -0.14563927989835548,\n", " 'disparate_impact_ratio': 0.373902592666064,\n", " 'base_rate_preds': 0.17469166608341952,\n", " 'equal_opportunity_difference': -0.16895288908532996,\n", " 'average_odds_difference': -0.11265476868348775,\n", " 'conditional_demographic_disparity': -0.04950064763812943,\n", " 'smoothed_edf': 0.9837598229874769,\n", " 'df_bias_amplification': 0.2956815628076588,\n", " 'balanced_accuracy_score': 0.7699920794533341}" ] }, "execution_count": 31, "metadata": {}, "output_type": "execute_result" } ], "source": [ "get_metrics(\n", " y_true = dataset_orig_panel19_train.labels[:,0],\n", " y_pred= dataset_nodebiasing_train.labels[:,0],\n", " prot_attr= dataset_orig_panel19_train.protected_attributes[:,0],\n", " sample_weight= dataset_orig_panel19_train.instance_weights\n", ")" ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'base_rate_truth': 0.22072904528726825,\n", " 'statistical_parity_difference': -0.16923836643546045,\n", " 'disparate_impact_ratio': 0.3035493858897022,\n", " 'base_rate_preds': 0.17473064073335087,\n", " 'equal_opportunity_difference': -0.22057827341954667,\n", " 'average_odds_difference': -0.15564589931117803,\n", " 'conditional_demographic_disparity': -0.05457126713047523,\n", " 'smoothed_edf': 1.1922106607200975,\n", " 'df_bias_amplification': 0.4887232675975284,\n", " 'balanced_accuracy_score': 0.7093721829285422}" ] }, "execution_count": 32, "metadata": {}, "output_type": "execute_result" } ], "source": [ "get_metrics(\n", " y_true = dataset_orig_panel19_val.labels[:,0],\n", " y_pred= dataset_nodebiasing_val.labels[:,0],\n", " prot_attr= dataset_orig_panel19_val.protected_attributes[:,0],\n", " sample_weight= dataset_orig_panel19_val.instance_weights\n", ")" ] }, { "cell_type": "code", "execution_count": 48, "metadata": {}, "outputs": [], "source": [ "sess.close()\n", "tf.reset_default_graph()\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Question 7: Redo the same (learn and Adversarial Debiasing) with the argument debias set to True\n", "\n", "Compare the metrics outputed" ] }, { "cell_type": "code", "execution_count": 49, "metadata": {}, "outputs": [], "source": [ "# Learn parameters with debias set to True\n", "sess = tf.Session()\n", "debiased_model = AdversarialDebiasing(\n", " unprivileged_groups=[{'RACE': 0.0}], \n", " privileged_groups=[{'RACE': 1.0}],\n", " scope_name='debiased_model',\n", " debias=True, \n", " sess=sess)" ] }, { "cell_type": "code", "execution_count": 50, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "epoch 0; iter: 0; batch classifier loss: 1.668031; batch adversarial loss: 0.675715\n", "epoch 1; iter: 0; batch classifier loss: 0.600276; batch adversarial loss: 0.719155\n", "epoch 2; iter: 0; batch classifier loss: 0.424414; batch adversarial loss: 0.682085\n", "epoch 3; iter: 0; batch classifier loss: 0.384530; batch adversarial loss: 0.741544\n", "epoch 4; iter: 0; batch classifier loss: 0.398656; batch adversarial loss: 0.689060\n", "epoch 5; iter: 0; batch classifier loss: 0.317573; batch adversarial loss: 0.713117\n", "epoch 6; iter: 0; batch classifier loss: 0.355454; batch adversarial loss: 0.701915\n", "epoch 7; iter: 0; batch classifier loss: 0.344063; batch adversarial loss: 0.713198\n", "epoch 8; iter: 0; batch classifier loss: 0.442193; batch adversarial loss: 0.692672\n", "epoch 9; iter: 0; batch classifier loss: 0.388846; batch adversarial loss: 0.704799\n", "epoch 10; iter: 0; batch classifier loss: 0.327973; batch adversarial loss: 0.658760\n", "epoch 11; iter: 0; batch classifier loss: 0.277305; batch adversarial loss: 0.649817\n", "epoch 12; iter: 0; batch classifier loss: 0.300734; batch adversarial loss: 0.691456\n", "epoch 13; iter: 0; batch classifier loss: 0.293729; batch adversarial loss: 0.673021\n", "epoch 14; iter: 0; batch classifier loss: 0.266237; batch adversarial loss: 0.683055\n", "epoch 15; iter: 0; batch classifier loss: 0.338164; batch adversarial loss: 0.676681\n", "epoch 16; iter: 0; batch classifier loss: 0.289851; batch adversarial loss: 0.667995\n", "epoch 17; iter: 0; batch classifier loss: 0.320636; batch adversarial loss: 0.648446\n", "epoch 18; iter: 0; batch classifier loss: 0.258184; batch adversarial loss: 0.667237\n", "epoch 19; iter: 0; batch classifier loss: 0.312935; batch adversarial loss: 0.670303\n", "epoch 20; iter: 0; batch classifier loss: 0.341342; batch adversarial loss: 0.660507\n", "epoch 21; iter: 0; batch classifier loss: 0.377482; batch adversarial loss: 0.645327\n", "epoch 22; iter: 0; batch classifier loss: 0.326349; batch adversarial loss: 0.622176\n", "epoch 23; iter: 0; batch classifier loss: 0.221210; batch adversarial loss: 0.638786\n", "epoch 24; iter: 0; batch classifier loss: 0.308485; batch adversarial loss: 0.673901\n", "epoch 25; iter: 0; batch classifier loss: 0.311263; batch adversarial loss: 0.638955\n", "epoch 26; iter: 0; batch classifier loss: 0.373543; batch adversarial loss: 0.607645\n", "epoch 27; iter: 0; batch classifier loss: 0.397615; batch adversarial loss: 0.677441\n", "epoch 28; iter: 0; batch classifier loss: 0.330878; batch adversarial loss: 0.643309\n", "epoch 29; iter: 0; batch classifier loss: 0.361054; batch adversarial loss: 0.618110\n", "epoch 30; iter: 0; batch classifier loss: 0.295726; batch adversarial loss: 0.637840\n", "epoch 31; iter: 0; batch classifier loss: 0.283701; batch adversarial loss: 0.652183\n", "epoch 32; iter: 0; batch classifier loss: 0.288939; batch adversarial loss: 0.645767\n", "epoch 33; iter: 0; batch classifier loss: 0.270137; batch adversarial loss: 0.647179\n", "epoch 34; iter: 0; batch classifier loss: 0.308144; batch adversarial loss: 0.613944\n", "epoch 35; iter: 0; batch classifier loss: 0.345032; batch adversarial loss: 0.633732\n", "epoch 36; iter: 0; batch classifier loss: 0.222320; batch adversarial loss: 0.655226\n", "epoch 37; iter: 0; batch classifier loss: 0.276656; batch adversarial loss: 0.658579\n", "epoch 38; iter: 0; batch classifier loss: 0.311304; batch adversarial loss: 0.644796\n", "epoch 39; iter: 0; batch classifier loss: 0.270764; batch adversarial loss: 0.681036\n", "epoch 40; iter: 0; batch classifier loss: 0.350736; batch adversarial loss: 0.637560\n", "epoch 41; iter: 0; batch classifier loss: 0.362649; batch adversarial loss: 0.667862\n", "epoch 42; iter: 0; batch classifier loss: 0.273371; batch adversarial loss: 0.616815\n", "epoch 43; iter: 0; batch classifier loss: 0.318770; batch adversarial loss: 0.655881\n", "epoch 44; iter: 0; batch classifier loss: 0.305059; batch adversarial loss: 0.677409\n", "epoch 45; iter: 0; batch classifier loss: 0.380711; batch adversarial loss: 0.638998\n", "epoch 46; iter: 0; batch classifier loss: 0.273439; batch adversarial loss: 0.666721\n", "epoch 47; iter: 0; batch classifier loss: 0.321953; batch adversarial loss: 0.611573\n", "epoch 48; iter: 0; batch classifier loss: 0.212225; batch adversarial loss: 0.635766\n", "epoch 49; iter: 0; batch classifier loss: 0.287412; batch adversarial loss: 0.643768\n" ] }, { "data": { "text/plain": [ "" ] }, "execution_count": 50, "metadata": {}, "output_type": "execute_result" } ], "source": [ "debiased_model.fit(dataset_orig_panel19_train)" ] }, { "cell_type": "code", "execution_count": 51, "metadata": {}, "outputs": [], "source": [ "# Apply the plain model to train and val data\n", "dataset_debiased_train = debiased_model.predict(dataset_orig_panel19_train)\n", "dataset_debiased_val = debiased_model.predict(dataset_orig_panel19_val)" ] }, { "cell_type": "code", "execution_count": 52, "metadata": {}, "outputs": [], "source": [ "sess.close()\n", "tf.reset_default_graph()\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'base_rate_truth': 0.21451724306616177,\n", " 'statistical_parity_difference': -0.03286073522352835,\n", " 'disparate_impact_ratio': 0.7829856100778496,\n", " 'base_rate_preds': 0.13835275181308068,\n", " 'equal_opportunity_difference': 0.08605263139629149,\n", " 'average_odds_difference': 0.05171283635752971,\n", " 'conditional_demographic_disparity': -0.013507679412370324,\n", " 'smoothed_edf': 0.2446409015764861,\n", " 'df_bias_amplification': -0.443437358603332,\n", " 'balanced_accuracy_score': 0.7304047954243789}" ] }, "execution_count": 53, "metadata": {}, "output_type": "execute_result" } ], "source": [ "get_metrics(\n", " y_true = dataset_orig_panel19_train.labels[:,0],\n", " y_pred= dataset_debiased_train.labels[:,0],\n", " prot_attr= dataset_orig_panel19_train.protected_attributes[:,0],\n", " sample_weight= dataset_orig_panel19_train.instance_weights\n", ")" ] }, { "cell_type": "code", "execution_count": 54, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'base_rate_truth': 0.22072904528726825,\n", " 'statistical_parity_difference': -0.043359614701920665,\n", " 'disparate_impact_ratio': 0.7255907209916608,\n", " 'base_rate_preds': 0.14051951585037967,\n", " 'equal_opportunity_difference': 0.05363167637931765,\n", " 'average_odds_difference': 0.026357224025350512,\n", " 'conditional_demographic_disparity': -0.016693321341970227,\n", " 'smoothed_edf': 0.3207690559619516,\n", " 'df_bias_amplification': -0.38271833716061754,\n", " 'balanced_accuracy_score': 0.6851884726452877}" ] }, "execution_count": 54, "metadata": {}, "output_type": "execute_result" } ], "source": [ "get_metrics(\n", " y_true = dataset_orig_panel19_val.labels[:,0],\n", " y_pred= dataset_debiased_val.labels[:,0],\n", " prot_attr= dataset_orig_panel19_val.protected_attributes[:,0],\n", " sample_weight= dataset_orig_panel19_val.instance_weights\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Question 8: Combine the Reweighing with the Adversarial Debiasing" ] }, { "cell_type": "code", "execution_count": 55, "metadata": {}, "outputs": [], "source": [ "sess = tf.Session()\n", "debiased_rw_model = AdversarialDebiasing(\n", " unprivileged_groups=[{'RACE': 0.0}], \n", " privileged_groups=[{'RACE': 1.0}],\n", " scope_name='debiased_rw_model',\n", " debias=True, \n", " sess=sess)" ] }, { "cell_type": "code", "execution_count": 56, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "epoch 0; iter: 0; batch classifier loss: 1.154897; batch adversarial loss: 0.658370\n", "epoch 1; iter: 0; batch classifier loss: 0.348980; batch adversarial loss: 0.688762\n", "epoch 2; iter: 0; batch classifier loss: 0.357638; batch adversarial loss: 0.687641\n", "epoch 3; iter: 0; batch classifier loss: 0.288130; batch adversarial loss: 0.680000\n", "epoch 4; iter: 0; batch classifier loss: 0.327123; batch adversarial loss: 0.679374\n", "epoch 5; iter: 0; batch classifier loss: 0.223153; batch adversarial loss: 0.675010\n", "epoch 6; iter: 0; batch classifier loss: 0.328404; batch adversarial loss: 0.664870\n", "epoch 7; iter: 0; batch classifier loss: 0.242467; batch adversarial loss: 0.671999\n", "epoch 8; iter: 0; batch classifier loss: 0.212294; batch adversarial loss: 0.665952\n", "epoch 9; iter: 0; batch classifier loss: 0.285957; batch adversarial loss: 0.673434\n", "epoch 10; iter: 0; batch classifier loss: 0.211140; batch adversarial loss: 0.635633\n", "epoch 11; iter: 0; batch classifier loss: 0.220927; batch adversarial loss: 0.650842\n", "epoch 12; iter: 0; batch classifier loss: 0.280934; batch adversarial loss: 0.642847\n", "epoch 13; iter: 0; batch classifier loss: 0.227307; batch adversarial loss: 0.658569\n", "epoch 14; iter: 0; batch classifier loss: 0.198046; batch adversarial loss: 0.640380\n", "epoch 15; iter: 0; batch classifier loss: 0.194001; batch adversarial loss: 0.624870\n", "epoch 16; iter: 0; batch classifier loss: 0.236342; batch adversarial loss: 0.592632\n", "epoch 17; iter: 0; batch classifier loss: 0.200407; batch adversarial loss: 0.629433\n", "epoch 18; iter: 0; batch classifier loss: 0.257939; batch adversarial loss: 0.635086\n", "epoch 19; iter: 0; batch classifier loss: 0.341120; batch adversarial loss: 0.637391\n", "epoch 20; iter: 0; batch classifier loss: 0.221999; batch adversarial loss: 0.617171\n", "epoch 21; iter: 0; batch classifier loss: 0.227264; batch adversarial loss: 0.632642\n", "epoch 22; iter: 0; batch classifier loss: 0.290112; batch adversarial loss: 0.639611\n", "epoch 23; iter: 0; batch classifier loss: 0.220929; batch adversarial loss: 0.598322\n", "epoch 24; iter: 0; batch classifier loss: 0.196794; batch adversarial loss: 0.614885\n", "epoch 25; iter: 0; batch classifier loss: 0.256587; batch adversarial loss: 0.592372\n", "epoch 26; iter: 0; batch classifier loss: 0.228333; batch adversarial loss: 0.586593\n", "epoch 27; iter: 0; batch classifier loss: 0.198462; batch adversarial loss: 0.633650\n", "epoch 28; iter: 0; batch classifier loss: 0.207675; batch adversarial loss: 0.582259\n", "epoch 29; iter: 0; batch classifier loss: 0.200531; batch adversarial loss: 0.638908\n", "epoch 30; iter: 0; batch classifier loss: 0.151845; batch adversarial loss: 0.660304\n", "epoch 31; iter: 0; batch classifier loss: 0.208599; batch adversarial loss: 0.700708\n", "epoch 32; iter: 0; batch classifier loss: 0.181661; batch adversarial loss: 0.667166\n", "epoch 33; iter: 0; batch classifier loss: 0.272511; batch adversarial loss: 0.699025\n", "epoch 34; iter: 0; batch classifier loss: 0.272945; batch adversarial loss: 0.735675\n", "epoch 35; iter: 0; batch classifier loss: 0.589232; batch adversarial loss: 0.787223\n", "epoch 36; iter: 0; batch classifier loss: 0.609764; batch adversarial loss: 0.686091\n", "epoch 37; iter: 0; batch classifier loss: 0.434527; batch adversarial loss: 0.676739\n", "epoch 38; iter: 0; batch classifier loss: 0.369904; batch adversarial loss: 0.642061\n", "epoch 39; iter: 0; batch classifier loss: 0.326249; batch adversarial loss: 0.616469\n", "epoch 40; iter: 0; batch classifier loss: 0.215591; batch adversarial loss: 0.635647\n", "epoch 41; iter: 0; batch classifier loss: 0.219332; batch adversarial loss: 0.644894\n", "epoch 42; iter: 0; batch classifier loss: 0.224591; batch adversarial loss: 0.591960\n", "epoch 43; iter: 0; batch classifier loss: 0.238024; batch adversarial loss: 0.686460\n", "epoch 44; iter: 0; batch classifier loss: 0.176898; batch adversarial loss: 0.611943\n", "epoch 45; iter: 0; batch classifier loss: 0.196063; batch adversarial loss: 0.668015\n", "epoch 46; iter: 0; batch classifier loss: 0.131907; batch adversarial loss: 0.608432\n", "epoch 47; iter: 0; batch classifier loss: 0.200755; batch adversarial loss: 0.655321\n", "epoch 48; iter: 0; batch classifier loss: 0.157614; batch adversarial loss: 0.644359\n", "epoch 49; iter: 0; batch classifier loss: 0.230438; batch adversarial loss: 0.645884\n" ] }, { "data": { "text/plain": [ "" ] }, "execution_count": 56, "metadata": {}, "output_type": "execute_result" } ], "source": [ "debiased_rw_model.fit(dataset_rw_train_scaled)" ] }, { "cell_type": "code", "execution_count": 58, "metadata": {}, "outputs": [], "source": [ "# Apply the plain model to train and val data\n", "dataset_debiased_rw_train = debiased_rw_model.predict(dataset_rw_train_scaled)\n", "dataset_debiased_rw_val = debiased_rw_model.predict(val_dataset_rw_scaled)" ] }, { "cell_type": "code", "execution_count": 61, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'base_rate_truth': 0.21451724306616174,\n", " 'statistical_parity_difference': 0.034066849708783875,\n", " 'disparate_impact_ratio': 1.2719115818733928,\n", " 'base_rate_preds': 0.13883535227369675,\n", " 'equal_opportunity_difference': 0.15386967241765637,\n", " 'average_odds_difference': 0.07760904056335116,\n", " 'conditional_demographic_disparity': 0.013962606237140575,\n", " 'smoothed_edf': 0.24052095710201438,\n", " 'df_bias_amplification': 0.24052094124953416,\n", " 'balanced_accuracy_score': 0.819415000779465}" ] }, "execution_count": 61, "metadata": {}, "output_type": "execute_result" } ], "source": [ "get_metrics(\n", " y_true = dataset_rw_train_scaled.labels[:,0],\n", " y_pred= dataset_debiased_rw_train.labels[:,0],\n", " prot_attr= dataset_rw_train_scaled.protected_attributes[:,0],\n", " sample_weight= dataset_rw_train_scaled.instance_weights\n", ")" ] }, { "cell_type": "code", "execution_count": 62, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'base_rate_truth': 0.22117821483019162,\n", " 'statistical_parity_difference': -0.02865933396734835,\n", " 'disparate_impact_ratio': 0.7577313861357575,\n", " 'base_rate_preds': 0.10670301961727642,\n", " 'equal_opportunity_difference': -0.0458395485628581,\n", " 'average_odds_difference': -0.03408427596721164,\n", " 'conditional_demographic_disparity': -0.013833463412643522,\n", " 'smoothed_edf': 0.2774261869825949,\n", " 'df_bias_amplification': 0.2574483939574663,\n", " 'balanced_accuracy_score': 0.6305208309403187}" ] }, "execution_count": 62, "metadata": {}, "output_type": "execute_result" } ], "source": [ "get_metrics(\n", " y_true = val_dataset_rw_scaled.labels[:,0],\n", " y_pred= dataset_debiased_rw_val.labels[:,0],\n", " prot_attr= val_dataset_rw_scaled.protected_attributes[:,0],\n", " sample_weight= val_dataset_rw_scaled.instance_weights\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This in-processing approach does not seem compatible withe the Reweighing, has the df_bias_amplification is high and the disparate impact ratio is not improved by the use of the reweighing has pre-processing.\n", "Although very efficient on the fairness metrics of the dataset, the Reweighing is not convenient for every kind of machine learning algo.\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Analysis of the influence of Reweighing \n", "\n", "### QUESTION 9 : Pour aller plus loin, étudier l'impact du Reweighing sur différents modèles notamment les arbres de décision" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [], "source": [ "from sklearn import tree\n", "DTclf = tree.DecisionTreeClassifier()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Apprentissage d'un arbre de decision sans Reweighing" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "DTclf.fit(\n", " X=train_dataset_scaled.features, \n", " y=train_dataset_scaled.labels[:,0],\n", " sample_weight=train_dataset_scaled.instance_weights)\n", "metrics_list=[]\n", "y_val_pred_scores = DTclf.predict_proba(val_dataset_scaled.features)\n", "\n", "for thr in [float(x/20) if x>0 else 0.01 for x in range(10) ]:\n", " y_val_pred = (y_val_pred_scores[:, -1] > thr).astype(np.float64)\n", " metrics = get_metrics(\n", " y_true=val_dataset_scaled.labels[:,0], \n", " y_pred=y_val_pred, \n", " prot_attr=val_dataset_scaled.protected_attributes[:,0], \n", " sample_weight=val_dataset_scaled.instance_weights)\n", " metrics['threshold'] = thr\n", " metrics_list.append(metrics)\n", "df_metrics_DT = pd.DataFrame.from_records(metrics_list)\n", "df_metrics_DT" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "fig = px.scatter(df_metrics_DT, x='balanced_accuracy_score', y='disparate_impact_ratio', color='threshold', hover_data=[\"threshold\"])\n", "fig.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We observe that the threshold has a low impact on the two metrics" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# with Reweighing\n", "DTclf_rw = tree.DecisionTreeClassifier()\n", "DTclf_rw.fit(\n", " X=dataset_rw_train_scaled.features, \n", " y=dataset_rw_train_scaled.labels[:,0],\n", " sample_weight=dataset_rw_train_scaled.instance_weights)\n", "metrics_list=[]\n", "y_val_rw_pred_scores = DTclf_rw.predict_proba(val_dataset_rw_scaled.features)\n", "for thr in [float(x/20) if x>0 else 0.01 for x in range(10) ]:\n", " y_val_pred = (y_val_rw_pred_scores[:, -1] > thr).astype(np.float64)\n", " metrics = get_metrics(\n", " y_true=val_dataset_rw_scaled.labels[:,0], \n", " y_pred=y_val_pred, \n", " prot_attr=val_dataset_rw_scaled.protected_attributes[:,0], \n", " sample_weight=val_dataset_rw_scaled.instance_weights)\n", " metrics['threshold'] = thr\n", " metrics_list.append(metrics)\n", "df_metrics_rw_DT = pd.DataFrame.from_records(metrics_list)\n", "df_metrics_rw_DT" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "With the Reweighing the Disparate Impact has improved, it is closer to 1 but the df_bias_amplification has increased, because the dataset is not biased anymore. But the model still over learned the bias, reweighing is not so powerful for decision trees." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "fig = px.scatter(df_metrics_rw_DT, x='balanced_accuracy_score', y='disparate_impact_ratio', color='threshold', hover_data=[\"threshold\"])\n", "fig.show()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "## naive_bayes\n", "from sklearn import naive_bayes \n", "NBclf = naive_bayes.GaussianNB()\n", "NBclf.fit(\n", " X=train_dataset_scaled.features, \n", " y=train_dataset_scaled.labels[:,0],\n", " sample_weight=train_dataset_scaled.instance_weights)\n", "metrics_list=[]\n", "y_val_pred_scores = NBclf.predict_proba(val_dataset_scaled.features)\n", "for thr in [float(x/20) if x>0 else 0.01 for x in range(20) ]:\n", " y_val_pred = (y_val_pred_scores[:, 1] > thr).astype(np.float64)\n", " metrics = get_metrics(\n", " y_true=val_dataset_scaled.labels[:,0], \n", " y_pred=y_val_pred, \n", " prot_attr=val_dataset_scaled.protected_attributes[:,0], \n", " sample_weight=val_dataset_scaled.instance_weights)\n", " metrics['threshold'] = thr\n", " metrics_list.append(metrics)\n", "df_metrics_NB = pd.DataFrame.from_records(metrics_list)\n", "df_metrics_NB" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "fig = px.scatter(df_metrics_NB, x='balanced_accuracy_score', y='disparate_impact_ratio', color='threshold', hover_data=[\"threshold\"])\n", "fig.show()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# NB with Reweighing\n", "NBclf_rw = naive_bayes.GaussianNB()\n", "NBclf_rw.fit(\n", " X=dataset_rw_train_scaled.features, \n", " y=dataset_rw_train_scaled.labels[:,0],\n", " sample_weight=dataset_rw_train_scaled.instance_weights)\n", "metrics_list=[]\n", "y_val_rw_pred_scores = NBclf_rw.predict_proba(val_dataset_rw_scaled.features)\n", "for thr in [float(x/20) if x>0 else 0.01 for x in range(10) ]:\n", " y_val_pred = (y_val_rw_pred_scores[:, -1] > thr).astype(np.float64)\n", " metrics = get_metrics(\n", " y_true=val_dataset_rw_scaled.labels[:,0], \n", " y_pred=y_val_pred, \n", " prot_attr=val_dataset_rw_scaled.protected_attributes[:,0], \n", " sample_weight=val_dataset_rw_scaled.instance_weights)\n", " metrics['threshold'] = thr\n", " metrics_list.append(metrics)\n", "df_metrics_rw_NB = pd.DataFrame.from_records(metrics_list)\n", "df_metrics_rw_NB" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "fig = px.scatter(df_metrics_rw_NB, x='balanced_accuracy_score', y='disparate_impact_ratio', color='threshold', hover_data=[\"threshold\"])\n", "fig.show()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# LR\n", "from sklearn.linear_model import LogisticRegression\n", "\n", "LR_clf = LogisticRegression(solver='liblinear', random_state=42)\n", "LR_clf.fit(\n", " X=train_dataset_scaled.features, \n", " y=train_dataset_scaled.labels[:,0],\n", " sample_weight=train_dataset_scaled.instance_weights)\n", "metrics_list=[]\n", "y_val_pred_scores = LR_clf.predict_proba(val_dataset_scaled.features)\n", "for thr in [float(x/20) if x>0 else 0.01 for x in range(10) ]:\n", " y_val_pred = (y_val_pred_scores[:, -1] > thr).astype(np.float64)\n", " metrics = get_metrics(\n", " y_true=val_dataset_scaled.labels[:,0], \n", " y_pred=y_val_pred, \n", " prot_attr=val_dataset_scaled.protected_attributes[:,0], \n", " sample_weight=val_dataset_scaled.instance_weights)\n", " metrics['threshold'] = thr\n", " metrics_list.append(metrics)\n", "df_metrics_LR = pd.DataFrame.from_records(metrics_list)\n", "df_metrics_LR" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "LR_clf_rw = LogisticRegression(solver='liblinear', random_state=42)\n", "LR_clf_rw.fit(\n", " X=dataset_rw_train_scaled.features, \n", " y=dataset_rw_train_scaled.labels[:,0],\n", " sample_weight=dataset_rw_train_scaled.instance_weights)\n", "metrics_list=[]\n", "y_val_rw_pred_scores = LR_clf_rw.predict_proba(val_dataset_rw_scaled.features)\n", "for thr in [float(x/20) if x>0 else 0.01 for x in range(10) ]:\n", " y_val_pred = (y_val_rw_pred_scores[:, -1] > thr).astype(np.float64)\n", " metrics = get_metrics(\n", " y_true=val_dataset_rw_scaled.labels[:,0], \n", " y_pred=y_val_pred, \n", " prot_attr=val_dataset_rw_scaled.protected_attributes[:,0], \n", " sample_weight=val_dataset_rw_scaled.instance_weights)\n", " metrics['threshold'] = thr\n", " metrics_list.append(metrics)\n", "df_metrics_rw_LR = pd.DataFrame.from_records(metrics_list)\n", "df_metrics_rw_LR" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "colab": { "provenance": [ { "file_id": "1reWieE41k1RU9_T08Chmufh9WOwBoIKZ", "timestamp": 1706013971768 }, { "file_id": "1fWzH-WkCZ9xcagC-8OY71_b_aNcXfdpM", "timestamp": 1705973751107 } ] }, "kernelspec": { "display_name": "env_adv", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.4" } }, "nbformat": 4, "nbformat_minor": 0 }