piwik-script

Intern
    Data Science Chair

    XAI Evaluation Datasets

    Boolean XAI evaluation datasets

    In our paper "Evaluation of post-hoc XAI approaches through synthetic tabular data", 
    we introduce an evaluation setting using synthetic data, in order to investigate 
    which explainable aritificial intelligence (XAI) approaches correctly explain 
    the decision making of deep neural networks that solve basic Boolean functions. 
    
    Finding that providing explanations on datasets proves no trivial task for the investigated
    XAI approaches, we publish the generated synthetic data as benchmark datasets.
    
    The datasets contain 12 Boolean features with every possible permutation
    being included exactly once in the dataset, resulting in 2*12=4096 data samples in each dataset.
    The labels 'y' are generated as described in the paper, with the first columns being used for label calculation.
    (i.e. for the XOR dataset the label is calculated by y = f1 XOR f2 )
    The 'explanation' column contains the relevant feature columns for each data sample,
    according to the definition given in the paper.
    
    You can download the datasets using this link.

    Publication

    • Evaluation of post-hoc XA... - Download
      Evaluation of post-hoc XAI approaches through synthetic tabular data. Tritscher, Julian; Ring, Markus; Schlör, Daniel; Hettinger, Lena; Hotho, Andreas in International Symposium on Methodologies for Intelligent Systems (2020).