publications

2021

1. CSCW

   Explainable Active Learning (XAL): Toward AI Explanations as Interfaces for Machine Teachers

   Ghai, Bhavya, Liao, Q. Vera,  Zhang, Yunfeng, Bellamy, Rachel, and Mueller, Klaus

   Proceedings of the ACM on Human-Computer Interaction Jan 2021

   Abs

   The wide adoption of Machine Learning (ML) technologies has created a growing demand for people who can train ML models. Some advocated the term "machine teacher” to refer to the role of people who inject domain knowledge into ML models. This "teaching” perspective emphasizes supporting the productivity and mental wellbeing of machine teachers through efficient learning algorithms and thoughtful design of human-AI interfaces. One promising learning paradigm is Active Learning (AL), by which the model intelligently selects instances to query a machine teacher for labels, so that the labeling workload could be largely reduced. However, in current AL settings, the human-AI interface remains minimal and opaque. A dearth of empirical studies further hinders us from developing teacher-friendly interfaces for AL algorithms. In this work, we begin considering AI explanations as a core element of the human-AI interface for teaching machines. When a human student learns, it is a common pattern to present one’s own reasoning and solicit feedback from the teacher. When a ML model learns and still makes mistakes, the teacher ought to be able to understand the reasoning underlying its mistakes. When the model matures, the teacher should be able to recognize its progress in order to trust and feel confident about their teaching outcome. Toward this vision, we propose a novel paradigm of explainable active learning (XAL), by introducing techniques from the surging field of explainable AI (XAI) into an AL setting. We conducted an empirical study comparing the model learning outcomes, feedback content and experience with XAL, to that of traditional AL and coactive learning (providing the model’s prediction without explanation). Our study shows benefits of AI explanation as interfaces for machine teaching–supporting trust calibration and enabling rich forms of teaching feedback, and potential drawbacks–anchoring effect with the model judgment and additional cognitive workload. Our study also reveals important individual factors that mediate a machine teacher’s reception to AI explanations, including task knowledge, AI experience and Need for Cognition. By reflecting on the results, we suggest future directions and design implications for XAL, and more broadly, machine teaching through AI explanations.

2. IUI

   Model LineUpper: Supporting Interactive Model Comparison at Multiple Levels for AutoML

   Narkar, Shweta,  Zhang, Yunfeng, Liao, Q. Vera, Wang, Dakuo, and Weisz, Justin D.

   In 26th International Conference on Intelligent User Interfaces Apr 2021
3. IUI

   Model LineUpper: Supporting Interactive Model Comparison at Multiple Levels for AutoML

   Narkar, Shweta,  Zhang, Yunfeng, Liao, Q. Vera, Wang, Dakuo, and Weisz, Justin D.

   In 26th International Conference on Intelligent User Interfaces Apr 2021

2020

1. NeurIPS

   Model Agnostic Multilevel Explanations

   Natesan Ramamurthy, Karthikeyan, Vinzamuri, Bhanukiran,  Zhang, Yunfeng, and Dhurandhar, Amit

   In Advances in Neural Information Processing Systems Apr 2020
2. AIES

   Data Augmentation for Discrimination Prevention and Bias Disambiguation

   Sharma, Shubham,  Zhang, Yunfeng, Ríos Aliaga, Jesús M., Bouneffouf, Djallel, Muthusamy, Vinod, and Varshney, Kush R.

   In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society Feb 2020

   Abs

   Machine learning models are prone to biased decisions due to biases in the datasets they are trained on. In this paper, we introduce a novel data augmentation technique to create a fairer dataset for model training that could also lend itself to understanding the type of bias existing in the dataset i.e. if bias arises from a lack of representation for a particular group (sampling bias) or if it arises because of human bias reflected in the labels (prejudice based bias). Given a dataset involving a protected attribute with a privileged and unprivileged group, we create an "ideal world” dataset: for every data sample, we create a new sample having the same features (except the protected attribute(s)) and label as the original sample but with the opposite protected attribute value. The synthetic data points are sorted in order of their proximity to the original training distribution and added successively to the real dataset to create intermediate datasets. We theoretically show that two different notions of fairness: statistical parity difference (independence) and average odds difference (separation) always change in the same direction using such an augmentation. We also show submodularity of the proposed fairness-aware augmentation approach that enables an efficient greedy algorithm. We empirically study the effect of training models on the intermediate datasets and show that this technique reduces the two bias measures while keeping the accuracy nearly constant for three datasets. We then discuss the implications of this study on the disambiguation of sample bias and prejudice based bias and discuss how pre-processing techniques should be evaluated in general. The proposed method can be used by policy makers who want to use unbiased datasets to train machine learning models for their applications to add a subset of synthetic points to an extent that they are comfortable with to mitigate unwanted bias.

3. FAccT

   Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making

   Zhang, Yunfeng, Liao, Q. Vera, and Bellamy, Rachel K. E.

   In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency Jan 2020

   Abs

   Today, AI is being increasingly used to help human experts make decisions in high-stakes scenarios. In these scenarios, full automation is often undesirable, not only due to the significance of the outcome, but also because human experts can draw on their domain knowledge complementary to the model’s to ensure task success. We refer to these scenarios as AI-assisted decision making, where the individual strengths of the human and the AI come together to optimize the joint decision outcome. A key to their success is to appropriately calibrate human trust in the AI on a case-by-case basis; knowing when to trust or distrust the AI allows the human expert to appropriately apply their knowledge, improving decision outcomes in cases where the model is likely to perform poorly. This research conducts a case study of AI-assisted decision making in which humans and AI have comparable performance alone, and explores whether features that reveal case-specific model information can calibrate trust and improve the joint performance of the human and AI. Specifically, we study the effect of showing confidence score and local explanation for a particular prediction. Through two human experiments, we show that confidence score can help calibrate people’s trust in an AI model, but trust calibration alone is not sufficient to improve AI-assisted decision making, which may also depend on whether the human can bring in enough unique knowledge to complement the AI’s errors. We also highlight the problems in using local explanation for AI-assisted decision making scenarios and invite the research community to explore new approaches to explainability for calibrating human trust in AI.

4. AIES

   Joint Optimization of AI Fairness and Utility: A Human-Centered Approach

   Zhang, Yunfeng, Bellamy, Rachel, and Varshney, Kush

   In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society Feb 2020

   Abs

   Today, AI is increasingly being used in many high-stakes decision-making applications in which fairness is an important concern. Already, there are many examples of AI being biased and making questionable and unfair decisions. The AI research community has proposed many methods to measure and mitigate unwanted biases, but few of them involve inputs from human policy makers. We argue that because different fairness criteria sometimes cannot be simultaneously satisfied, and because achieving fairness often requires sacrificing other objectives such as model accuracy, it is key to acquire and adhere to human policy makers’ preferences on how to make the tradeoff among these objectives. In this paper, we propose a framework and some exemplar methods for eliciting such preferences and for optimizing an AI model according to these preferences.

5. NeurIPS

   Model Agnostic Multilevel Explanations

   Natesan Ramamurthy, Karthikeyan, Vinzamuri, Bhanukiran,  Zhang, Yunfeng, and Dhurandhar, Amit

   In Advances in Neural Information Processing Systems Feb 2020
6. AIES

   Data Augmentation for Discrimination Prevention and Bias Disambiguation

   Sharma, Shubham,  Zhang, Yunfeng, Ríos Aliaga, Jesús M., Bouneffouf, Djallel, Muthusamy, Vinod, and Varshney, Kush R.

   In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society Feb 2020

   Abs

   Machine learning models are prone to biased decisions due to biases in the datasets they are trained on. In this paper, we introduce a novel data augmentation technique to create a fairer dataset for model training that could also lend itself to understanding the type of bias existing in the dataset i.e. if bias arises from a lack of representation for a particular group (sampling bias) or if it arises because of human bias reflected in the labels (prejudice based bias). Given a dataset involving a protected attribute with a privileged and unprivileged group, we create an "ideal world” dataset: for every data sample, we create a new sample having the same features (except the protected attribute(s)) and label as the original sample but with the opposite protected attribute value. The synthetic data points are sorted in order of their proximity to the original training distribution and added successively to the real dataset to create intermediate datasets. We theoretically show that two different notions of fairness: statistical parity difference (independence) and average odds difference (separation) always change in the same direction using such an augmentation. We also show submodularity of the proposed fairness-aware augmentation approach that enables an efficient greedy algorithm. We empirically study the effect of training models on the intermediate datasets and show that this technique reduces the two bias measures while keeping the accuracy nearly constant for three datasets. We then discuss the implications of this study on the disambiguation of sample bias and prejudice based bias and discuss how pre-processing techniques should be evaluated in general. The proposed method can be used by policy makers who want to use unbiased datasets to train machine learning models for their applications to add a subset of synthetic points to an extent that they are comfortable with to mitigate unwanted bias.

7. FAccT

   Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making

   Zhang, Yunfeng, Liao, Q. Vera, and Bellamy, Rachel K. E.

   In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency Jan 2020

   Abs

   Today, AI is being increasingly used to help human experts make decisions in high-stakes scenarios. In these scenarios, full automation is often undesirable, not only due to the significance of the outcome, but also because human experts can draw on their domain knowledge complementary to the model’s to ensure task success. We refer to these scenarios as AI-assisted decision making, where the individual strengths of the human and the AI come together to optimize the joint decision outcome. A key to their success is to appropriately calibrate human trust in the AI on a case-by-case basis; knowing when to trust or distrust the AI allows the human expert to appropriately apply their knowledge, improving decision outcomes in cases where the model is likely to perform poorly. This research conducts a case study of AI-assisted decision making in which humans and AI have comparable performance alone, and explores whether features that reveal case-specific model information can calibrate trust and improve the joint performance of the human and AI. Specifically, we study the effect of showing confidence score and local explanation for a particular prediction. Through two human experiments, we show that confidence score can help calibrate people’s trust in an AI model, but trust calibration alone is not sufficient to improve AI-assisted decision making, which may also depend on whether the human can bring in enough unique knowledge to complement the AI’s errors. We also highlight the problems in using local explanation for AI-assisted decision making scenarios and invite the research community to explore new approaches to explainability for calibrating human trust in AI.

8. AIES

   Joint Optimization of AI Fairness and Utility: A Human-Centered Approach

   Zhang, Yunfeng, Bellamy, Rachel, and Varshney, Kush

   In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society Feb 2020

   Abs

   Today, AI is increasingly being used in many high-stakes decision-making applications in which fairness is an important concern. Already, there are many examples of AI being biased and making questionable and unfair decisions. The AI research community has proposed many methods to measure and mitigate unwanted biases, but few of them involve inputs from human policy makers. We argue that because different fairness criteria sometimes cannot be simultaneously satisfied, and because achieving fairness often requires sacrificing other objectives such as model accuracy, it is key to acquire and adhere to human policy makers’ preferences on how to make the tradeoff among these objectives. In this paper, we propose a framework and some exemplar methods for eliciting such preferences and for optimizing an AI model according to these preferences.

2019

1. arXiv

   One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques

   Arya, Vijay, Bellamy, Rachel K. E., Chen, Pin-Yu, Dhurandhar, Amit, Hind, Michael, Hoffman, Samuel C., Houde, Stephanie, Liao, Q. Vera, Luss, Ronny, Mojsilović, Aleksandra, Mourad, Sami, Pedemonte, Pablo, Raghavendra, Ramya, Richards, John, Sattigeri, Prasanna, Shanmugam, Karthikeyan, Singh, Moninder, Varshney, Kush R., Wei, Dennis, and Zhang, Yunfeng

   arXiv:1909.03012 [cs, stat] Sep 2019

   Abs

   As artificial intelligence and machine learning algorithms make further inroads into society, calls are increasing from multiple stakeholders for these algorithms to explain their outputs. At the same time, these stakeholders, whether they be affected citizens, government regulators, domain experts, or system developers, present different requirements for explanations. Toward addressing these needs, we introduce AI Explainability 360 (http://aix360.mybluemix.net/), an open-source software toolkit featuring eight diverse and state-of-the-art explainability methods and two evaluation metrics. Equally important, we provide a taxonomy to help entities requiring explanations to navigate the space of explanation methods, not only those in the toolkit but also in the broader literature on explainability. For data scientists and other users of the toolkit, we have implemented an extensible software architecture that organizes methods according to their place in the AI modeling pipeline. We also discuss enhancements to bring research innovations closer to consumers of explanations, ranging from simplified, more accessible versions of algorithms, to tutorials and an interactive web demo to introduce AI explainability to different audiences and application domains. Together, our toolkit and taxonomy can help identify gaps where more explainability methods are needed and provide a platform to incorporate them as they are developed.

2. IUI

   Explaining Models: An Empirical Study of How Explanations Impact Fairness Judgment

   Dodge, Jonathan, Liao, Q. Vera,  Zhang, Yunfeng, Bellamy, Rachel K. E., and Dugan, Casey

   In Proceedings of the 24th International Conference on Intelligent User Interfaces - IUI ’19 Sep 2019

2018

1. arXiv

   AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias

   Bellamy, Rachel K. E., Dey, Kuntal, Hind, Michael, Hoffman, Samuel C., Houde, Stephanie, Kannan, Kalapriya, Lohia, Pranay, Martino, Jacquelyn, Mehta, Sameep, Mojsilovic, Aleksandra, Nagar, Seema, Ramamurthy, Karthikeyan Natesan, Richards, John, Saha, Diptikalyan, Sattigeri, Prasanna, Singh, Moninder, Varshney, Kush R., and Zhang, Yunfeng

   arXiv:1810.01943 [cs] Oct 2018

   Abs

   Fairness is an increasingly important concern as machine learning models are used to support decision making in high-stakes applications such as mortgage lending, hiring, and prison sentencing. This paper introduces a new open source Python toolkit for algorithmic fairness, AI Fairness 360 (AIF360), released under an Apache v2.0 license {https://github.com/ibm/aif360). The main objectives of this toolkit are to help facilitate the transition of fairness research algorithms to use in an industrial setting and to provide a common framework for fairness researchers to share and evaluate algorithms. The package includes a comprehensive set of fairness metrics for datasets and models, explanations for these metrics, and algorithms to mitigate bias in datasets and models. It also includes an interactive Web experience (https://aif360.mybluemix.net) that provides a gentle introduction to the concepts and capabilities for line-of-business users, as well as extensive documentation, usage guidance, and industry-specific tutorials to enable data scientists and practitioners to incorporate the most appropriate tool for their problem into their work products. The architecture of the package has been engineered to conform to a standard paradigm used in data science, thereby further improving usability for practitioners. Such architectural design and abstractions enable researchers and developers to extend the toolkit with their new algorithms and improvements, and to use it for performance benchmarking. A built-in testing infrastructure maintains code quality.\vphantom}

2017

1. arXiv

   CELIO: An Application Development Framework for Interactive Spaces

   Shrinivasan, Yedendra B., and Zhang, Yunfeng

   arXiv preprint arXiv:1710.01772 Oct 2017
2. arXiv

   Combining Absolute and Relative Pointing for Fast and Accurate Distant Interaction

   Zhang, Yunfeng

   arXiv preprint arXiv:1710.01778 Oct 2017
3. arXiv

   CELIO: An Application Development Framework for Interactive Spaces

   Shrinivasan, Yedendra B., and Zhang, Yunfeng

   arXiv preprint arXiv:1710.01772 Oct 2017
4. arXiv

   Combining Absolute and Relative Pointing for Fast and Accurate Distant Interaction

   Zhang, Yunfeng

   arXiv preprint arXiv:1710.01778 Oct 2017

2015

1. ICCM

   Visual Search of Displays of Many Objects: Modeling Detailed Eye Movement Effects with Improved Epic

   Kieras, David E., Hornof, A. J., and Zhang, Yongfeng

   In Proceedings of the 13th International Conference on Cognitive Modeling Oct 2015
2. arXiv

   Interacting with Large Display Walls in Commercial Environments

   Shrinivasan, Y., Melville, D.,  Zhang, Yunfeng, Lenchner, Jonathan, and Bellamy, R.

   In Commercial Environments, Workshop on Interaction on Large Displays, ACM ITS 2015 Oct 2015
3. CHI

   Designing Information for Remediating Cognitive Biases in Decision-Making

   Zhang, Yunfeng, Bellamy, Rachel KE, and Kellogg, Wendy A.

   In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems Oct 2015
4. TOPICS

   Reinforcement Learning and Counterfactual Reasoning Explain Adaptive Behavior in a Changing Environment

   Zhang, Yunfeng, Paik, Jaehyon, and Pirolli, Peter

   Topics in cognitive science Oct 2015
5. Visual Search of Displays of Many Objects: Modeling Detailed Eye Movement Effects with Improved Epic

   Kieras, David E., Hornof, A. J., and Zhang, Yongfeng

   In Proceedings of the 13th International Conference on Cognitive Modeling Oct 2015
6. arXiv

   Interacting with Large Display Walls in Commercial Environments

   Shrinivasan, Y., Melville, D.,  Zhang, Yunfeng, Lenchner, Jonathan, and Bellamy, R.

   In Commercial Environments, Workshop on Interaction on Large Displays, ACM ITS 2015 Oct 2015
7. CHI

   Designing Information for Remediating Cognitive Biases in Decision-Making

   Zhang, Yunfeng, Bellamy, Rachel KE, and Kellogg, Wendy A.

   In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems Oct 2015
8. TOPICS

   Reinforcement Learning and Counterfactual Reasoning Explain Adaptive Behavior in a Changing Environment

   Zhang, Yunfeng, Paik, Jaehyon, and Pirolli, Peter

   Topics in cognitive science Oct 2015

2014

1. BICA

   Counterfactual Reasoning as a Key for Explaining Adaptive Behavior in a Changing Environment

   Paik, Jaehyon,  Zhang, Yunfeng, and Pirolli, Peter

   Biologically Inspired Cognitive Architectures Oct 2014
2. ETRA

   Easy Post-Hoc Spatial Recalibration of Eye Tracking Data

   Zhang, Yunfeng, and Hornof, Anthony J.

   In Proceedings of the Symposium on Eye Tracking Research and Applications Oct 2014
3. CHI

   Understanding Multitasking through Parallelized Strategy Exploration and Individualized Cognitive Modeling

   Zhang, Yunfeng, and Hornof, Anthony J.

   In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Oct 2014
4. BICA

   Counterfactual Reasoning as a Key for Explaining Adaptive Behavior in a Changing Environment

   Paik, Jaehyon,  Zhang, Yunfeng, and Pirolli, Peter

   Biologically Inspired Cognitive Architectures Oct 2014
5. ETRA

   Easy Post-Hoc Spatial Recalibration of Eye Tracking Data

   Zhang, Yunfeng, and Hornof, Anthony J.

   In Proceedings of the Symposium on Eye Tracking Research and Applications Oct 2014
6. CHI

   Understanding Multitasking through Parallelized Strategy Exploration and Individualized Cognitive Modeling

   Zhang, Yunfeng, and Hornof, Anthony J.

   In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Oct 2014

2013

1. ICCM

   Using Model Tracing and Evolutionary Algorithms to Determine Parameter Settings for Cognitive Models from Time Series Data Such as Visual Scanpaths

   Zhang, Yunfeng, and Hornof, Anthony J.

   In Proceedings of the 12th International Conference on Cognitive Modeling Oct 2013
2. ICCM

   Using Model Tracing and Evolutionary Algorithms to Determine Parameter Settings for Cognitive Models from Time Series Data Such as Visual Scanpaths

   Zhang, Yunfeng, and Hornof, Anthony J.

   In Proceedings of the 12th International Conference on Cognitive Modeling Oct 2013

2012

1. HFES

   A Discrete Movement Model For Cursor Tracking Validated in the Context of a Dual-Task Experiment

   Zhang, Yunfeng, and Hornof, Anthony J.

   In Proceedings of the Human Factors and Ergonomics Society Annual Meeting Oct 2012
2. HFES

   A Discrete Movement Model For Cursor Tracking Validated in the Context of a Dual-Task Experiment

   Zhang, Yunfeng, and Hornof, Anthony J.

   In Proceedings of the Human Factors and Ergonomics Society Annual Meeting Oct 2012

2011

1. BRM

   Mode-of-Disparities Error Correction of Eye-Tracking Data

   Zhang, Yunfeng, and Hornof, Anthony J.

   Behavior research methods Oct 2011
2. BRM

   Mode-of-Disparities Error Correction of Eye-Tracking Data

   Zhang, Yunfeng, and Hornof, Anthony J.

   Behavior research methods Oct 2011

2010

1. CHI

   Knowing Where and When to Look in a Time-Critical Multimodal Dual Task

   Hornof, Anthony J.,  Zhang, Yunfeng, and Halverson, Tim

   In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Oct 2010
2. ICCM

   Task-Constrained Interleaving of Perceptual and Motor Processes in a Time-Critical Dual Task as Revealed through Eye Tracking

   Hornof, Anthony J., and Zhang, Yunfeng

   In Proceedings of the 10th International Conference on Cognitive Modeling Oct 2010