Introduction

SQuARE is an open-source QA platform that enables researchers and developers to:

• Use QA Skills in a web browser without writing any code. A QA Skill or simply Skill is a QA pipeline, i.e., a datastore, a retrieval model, a reader model, and a data preprocessing and postprocessing steps. All components are optional, allowing maximum flexibility to the user.
• Study the strengths and weaknesses of existing Skills by comparing them in parallel with a user-friendly interface.
• Analyze the exaplanability of the Skills through Saliency Maps.
• Investigate the robustness to Adversarial Attacks.
• Visualize graphs for Skills using graphs.
• Study the behavior of the Skills through behavioral tests.
• Combine multiple Skills with Meta-Skills.

NOTE: To try out SQuARE directly, please visit the SQuARE Website.

Why SQuARE?

The recent explosion of Question Answering (QA) models and datasets is difficulting the holistic analysis of new research. Recent advances in NLP and information retrieval have given rise to a diverse set of question answering tasks that are of different formats (e.g., extractive, multiple-choice), require different model architectures (e.g., generative, discriminative), and setups (e.g., with or without retrieval). SQuARE aims to be an entry-point to QA research providing a platform to run models to simplify reproducibility, analysis and comparisons. In addition, this platforms also provides the first ecosystem for QA research with explainability, adversarial attacks, behavioral tests, and graph visualizations.

NOTE: Find out more about the project on UKPs Website.