Automated Political Stance Identification Tool Released
A new tool designed to score political language and classify positions has been launched, offering researchers and the public a systematic way to evaluate political discourse.
The Automated Political Stance Identification (APSI) Tool was developed by the Path to Power Project members, Dr. Juan S. Gómez Cruces and Ewan Thomas-Colquhoun, together with HPI Master’s Student Yorik Scheffler. The tool is now publicly accessible at the https://apsi.sc.hpi.de/.
APSI uses Natural Language Inference to classify political stances in text. It is a method where the model tests how strongly a text supports or contradicts a set of predefined statements based on political science concepts.
The tool scores the text across three dimensions, including economic positioning on the left–right spectrum and the presence of populist rhetoric. A third dimension, support for liberal democratic values, is currently under development. In addition, APSI provides interpretation labels, confidence levels, contradiction indicators, and information on which hypotheses most influenced the results.
How the APSI Tool works
The tool can be applied to a wide range of materials, from political speeches and policy documents to social media posts and interviews. Its aim is to make the analysis of political discourse more transparent, reproducible, and accessible to researchers, journalists, policymakers, and the public.
The APSI Tool can be used for media monitoring, civil society advocacy, disinformation research, or to better understand how political language is constructed and how it shifts over time.
“We want to bring scientific knowledge closer to all people, regardless of their previous knowledge of political concepts”
The base model used by APSI was trained on 2.73 million hypothesis–premise pairs across 26 languages. It was fine-tuned on 201.7K political examples. To validate the APSI scores, the researchers relied on the opinions of 147 leading political scientists, who had evaluated a set of political texts (20 texts per dimension).
Speaking on the idea behind the project, Dr. Gómez Cruces says that within the Path to Power project, the researchers noticed that many reputable expert surveys and datasets do not cover a large number of countries in the Global South. “We decided to build a tool that classifies political information in a standardized, transparent, and automated way,” Gómez Cruces explained. However, the development process led the team in an unexpected direction.
‘We started with a common approach in the social sciences: fine-tuning an existing model. We assumed there would be far more labeled data available to build on. It quickly became apparent that annotations from political manifestos did not transfer well to the classification of individual sentences. This pushed us away from the standard training pipeline and toward a different approach entirely.
Ewan Thomas-Colquhoun, Dr. Juan S. Gómez Cruces, Yorik Scheffler (left-to-right)
Photo: Natasha Kondrashova
We expected the supervised training approach to perform best in terms of accuracy. When the hypothesis-based method actually outperformed it, we were genuinely surprised’, admit the researchers.
The developers emphasize that it is intended for research purposes only and should not be used for surveillance, profiling, or decision-making about individuals. As with any AI-based system, they caution that results may reflect biases in training data and should be interpreted critically.