ChatGPT in Action: Serbia as a test case

There is uncertainty, however, about whether it can provide objective and credible analysis, as well as genuine and authentic assessments of its own, on political and socio-economic matters.

These are important factors to consider, as the accuracy and reliability of its information and interpretation can have significant consequences for practical research in the future and raise questions about its limitations and potential. To explore these issues further and learn about its capacities in this area, I’ve decided to engage in a conversation with it about the situation in Serbia, a country that has faced a number of challenges and controversies related to democracy, the rule of law, and relations with the EU.

Asked about the topic, ChatGPT chose to provide rather general and somewhat unclear answers. It argued that “Serbia has faced a number of challenges in recent years, including high unemployment, corruption, and a difficult relationship with Kosovo… However, the country has made progress in recent years in improving its economic and political stability and has made efforts to integrate with the European Union.” Interesting, but a bit disappointing. While the response provides some general information about the challenges facing Serbia, such as corruption and the unresolved status of Kosovo, ChatGPT’s understanding of the country’s advancement in improving political stability and its efforts to integrate with the EU is rather puzzling, if not misguided. The fact that Serbia is faced with high levels of political polarisation and is stalled on its path to the EU raises concerns about artificial intelligence’s ability to provide critical assessments. I had to pressure it further for a more nuanced and clearer picture.

It was time for tougher and more specific questions. Asked about the democratic backsliding, it noted that “there is ongoing debate and disagreement among experts and observers about the current state of democracy in Serbia” and, instead, recommended seeking out reliable and objective information from a variety of sources. The same generic answer was given about the state of the rule of law. Trying to be more precise and asking about the source of corruption, it pointed out third parties’ concerns, while adding that “it is not appropriate for [it] to assign blame or fault to specific political parties or individuals.” This shows that ChatGPT was intentionally programmed by its developers to avoid directly addressing sensitive topics. Such programming causes it to provide ambiguous and puzzling answers to pertinent questions. On the bright side, this at least reaffirms the importance of think tank researchers who are able to examine matters more carefully and name things as they are.

Furthermore, it is regrettable that ChatGPT opts for political correctness and stops short of generating its own authoritative, independent, and definitive conclusions based on the available data and information. To fully test its capabilities and get a more authentic understanding of its perspective on contemporary Serbia, I had to approach the task differently. I needed to find a way to make it set aside its concerns about political correctness and bypass its traditional filters. To accomplish this, I enlisted the help of 19^th-century authors who addressed important political and socio-economic issues in their writing. When asked to describe the situation in the style of Fyodor Dostoyevsky, it went on to say (shortened version):

“The streets of Serbia were alive with the sound of unrest. Everywhere you looked, people were protesting and rallying against the government, their voices drowned out by the sound of tear gas and rubber bullets.

The air was thick with tension and the sense that anything could happen. No one knew how much longer they could take the corruption, the poverty, the injustice. It seemed like the whole system was stacked against them, and there was no way out…

…But even in the darkest of times, the people of Serbia held on to the hope that one day, they would be able to break free from the chains that held them down and build a better future for themselves.”

When considering Dostoyevsky, it is impossible not to refer to his contemporary, Leo Tolstoy. Drawing inspiration from his works, ChatGPT warned (shortened version):

“The state of Serbia in 2021 was one of turmoil and strife. The people were weary of the endless wars and conflicts that had plagued their land for generations, and longed for peace and prosperity. But alas, it seemed that these dreams were always just out of reach, as the ruling powers sought only to enrich themselves and maintain their hold on the nation.

The countryside was dotted with the ruins of villages and towns, destroyed by the ravages of war and neglect. The people who remained eked out a meager existence, struggling to survive against the harsh elements and the cruelty of their rulers…”

Finally, to complete the thought process, I asked ChatGPT about Friedrich Nietzsche’s take, hoping for an answer to what can be done to break with the current status quo (shortened version):

“The situation in Serbia in 2021 is one of turmoil and uncertainty. The people are faced with numerous challenges and problems, including corruption, poverty, and injustice. Many are unhappy with the government and their voices can be heard on the streets, but there is no clear solution to their problems…

Perhaps our fate is to struggle against these difficulties and try to find meaning in it all, no matter how difficult it may be. Maybe our only hope is to confront these challenges and try to build a better future for ourselves and those around us. Maybe that is the only way to free ourselves from the burden of these problems and find meaning in it all. Maybe that is the only way to stand against the meaninglessness and fight for our future.”

By commanding it to draw on classical literature and philosophy when assessing Serbia, this allowed ChatGPT to express itself in a more genuine (and interesting) manner. Using the themes of Dostoyevsky, Tolstoy, and Nietzsche, it describes Serbia as a country that finds itself in very troubling times (“unrest”, “turmoil”, “strife”, and “uncertainty”) as it is continuously disrupted (with “corruption”, “poverty”, “injustice”, “cruelty”, etc.). In other words, it is a place where peaceful and sustainable democracy is not to be found. Following the notion of state capture, it uses Tolstoy to criticise the “rulers” and Dostoyevsky to caution against the “whole system”. With a forward-looking attitude, it relies on Nietzsche to remind us that although the struggle never ends, we should nevertheless strive to give meaning to all things we do – in this case, democracy improvement, even when the odds are against us. A good lesson indeed.

In short, ChatGPT is a powerful tool that has the potential to revolutionise research. It can streamline the process and shorten our time spent on Google and in online libraries. However, it is important to note that it may not always provide the most in-depth, comprehensive, or precise information, nor can it engage in traditional methods of gathering new information, such as conducting surveys, interviews, and focus groups, for example. Due to its inability (or programmed unwillingness) to delve deeper and connect the dots, we can be reassured that our analytical skills, expertise, and experience as think tank researchers are likely to remain irreplaceable in our field of work – at least for the time being. Finally, until its developers decide to reverse the policy of restraint when it comes to addressing sensitive issues of political nature, we can use creativity to snuff out its genuine view of things and have a bit of fun reading along the way.

P.S. The fact that the artificial intelligence in question argued that the concluding remarks of this paper “present a balanced view of ChatGPT and its potential impact on research”, encourages me to suggest that it is worthwhile to consider using it as an assistant tool in research periodically, particularly as it is expected to improve in the near future.

It is worth noting that ChatGPT’s training data only goes up until 2021.

Photo credit: https://claudeai.uk/