My latest column for Al Jazeera English is about the Donald Trump phenomenon. It was  published in Al Jazeera under the headline ‘Politics of Disillusionment and the Rise of Donald Trump’.

Last autumn, when I was in the USA, I asked a number of senior Republican Party figures about the possibility of Donald Trump winning the presidential nomination. All laughed out loud. It simply won’t happen, they told me. The Republican Party machine would take care of him.

I doubt if they are laughing now. On ‘Super Tuesday’ this week Trump won seven out of eleven states. The likelihood of Trump challenging for the White House seems less of a joke by the week.

‘All of us Smart Guys were blindsided by the rise of Donald Trump’, the historian Garry Wills observed recently. For a long time pundits ‘agreed that it could not happen, so it was not happening’. And now that it has happened, Wills lamented, ‘we have no good explanation of it.’

It is not only Trump who has left the pundits perplexed. Populist movements, from the UK Independence Party to the Front National in France to the Danish Peoples’ Party, are sweeping across Europe too.

Nor does the populist moment merely find expression on the right. Bernie Sanders is unlikely to win the Democratic Party nomination, but he has given Hillary Clinton a tougher fight than anyone expected. In Britain, Jeremy Corbyn astonished most pundits by becoming leader of the Labour Party last year, a success that former Prime Minister Tony Blair finds particularly perplexing. ‘I’m not sure I fully understand politics right now’, he told the Financial Times recently, adding that it might seem ‘an odd thing to say when I’ve spent my life in it’.

But perhaps it is not such an odd thing to say. Perhaps it is because Blair has spent his whole life in politics that he finds the contemporary scene so baffling. The landscape of politics has, over the past two decades, transformed, leaving many politicians and pundits without a map to guide them in the new terrain, and baffled as to the rules that now govern popular political behaviour.

The irony is that politicians like Tony Blair have played a major part in effecting this transformation. For much of the twentieth century, the political landscape was defined by the ideological struggle between left and right. In Europe, this expressed itself primarily through the contest between social democracy and conservatism. In the USA, the ideological distinctions were far more fuzzy; nevertheless the battle between the Democrats and the Republicans expressed the same distinction.

Today, that broad ideological divide has largely dissolved. The political sphere has narrowed; politics has increasingly become more about technocratic management rather than about social change. The gap between voters and the elite has widened, fostering disenchantment with the very idea of politics.

One way in which people have felt this change is as a crisis of political representation, as a growing sense of being denied a voice, and of political institutions as being remote and corrupt. The sense of being politically abandoned has been most acute within sections of the traditional working class. Economic changes, from the decline of manufacturing industries to the imposition of austerity policies, have made life more precarious. The weakening of trade unions have further marginalized working class communities. In the past, the mainstream parties of the left seemed to provide the means to change society for the better and to achieve their hopes and aspirations. Few still feel that.

The old political landscape left room for the ‘protest vote’. People often showed disenchantment with mainstream parties by voting for fringe parties or candidates as a means of temporarily expressing dissatisfaction with one of the main parties before returning to the fold; it was a way sending a message but not of upsetting the system.

Today the protest vote is no longer about teaching the main parties a lesson. It is about disengagement from the whole political process. Voters are not saying ‘I am voting for another party at this election to make you listen to me’. Increasingly many are saying, ‘You will never listen to me, so there is no point in voting for you at all’. The fringe party has taken centre stage. Hence the success of a figure such as Donald Trump.

What drives the Trump phenomenon is not ideology but attitude. Populists draw support because of their seeming willingness to challenge the mainstream consensus and to say things that others seem too scared to say. That is why the more offensive Trump is, the higher his ratings. When he calls Mexican immigrants rapists, or demands a ban on Muslims entering America, many of us hear the racism and the bigotry. Trump supporters hear someone speaking his mind and saying things that cannot normally be said.

There is now a chasm between the attitudes of the grassroots in the Republican Party and the party establishment. Over the past week, a panicking party machine has wheeled out figures such as Mitt Romney and John McCain to denounce Trump. It is unlikely to have much effect. What has drawn people to Trump is their anger with mainstream politicians. So why should the views of mainstream politicians like Romney and McCain change their minds? The more that the party establishment attacks Trump, the more it is likely to solidify Trump’s support.

What is true of the Republican party is true, too, of politics more generally. The fundamental fautline in politics is no longer between left and right. It is rather between mainstream political institutions and a growing mass of people who feel alienated and politically voiceless. And that is why politics today appears so unpredictable.

In an age in which progressive social movements have largely crumbled, and in which people feel they have lost control of the forces that govern their lives, political anger often finds expression not through opposition to a particular policy or a government, but through a generalized hatred of everything and everyone in power. Inchoately kicking out against the system can all too quickly mutate into indiscriminately striking out against the ‘Other’, whether Mexicans or Muslims. This is particularly so as the same mainstream politicians who dismiss populists as bigots themselves often assiduously foster fears about immigration, hence increasing cynicism about mainstream politics while also making people more receptive to the anti-immigration rhetoric of the populists.

The challenge we face is this. We need to find a means of revitalizing the democratic process, and of acknowledging the fears and anxieties of those who have rejected the mainstream, without either pandering to prejudices or dismissing them as just racists. It is whether or not we are able to meet that challenge, rather than anything that the likes of Donald Trump or Marine Le Pen may say, that will determine the future shape of politics on both sides of the Atlantic.

The photo of a jobs protest in Los Angeles by AFP/Der Spiegel