This essay, on the dangers of unthinking pessimism, was my Observer column this week. It was published on 7 November 2021, under the headline “Want to change the world? Then you’d better give up on self-defeating pessimism”.

He quickly apologised for his crassness. But the Archbishop of Canterbury’s comparison of politicians who fail to tackle climate change with those in the 1930s who appeased the Nazis was not simply crass. It illustrated how many imagine that the best means of arousing political concern is by painting as dark a picture as possible of social problems. And few things today speak more of evil than the Holocaust.

The kind of facile optimism that a figure such as Boris Johnson exudes is deeply obnoxious. It’s a way of avoiding the issues, of pretending that we can resolve our problems by not thinking deeply about them, but by simply asserting “we can do it”.

There is something equally objectionable about unthinking pessimism. About the insistence that a social problem is so beyond control that we cannot avoid catastrophe or so deeply rooted that it cannot be dug out. Yet this often seems to be the default setting for so many political discussions, from racism to the refugee crisis. And on no issue more so than climate change.

Roger Hallam, the co-founder of Extinction Rebellion, has given “Advice to Young People as They Face Annihilation”. Describing social collapse after unchecked climate change, he warned in graphic terms that “this is what is going to happen to your generation”: “A gang of boys will break into your house demanding food. They will see your mother, your sister, your girlfriend and they will gang rape her on the kitchen table… They’ll take a cigarette and burn out your eyes with it… That is the reality of climate change.” It’s difficult to know what such nihilistic despair is supposed to achieve or how imagining the world as a Mad Max film would make people more willing to take action.

The novelist Jonathan Franzen insisted in the New Yorker magazine that people need to stop “denying reality” and tell “ourselves the truth” that climate change “can’t be solved”. “Every day, instead of thinking of breakfast,” he suggested, “they have to think about death.”

Not surprisingly, this sense of bleakness and futility has seeped into wider culture. A recent international survey of young people found that 75% believed “the future is frightening”, 56% thought “humanity is doomed” and 39% were “hesitant to have children”.

Climate change is a critical issue and one that will require considerable political will and social resolve to challenge. Hallam and Franzen and similar thinkers insist that only an apocalyptic vision will persuade people to take action. In reality, as the environmental journalist Hannah Ritchie has observed: “Once anger transitions into hopelessness, we struggle to achieve much at all”. Telling people that there is no future is hardly conducive to getting them to act to change it.

It’s not just on climate change that pessimism has colonised the debate. Consider discussions of racism. From the widening scandal over Yorkshire County Cricket Club’s treatment of former player Azeem Rafiq to the trial in America of three white men for the death of jogger Ahmaud Arbery, racism dominates much of the news. Central to the debate is the insistence on the need to root out “institutional racism”. And yet, for all such talk, many, whether columnists or corporations, see racism as the property not of social structures but of white people. Once racism is rendered the product of whiteness, it becomes, in the words of Robin DiAngelo, author of White Fragility and doyenne of the “white people are the problem” school, “unavoidable”, something that will endure as long as white people exist. That is why “progressives” confronting racism “cause the most damage to people of colour” because of their dreams of eradicating racism.

DiAngelo has a nice line in monetising white guilt, but serious black thinkers have equally made the case for taking a bleak view. The US writer Ta-Nehisi Coates was acclaimed by the late Toni Morrison as the new James Baldwin for his challenging arguments and exquisite prose. White supremacy, Coates suggests, is like a natural force beyond reach of human solutions. “The earthquake cannot be subpoenaed. The typhoon will not bend under indictment”, as he puts it in his book Between the World and Me. Coates is part of a school of thought, sometimes called “Afropessimism”, nurtured by writers such as Frank Wilderson and Patrice Douglass, who argue that little can change for black people and that, in Wilderson’s words, “Blackness is coterminous with slaveness”.

Such arguments are not, of course, part of everyday, mainstream discussion. Yet the success of figures such as DiAngelo and Coates and the entrenchment of the view of racism as an issue of whiteness show how these ideas can infect wider circles.

It might seem odd to talk of pessimism or despair in the age of Black Lives Matter and Extinction Rebellion. However, a paradox of our time is that passionate protest often coexists with a sense of fatalism about possibilities of change. Much has been made of the disruption caused by Insulate Britain’s tactics of closing off motorways. All protests cause some disruption. The real problem with Insulate Britain is that the disruption has become an end in itself, protesters seemingly unconcerned about alienating people, rather than winning them over to their side.

Underlying much of this is a sense of despair about human beings, who, in the the words of philosopher David Benatar, not only constitute “the most destructive species” but whose lives are “filled with dishonesty, betrayal, negligence, cruelty, hurtfulness, impatience, exploitation”. It is, he says, best that no human is born because of “the harm that the created person will (likely) do”. Again, most people probably abhor the better-never-to-have-been-born argument and yet that sense of cynicism about human activity finds a wider hearing.

There is a long history of misanthropic pessimism, largely within the conservative tradition. For conservatives, the flaws of human nature are reason to shun what they regard as utopian schemes for social change. “Those who struggle to change the world,” the philosopher John Gray suggests, are merely seeking “consolation for a truth they are too weak to bear”.

What is different today is that such pessimism and misanthropy have come to colonise much of the left, too. This, as much as climate change or racism, is an issue to confront. Otherwise, despair about challenging such problems will become a self-fulfilling prophecy.