This essay, on vaccine hesitancy, was my Observer column this week. (The column included also a short piece on Chicago libraries’ utopian experiment.) It was published on 15 November 2020, under the headline “Vaccination hesitancy is about lack of trust. Compulsion is not the answer”.

“If a strain as deadly as the 1918 influenza emerges and people’s hesitancy to get vaccinated remains at the level it is today, a debilitating and fatal disease will spread.”

So wrote Heidi Larson in 2018. Larson is director of the London-based Vaccine Confidence Project and probably the most knowledgeable person on the question of “vaccine hesitancy” – the unwillingness of some to get vaccinated.

Two years after Larson wrote those words, we do have a pandemic that so far has taken more than a million lives, including at least 50,000 in the UK. We also have the possibility of a vaccine, the first of a number that could transform the Covid-19 landscape. Whether they do depends not just on how effective they are, but also on the willingness of people to be vaccinated. In the US, just half the population seems so inclined. In Britain, the figure is higher – around 70% – but still probably insufficient to generate herd immunity.

Such hesitancy has opened up a debate about whether Covid-19 vaccination should be compulsory. Like much else today, it’s a debate polarised between those absolutely for and absolutely against, between those who insist on the importance of individual freedom and those who point out that a refusal to be vaccinated is to put at risk not just one’s own life but also those of people in the wider community.

And, like much else today, it’s a more complicated argument than the “for” and “against” polarisation suggests. Despite the cries that mandatory vaccination programmes are an affront to liberty, they are not uncommon. Every US state requires vaccinations before a child can attend school, though most allow for religious and philosophical exemptions. France and Italy both have mandatory programmes. In Australia, some welfare benefits are conditional on vaccination. The evidence for the efficacy of such programmes is mixed, some studies finding that they work, others showing no link between mandatory vaccination and rates of immunisation.

There are many reasons for vaccine hesitancy. The greatest support for vaccination lies in the global south, in countries still ravaged by infectious diseases. High-income countries are often the least enthusiastic. The very success of vaccination programmes has occluded the social memory of such diseases.

It’s not just historical amnesia or factual ignorance, though, which drives vaccine hesitancy. The key, as Larson observes, is also the question of trust. We live in an age of growing distrust of expertise, of public institutions, of the “elite” and of other people. Inevitably, this shapes attitudes to vaccination.

Studies have shown that low levels of confidence in national institutions increases vaccine hesitancy, as does greater inequality. And then there’s the role of pharmaceutical companies. Those worried about vaccines, the science writer Leigh Phillips observes, often view big pharma as “corrupt, only out to make money for their shareholders and that their lobbyists have bought our political leaders”. From the role of these companies in creating America’s opioid crisis to their contemptuous treatment of poorer nations, there is truth to these claims. On the day that Pfizer made its big announcement about its Covid-19 vaccine being 90% effective, its CEO sold 62% of his shares, just as the share price surged. Pfizer claims that the sale had long been planned but it also fits a historical pattern of venality.

Coronavirus has wreaked havoc, destroying lives and livelihoods. The desire to enforce vaccination is understandable – I often feel it, too. But the background to vaccine hesitancy is complex. We should not dismiss fears as simply ignorance or irrationality. Still less should we treat it as a matter of law and order requiring punitive policies or the policing of the population.

The question of trust is important not just in this pandemic, but for the next one too. And for the success also of wider political projects. The impact of Covid-19 is devastating. But so, too, may be the long-term consequence of the draining away of public trust.

There may be a time, if voluntary vaccinations do not work, when the balance between individual freedom, maintaining public trust and the need to protect the community tips in the direction of making vaccination mandatory. But it’s a step we should take with caution.

The vaccination debate is really one about social solidarity and how to build it. We need to engage on many fronts, from challenging disinformation, to reducing inequalities, to rethinking the pharmaceutical market, to making institutions more responsive to people’s needs. There’s a line that Larson quotes that gets to the heart of the issue: “People don’t care about what you know, unless they know that you care.”