Doctors, hospitals, nurses and midwives are rarely off our TV screens. So it’s been since the 1960s when Dr Kildare and Emergency Ward 10 topped audience ratings. In the 1990s, Casualty was peak time viewing for millions and more recently we’ve had the likes of Call the Midwife (set in the 1950s and 60s) and E R. The public appetite for medical dramas in all countries seems insatiable.

Only occasionally do the political challenges of providing health services intrude – resource constraints, rationing, restricting clinicians’ autonomy, concerns about the quality of care, inequity. And when they do, it’s as a marginal attempt to explain the behaviour of staff or patients. Like most screen drama these shows dwell on personal stories and relationships.

Literature is no different. Few novelists have concerned themselves with how health care is funded, how staff are paid, how patients are treated or how services are organised and managed. Maybe it reflects a lack of awareness of the dramas being played out behind the scenes in hospitals, dispensaries and community services. Or maybe it’s a lack of interest. Novelists tend to be absorbed by personal stories and may see little in the challenges and intrigue of policy and politics.

Ever since the early decades of modern health care (following the Regency revolution that initiated the transformation of so many aspects of public life), novelists have shown no interest despite the woeful state of medical practice they must have seen all around them. As improvements in the conduct of doctors and the governance of hospitals got underway in the nineteenth century, these changes hardly got mentioned in novels.

When novelists wrote about doctors or nurses, it was their personal lives they focused on. Their clinical practice rarely got mentioned, let alone how services were funded, organised and provided. In the main, doctors remained marginal characters (and nurses and midwives were invisible). In Emma, Jane Austen doesn’t even let the apothecary, Mr Perry, speak. Despite frequent appearances, all we learn is that he is the main source of comfort for Emma’s father. And when doctors are centre stage, such as Alan Woodcourt in Bleak House (written in the 1850s but set in the 1830s), Dickens confines his interest to the doctor’s love life. This is all-the-more surprising given the writer’s promotion of new hospitals in London, such as that for children in Great Ormand Street.

Equally perplexing is Anthony Trollope, a novelist who wrote extensively on political, social, and gender issues. Yet on the one occasion his main character was a general practitioner, Dr Thorne, all we learn about him are his diplomatic skills in manoeuvring his way through the vicissitudes and clashes of members of the upper echelons of society. Only when Dr Fillgrave appears, a physician from the nearby city, do we glimpse the animosity between these two strands of the medical profession, a key factor that shaped medical care and, to some extent, can still be detected today. The name Trollope gives the physician reveals something of his level of confidence in the medical profession.

One exception is Harriet Martineau who, in Deerbrook, set in the 1830s, reveals how precarious a general practitioner’s position was. Deemed a tradesman or shop-keeper, he was expected to cast his newly acquired right to vote according to the wishes of the rich land-owners. When Edward Hope, the general practitioner, refuses to support the incumbent Tory, the principal land-owner sees it as “an unpardonable impertinence” and proceeds to orchestrate rumours and gossip that destroy the doctor’s medical and personal reputation.

The one nineteenth century novel that really stands out for its insight and understanding of intra-professional rivalries and early attempts to reform medicine is Middlemarch. Published in 1871-2, George Elliott is thought to have been influenced by Martineau’s earlier work but surpasses it in her grasp of the scientific revolution that was starting to threaten the established order in the 1830s. Through Tertius Lydgate, the young general practitioner, she presents the challenges that such derided ‘Lancet men’ posed to medical complacency.

It was 1937, almost 70 years later, before another, comparable work of fiction appeared. The Citadel, set in the 1920s, not only enlightened its readers, it was seen as influencing the formation of the British NHS. This must have pleased the author, Dr A J Cronin, who made clear his motivation and intention:

“I have written in The Citadel all I feel about the medical profession, its injustices, its hide-bound unscientific stubbornness, its humbug … The horrors and inequities detailed in the story I have personally witnessed. This is not an attack against individuals, but against a system.”

This was ground-breaking, demonstrating the influence fiction could have on policy-making. Cronin went on to write the much-loved TV series, Dr Finlay’s Casebook, which explored primary care and public health practice in the 1920s. But few other writers have followed his lead. And when policy and management of health care has been included, it has been for its comic value – Green Wing in the UK, Scrubs in the USA.

Before bemoaning the lack of attention to the behind-the-scenes workings in health care, we need to remember that this is not unique to the health sector. There aren’t a lot of tales from the worlds of engineering, legal firms or accountancies. The one exception is higher education where in the 1970s, once again, comedy was the vehicle to expose the writers’ critical views of the way universities were run. David Lodge, Malcolm Bradbury and Tom Sharpe created enduring images of intrigue, battles and incompetence, shattering innocent views of our palaces of learning.

Novelists and policy-makers inhabit contrasting paradigms. Is it fanciful to think fiction could contribute more to expanding people’s understanding of policy, let alone achieving what AJ Cronin may have done and influence policy? The power of fiction to move, to grip readers’ attention and transport them to previously unimagined worlds is so different to dry, technical policy reports. But what if fiction could be harnessed to the world of policy? Is fiction an unrealised, or at least under exploited, means of instilling policy debates with greater reality and, as a result, helping create better policy?

I knew that writing fiction would be different. After years of writing scientific papers, editorials, essays and reports, it was the very challenge of trying something new that attracted me. That, and the thought that maybe my lifelong interest in enhancing public understanding of health care might best be tackled through story-telling.

Released from the constraints of non-fiction – rigorous adherence to accuracy, avoidance of personal opinion straying into scientific discourse, keeping to long-established structure and order – it took time to break free from those familiar and comforting shackles. But when I did, the sense of liberation, the scope to imagine, to create new realities, to shape the narrative was exhilarating. To take the historical, documented facts and have licence to conjure up plausible explanations to link disparate events was wonderful.

In some ways, there were still parallels with non-fiction. After all, scientific research papers require you to offer explanations, based on explicit theory, of how what you’ve observed may have come about. And just as non-fiction requires a lot of preparatory research, so did fiction – background reading on the age of reform, trawling through medical journals and newspapers, examining hospital archives, searching college and society registers, scrutinising maps and visiting sites. It didn’t end there. Having assembled a mountain of evidence, a narrative had to be constructed that could bring order and clarity, a task that hardly differed from the simple guidelines I had spent years teaching junior faculty and doctoral students about how to write up their research.

But from there, the similarities ended. Tentatively tip-toeing into the first draft, I became aware that the characters I was fumbling with, trying to make coherent, consistent and believable were starting to take on a life of their own. I was discovering something most novelists know. I could start to sit back and let my characters speak for themselves. They were helping me work out what they’d do or say. All I had to do was to put them in situations and then watch and listen. The writing was still challenging, at times tough going, but it felt as though I was now sharing the task. I was on a journey with them, a journey in which only I knew the destination.

I wasn’t surprised by the way my characters were taking on an almost independent existence. The limited amount I’d read about fiction writing had prepared me for this, though until it happened I had remained somewhat sceptical and doubted it would happen for me. What I hadn’t been prepared for was my emotional involvement with them. I felt protective towards them. I found myself sharing their joy and their despair as if they were close friends, real people. I shouldn’t have been surprised given the intensity of our relationship. Not just measured in terms of the hours spent writing but the days, weeks, months spent living with them. They were ever present as I walked by the sea, stood in the shower, chopped vegetables, pruned roses. As if by magic, solutions to dilemmas I had been grappling with would offer themselves, resolving obstacles that had concerned me for weeks.

At times, I wondered if I had become too involved with the main characters. Maybe greater detachment would enhance my writing. But that didn’t seem to be an option, even if I wanted it. Sometimes my involvement was overwhelming. Without giving anything away about the plot, there were scenes that reduced to me tears as I wrote them. Even now, months later, I have to brace myself to read those sections. So, yes, fiction and non-fiction share much in common but there are striking differences as well. Forty years of non-fiction writing never made me cry.