Public sector borrowing fell to £5bn in May, down £2bn from a year earlier, official figures show.
The fall was bigger than expected and brings borrowing for the financial year to date to £11.8bn, £4.1bn less than in the same period in 2017.
At the same time, the Office for National Statistics (ONS) revised down its figure for government borrowing in 2017-18 to £39.5bn.
The total was the lowest annual level of borrowing in 11 years.
The figures come as Chancellor Philip Hammond prepares to reaffirm his promise to reduce public debt, despite Prime Minister Theresa May’s promise of increased spending on the NHS.
In a speech later on Thursday, Mr Hammond will say that taxes must rise, although increases will be implemented in a “fair and balanced way”.
Public sector net debt, excluding public sector banks, was £1,781.4bn at the end of last month, equivalent to 85% of GDP, the ONS said.
That is £44.7bn higher than a year earlier, but 0.4 percentage points lower as a percentage of GDP.
Analysis: Kamal Ahmed, economics editor
As the chancellor grapples with how to raise £20bn of extra spending on the NHS in England by 2023, today’s public finance figures do provide some help.
Borrowing is lower than forecast by the Office for Budget Responsibility in March, giving Philip Hammond a little more “wriggle room” whilst still hitting his target of reducing the deficit – the amount of money the government spends above the amount it gains in revenues from taxes and investment.
The chancellor knows that as the public finances begin to look healthier, pressure will grow to open the borrowing taps – thereby enabling more spending on not just health, but the police and defence, for example.
But that misunderstands Mr Hammond’s economic DNA.
At £1.8tn, or 85% of the value of the total UK economy, the chancellor believes Britain’s debts are still too high.
Yes, they might be falling as a proportion of the overall economy, but they are still double the levels before the financial crisis.
Levels so high that Britain remains exposed to risks in the event of another significant economic downturn, Mr Hammond believes.
If that were to happen – and the first few months of the year have been economically weaker than many expected – then the better state of the public finances announced today could soon sour.
“May’s public finances figures not only confirmed that the new fiscal year got off to a good start, but revealed that borrowing in 2017-18 was also a little lower than previously thought,” said Andrew Wishart, UK economist at Capital Economics.
“It’s early days yet, but if this is sustained, borrowing would undershoot the [Office for Budget Responsibility’s] 2018-19 forecast by £9bn or so over the year as a whole.
“What’s more, if the economy holds up as we expect, borrowing is likely to undershoot the OBR’s forecast by a more significant margin in subsequent years.
“This would allow the chancellor to deliver the recently promised increase in health spending over the next five years while still meeting his fiscal target.”