Welfare fraudsters will be dealt with faster under new legislation expected to save the UK taxpayer £1.6bn over the next five years, the prime minister is set to announce.

Keir Starmer is expected to promise to “leave no stone unturned” as his government seeks to “rebuild our public services” during his speech to the Labour party conference on Tuesday, his first conference address as prime minister.

Part of his speech will include the announcement of a new fraud, error and debt bill to modernise the Department for Work and Pensions (DWP), allowing it to recover money lost to fraud while protecting vulnerable claimants from mounting debts.

It will also contain powers to help officials keep up with more sophisticated fraud. Labour said fraud and error in the social security system cost the country £10bn a year.

Critics within the party reacted to reports of the proposals, saying it was “almost like George Osborne speaking again”.

Alongside the bill, the prime minister is expected to announce plans for wider welfare measures to help get people back into work, with details to be unveiled later this year.

Starmer is expected to say: “We will get the welfare bill down, because we will tackle long-term sickness and get people back to work. We will make every penny work for you because we will root out waste and go after tax avoiders.

“There will be no stone left unturned, no innovation ignored and no return to Tory austerity. We will rebuild our public services, protect working people and do this in a Labour way. That is a promise.”

Tackling welfare fraud forms part of the government’s wider war against corruption, fraud and tax evasion, with plans for a Covid corruption commissioner already announced.

Starmer is expected to argue that savings made by tackling fraud can be redirected into other public services as his government seeks to fill what it describes as a £22bn “black hole” left by its predecessors.

Among the measures included in the proposed bill are new powers to allow the DWP to take control of fraud investigations and to recover money from those who could pay it back but have avoided doing so.

Labour also stressed that the bill would contain safeguarding measures for vulnerable welfare claimants, and staff would be trained to the highest standards on the appropriate use of new powers.

Labour has also promised a crackdown on tax avoidance, with the chancellor, Rachel Reeves, saying the government would hire 5,000 more compliance officers at HMRC.

The former Labour shadow chancellor John McDonnell, who is firmly on the left of the party, said Starmer’s messaging about the economy and spending was the same as that from the Conservative party during austerity.

Speaking to BBC Radio 4’s Today programme, McDonnell, who had the whip removed from him by Labour when he voted against the government over the removal of the two-child benefit cap, said: “If you close your eyes and listen to the language being used, it’s almost like George Osborne speaking again in 2010.

“When you hear politicians talk about tough choices or painful decisions, and then you hear some of the rhetoric around fraud and social security, literally that’s a replica of a speech made by George Osborne in 2010.

“What I’m worried about is the first measure the government has actually introduced is an austerity measure of winter fuel allowance, which for many of my constituents will cause immense hardship.”

He added: “We were desperate to get rid of the Tories. I was so elated at the election of a Labour government, but I don’t believe that actually what we’re hearing this week is going to inspire people around the sort of objectives that we have as a party, of changing our society, ending austerity.”

Speaking on Sky News, the chancellor of the duchy of Lancaster, Pat McFadden, defended the focus on benefit fraud. He said: “It’s really important that if money is spent on benefits, it goes to those who are genuinely in need of it, and where there’s fraud in the system that we try to root that out.”