Tory former Brexit minister Lord David Frost has called on Boris Johnson to quit as Prime Minister before he takes the Conservative Party and the Government “down with him”.

Rishi Sunak quit as chancellor on Tuesday, alongside Sajid Javid who resigned as health secretary, in a move that came just as the Prime Minister was being forced into a humiliating apology to address the row over scandal-hit former deputy chief whip Chris Pincher.

Lord Frost wrote in The Daily Telegraph that Mr Johnson’s place in history will be secure as “one of the past century’s most consequential prime ministers”.

“If he leaves now, before chaos descends, that reputation is what will be remembered. If he hangs on, he risks taking the party and the Government down with him,” the former Cabinet minister said.

“That’s why it is time for him to go. If he does, he can still hand on to a new team, one that is determined to defend and seek the opportunities of Brexit, one that is able to win the next election convincingly. That is in the Conservative Party’s interest, in Leave voters’ interest, and in the national interest. It needs to happen.”

A series of significant Government exits have threatened to force Mr Johnson from Number 10.

Mr Pincher quit as deputy chief whip last week following claims that he groped two men at the upmarket Carlton Club, but Mr Johnson knew about allegations against him as far back as 2019.

Mr Sunak and Mr Javid, both potential leadership rivals, offered sharp criticisms of Mr Johnson in their resignation letters.

The pair were swiftly replaced on Tuesday night, with Nadhim Zahawi promoted to be the new Chancellor and Steve Barclay becoming Health Secretary.

Lord Frost quit the Government with immediate effect in mid-December, citing its “current direction of travel” as well as fears over “coercive” Covid measures and the wish for the UK to become a “lightly regulated, low-tax” economy.

“I resigned from the Government on a matter of principle,” he wrote in the Telegraph on Wednesday.

“Last night Sajid Javid and Rishi Sunak did the same. Other Cabinet ministers now need to consider whether they are truly happy with the current direction of travel.”

The 57-year-old said the Pincher affair had been a “real-life case study” of Mr Johnson’s decision-making.

“Confronted with a problem which appeared to reflect badly on the Prime Minister’s judgement, we saw once again the instinct was to cover up, to conceal, to avoid confronting the reality of the situation,” Lord Frost wrote.

“Once again that instinct, not the issue itself, has become the story and the problem. Worse, this time round, ministers have been sent out repeatedly to defend suspect positions that came apart under closer examination. This is no way to run a government.”