Women emerging from the Libyan embassy in London cover their faces as a British policeman looks on
Tripolihas condemned the UK for recognising the rebels as Libya's "sole governmental authority" after similar moves by France and the US.
Khaled Kaim, deputy foreign minister in Muammar Gaddafi's government, told reporters the decision was unprecedented and irresponsible.
Libyawould seek to reverse the decision through the courts, he said.
Britainhas ordered the expulsion of all eight remaining Gaddafi diplomats in the UK.
The rebel leadership, the National Transitional Council (NTC), has put forward Mahmud al-Naku, a writer and journalist, as the new Libyan ambassador in London.
He told the BBC he has been in exile for 33 years because of his opposition to the Gaddafi regime.
Foreign Secretary William Hague said the NTC had shown its commitment to a "more open and democratic Libya... in stark contrast to Gaddafi whose brutality against the Libyan people [had] stripped him of all legitimacy".
The green flag of the Gaddafi government was still flying outside the Libyan embassy in Knightsbridge on Wednesday afternoon as protesters carrying the red, green and black flag of the rebels gathered outside.
Folllowing the US decision to recognise the NTC two weeks ago, the BBC has been told the US has now received an "official request" from the rebels to reopen the Libyan embassy in Washington.
US officials say they are "reviewing" the request.
Libyan rebels and pro-Gaddafi forces are still locked in battle, five months after an uprising began against Muammar Gaddafi's 42-year rule, as NATO continues to enforce a UN-backed no-fly zone over the country.
Kaim said the British decision was "unprecedented in diplomatic history".
"It's illegal, it's irresponsible and for us it was a surprise to happen from the British Government because, I mean, if other countries will [follow] Britain, then the international diplomacy will be chaos," he told reporters in Tripoli.
It's difficult to tell how damaging the expulsion of the Libyan diplomats in London will be to the authorities here in Tripoli.
It is in the nature of the government here not to admit to feeling any pressure.
Officials insist that Colonel Gadaffi and his supporters will win the conflict with NATO and the rebels - no matter how long the war may last.
"I personally consider it a stain on the forehead of Britain," he said. Libya, he added, would try to reverse the decision by taking legal action in both British courts and the International Court of Justice.
The deputy foreign minister also condemned Britain and France for supporting the rebels in eastern Libya. They were, he said, "flogging a dead horse".
Col Gaddafi's charge d'affaires, Khaled Benshaban, is being given three days to leave while his colleagues' fate will be decided on a "case-by-case basis", the Foreign Office said.
Hague told reporters in London that a recent meeting of the Libya contact group in Istanbul had decided to treat the NTC as the legitimate government authority in Libya - and he was outlining the UK's response.
The UK previously said it recognised "countries not governments" but the British foreign secretary argued it was a "unique situation" and recognising the NTC could help "legally in the unfreezing of some assets".
The British government will now be able to unfreeze £91m ($149m, 102m euros) in assets from a Libyan oil company but not other Libyan assets frozen in the UK, which total about £12bn.