To clarify, the ‘Romans’ didn’t leave Britain.
With Rome fighting civil wars and then invasions from the Visigoths (including the siege of Rome itself), troops were removed from Britain and never really returned. It was not an over night process, but a series of events over a course of a number of Emperors.
All Romans had been recalled to Rome and the Emperor Honorious told the people of Britain that they no longer had a connection to Rome and that they should defend themselves.