The Ministry of Justice has admitted 37 prisoners were wrongly released under Labour’s plans to reduce overcrowding in jails.
The individuals were released in error on 10 September because their offences were wrongly logged under outdated legislation.
A Ministry of Justice spokesperson said: “Public safety is our first priority. That is why we took decisive action to fix the broken prison system we inherited and keep the most dangerous offenders locked up. This included blocking the early release of domestic abuse offences such as stalking and controlling behaviour.
“We are working with the police to urgently return a very small number of offenders - who were charged incorrectly and sentenced under repealed legislation - to custody.
“The convictions remain valid with offenders monitored since their release and will soon be back behind bars."
Robert Jenrick posted on Twitter: "Chaos in Downing Street. Chaos on our streets. Labour are already a complete shambles."
Five prisoners out of the 37 have not yet been returned to jail but most have been brought back into custody.
Days after the mass early release of prisoners, it emerged a man freed was alleged to have sexually assaulted a woman on the same day he was released.
It is understood the man was released as part of the policy, which aims to ease prison overcrowding, on its first day.
He is alleged to have reoffended in Sittingbourne in Kent on "the same day released", according to court documents.
He was later arrested at an address in south London.
He appeared at a magistrate’s court on Thursday charged with sexual assault, and is due to appear at the crown court next month. The man has been recalled to prison.
A Ministry of Justice spokesperson said: "The Government inherited prisons in crisis and on the brink of collapse. Had that happened, the courts would have been unable to hold trials and the police to make arrests.
"We had no choice but to introduce emergency measures, releasing some prisoners a few weeks or months early, while excluding a number of offences and imposing strict licence conditions.
"While we cannot comment on the details of any single case, those who break their licence conditions or commit further crimes will be punished."