The historic problem in the DC▸ supply area (hinted at in the earlier mention of Woodhead locos) is that the supply rectifiers were (and I think generally still are) usually one way devices. They could not pass energy back to their supply side. Effectively DC trains regenerate into the third rail only, but because it is a wide area operating with supplies in parallel the energy can be dissipated into other trains that happen to be motoring. AIUI▸ DC testing has usually taken place in outlying areas, because the problem comes when there are no 'motoring' trains to absorb the power. If the third rail voltage rises too high the energy must then be diverted into onboard resistors.
There are few other issues with regen on the third rail system, the rail voltage must not exceed 1000V Within the London area the rectifier voltage was set to 650V and regen braking not permitted this was due to the interfaces with
LUL▸ now that all of their older stock has gone (C and D stock)
NR» are increasing the rectifiers voltage to 750V.
Another problem is balancing the regen braking against the rectifiers, NR rectifiers are at a set output, that is there is no voltage regulation so if the rectifier offers a lower source impedance than a trains regen the train will switch its regen to on board resistors.
The best solution to DC rail system use of regen braking would be on board storage (eg batteries)
The AC system is always supplied by transformers, and these are inherently two way devices, and regeneration is straightforward as long as the protection devices can be altered to allow for the two way power flow.
The "Distance" protection relays need to have the phase angle set correct to around 70 deg. Generally the AC regen breaking works better than DC because there is a bigger receptive load ie the National Grid.