There's really no need for train crew changes mid-route on a line that is only 3 hours long – if they were done at Paddington and Swansea instead, no-one would even notice them. Sounds very much like the crew booking on place is in the wrong location.
That is fine as a principle but you need to ensure all Swansea crews end their shift in Swansea and Paddington crews end in London. If you can't then splitting a journey between two crews may be a way of achieving it.
Or alternatively, end up with crews from Swansea working trains to and from Cheltenham/Gloucester/Hereford and being taxied to/from Swansea for their shifts.