One of the striking things about this question is how the arguments have remained largely the same for over 100 years, despite the changes in technology and other areas.
There are only two systems in large-scale use here, but historically there were three choices to be made -
DC▸ or AC, and what frequency?
low volts (750 up to a few kV) or high volts (25 kV outside Germany)?
overhead or third rail?
If you choose DC it used to be hard to change voltage. Now every train does it, though we can still only handle a few kV. I think converters (DC-DC and DC-AC) still cost more than transformers at this power level (around a MW) - at low powers that's not true, and all consumer electrics/electronics is stuffed full of them, even (LED) light bulbs.
We in Britain are unique in persisting with top-contact third rail on a large scale outside metros. Many places tried it and gave up around 100 years ago - often changing to side- or bottom-contact to avoid the snow and ice, or overhead then or later. Third rail is inevitably hard to insulate, so only feasible at low voltages (apart from the safety issues).
There is an issue with what kind of electricity motors work on, partly fundamental but changing with technical evolution. When the early decisions were made it was common to use wound-field DC motors for both - they don't care which way the current flows, but object if it changes over too often. Hence Germany chose 16.67 Hz AC as the highest feasible at the time. Now we can convert power to AC or DC at variable volts or frequency to suit the motor.
Low-voltage (DC) OHLE is rare in Britain, other than in trams. It is surprisingly common elsewhere - e.g. 3 kV in Belgium and many others, 1.5 kV in the Netherlands and half of France. That's for main lines, so the current per train is at least as high as for suburban 750V. So the problems of feeding many kAmps of DC to long-distance tracks can be overcome.
In these countries newer lines are often 25 kV, and high-speed lines all are. There are moves to change over DC OHLE to 25 kV AC, though not very fast despite the fact that a high proportion of motive power is dual-standard or more. The current standard design (transform and convert AC to around 1 kV DC, and then convert to suit the motors) makes multi-standard much easier to do than in the past.
If you want a thorough analysis of the costs, there is an
RSSB▸ research report on the implications of changing over (as is planned for Southampton to Basingstoke):
http://www.rssb.co.uk/RESEARCH/Lists/DispForm_Custom.aspx?ID=965This page summarises a recent meeting, giving an idea of the views current in the industry (but I'm dubious of its take on the history):
http://www.therailengineer.com/2013/04/10/has-third-rail-had-its-day/And finally, all of these arguments (plus the merits of three-phase versus single phase AC and many others) are dealt with in "Electrical Engineering" by Harold H Simmons* (Cassel, 1908 - but my Grandfather's copy is dated 1909 and was bought in 1914). So they are not new! And note that even then the use of low voltage DC third rail on inter-city lines was dismissed as uneconomic.
The real puzzle in some ways is that 1.5 or 3 kV DC overhead was and is still so widely used over long distances, despite the cost of frequent feeder stations and the big wires that have to run alongside the track between them.
*This is available online at:
https://archive.org/details/outlineselectri00simmgoog though the digitisation failed to capture many of the pictures. Section XI, chapter IV "Railways" is on p 815, which is p 860 in Google's online version.