Buses at the station ... combined with rail timetable

[whole diary]

[grahame]

Question asked of me ... how does the Melksham Rail User Group's web site work to produce a combined live bus and train feed?



Well - we start off by identifying the bus stops and station we're interested in. The National Public Transport Access Nodes database - read about it at http://naptan.app.dft.gov.uk/DataRequest/help and download the data from http://naptan.app.dft.gov.uk/DataRequest/Naptan.ashx?format=csv tells you about just about every bus stop and then you hunt through to find the ones that you want (a needle in a whole line of haystacks!) and get the 7 letter code for each stop wanted. Here are the codes for the two stops at the Railway Inn in Westbury, at the bottom of Station Approach.

"4600WIA10362","wilagtwa","","","Railway Inn","en","","","","","Station Road","en","","","NW-bound","en","NW","E0056892","Westbury","","","","","","","0","U",386481,151896,-2.1951501123,51.2660577345,"BCT","MKD","OTH","","","","108","2020-04-07T12:46:51","2020-02-25T08:31:54",0,"new","act"

"4600WIA10363","wilagtwp","","","Railway Inn","en","","","","","Station Road","en","","","SE-bound","en","SE","E0056892","Westbury","","","","","","","0","U",386524,151872,-2.1945328891,51.2658429590,"BCT","MKD","OTH","","","","108","2020-04-07T12:46:51","2020-02-25T08:31:54",0,"new","act"

You can put those codes into sites such as nextbuses to give you a web page of departures:

https://www.nextbuses.mobi/WebView/BusStopSearch/BusStopSearchResults/wilagtwa

https://www.nextbuses.mobi/WebView/BusStopSearch/BusStopSearchResults/wilagtwp



Then go to "Tiger" - http://iris2.rail.co.uk/tiger/ - and find the station you want; grab the page in the format you want (or in XML if you're going to program it into a web page) - you'll get a URL such as http://iris2.rail.co.uk/tiger/303244.xml or http://iris2.rail.co.uk/tiger/rendercissod.asp?file=303244.xml .



If Tiger isn't available at your station, Real Time Trains and Network Rail's Darwin also offer feeds.

As a simple solution, you could offer links to each of these feeds you identify on your site.   For an integrated solution, you'll need to do a bit of programming - the Melksham Rail User Group one is in PHP, but you could use other languages.  If you ask me "does it take a long time to do" the answer is "no - it probably only took me a day", but if you ask "and how long does it take to learn what to do" ... that can take years!

A word of warning - the results on this Maundy Thursday, with lots of late changes to buses because of the pandemic crisis, are scattered with errors ... the bus companies have much else to do to plan the services they're running and you may see some suspended services shown, and some of their thinner replacements missing.

[davidfletcher]

Thank you for explaining this. It is very useful.
I have found and set up some bus records for my area in Cheshire and am now experimenting with display on our web pages.
I see that in your display you have stripped out the graphics and just showing text. Is there a simple way to do this?
I started with iframes in wordpress. Any thoughts appreciated.

[grahame]

Thank you for explaining this. It is very useful.
I have found and set up some bus records for my area in Cheshire and am now experimenting with display on our web pages.
I see that in your display you have stripped out the graphics and just showing text. Is there a simple way to do this?
I started with iframes in wordpress. Any thoughts appreciated.

Welcome to the forum, David.   Simple if you know how and have done similar before, nightmare if you don't.

I used PHP ... my own function called "grabstop" to download from the nextbuses site to a local file (but only if I haven't already grabbed a local copy within the last three minutes) then "scrape" data from the pages.  I have hardcoded the bus stops I want by NaPTAN code, but grab the stop name from the page, as well as the times.  I make up an array of bus calls ($leaving) which later in the program I can add train and National Express times to from other sources, then sort them by time before reformatting.

Code:

function grabstop($which) {
	$lag = time() - filemtime($which);
	if ($lag > 179) {
		$slurp = unlink($which);
		$perform = "wget --no-use-server-timestamps https://www.nextbuses.mobi/WebView/BusStopSearch/BusStopSearchResults/".$which ;
		$action = `$perform`;
		$lag = 0;
		}
	$slurp = file_get_contents($which);
	return array($slurp,$lag);
}

foreach (array("wilamajm","wilamagt","wildtdmp","wildptwm") as $current) {
	$slurp1 = grabstop($current);
	preg_match('!<body.*?>(.*)</body>!sm',$slurp1[0],$gotten1);
	preg_match_all('!<h2>Departures for .*?\[(\w+)\]</h2>!sm',$slurp1[0],$gotten);
	for ($k=0; $k<count($gotten[0]); $k++) { $stopper = $gotten[1][$k]; }
	preg_match_all('!<tr>\s*<td class="Number">(.*?)</td>\s*</tr>!sm',$slurp1[0],$gotten);
	for ($k=0; $k<count($gotten[0]); $k++) {
		$recs = explode("\n",$gotten[1][$k]) ;
		$px = explode(" at",strip_tags($recs[5]));
		$rz .= "%" . strip_tags($recs[1]) . "<br>" . $px[0] . "<br>" . $px[1] . "<br>";
		array_push($leaving,array(trim($px[1]),trim($stopper),trim(strip_tags($recs[1])),trim($px[0])));
	}
}

A few a couple of forum members such as Tim and Tom (if he still reads - it's been a while and he's move on to greater things) will "get" this ... as will PHP web site geeks; others will say oh sh**.

Co-incidentally with you posting, I was looking up some bus API stuff - it's come forward in the time since I wrote this and there is now the Open Data requirement in the 2018 (was it?) Transport Act.  Life has been very, very busy for the last month but looks to be settling down a bit and I may have a go with the bus data via https://www.travelinedata.org.uk/traveline-open-data/nextbuses-api/ and also with the Network Rail (Darwin) train stuff. "Month of Sundays" I have said ... but it looks like May will be a month of Sunday bus services here!

[TonyK]

  Oh sh**

Being serious, this could be extremely useful in a lot of places, but rather needs a bit of certainty around arrival times for both modes, so would be tricky in, say, Bristol.

[davidfletcher]

thanks for the update
time to ponder!!

[davidfletcher]

PHP not my game. I found a bit of wordpress app to insert the code, but nothing is happening
What am I missing? Any clues appreciated

[grahame]

Don't be mislead by how quickly I (or somewhat who's been at it for years) can throw together a PHP page shell. It takes a lot longer the first time, and you'll be reading up / asking lots of questions, pulling your hair out with frustration during that time.

The first time to make and bake a Victoria sponge, perhaps it would take you 90 minutes; by the time it becomes a regular routine and you know what you're doing, you may have reduced that to 45 minutes.   That's a 2:1 ratio, over perhaps a couple of dozen cakes.

I can throw together a web page with PHP variable content on the MRUG^» (Melksham Rail User Group - site) or Coffee Shop websites in as short a time as 45 minutes (clearly, prepared ingredients needed as would be the case with the sponge), but the very first time I did it would have taken me all day and longer - not a 2:1 ratio but a 12:1 ratio. 

So if you're just doing one PHP page set, then, advise would be to find someone who knows ... to help set up the framework of it.   Not always easy, but by way of example I've set up a number of such initial applications / pages through my career and still look after a handful of clients.  You will appreciate I can't say "pop in and I help with the first one" at the moment though.

There is a case for much more general code that picks up rail and road data (from the APIs described above) and produces a composite feed.  Once you start generalising code and making it user friendly, easy to use and secure in its uses, you add an order of magnitude to the time taken to write, test and document ... then support ... what you have.  Given a month of Sundays (and if we're still locked down in June, I may in effect have that!) ...

[davidfletcher]

I got in touch with the API people and started to look at that. Unfortunatley the guides are no doubt correct, but totally useless. So I am no further forward. Very frustrating.

[grahame]

I got in touch with the API people and started to look at that. Unfortunatley the guides are no doubt correct, but totally useless. So I am no further forward. Very frustrating.

How come "useless" ... was it like trying to learn Turkish starting from an English - Turkish technical dictionary?  There are  people around who know a bit of Turkish and could work with you on the specific technical bits of the API, I'm sure. Alternative is for you to learn Turkish.

Oh - for "Turkish" read "PHP" ;-)

[TonyK]

How come "useless" ... was it like trying to learn Turkish starting from an English - Turkish technical dictionary?  There are  people around who know a bit of Turkish and could work with you on the specific technical bits of the API, I'm sure. Alternative is for you to learn Turkish.

Onu çok severim!

Oh - for "Turkish" read "PHP" ;-)

Unless you mean the Gong album about flying teapots, I am utterly lost.

[davidfletcher]

There you have the first problem. It doesnt tell me what the language is. If I knew it was php I could make a start.
And then it doesn't say what to do with the code examples it provides.
POSTing for me means printing it out, and putting it in a red box, perhaps a stamp?
It makes a broad assumption that I know what I am doing. At my age, new skills have to pay back quickly.
No time to buy green bananas!

[grahame]

There you have the first problem. It doesnt tell me what the language is. If I knew it was php I could make a start.
And then it doesn't say what to do with the code examples it provides.
POSTing for me means printing it out, and putting it in a red box, perhaps a stamp?
It makes a broad assumption that I know what I am doing. At my age, new skills have to pay back quickly.
No time to buy green bananas!

Browsers can ask for data from a server in a handful of ways and the reference to POST is one of those. I fear that if I direct you to https://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html I'll have answered one question but raised a couple of dozen more.  It's either a new skill for you - and so much to that skill that you'll be set for a new career, or finding someone you can tap into who's already jumped the learning hurdle.

Part of me is so, so tempted to say "send the stuff to me and I'll take a look" as I suspect I could sort it out.  I am, however, painfully award that a project like this divides into three sections - development, code maintenance and data maintenance, and that the time put into development is followed - if something is successful - by a bigger wedge of time in lifetime maintenance, and by an even bigger wedge looking after the data - and I know if I jumped in, I would find it hard to commit to the long term code maintenance even with you or others taking up the data issues.

Were I 30 years younger, and looking to make a profession out of this, I suspect I could earn a good living. As it is, much these days is adopt and adapt.  Should an open bus data feed get built into any of the stuff we're doing in Wiltshire, happy to publish the code used but it would be on a "not guaranteed / up to you to check it's tight for you" basis; only redactions would be security keys ("put your API password / ID here") type stuff.