Search facility, document and post archives on the Coffee Shop forum

ellendune

Transport Scholar
Hero Member

Posts: 4512

Search facility, document and post archives on the Coffee Shop forum

« on: October 07, 2021, 08:04:03 »

I have used this to find the Lorry Driver Shortage Thread. I found the name of the thread. It tells me this.

Quote

+ 30 Sep 2021 Thread - Lorry driver crisis made worse by Covid - Shapps 9

e StokeonTrentLive:. Tesco beats HGV driver shortage by importing fresh produce from Spain b

Unfortunately I still have no idea which board it is on. How can I find out?


	Logged

grahame

Administrator
Hero Member

Posts: 43324

Re: Search Function

« Reply #1 on: October 07, 2021, 08:11:41 »

Quote from: ellendune on October 07, 2021, 08:04:03

I have used this to find the Lorry Driver Shortage Thread. I found the name of the thread. It tells me this.

Quote

+ 30 Sep 2021 Thread - Lorry driver crisis made worse by Covid - Shapps 9

e StokeonTrentLive:. Tesco beats HGV driver shortage by importing fresh produce from Spain b

Unfortunately I still have no idea which board it is on. How can I find out?

Click on the "+" to the left of the result you have quoted above and it will take you to the thread. If you look in the top bar above the thread when you're there, you'll see which board it is on.


	Logged

Coffee Shop Admin, Chair of Melksham Rail User Group, TravelWatch SouthWest Board Member

ellendune

Transport Scholar
Hero Member

Posts: 4512

Re: Search Function

« Reply #2 on: October 07, 2021, 22:05:30 »

Thank you


	Logged

grahame

Administrator
Hero Member

Posts: 43324

Research corner - finding archived documents and historic forum content

« Reply #3 on: January 13, 2022, 10:19:05 »

( Earlier discussions at http://www.passenger.chat/24635 )

There are now 2,000 documents in the Coffee Shop's archive collection, of which 900 can be found through a default search using the forum's deep search facility at http://www.passenger.chat/research.html and also accessible from the box near the top of forum pages. Location shown in greeh here:

Default searches will also find matches within public forum posts which are less that three years old (there are some 50,000 posts that are searched) and stations who's names match (from 2,500 National Rail stations)

* A further 900 of the archived documents can be found if you select to report on "All resources (including archive)"; the final 200 documents are stored in various conditions awaiting attention.

* If you wish to look back to earlier forum threads, there's a option on the search page ( http://www.passenger.chat/research.html ) that lets you look into all public posts since the forum started in 2007

There are copyright issues with archived documents, and rather than make the content of them public ("publish them") ourselves, most of them are accessible to members only . If you are a guest, please remember that registration doesn't cost anything and if you have a public / sustainable transport interest with a significant UK^▸ element, you'll be welcome.

Around 100 documents in our archive are "whitelist"ed, and those are available to guests without registration as well as to members. These documents are listed at http://www.passenger.chat/whitelist.html

If you feel that our archive is missing a key document, please let me know - chosing what's in there is somewhat subjective and there will be things I have missed (and probably things included that you will wonder "why?"). All documents are stored as .pdfs - if necessary, other formats such as .doc and .html can be "printed" to .pdf and archived in that way, but with some formats and security features that might not be practical. Spread sheets in particular can be awkward to 'reduce' to printable documents.


	Logged

Coffee Shop Admin, Chair of Melksham Rail User Group, TravelWatch SouthWest Board Member

grahame

Administrator
Hero Member

Posts: 43324

Re: Research corner - finding archived documents and historic forum content

« Reply #4 on: January 31, 2022, 11:54:45 »

An update - Deep Search now defaults to reporting the resource with the most hits first rather than sorting by date. This is what the small print has always told you the default is, but it hasn't actually done that. The issue came to light when I was adding the Bristol to Birmingham report in this morning.

Try entering Bristol*Birmingham into the Deep Search box - you should find the resource that I added this morning high up the list now, with 85 mentions!


	Logged

Coffee Shop Admin, Chair of Melksham Rail User Group, TravelWatch SouthWest Board Member

grahame

Administrator
Hero Member

Posts: 43324

Re: Research corner - finding archived documents and historic forum content

« Reply #5 on: July 10, 2022, 19:13:16 »

We have now reach 1000 indexed documents in our .pdf library, with another 1000 or do still to be indexed.

1 document dated 1845
1 dated 1855
1 1874
1 1907
1 1928
1 1931
1 1941
1 1953
1 1955
2 1957
2 1960
4 1963
2 1964
3 1965
1 1979
2 1981
1 1982
1 1983
2 1984
2 1987
1 1988
1 1989
1 1990
1 1991
3 1992
1 1993
2 1996
3 2002
1 2003
4 2004
4 2005
11 2006
4 2007
61 2008
21 2009
19 2010
26 2011
21 2012
57 2013
61 2014
72 2015
91 2016
77 2017
107 2018
81 2019
104 2020
107 2021
27 2022

Indexed documents accesible to members are categoried as follows
324 3 - of local interest
355 5 - General documents
214 7 - Important documents
48 9 - Key documents
with the remaining documents being only accessible to admins, largely for copyright reasons. 110 of the documents included in the counts above are "whitelisted" so that they are available to the public even without an account.


	Logged

Coffee Shop Admin, Chair of Melksham Rail User Group, TravelWatch SouthWest Board Member

grahame

Administrator
Hero Member

Posts: 43324

Document and post archives and searching

« Reply #6 on: October 08, 2023, 10:28:48 »

We now have over 1,900 documents in our archive of which 1,075 searchable indexed documents. Most of these are available to members only, but just under 150 are available in public where we are clear there's public interest and no copyright concerns. We also have 25,000 threads with over 335,000 individual posted contributions - and the vast majority of those ARE public.

Just this morning I have added three documents:

http://www.passenger.chat/NN_at_announcement.pdf - "Network North" as it was announced by the Prime Minister on 3rd October

http://www.passenger.chat/NN_20231008.pdf - "Network North" as it was updated on 4th October

http://www.passenger.chat/hs2program_20231001.pdf - A recent (but pre-changes) description of HS2^▸ and the various phases from "HS2" themselves.

Anyone can access the public (white list) documents via http://www.passenger.chat/whitelist.html ; they are categorised by importance though and it's not always easy to find what you want. Indeed a great deal of work could usefully be done to make the entire archive easier to use.

If you want to search for something, the "deep search" box and button at the top of forum pages is a good place to start, as is http://www.passenger.chat/research.html - if you are a member, log in first and you'll then be able to view the documents you find.

Note that some resources are left out of default searches. If you want to go back to forum threads prior to 2019, the "Dated" option lets you add in a seach right back to our foundation in 2007, and if you want to look in the documents in our archives but not yet indexed, use the "Report on" option and select "All Resources".

Coverage in our archive is patchy as documents need to be added when they crop up, and it's something of a personal project. However, I try to make it useful by adding things as I see them crop up. Many are never accessed at all - but at least we have backup copies should the original resource get lost or changed.

* Please let me know / send me a copy or link of documents you feel are missing - even if I have them, you might be helping me with poor indexing

* Some documents we have are locked .pdf files and that has made it impractical for us to extract the textual content to search


	Logged

Coffee Shop Admin, Chair of Melksham Rail User Group, TravelWatch SouthWest Board Member

infoman

Hero Member

Posts: 1496

Re: Document and post archives and searching

« Reply #7 on: October 08, 2023, 17:54:18 »

can previous copies of the Nathional rail timetable be published/shown/downloaded?


	Logged

grahame

Administrator
Hero Member

Posts: 43324

Re: Document and post archives and searching

« Reply #8 on: October 08, 2023, 19:45:18 »

Quote from: infoman on October 08, 2023, 17:54:18

can previous copies of the Nathional rail timetable be published/shown/downloaded?

We have some

May 2019 - http://www.firstgreatwestern.info/nr_timetables
December 2019 - http://www.firstgreatwestern.info/nrtt20
May 2020 - http://www.firstgreatwestern.info/nrtt20summer
May 2021 - http://www.firstgreatwestern.info/nrtt21summer
December 2021 - http://www.firstgreatwestern.info/nrt_dec21
December 2022 - http://www.firstgreatwestern.info/nrtt1222
May 2023 - http://www.firstgreatwestern.info/nrtt_may23

Edited to add these other places you could try:
https://timetableworld.com
https://www.railtables.co.uk/all-services/
https://britishrailwaybooks.co.uk/wtt/wtt.php
https://discovery.nationalarchives.gov.uk/details/r/C13103
https://www.dpsimulation.org.uk/timetables.html


« Last Edit: October 08, 2023, 20:04:36 by grahame »	Logged

Coffee Shop Admin, Chair of Melksham Rail User Group, TravelWatch SouthWest Board Member

grahame

Administrator
Hero Member

Posts: 43324

Re: Document and post archives and searching

« Reply #9 on: November 19, 2023, 11:13:15 »

An update from my blog - http://grahamellis.uk/blog1052.html - "Melksham needs appropiate "Siftware"" ... looking at the issues managing and finding data in the town. I have include Coffee Shop stats ...

Quote

The Great Western Coffee Shop forum - founded in 2007 - has an incredible amount of data on it, and over the years our team of moderators (data managers) and contributors have helped us build a searchable archive. I'm rather proud of this - open source software with code I have added and the search will help you navigate though:
* 1933 Mirrored documents
* 2535 Stations
* 25340 Forum topics
* 137143 "Did you mean?" words
* 337789 Individual posts
It's not just searching - these resources are indexed by category, date, security level, and geographic area too ... and there are 1562 timetables and 28819 images which come up within other documents or just by geographic and date index. Here is an example ... searching for "Rail Link Bus" if you wanted the history of the shortlived bus service to Melksham Station


	Logged

Coffee Shop Admin, Chair of Melksham Rail User Group, TravelWatch SouthWest Board Member

grahame

Administrator
Hero Member

Posts: 43324

Re: Document and post archives and searching

« Reply #10 on: December 11, 2023, 17:14:42 »

Quote from: grahame on October 08, 2023, 19:45:18

Added
December 2023 - http://www.firstgreatwestern.info/nrtt_202312/


	Logged

Coffee Shop Admin, Chair of Melksham Rail User Group, TravelWatch SouthWest Board Member

Chris from Nailsea

Administrator
Hero Member

Posts: 19254

Re: Document and post archives and searching

« Reply #11 on: February 10, 2025, 15:33:33 »

Quote from: grahame on February 10, 2025, 14:28:14

Quote from: GBM on February 10, 2025, 10:28:53

Quote from: grahame on February 10, 2025, 08:23:36

Quote from: GBM on February 10, 2025, 08:04:19

Had to spend quite a while using the search function to find this topic in the forum.
No results found!
Next search found threads from 2020 and earlier, but not this topic.
Had to go into my posts section and scroll through....... Sad

Please let me know the search terms you used and which search function - the one in the top bar after "Home, Help ..." or Deep Search?

Two emails with attachments sent.

Thank you ... and an answer / explanation - doing it in public since this is of general interest.

Our search tools are - if you like - a very basic form of AI and for "intelligence" have a rather low IQ and a frustrating pedanticism ... they search the bodies of public threads, the extractable text from .pdf documents and station names.

1. In the case of "Cornish delays" the words aren't mentioned as a pair anywhere in the thread - just in the subject line - so you drew a blank. The fact that I have now mentioned them means that this tread and post will be found in the future. clearly a design shortcoming - because at times some of our member don't repeat the words of the subject in their threads - something I have been aware of for a time and I have just applied a patch so it searches both. I hope I have not caused a problem applying the law of unintended consequences.

2. Where a search uses a pseudonym the search is only on the word you put in - so look for "Cornish delays" and our system is not bright enough to look for "Cornwall delays' and will certainly not offer you "Western Peninsular delays". If you put in a single word which does not match, it WILL offer you close spellings - "did you mean". Put in "Sinday" and it will suggest Sunday, even if you were searching for a naughty overnight destination with the girlfriend.

3. If you put a space between word in the search we look for the words consequetively. If you put an "*" between two words it will look for both words in the post / document and in that order, but not necessarily together

Our receptionist server is more powerful than our worker that does all the searching at present, and there is scope for enhancement - the sort of code I enjoy writing. All sorts of consideration for - well - some point in the future

I have added a copy of the above post here, in this general guidance for the use of the Coffee Shop forum's search facility.

Personally, I hate our forum 'search' facility with a passion: that's why I spend time moving and merging existing topics wherever possible - to avoid the need to 'search'.

That is in no way any criticism of grahame: he has done a brilliant job, working to improve upon the software restrictions this forum uses.


	Logged

William Huskisson MP^▸ was the first person to be killed by a train while crossing the tracks, in 1830. Many more have died in the same way since then. Don't take a chance: stop, look, listen.

"Level crossings are safe, unless they are used in an unsafe manner." Discuss.

grahame

Administrator
Hero Member

Posts: 43324

Re: Document and post archives and searching

« Reply #12 on: February 11, 2025, 08:01:48 »

Quote from: Chris from Nailsea on February 10, 2025, 15:33:33

Personally, I hate our forum 'search' facility with a passion: that's why I spend time moving and merging existing topics wherever possible - to avoid the need to 'search'.

That is in no way any criticism of grahame: he has done a brilliant job, working to improve upon the software restrictions this forum uses.

It's a bit like the Melksham Train Service. 15 years ago, the facility supplied by default was broken, and we were left without any useable search. I have worked to provide something and make it extendable useable, but there is a lot more to be done and users need to be aware and work with the limitations at present. There is a lot more that could be done .. recall 3000 journeys a year past ... 75,000 journeys a year pre-covid peak, 400,000 journeys a year that is should be for the size and shape of the town. And a huge "thank you" and no criticism for the people who have helped the rise from 3k to 75k, whilst being reminded there is still work to be done.


	Logged

Coffee Shop Admin, Chair of Melksham Rail User Group, TravelWatch SouthWest Board Member

Do you have something you would like to add to this thread, or would you like to raise a new question at the Coffee Shop? Please [register] (it is free) if you have not done so before, or login (at the top of this page) if you already have an account - we would love to read what you have to say!

You can find out more about how this forum works [here] - that will link you to a copy of the forum agreement that you can read before you join, and tell you very much more about how we operate. We are an independent forum, provided and run by customers of Great Western Railway, for customers of Great Western Railway and we welcome railway professionals as members too, in either a personal or official capacity. Views expressed in posts are not necessarily the views of the operators of the forum.

As well as posting messages onto existing threads, and starting new subjects, members can communicate with each other through personal messages if they wish. And once members have made a certain number of posts, they will automatically be admitted to the "frequent posters club", where subjects not-for-public-domain are discussed; anything from the occasional rant to meetups we may be having ...

Pages: [1]

« previous next »