The only advance fares in the LTV▸ area are on PAD» <>OXF» standard class only.
At the weekend/bank holiday splitting at Oxford and combining with Weekend First can make the fare a bit more agreeable to/from Worcester or Hereford.
As an example, travelling this Saturday on the 1213 HST▸ from Hereford:
HFD» -OXF Off Peak Day Single ^19.80
OXF-PAD Standard Advance ^4.00
Weekend First upgrade ^15.00
Total: ^38.80.
That's only a fiver more than the through Standard fare and half the price of the through First Class fare.
Unless you absolutely must travel first class, quite frankly, you might just as well go on one of the weekday HSTs out to Hereford. There's always lots of room on them in standard west of Oxford, even unoccupied table seats. Those ^4 fares can be found from mid-morning on weekdays and it only tends to go up to ^5 for afternoon peak services heading into London from Oxford. You can also get ^6.50 advance singles to Oxford from Paddington on the 17.22, 17.50 and 18.22 if you're booking far enough ahead, a heck of a saving on the ^17.15 anytime single.
And if you have one, using a Network Railcard for Oxford to Worcester Foregate Street and buying a Wof-Hereford ticket would trim another ^1.95 off the example above.
^7.80 Wof-Hfd
^10.05 Wof-oxf using railcard
^4 oxf-pad
^15 weekend first
=^36.85
And if you don't mind doing some of the run on Arriva,
FGW▸ will sell you an ^11.50 advance single for some weekday off-peak journeys between London and Hereford via Newport.