In my experience the problem is in part due to boarding and disembarking times at Bath station at busy times. Never seems quite so bad at Temple Meads. As to why....?
Purely boarding time in the stats, I suspect ... how long it takes to disembark, get to the exit area, queue to buy a ticket if no other opportunity has been offered ... probably doesn't form part of the stats, even though that extra time is a necessary part of a journey.
Bath is a through station in every sense, where most of the trains arrive as short a time as possible before their due to leave for the benefit of through passengers. But that's not really the case at Temple Meads, where many trains reverse, terminate or have a significant pause for operational reasons.
Bath has 6.2 million journeys per annum - 3.1 million per platform; Bristol Temple Meads has 10.1 million - that's between 0.77 and 1.26 million per platform, depending on whether you count the long platforms with 2 numbers as 1 or 2 platforms.