As an amateur, you are strictly limited as to where you can fly. No flying over towns and villages, within 50M of any vehicle, boat or person etc. As a commercial drone operator, you will probably need to fly closer to things and people, so it is reasonable that you have a higher level of training, as well as full insurance etc.
I believe the current issue over 5.8GHz is that it is not currently recognised as an airborne R/C system and there are some concerns over its suitability, due to limited and unpredictable range in some atmospheric conditions. Many UK owners are switching their TXs up from the CE setting to get usable range, so this kind of proves the point.
The CAA treats all airborne vehicles with a recording camera the same, whether they weigh 0.5Kg or 100Kg. The rules are there to protect privacy as well as to avoid accidents and incidents.
For my book, I researched Youtube videos and came to the conclusion that around 80% of UK clips showed illegal flying - mainly close to houses and over towns. This really brings home that there needs to be more education and more information when people buy these machines.
I am sure there is a suspicion that commercial operators are protecting their interests by keeping out the amateurs through high costs, but this is not the case. It's the same in lots of industries - I work for a large events company and we compete with lots of one man bands. We have full truck operating licences, carry out daily vehicle checks, only work our drivers within the rules etc. but find that many one man bands ignore these legal requirements and are therefore are cheaper... Right up until they get caught and prosecuted!