point of view

Hi All i used the point o view for the first time properly recently and i find that the subject seems to move out of the middle as it pans around. Any ideas why? it wasnt even windy as thought that may be the case. I was able to cut out the best bits to make my video but could not get a full...