drifting back

I mainly fly in GPS mode but last night decided to give ATTI mode a try as i like the sound of filming whilst drifting there was no wind a very calm evening, i let the phantom hover in a stationary position then flipped the switch to ATTI mode and it began moving backwards on its own .Now I know...