Look at it from the deer's perspective... in a normal environment deer have no predators from the sky. So normally they aren't threatened from a small aircraft (not saying never). This is even more the case when they are used to human sounds (motors, cars etc). If the deer is already skittish then the drone "might" be more spooky to the animal.
During the spring I'm not sure I would be so quick to address a deer with a broomstick. She may have little ones she is protective of and a deer can do more damage than you'd imagine with their hooves. Check out the video below... poor dog LOL