I had a similar incident with the poster above..
It's hard to argue with authorities and people who think they have authority. I still It is best to move even if you are in the right as for now as drones have negative connotations in every community. Any one who doesn't own a drone seems to have prejudice and think they are all evil until they own one.
I was at Sculptures by the Sea last week, where artworks were displayed at on a public beach and on coastal legdes. I decided to arrive before sunrise and get a few happy snaps of an artwork that was in accessible and far from people. After 5 minutes of aerial, the staff/guard told me land. It's 530am and there isn't a council person at work, there was no signs, no people to endanger and its a public beach. I could of argued but I decided not fly as I don't want Australia to become like Sweden where its illegal to use a selfie stick because it's technically a form of surveillance.
After getting some grounded snaps, another 3 phantoms whizzed. I saw the guard trying to track down the users. You could only spot users by the white controllers, or wait till they land.
Moral here is to take advantage of the light bridge system and camo your controller.
Another tip is to look like you have authority. Wear a glow safety vest. For some silly reason, it looks like you are serious. I happen to forgot that I hade one the car.