Same happened to me but not as bad.
I was out flying with FPV headset. Drone at 150 ft in a park (just a small dot in the sky no sound audible). Perfectly legal. Dogs barking, motor bikes roaring. Got a tap on the shoulder by a young woman who said her piece of mind was disturbed by the drone...