I certainly feel anxious when filming/flying in certain situations. This is largely because of a self-imposed pressure to deliver results to a high standard. It’s a good thing. Today, for example, I had to fly our drone off the edge of a mountainside and drop down a long way to get to the location we needed to film - with high winds causing warnings during all of the flight. It was far from relaxing and I was relieved after it was done (as well as being pleased we’d likely achieved a good result).
Whereas we only fly a drone for work, I’m guessing that if I did it as a hobby there would be a similar feeling of being under pressure - because I’d still be the same person and I value consistency.
We only fly on our own land, whether in South Africa or in the UK. So there’s no pressure or distraction from other people (other than colleagues who know what is happening with the drone flying). Personally, I wouldn’t like being interrupted by other people if I was flying in a public place.
What you seem to be describing is a healthy level of conscientiousness. And, yes, I’d recommend finding a secluded place in natural surroundings. Also, flying at a time of the day when most people are still in bed, such as at dawn in summertime.