Most the time I try to fly away from people and homes in general, minimizing risk when possible. I definitely don't hover around homes, I keep it moving when homes are nearby. People think the worst when a drone hovers, but if you keep moving, emotionally they think of it no different than a model plane flying overhead. But when you hover..... oh boy, their minds start going berserk.
One of the secrets to no hassle flying is to be inconspicuous while piloting. If the public can't see the pilot, they can't call a Leo with specifics. Police depts typically won't respond to a call without specifics telling them where to go. So I'll take my backpack on my back, like I'm hiking somewhere, this raises no eyebrows. Then I try to find a place to fly from that isn't so visible to the public. I learned this from a policeman that cited me for trespassing, saying I was reported by someone that saw me flying. Fortunately the $300 citation was thrown out due to the area not being adequately posted, "no trespassing, closed area". I wasn't sited for flying, only trespassing (Crystal Cove, Laguna Beach).
My rule of thumb is, the less people there are that can SEE ME, the better chances of the public not having any luck with Leos responding. The public might see the drone, and they may be annoyed, but if they don't see me they will assume there is nothing that can be done about it.
Flying from inside my car is a common tactic in new areas, with air conditioning running on hot days (needed with iPad Mini2). Most people won't be looking for a drone pilot inside a car. Surprisingly the range isn't affected much from inside the car. I try to land swiftly and leave swiftly sometimes, if I see people on the phone looking at me when I land. Yes, I do look all around before I come in to land. Sometimes I feel like such a criminal, intentionally evading the law.
