Ever since 9/11, rail fans have been harrassed by various authorities for taking photographs of trains, including threats of imprisonment and confiscation of camera and film. Here are two short exerpts from a column on the subject in the August 2013 issue of Trains Magazine that presents some parallels with the harrassment difficulties, both real and anticipated, by UAV pilots:
"In the months after the terrorist attacks of Sept. 11, 2001, a lot of panicky American institutions invoked new policies that did nothing more than attack the rights of Americans. They accomplished absolutely nothing, and bit by bit, cooler heads have mostly pulled back from this precipice. What these panicky and fearful people did post-9/11 was at least partly understandable, if dumb. In the railfan community, these “policies” mostly involved banning photography of trains and railroad facilities. These policies were clearly un-Constitutional and have repeatedly been knocked down in court."
One of the antagonists (that I've kept nameless here) "seems to think of itself as the victim of 9/11, but that is nonsense. The victim was the United States of America. Our USA has something called a Constitution that guarantees our rights, and the Supreme Court has repeatedly ruled that those rights include the right to take pictures in public places and on public property – pictures of ANYTHING. The greatest thing the USA and its institutions can do to honor the victims of 9/11 is to jealously guard our rights."
The upshot of several opinion columns in Trains Magazine is just as has already been stated by several people in this thread: as long as you are in a public place, don't do anything dumb, be calm, be respectful, don't argue, ask politely if you are being detained, and ask if you are free to leave.
You may find it very instructive to read the safety code of the Academy of Model Aeronautics (AMA) at this website:
https://www.modelaircraft.org/files/105.pdf and then decide if you wish to join AMA and be covered by its $2,500,000 personal liability insurance coverage, $25,000 Medical Coverage-AD&D Policy ($10,000 Death Benefit), $1,000 Fire, Theft, and Vandalism coverage for $58 per year ($48 if you are over 65, junior members free), and right now AMA is offering a 25% discount on new memberships. Sounds like a good deal to me!
http://www.modelaircraft.org/