That's my understanding as well, 400 feet reduces the risk of conflict with manned aircraft, manned aircraft have to comply with 14 CFR Part 91.119 which states that they can't fly below 500 feet in non-congested areas, and 1,000 feet in congested areas. Of course, near airports they would fly...