I don't think that a wifi crowded area would affect the RC, which is operating at 5.8GHz. But it would be an option to try and go to a less-urban area.
Also, in which position are you using this antenna on your controller? These Spironet antennas do not transmit very well to the top of it (which is this region with larger area). Instead, you should try and use it in 90° with the corner facing to the phantom.
You don't explain where you got signal loss.
It's complicated and affected by a number of factors, particularly the geometry of the angles between your antenna and the Phantom's.
It varies but with standard equipment you can get 1500 - 3200 feet.
Flying straight up is always the trickiest angle for good signal.