UPDATE: I did another litchi waypoint mission and this time I didn't manually mess with the gimbal at all, I just let it fly and come back. It seems like the root issue is that when it should be looking down at the POI (which is set a few feet off the ground), it's actually looking straight out...