I don't think there is any right time to use graduated colour filters. There is nothing they do which you can't achieve in post and you have little chance of undoing any effect they introduce later.
ND's are very useful to maintain your shutter speed at or close to twice the frame rate for video. Obviously the darker ones are used in brighter conditions. They are of little use in still shooting unless you are aiming for slower shutter speeds to introduce some creative blur (seascapes, waterfalls etc).
Polarising filters are good for arjening skies, introducing saturation and reducing reflections. Most of this effect can be realised in post (with the exception of reflection reduction) and you will often find skies vary greatly in overall brightness due to the wide FOV of the lens (can look very off and exceptionally difficult to correct).
UV is a good general purpose leave on to protect the lens when not using other filters.
Hi together. I am new here but my profession is dop in German TV. About ND Filters I can say is useful on brighter days. Hard to say which on which light situation, because on normal cameras is it easier to handle with ND Filters. You are on the ground an normally you have your camera in your hands. For shooting with your phantom I recommend better to have a higher ND number as to less. There is always some picture information on dark areas in the picture so you can fix it in the postproduction. If you are not really sure, use a higher one than really needed.
On really sunny bright days I fly with ND16. On normal days with ND8. I have a ND4 too but I think I don't really need it.
I fly with a PLND8 most days. Swop to PLND4 on cloudy / dull days and go up to PLND16 on bright sun. Not sure about the polariser bit, can give different results when you turn. QUICK TIP FOR REMOVING 'stock' filter - use dry washing up gloves! Tried everything, no joy, then immediately worked (after asking wife's permission...)