When your home point is recorded, everything is relative to that location in space, so that is why it is height and not altitude.
I flew down into a huge copper mine in Bisbee, Arizona recently, and it was over 800' down to the small lake at the bottom I hovered over. So, I was over 800' negative height to my home point.
I have some Litchi missions where I have to use negative height, which is scary to me to plan since I'm going by the map's altitude reading. So far, I've made it back safe and sound.