For all of the normal map calculations, I had assumed that the light was all coming from one direction. Obviously this was deficient as obviously there were times when a surface is surrounded by light. I fixed this by adding a calculation into the baking that determines how much light is coming from the dominate direction and how much is ambient. This ratio is stored, and used in the normal calculations as a scalar, so if a plane is evenly lit up from all directions, normal maps should essentially disappear.

I also made it a rendering option whether or not to account for normal maps

I also had a duh moment today. I've been thinking about how to handle the fact that I want to be able to open and close doors in playing mode, but not have that affect the save file. I was playing with having separate values but then synchronizing them was a bitch and I would then also have to preprocessor some of it out when not in the Editor build. Then I realized that everything I wanted could be accomplished by pushing a save json onto the undo stack, playing the level as needed, then use popping back the top undo point. I already have an undo system that works exactly this way. Duh, I can use that with only two lines of code to switch between editing and play mode and have all the changes in play mode revert instantly.