I have a very similar looking setup (such substrate, the plants, the branch, a climbable background - yes!), and also my bearded dragon Taco brumated in a cave he dug himself. I intentionally have no huts that can be lifted up, but deep substrate and hiding spots just like in your dragon's enclosure enabling the dragon to dig a burrow just in nature and also preventing me from being tempted to simply "take a look how he does".
I did not dig up/ wake up my bearded dragon. What I did was on warmer and brighter days when he came out a bit (that's a normal, natural behavior during brumation - usually just he head or his body up to the front feet, but not more, was sticking out) I dropped some water on him.
I was able to see somewhat in his cave that he is alive, slowly breathing (normal) and not getting thinner, so I let him be. I would also guess being dug up during brumation would be kind of traumatic - like being dug up by a predator, even if he then recognizes it's just you.
Btw.: Love your setup