Quite OK to handle him, but keep it minimal until he has settled in and starts to show he trusts you.
Maybe make a very predictable routine of it and make it a bonding ritual . This is what I've been doing for all my skinks and beardies.
Essentially my new lizard/s are nearly always babies (skinks produce live babies) or hatchlings (dragons lay eggs and after incubation hatchlings emerge).
My approach is :
First thing each day (before I have my brekky) I feed the hatchlings/babies.
I handfeed them their first 2 or 3 insects as a bonding exercise, and they get the rest in their rearing tub/s a few at a time (so as not to overwhelm them, I keep offering live insects until the hatchling/baby looses interest ==> cue = I'm now full , no more insects please.
The bonding ritual involves
>>either waking the baby/hatchling up if it's still asleep in it's hide (gently by lifting the hide off from over them or picking them up from their basking spot if they are already awake and basking (my lights are on a timer and come on 6am , brekky is about 9-10am)
>> I pet them for a little while before slipping my fingers under their body and lifting them out to deposit them on my chest/tummy facing my face,
>> more petting while I my left wrist resting on the skinny part of their tail - this discourages them dashing off if they see sometime they must check out immediately ,
>> I then offer a insect who's head has been squashed (makes it tasty and juicey) and repeat this for 2 or 3 insects, with petting inbetween.
The advantage is they soon learn to associate being picked up and handled with getting something they enjoy, FOOD and they soon get the hang of handfeeding , be prepared for a few accidential finger tip nips in the training period. This builds trust and soon your beardie will come to you in anticipation when it see you (and the insects).