Are you afraid to hold him because you're afraid to hurt him? Or are you afraid because you don't understand his movements?
I was nervous because of the second, I'd read and watched videos on how to hold her, but I was still learning what various body language meant. To counteract this, I sat in the bathtub with her (without water, lol) and let her crawl on me at her own comfort level. This was a controlled environment, as she can't escape the tub and run off like on a couch. With this I learned the signals for when she was going to jump, climb, turn around, or just sit.
But like everyone else said, make sure he's supported when you do hold him and you should be fine.