Maybe something happened that scared him while you were in contact or maybe you had something on your hands he didn't like, idk.
The only other advice I can give is don't rush it. Keep trying to touch, handle etc. and even hand feed. Then hopefully he will learn to trust you.