4 year old wetting pants

Just wondering if anyone else has been in this situation or similar. I will be getting professional advice (waiting on an appointment).

I have a 4 year old son. His dad is in and out of his life. He doesn't go for regular visits nor sleeps at his dad's (his dad's choice not mine). My son is a sensitive little boy who even sometimes cries when I drop him off at daycare. In the past when he has gone and stayed at dad's (very very rare occasions) he would cry really badly to the point he couldn't breathe properly.
My son is toilet trained (but took longer than normal). He has the odd little dribble on his pants on occasion but it has settled down. When his dad doesn't see him for a while , (last time was 3 weeks in between seeing him with no phone calls etc and the times before that could be a couple of months or so) , he doesn't have any wet pants and is so good with going to the toilet before he is too busting. But when his dad pops in to visit him for some reason my son resorts back to wetting his pants and can sometimes have about 4-5 wet pants a day afterwards for a week or two. And when he is at daycare when this happens he won't change his pants he will just run around with them on until they dry and it's obvious when I pick him up due to the smell.
So my concern is why is he doing this. Could it be related to his dad being in and out of his life? What do I do about it (apart from get professional advice). I can't stop his dad from seeing him and I would never do that. But when his dad isn't around my son goes back to normal after a week or two. It's not just one or two times this has happen , it has happened on several occasions and too me it's too much of a coincidence for it not to be the problem.