Am I Heartless?

Bit of a back story,about 5 years ago my husband started working with this woman, they went to a party and slept together and we ended up separating. After a few weeks we began talking and he suggested we try marriage counselling to see if we could maybe work out some issues and try and get back together (We had a child together) During our separation this woman told my husband that she was told she may never be able to have children, however after a few weeks of them being together she was pregnant, i said that since we were separated already for him just to stay with her, so he did. They found out they were expecting twins, however they ended up losing the babies. He asked me to attend the funeral, and to even wanted our child to go as well and i flat out said No, i know he was hurt, he asked me to go because it was hard for him and since he didn't have any family near by he wanted someone close to him to be there but I couldn't do it. This started argument with his friends about me being heartless and i wasn't being sympathetic enough to them after going through such a hard loss. Fast forward a few months and he and the woman are still together but on and off. So my husband and I start talking again, he hasn't been with the woman for a little bit, things are going well and we talk about counselling again and seeing if maybe we could make it work but take it step by step. So we are in the middle of working things out when the woman finds out and pops up saying she is pregnant again and its my husbands. He tells her if its true then he will help her with the baby but doesn't want to be with her, she gets the shits and stops speaking to him. Then not long before she is due, she starts talking about if he doesn't go into the delivery room with her then she is going to take all his money and he would never see the child. I told him id rather him not go in but be up there for support but in the end it was up to him, he decided he wouldn't go in and then we didn't hear from her again until one night he got a text saying the child had been born. He text her asking when he could see the baby and she said she would text back, about a week later he got a text saying that they were at home and they could work out a time to meet. She told him that he had to go to her house since she had a C section and couldn't drive. I wasn't happy they would be alone but he went. When he gets there she tells him that me and my child are not allowed near her baby ( which was fine with me, i didn't want anything to do with her ) and he has to go to her house to see the baby and he wouldn't be allowed to take the baby and he had to text her to see if they were available etc If he didn't pay her then he couldn't see the baby, it would never be allowed to go to his house, only her house or the park that's it. He said he would pay for the kid but as it got older he at least wanted a night a week or holidays with it and she said if i was there then he would never be able to have it. He only saw the baby once after that, partly because of her and her family and because i said i never wanted anything to do with them.

So i had an argument with a mutual 'friend' of my husbands and hers. He was telling me that i am a heartless cow because i stopped my husband from seeing his child. I said I never said he couldn't see the kid, i just wanted nothing to do with it. His friend said i was the one who gave him the ultimatum me and our child or the new child. I said no that's not it at all, but apparently that's what her family and people who stopped being friends have been saying. He said to me that my husband resents me for it and that it will be the end of our marriage if i don't let him have anything to do with the child and that my kids are his other child will all hate me when they grow up and realise i am the one who stopped them being a family because of my petty feelings for the woman. I've asked my husband if that's how he feels and he said he doesn't resent me but does get upset when he thinks about the babies they lost and that he has a child he doesn't get to grow up but he made a decision and that's that.

She now has a new child to her new partner and I've told him if he wants to speak to her then he should but he avoids talking about it.

Am i in the wrong? Am I heartless because I didn't push for him to be in the child's life? What about my children? Whenever someone brings it up its poor her this and poor him that but do my feelings not count either?