TTC with new partner?

Hi IM's, I have a question about trying to conceive with new partners..
I have a 6 year old who was conceived unintentionally and then struggled for 2 years trying to conceive another with the father - it didn't happen - I had all the regular tests done and nothing came back wrong but the father did have a low count..
Now new husband and I want to start 'not preventing' with but I'm worried it won't happen..
Does anyone want to share their stories to give me some hope?
Thanks in advance