Successful IVF pregnancy with sperm that can't survive being frozen

Hello ladies. My partner & I are going through IVF and he recently had sperm retrieved directly from his testes via needles, however they weren't able to get much and it couldn't be frozen as it wouldn't have survived. We have been told they need to do the sperm retrieval from the testes again at the same time they do the egg retrieval and use it fresh. I guess what I am wanting to ask is if anyone has had a similar situation where the sperm wasn't great however it's resulted in a successful pregnancy? I am trying to stay positive however the odds seem to be against us.