Hey. hey. hey. It's going to be okay. Is the humane society going to have him neutered? Do you know if that's in his plan at all? Or are they leaving him up for adoption for a limited time instead? This poor guy's world got rocked recently and even though I don't know much about dogs, I wonder if he'd act differently in another setting.
When we were debating about adopting our 3rd cat we left our number with the office and said "if he's still here in a week, call us". This is a decision only you and your family can make. Sleep on it, but please don't feel guilty. Whatever is meant to happen with this guy will happen.
Did the humane society mention anything about being contacted by a rescue group?