IMO nothing wrong with the guy asking for the dog back. Things might have changed personalky for you in the time you had the dog and it could work out best for both parties.....
BUT they guy must leave you be once you say 'no thankyou, the dog is staying'.
Chances are, the girls jealous and now wants the dog back. In a couple of months when she is fed up then he would be back. In the meantime you would have got another pet.
In short, your right.