Well, he's already fighting two stone above his ideal weight, so, and this is pretty much what I've read, perhaps he's finding the training camps etc etc, up at heavyweight, a bit too taxing?
To unify the belts, means he'd be at roughly a 4 stone disadvantage to fury.
Maybe a step too far?