Tell him that there won't be another date to his face. It's better than telling him a few days later on text. You aren't doing anyone a favour by avoiding confrontation, but just adding stress to both your lives.