Consent for touch/affection/sex is never owed in a relationship, and can be withdrawn at any time for any reason, whether this is expressed verbally or not. If comfort with touch changes over time between two people, there is no obligation on the uncomfortable person to justify it to anyone.