Speaking in libertarian terms (and as a social moderate myself), the question of whether it’s proper to deny a restaurant from operating on a college campus because it has an associated nonprofit that gives money to social conservatives* is a fairly easy one, yes? Right of association, and all that: if you don’t like a company that supports a moral and ethical agenda that doesn’t agree with you, by all means vote with your feet and your pocketbook. Assuming, of course, that no federal money is involved; if it’s strictly a private relationship, then you should be perfectly within your rights to do business how and as you please**.
Of course (still speaking as a social moderate), you’re also an idiot for blaming the local Chick-fil-A – and more importantly, denying yourself a damned good chicken sandwich – for the actions of an affiliate of its corporate management. Fortunately, while ‘being an idiot’ is not actually an enumerated right under the Constitution it is generally conceded that American citizens have a reasonable expectation of being able to be idiots. If that’s what they choose to be, of course.
*They’re actually saying it’s because of said groups’ attitudes towards homosexuality, but that’s just a sleazy rationalization. And I’ll concede otherwise the day that universities start sanctioning African-American organizations which officially share social conservatives’ opinions on same-sex marriage.
**This is often not strictly true, in a practical sense. Then again, we are talking about points of libertarian order, which means that practicality is more or less optional. Or would that be ‘orthogonal?’ I don’t have time to really look it up.