My heart says no, but there's a reason humans have minds, and my mind says yes, rights belong to those who reject them, too.
A note for people who know so little of the principle of free speech that they think mocking it as "freeze peach" is funny: The First Amendment covers the legal principle, but the moral principle is always greater than the law. You have a legal right to try to get people fired or blacklisted for saying what they think, but you have a moral obligation to defend them, no matter how reprehensible you may think their words.
I highly recommend Frederick Douglass's "A Plea for Free Speech". He talks about—but does not use the phrase—the heckler's veto being used to silence speakers. A few bits:
"These gentlemen brought their respect for the law with them and proclaimed it loudly while in the very act of breaking the law. Theirs was the law of slavery. The law of free speech and the law for the protection of public meetings they trampled under foot, while they greatly magnified the law of slavery."
"Even here in Boston, and among the friends of freedom, we hear two voices: one denouncing the mob that broke up our meeting on Monday as a base and cowardly outrage; and another, deprecating and regretting the holding of such a meeting, by such men, at such a time. We are told that the meeting was ill-timed, and the parties to it unwise."
"To suppress free speech is a double wrong. It violates the rights of the hearer as well as those of the speaker."
It's short. Read it all.