Freedoms vs rights

I have been reflecting on freedoms, especially that of speech recently. I have come against a bit of a brick wall.  

I'm a libertarian, in that I favour freedom and responsibility over constraints. But my freedoms might well impinge upon your rights. If you consider that you have the right to live until a random accident or nature takes you off, then my right to kill you is curtailed. Generally, the more rights you have, the fewer freedoms I have, and so, if we assume that you and I both have the same rights and freedoms, then the more rights I have, the fewer freedoms I can enjoy.

 States regulate the trade-offs between rights and freedoms by means of laws (and by implication, crimes). Different states balance things differently, which, at fundament, is the difference between states.

 It follows logically from the existence of laws, that free speech must be curtailed: incitement to commit a crime is an attempt to commit a crime by proxy, and must, logically, itself be a crime. Similarly threats. Not threats of the kind "If you don't buy me an ice cream I'll hate you forever", but threats of the kind "I'm going to do something illegal to your detriment" (whether or not the threat is made in an attempt to coerce a certain behaviour from the target) must be a crime.

 Both of the above fall logically straight out of the existence of laws.

There's a third category of restriction of freedom of speech, that is libel and slander, that arise out of a reasonable right of a person not to be harmed by lies created or circulated about them.

Now, we don't seem to have too much trouble with the idea that one can be "free" despite one's day-to-day existence being hedged around with many and complex laws.  Therefore I'm wondering why freedom of speech has to be binary: you can either say whatever you want, or there's no freedom of speech.

 So I'm shouting at Manuel Valls on the telly the other day, when he is explaining to us how freedom of speech for Charlie Hebdo is 100% compatible with jailing people for celebrating the attacks on the newspaper, and then I find myself wondering if he's right.

Given that the existence of the above three types of constraint on free speech are OK, why not others, that arise from different "rights"? .........

The only argument I can come up with (in favour of complete freedom of speech) is that the effect of only allowing people to say whatever the state happens to agree with, is worse than allowing people to say whatever they like.   Which is more of an opinion, rather than a fact.