I keep reading people talking about quality they get from this H.264 ...
Do you have to be a mac user to use this? I use Sony Vegas for my encoding and I was looking around and it has H.261 and H.264 and I tried them and the encoder failed for whatever reason...

So what is H.264 and how do I get to use it? Or What program outputs to it?

http://en.wikipedia.org/wiki/H.264

ok so the quicktime encoder in Vegas won't do it then?

http://vegas.digitalmedianet.com/articles/viewarticle.jsp?id=39269

Oh I see it, it's under MainConcept AVC/AAC
So AVC and H.264 are the same thing then, correct?

Yes, Mike, that is correct. You can also encode to h.264 from Quicktime pro.