Can someone remind me the setting to stop the capture window from canceling when hidden. If hide the capture window the capture stops.

What do you mean by "hide the capture window"?

For some odd reason (after a reinstall of Vegas,) NOW when I am capturing footage, my capture will stop if the capture window gets hidden. For instance by let's say behind a simple my documents window. Not sure why.

You just need to un-tick the box.