VGA is a better quality signal than component, also alot of TVs remove any picture processing when they are connected Via VGa, this will often result in a cleaner image.
Also some Tvs will sometimes cause slight jerkiness when using VGA.
The only way is to try it.
As for the upscaling, it again varies between Tvs, if it is very good quality set, the Tvs scaler will most likely be better, if it is a lower priced set the 360 might be better off doing it.
I'd just say give the VGA lead a try and see which you prefer, I can REALLY see the difference on my set whether I use it in 1280*720 or whether I crank it up to 1366*768.
The colours being washed out aren't actually washed out, by comparison to the component output it looks that way, but it is just down to component being too colourful and oversaturated. VGA looks more natural.