Sorry for that.
Okay, so update time. I got it working!!
Apparently when you setup the xbox with MCE, it creates a new user account and by default, only the contents of that user (MCX1) folder gets added.
I was able to fix things by choosing to add folders, on the xbox, under Videos, Music, etc. I think I got confused because I tried this once with music and it was going so slowly traversing every directory that I thought it was copying things over to the xbox, which I didn't want. I now realize that it was not doing that and in fact it's only when adding music folders that it seems to take a bit of time to add artwork, etc.
So anyway, by going into Media Center on the 360, then choosing "more" and "add folder" I could browse the folders on my MCE and add my correct video folder

Hopefully this helps some other newbs who run into the same problem that I did.