I use smart FTP, and my xbox/pc are connected to a linksys hub, and also running through a 10/100 hub, and it takes like 5 seconds to transfer a map. So you definitly have a problem somewhere.
But about storing the cache files right on the x drive, that won't necessarily work. It is a cache directory, so they may be deleted by other programs and games that need to run, and also by that method you would only be able to store 3 maps on the HDD each of a different level, because MP maps can only be cache003, cache004, and cache005.
But if you want to do something like that, there's nothing wrong with just transferring the cache003 map file to your e or f drive into specific folders so you know which maps are which, then transferring them to the x drive (or whatever drive halo uses at the time for the cache, as it could also use y and z) when you want to use them.
However this takes a lot of useless time and a lot of drive space, and since your having connection problems anyway, an even better option would be to put HCE on your xbox. then just take the ppf patches of the mods you want and put them on your xbox, and patch whenever you need to with HCE right on your xbox.