Post by: Xbox-Scene on November 19, 2008, 02:15:00 PM
Halo 3 and NXE: HDD Install Causes Slower Loading Times ... Posted by XanTium | November 19 16:15 EST | News Category: Xbox360 |
Quote
Full Story: bungie.net Post by: Devedander on November 19, 2008, 03:20:00 PM Post by: MilkyNL on November 19, 2008, 03:26:00 PM QUOTE(Devedander @ Nov 19 2008, 11:56 PM)  ACK... boggles the mind how that could happen... only thing I can think of is halo3 is built with HUGE continuous chunks of data that take advantage of the DVD read speed and the 5400rpm laptop drives can't keep up. I would never have expected that though... If you read the full story, you'll know the exact reason... Post by: Landlocked on November 19, 2008, 03:30:00 PM Post by: johnnyrico on November 19, 2008, 03:34:00 PM Post by: riahc3 on November 19, 2008, 03:58:00 PM Disk fragmentation. With putting/removing movies, music, dashboard updates, etc data gets fragmentated. This also happened on Xbox 1 (even if most say it isnt possible because of the FATX file system) but wasn't as noticable (since there was no "content" for Xbox 1) Post by: C_Ripper on November 19, 2008, 04:01:00 PM QUOTE(riahc3 @ Nov 19 2008, 11:34 PM)  I havent read the story but I call fake. Ill explain why it may load slower (its simple): Disk fragmentation. With putting/removing movies, music, dashboard updates, etc data gets fragmentated. This also happened on Xbox 1 (even if most say it isnt possible because of the FATX file system) but wasn't as noticable (since there was no "content" for Xbox 1) Fake what? Like was stated before, read the article. The reason for the longer load times is a very basic, straightforward concept. Post by: riahc3 on November 19, 2008, 04:03:00 PM Just a patch that checks if the maps are loaded in the cache. If it is, continue. If it isn't, load it. Post by: ir0nw0lf on November 19, 2008, 04:12:00 PM Post by: dokworm on November 19, 2008, 04:30:00 PM Still I think they should try to fix it, and I think MS should keep their developers more in the loop! Post by: spinr34 on November 19, 2008, 04:34:00 PM QUOTE(riahc3 @ Nov 19 2008, 04:39 PM) And it can be fixed: Just a patch that checks if the maps are loaded in the cache. If it is, continue. If it isn't, load it. the game does this now. if it didn't it would always load the maps. the issue is that the game doesn't know the difference between the dvd and the hdd. and copying from a device to the same device is fairly slow. especially dealing with a 5400 rpm hard drive. there is only one mechanical piece in a hard drive that can read/write data to the drive. so it has to read data into the hard drives cache (an onboard chip) then it has to write it back to another part of the drive. rinse and repeat. this is what causes the longer load times. if the game knew it was on the hard drive, then it wouldn't copy anything, it'd just load it. but checking cache doesn't do you any good in this case. and i agree that it isn't in bungie's best interest to patch this. also, since you have to have the game in the drive anyways, it isn't a big deal if they don't. you wouldn't gain much of a benefit anyway in load times. Post by: neoxalucard on November 19, 2008, 04:36:00 PM I wonder if there is more to this story than meets the eye Post by: Flagg3 on November 19, 2008, 04:41:00 PM QUOTE(ir0nw0lf @ Nov 19 2008, 07:48 PM) Can be fixed and if/will they fix it are certainly two different things. I don't personally see the incentive for them to fix this and have to spend the time/$$$/manhours on it. Now if it's a simple fix, I could see them doing it. I'm no master coder, but I literally can't see how this is not a 10 minute fix. Keep in mind, the game is ALREADY coded to NOT cache to the hard drive if the hard drive isn't present. The simple fact that Microsoft required all games to be coded this way makes a fix very easy to implement. Had the Xbox 360 always come with a HD standard, the fix would be much more complicated, as all games would expect the hard drive to be there, and would always cache files. As the HD install is basically creating a virtual DVD, what the HD install SHOULD do is report back that there is no HD available for caching whenever a game is installed to the HD. Game saves would be the exception, but this should also be a trivial fix as game saves are done via a separate method. Ideally, this can be patched by Microsoft, and it ABSOLUTELY SHOULD BE. Anyone using HD install will be causing unnecessary wear and tear on their HD, and the last thing that Microsoft needs is to cause the console to become even more unreliable when the fix is so laughably simple. Post by: spinr34 on November 19, 2008, 05:24:00 PM QUOTE(neoxalucard @ Nov 19 2008, 04:12 PM) I have been play H3 just about everyday since I got it back in march. Today I have been playing H3 on my HD all day, I’m not experiencing these “longer load times” that bungie is complaining about. I’ve had about 7 er 8 matches in the last hour, it’s working great. If it is longer it certainly is not noticeable at all. I wonder if there is more to this story than meets the eye… that's because the maps are already cached. you only get longer load times when the maps aren't cached and they have to copy hdd -> hdd. Post by: spinemangler on November 19, 2008, 05:39:00 PM QUOTE(neoxalucard @ Nov 19 2008, 08:12 PM) I have been play H3 just about everyday since I got it back in march. Today I have been playing H3 on my HD all day, Im not experiencing these longer load times that bungie is complaining about. Ive had about 7 er 8 matches in the last hour, its working great. If it is longer it certainly is not noticeable at all. I wonder if there is more to this story than meets the eye This is from the article, which I am guessing was too long? "The only difference in load times occurs when we copy a map that has not been cached to the utility partition or has been evicted from the utility partition because some other maps were run. Once a map has been cached, the time to load it will be identical to running Halo 3 off the DVD with a HDD." Post by: frieko on November 19, 2008, 06:51:00 PM QUOTE It would be a significant undertaking to try and retroactively patch/update Halo 3 to be optimized to take advantage of the HDD install features of NXE. Wow, yes, a one line fix is so hard: CODE --- if ( hard_drive_present ) load_map_to_scratch_space(map_file); +++ if ( hard_drive_present && !( installed_to_hd ) ) load_map_to_scratch_space(map_file); else run_map_directly(map_file); Post by: Nepth on November 19, 2008, 07:20:00 PM 1) everyone, RTFA. Stop postulating and guessing as to why this happens. The article says it very clearly. 2) Yes, the game does have a check for the hdd, and caches if there is one present. The game DOES NOT have a check for if the game is running off the hdd. These are completely different checks. 3) The Halo3 cache is more than just maps. Map vs maps is just different textures. There's so much more that is caches that doesnt change between maps, ie: the engine, the player models, etc. Just switching a map won't rewrite the entire cache. Go play like 4 or 5 different games, then play halo 3. You'll notice the load diference. 4) Post by: somenutter on November 19, 2008, 07:25:00 PM didnt they sort this out ages ago they knew this was going to happen what a load of b8ll*ks this game is like the flagship title for 360 and its not been patched to address this issue what a sham  If i played halo anymore id be really p7ss8d off Come on bungie dont mug yourselves get this sorted now!! Post by: ska_drummer85 on November 19, 2008, 08:05:00 PM Post by: CoLD FiRE on November 20, 2008, 12:50:00 AM is to: 1- Run the game from DVD and let it load the maps 2- Exit to the Dashboard 3- Run the game off the HDD 
Post by: Mushy2000 on November 20, 2008, 03:05:00 AM Post by: CsL on November 20, 2008, 03:07:00 AM I think most games areusing this scratch-partition. Xbox1 already had this feature. Post by: luther349 on November 20, 2008, 04:18:00 AM Post by: frieko on November 20, 2008, 09:24:00 AM QUOTE(CsL @ Nov 20 2008, 11:43 AM) There isn't a possibility to turn the scratch function off? I.e. clear and write-lock the partition, which is used as scratch. If there's a hard drive present, Halo expects it to be writable. When the write failed it would think "unknown error" and exit/reboot. Post by: g8crapachino on November 20, 2008, 01:21:00 PM QUOTE(riahc3 @ Nov 19 2008, 11:34 PM) I havent read the story but I call fake. Ill explain why it may load slower (its simple): Disk fragmentation. With putting/removing movies, music, dashboard updates, etc data gets fragmentated. This also happened on Xbox 1 (even if most say it isnt possible because of the FATX file system) but wasn't as noticable (since there was no "content" for Xbox 1) The article gave a very plausible, simple, and real explanation for the slowdown. There is absolutely nothing for Bungie to gain by making up a fake story so why would they? Why is there always some cynical moron like you who jumps to wrong conclusions and assumes that everything they read is a fake? What even makes you qualified to know that it's fake? As I see it the only thing fake is you pretending to know more then you actually do. Post by: Mozbius on November 20, 2008, 01:32:00 PM QUOTE(Nepth @ Nov 20 2008, 02:56 AM) OK, 1) everyone, RTFA. Stop postulating and guessing as to why this happens. The article says it very clearly. 2) Yes, the game does have a check for the hdd, and caches if there is one present. The game DOES NOT have a check for if the game is running off the hdd. These are completely different checks. 3) The Halo3 cache is more than just maps. Map vs maps is just different textures. There's so much more that is caches that doesnt change between maps, ie: the engine, the player models, etc. Just switching a map won't rewrite the entire cache. Go play like 4 or 5 different games, then play halo 3. You'll notice the load diference. 4) +1 RTFA !!!!!!! It's annoying to see people arguing about stuff that are already answered. But it's even more when people start to talk about "simple programming" when they obviously don't seem to know much. Rule of thumbs for all you "PRO" programmers... If it look easy to you than most probably it isn't that easy! Post by: g8crapachino on November 20, 2008, 01:55:00 PM QUOTE(frieko @ Nov 20 2008, 02:27 AM) Wow, yes, a one line fix is so hard: CODE --- if ( hard_drive_present ) load_map_to_scratch_space(map_file); +++ if ( hard_drive_present && !( installed_to_hd ) ) load_map_to_scratch_space(map_file); else run_map_directly(map_file); Please, spare everyone your smug arrogance. You're just anothing self proclaimed "genius" who thinks that just because he can write a few simple apps in Visual Studio that he has the answers to to everything. Unless you work at Bungie you probable haven't got the slightlest clue as to what it would really take. In the real world there are things to worry about beyond just the actual code when providing a fix to any application. Post by: Devedander on November 20, 2008, 05:56:00 PM QUOTE(MilkyNL @ Nov 19 2008, 03:02 PM) If you read the full story, you'll know the exact reason... I mistook this part QUOTE To help explain why this is the case I tapped our uber engineer and Chief Caching Officer, Mat Noguchi, to answer a few questions. As some kind of stayed tuned and we will get back to you later thing. Didn't realize the story went further. Anyhow makes total sense... exactly why I have multiple drives in my computer, keeps large file size copying snappy. Post by: MrFish on November 20, 2008, 06:28:00 PM Would you like some cheese with your whine, Bungie? So sorry that you're faced with having to do some engineering when a major OS update comes out and you'd really rather be doing fun new games. You must be the only company ever to find themselves in a situation like this. Suck it up, stop blaming the user, and fix your game. Halo 2 had this exact same problem when you 'installed it to the hard drive'. The root cause was that it used a teeny-tiny block size when copying maps over. It would read a teeny-tiny amount, then immediately write it to the cache partition, then read the next teeny-tiny amount. Ostensibly this was because the Xbox only had 64MB to play in, but really it was so that the game would run well off the DVD, but mysteriously break off the hard disk. The reason, of course, being that reading from and writing to different devices means that most of the million seeks are to a place the heads are already at, but reading from and writing to the same device means it has to actually perform them and gets clobbered. There's no reason Halo 3 has to be modified in any kind of hacky, 'detect and catch the corner-cases'way; all it needs to do is instead of reading a half-k or so, then writing a half-k or so, then reading a half-k or so, then writing a half-k or so, then writing a half-k or so, then reading a half-k or so, then writing a half-k or so, then writing a half-k or so, then reading a half-k or so, then writing a half-k or so, then writing a half-k or so, then reading a half-k or so, then writing a half-k or so, then writing a half-k or so, then reading a half-k or so, then writing a half-k or so, then writing a half-k or so, then reading a half-k or so, then writing a half-k or so, then writing a half-k or so, then reading a half-k or so, then writing a half-k or so, simply use a bigger buffer the exact way Halo:CE did. I mean, you could do a benchmarked mad thrash-copy to work out whether you're running off the hard disk* (or a straight benchmarked read to work out if the device you're running off is fast enough to not need the cache), and then copy or not copy maps**, but there's no real point in doing so when you can just use a sensible buffer size like every other program in the known universe***. Copying files is not hard. Getting a few million people to change their mind 'for the good of the community' is hard. * the way the new OS implements the HD install is to use a disk image to emulate running off the disk, so the game can't just ask if it's on the hard disk or not ** Another solution would be to release a 'Halo 3 maps on your hard disk' bit of DLC: Halo doesn't cache DLC maps, because it can tell that they're already on the hard disk. *** Does Windows go batshit when you (copy c:\large_file c:\copy_of_large_file)? No it does not. Post by: neoxalucard on November 20, 2008, 07:14:00 PM Post by: MrFish on November 20, 2008, 08:01:00 PM QUOTE(neoxalucard @ Nov 20 2008, 11:20 PM) if you play Halo 3 online, please be aware that increased map loading times can have a negative impact to everyones experience. In some cases, it could take considerably longer to load multiplayer maps, meaning that everyone you are matched with is also stuck waiting for your maps to load. The corollary, of course, is that if even one person in the match has installed Halo 3, then you're all going to have to wait anyway, so you might as well install it too and save yourself the earache. It's the prisoners' dilemma, except the Rat gets a cookie. Post by: ssj4android on November 20, 2008, 09:06:00 PM QUOTE(g8crapachino @ Nov 20 2008, 04:31 PM) Please, spare everyone your smug arrogance. You're just anothing self proclaimed "genius" who thinks that just because he can write a few simple apps in Visual Studio that he has the answers to to everything. Unless you work at Bungie you probable haven't got the slightlest clue as to what it would really take. In the real world there are things to worry about beyond just the actual code when providing a fix to any application. Well, without any real information all we have are these guesses. What do think would make it so hard? The 360 API isn't publicly accessible, is it? Is there a way to check if the game is running from HDD? Only thing I can really think of is Microsoft didn't backport that function to the older libraries Halo 3 is using. Post by: g8crapachino on November 21, 2008, 01:31:00 AM QUOTE(neoxalucard @ Nov 20 2008, 10:20 PM) ...clip... Some people have argued that if you have less wait in loading it's because you've been playing and it has already cashed so you will never notice it unless you play 7 games a day before you play H3. The problem with that is most people don't, quite a few people play nothing but H3 (and some other games every once in a great while). Do you serious think those hardcore H3 only players will quite playing because the map load time went up 5~7 seconds?? No matter how you argue it, the premise is very shallow. Post by: neoxalucard on November 21, 2008, 03:56:00 AM QUOTE(g8crapachino @ Nov 21 2008, 10:07 AM) Do you serious think those hardcore H3 only players will quite playing because the map load time went up 5~7 seconds?? No matter how you argue it, the premise is very shallow. No, I don't and that's exactly my point. Post by: Devedander on November 21, 2008, 12:24:00 PM Post by: neoxalucard on November 21, 2008, 07:11:00 PM QUOTE(Devedander @ Nov 21 2008, 09:00 PM) Here's what I am wondering... don't people who don't have a hard drive already slow down the game as they have to load from disc every time? So aren't H3 users already used to potentially slow load times every now and then? Why would this cause them to leave if they have already been experiencing it before? That's another good point, it can take up to 5 er 6 min (on a good day)to get into a game on live anyway. Sitting and waiting is part of the game unfortunately. I really think Bungie was just trying to avoid mass panic and a snowball effect. In hind sight, it would have been foolish of Bungie no to have tested harddrive play and made a statement/warning to the public regarding their findings. Post by: MrFish on November 22, 2008, 11:41:00 AM QUOTE(Devedander @ Nov 21 2008, 09:00 PM) Here's what I am wondering... don't people who don't have a hard drive already slow down the game as they have to load from disc every time? So aren't H3 users already used to potentially slow load times every now and then? Why would this cause them to leave if they have already been experiencing it before? Because a hard disk doing a full-blown thrash-copy is way, way, way slower than a DVD drive doing sequential reading. The DVD spends all its time reading at a pretty steady speed, whereas the hard disk spends the majority of its time flinging the heads back and forth accomplishing nothing. Post by: llsTixll on November 25, 2008, 08:32:00 PM "So then the real question is why is copying from HDD to HDD slower than copying from DVD to HDD? In the first case, you are reading from one I/O device (HDD) and writing to the same I/O device (HDD). In the second case, you are reading from one I/O device (DVD) and writing to a different I/O device (HDD). In the first case, because we are reading and writing to the same device, the total copy time is the amount of time it takes to read the map plus the time it takes to write the map. Ultimately this is because for the HDD, you read and write through the same mechanism, i.e., the hard drive read/write head, and those reads and writes cannot occur simultaneously through a single mechanism." As a test on my PC, I copied some data from my optical drive to the hard drive and it took 4 mins. Copying the exact same data to another location on my hard drive took 45 seconds. How can they say copying from DVD to HDD is slower than copying HDD to HDD? Post by: yaazz on November 26, 2008, 04:38:00 PM QUOTE(llsTixll @ Nov 26 2008, 12:08 AM) Can someone please help me understand this statement: "So then the real question is why is copying from HDD to HDD slower than copying from DVD to HDD? In the first case, you are reading from one I/O device (HDD) and writing to the same I/O device (HDD). In the second case, you are reading from one I/O device (DVD) and writing to a different I/O device (HDD). In the first case, because we are reading and writing to the same device, the total copy time is the amount of time it takes to read the map plus the time it takes to write the map. Ultimately this is because for the HDD, you read and write through the same mechanism, i.e., the hard drive read/write head, and those reads and writes cannot occur simultaneously through a single mechanism." As a test on my PC, I copied some data from my optical drive to the hard drive and it took 4 mins. Copying the exact same data to another location on my hard drive took 45 seconds. How can they say copying from DVD to HDD is slower than copying HDD to HDD? This is a good question but a completely different circumstance. On a PC, the computer knows its copying from itself to itself so instead of copying the data, it just modifies the file system to tell the Hard drive the data is located at this new location. No ones and zeroes of the actual data are moved, just file system tags are changed. This is also the reason Fragmentation happens. On the xbox, Halo 3 takes the actual map data and loads it to a special section on the hard drive set aside for JUST game data, nothing else. Halo 3 doesn't know its loading off of the hard drive, and just checks to see if the game data is in this special hard drive area. If its not there it copys it over, even though its already on the hard drive. Since a Hard drive cant read and write at the same time (And for that matter, a DVD burner couldnt either), it takes longer then reading from one device and transfering to another. Post by: Libtoem on November 28, 2008, 02:39:00 AM |