Print

[Devenic]

QUOTE (fedge @ Nov 2 2003, 10:06 PM)

I have a hard time believing that MS (who makes both the XBOX and GDFImage Utility) would leave room for improvement in the ISO format for their own system.  In theory, ISO's made by Qwix are better organized, as I have discovered via research, but, when it comes down to the reason for making ISO's for the XBOX in the first place, better organized or not, I would believe that MS's Layout via GDFIMage is superior for Gameplay.  Otherwise, why would they give GDFImage to developers to pack their games?  Just a thought.

They didn't give GDFImage to developers to pack their games.  They gave the Game Disc Authoring Tool to developers, which allows them to fully optimize their games beyond anything we can automatically do with just our software.  They can run a DVD emulator that can mount an ISO file, and it then watches all of the file activity so that file placement can be optimized based on actual file usage during gameplay.  Then the Game Disc Authoring Tool can be re-run with the emulation log, and a final image can be submitted to MS.

In other words, GDFImage is a quick and dirty tool, and is not at all what MS wants developers to use to submit final disc images.  Trust me, it is not the standard.  It's just _a_ tool, not _the_ tool.  It's a very easy way for a developer to quickly make a disc, but MS would not accept that image for manufacturing.

[Xeero]

QUOTE (Devenic @ Nov 3 2003, 06:38 AM)

It's a very easy way for a developer to quickly make a disc, but MS would not accept that image for manufacturing.

Actually, I'm pretty sure that's one of the forms in which MS accepts projects for submission - tape, electronic package, and GDF ISO.  Granted, as you said, the developers are first given the option to manually arrange the layout using the Game Disc Authoring Tool (and are able to place more critical files on the parts of the disc with a higher velocity), but the images are THEN created in GDFimage using the saved preset from the Authoring Tool.  So unless you can create a tool that will read the developers' minds for file organization, GDFimage doesn't do a bad job at emulating how developers have arranged the files.

Also, know that I haven't tested any performance gains with Qwix, but I'd have to believe that straying from the directory structure used on original game discs would generally not lead to performance gains.

[Devenic]

QUOTE (Xeero @ Nov 3 2003, 08:25 AM)

Also, know that I haven't tested any performance gains with Qwix, but I'd have to believe that straying from the directory structure used on original game discs would generally not lead to performance gains.

Without the emulation log, gdfimage can't do anything slick.  It just creates a linear list of files in alphabetical order.  I'm surprised at how much faith everyone puts in that little tool.

Also, I'm not straying from the directory structure used on original discs any more than any other tool.  As far as I know, no one has figured out how to determine the layout on original discs because you can't mount the disc and read the directory table sectors directly.  You can only get the list of files (which says nothing about where they're located on the disc).

[fedge]

QUOTE (Devenic @ Nov 3 2003, 01:34 PM)

Without the emulation log, gdfimage can't do anything slick.  It just creates a linear list of files in alphabetical order.  I'm surprised at how much faith everyone puts in that little tool. Also, I'm not straying from the directory structure used on original discs any more than any other tool.  As far as I know, no one has figured out how to determine the layout on original discs because you can't mount the disc and read the directory table sectors directly.  You can only get the list of files (which says nothing about where they're located on the disc).

You are right about the linear list of files that GDFImage creates, only using the RTable entries... However, why change at all from that format to a 2 sided tree when you dont know what sectors the files were laid out on the original disc in the first place? I just dont understand how that is an "optimized" ISO format when the files are not laid out in a linear order.  Yes, you can search for a file name faster if your files are listed in a binary tree, but once it's extracted/burned, what difference does it make?  And how can you claim that it makes for a better game disc after burning an "optimized" image when no one has had performance issues when burning GDFImage ISO's but people have run into problems when other ISO formats are implemented, such as choppy audio/video.  I just dont understand the logic behind straying from something that you know works.  Im not saying that building an ISO the way that you have, which is very smart and I efficient, is not useful for extracting, etc, I just dont understand the advantages over linear layout (GDFImage) when it comes to the real reason for making iso's for XBOX which is to burn and play on the xbox.

note: Why cant the apps that extract the info from the game disc to teh xbox record the layout of how the files were organized (sector info, etc)?

As of today, I've added support for Qwix &  ZXBTools ISO's in Craxtion's ISOExplorer and extraction engine.  I think the only ISO's im having trouble with now are Yursoft iso's, but then again, who isn't?  I didn't want to have to add an external extraction utility for any iso's, but, if I cant get Yursofts ISO's to extract, i might have to include one incase.  I'll keep people posted.

[Xeero]

I agree with fedge here.  I'm just a little bit confused as to how an ISO created with extract-xiso or Qwix is actually deemed to be an optomized ISO.  Don't get me wrong, I'll be the first to try out new backup methods, but I also like to see ample justification behind claims like that.  I tried extract-xiso, but it created corrupt backups that would not boot in the Xbox and also created images not compatible with other ISO utilities (not even gdfimage).  Maybe I'm just bitter.  

Anyway, Devenic, I'm genuinely curious (and I'm not trying to sound over critical) as to what constitutes your images as being optimized?

[Xeero]

QUOTE (fedge @ Nov 3 2003, 11:31 AM)

note: Why cant the apps that extract the info from the game disc to teh xbox record the layout of how the files were organized (sector info, etc)?

Simply because they were originally only coded to back up the games to the hard drive via a file copy, in which case the layout of the files is unimportant.  If someone (the Avalaunch team, perhaps) wants to create a utility that would generate a FLD file from the disc in the drive, then we could get some REAL optimization.  That would also give legal game owners another edge over those who download games.

[Devenic]

QUOTE

I agree with fedge here. I'm just a little bit confused as to how an ISO created with extract-xiso or Qwix is actually deemed to be an optomized ISO. Don't get me wrong, I'll be the first to try out new backup methods, but I also like to see ample justification behind claims like that. I tried extract-xiso, but it created corrupt backups that would not boot in the Xbox and also created images not compatible with other ISO utilities (not even gdfimage). Maybe I'm just bitter.  Anyway, Devenic, I'm genuinely curious (and I'm not trying to sound over critical) as to what constitutes your images as being optimized?

FWIW, Qwix doesn't use any code from, nor was any code analyzed from extract-xiso.  If extract-xiso makes corrupted ISO files, that's a separate issue, and not anything I can help with.  

See my response above to fedge.  What this comes down to is 2 completely separate optimizations.  One is the internal structure of the directory tables, which AVL trees will be faster than linear lists, no matter how anyone would like to argue it.  Every Xbox can read the AVL trees (if they're constructed properly) and will be faster at navigating the directory tables if they're used.  ISO extraction programs having a problem with AVL trees are a separate matter, and one I don't care much about.    AVL trees are not what are causing the access time problems, I'd put money on it.

The other optimization is directory and file placement.  This is something we tested extensively, and found no reason not to place directories at the beginning of the ISO.  If some drives are having problems, it's most likely a seek time and spindle speed issue.  Theoretically, a DVD drive should be faster reading data at the outside edges of the disc.  But in reality, many drives are unaffected regardless of the placement.  Apparently, however, some people are running into problems with this second optimization.  I have a couple of ideas on how to handle this, one of which (very easy) is to just un-hide my options that allow you to turn off the optimized layout.  It was hidden in 1.0 because we found no reason to provide the option in our testing.

Keep in mind, this is 1.0 software.  I'm not claiming to have the end-all-be-all ISO maker.  There's room for improvement, and feedback.  My goal with Qwix was to make an ISO creation tool that I had complete control over, instead of passing control to an external application.  All of the extraction and ISO creation code (along with everything else) in Qwix is 100% mine.  This gives me a lot of control over each phase, and allows me to do things that can't be done with external tools.  What I don't like is the instant attack on Qwix, and any attempt to make things better than they already are.  I'm surprised that everyone would rather maintain the status quo.  Truly very sad.

[Xeero]

Nobody's attacking.  We're questioning.  You're providing thorough and informative answers, which is great.    That's the first step of innovation.

Concerning your association with extract-xiso, I realize there is no connection.  I was just saying I jumped onto the very scant "optimized ISO" bandwagon when extract-xiso for win32 first arrived, and it produced nothing but corrupt files for me, so I was somewhat bitter.  The statement served no purpose other than to explain my need for such extensive information on what goes into an optimized ISO.

UPDATE:  I'll give this a pin for now while there is so much great discussion going on.  After discussion has concluded, I'll list it in the "Key Threads" pin.

[feflicker]

Wow. I, personally, would really love to see some "benchmark" type tests done with a GDFImage'd ISO and a Qwix ISO (using each of the three Xbox DVD drives, and several games of different sizes...) The difficulty would lie in maintaining integrity in the testing (same xbox, same drive, same media, same burn speed, etc.). One person (or one team) would most likely have to do all of the testing... And that could take a lot of time and money!

Devenic mentions that 200 ISO's were tested to validate his optimized format used in Qwix. Maybe that test data could be a start?

Good idea Xeero splitting this into a seperate thread. I am sure the Craxtion authors were feeling that their 3.0 thread was being overrun  

This has certainly been an interesting read  

[feflicker]

Ok, sounds like a plan. I think we can all agree that in theory the "optimizations" in Qwix make sense, but I personally would like to see the real world data backing this (I am sure others would as well)...  Who would have thought that in Nov. of 2003 there would still be talks about the Xbox ISO format and possible improvements

[the joker]

shouldn't need to be backed up - nor proved.

Afaik most tools make this AVL lookup linear - which everyone gotta agree is the easy way.

Balanced AVL trees is decently complex stuff, and they are there for a reason.  If it wouldn't been faster, why would someone invent the system at all ?   If not to save time - then what other reason(s) would there be ?  

[Xeero]

QUOTE (the joker @ Nov 4 2003, 06:52 AM)

shouldn't need to be backed up - nor proved. Afaik most tools make this AVL lookup linear - which everyone gotta agree is the easy way. Balanced AVL trees is decently complex stuff, and they are there for a reason.  If it wouldn't been faster, why would someone invent the system at all ?   If not to save time - then what other reason(s) would there be ?

To the best of my knowledge, Xbox originals are laid out in a list rather than a tree (pure speculation), and I guess we just don't have any idea whether the Xbox would be able to take advantage of AVL trees.  I can't imagine I'm the only one that hears "it's there so it must be better" and doesn't just go along with it.

Again, I'm not criticizing...I'm just trying to get some data on what makes these optimized ISOs with respect to the Xbox.  I realize you can FXP them more quickly.....great.  But I'd really like to see some difference on burned DVD before I refer to them as "optimized".

For instance, I picked up a copy of Morrowind:GOTY, and because I know it always took forever to load on the PC, I backed it up using SimpleX and Qwix and burned both.  I cleared out the cache between boots to make it fair.  Both took a tremendous amount of time to load a new game, and there was no distinguishable increase in performance out of the Qwix disc.

[the joker]

simplified question.

what is fastest - 11 seeks, or 4096 seeks ?


edit :

QUOTE

To the best of my knowledge

that means you've just made your mind about it - or is it a fact ?

[LepPpeR]

He is not Criticizing you but giving you some information that he has found.  He tested a game with rather long load times, made a disc with both simplex (perfect gdf) and QWIX and had the same results.  Neither ran faster than the other.  I have made several tests now myself and have noticed no difference in loading times, gameplay, audio or video.

To the best of my knowledge is not used when you have made up your mind.  It is used when you are unsure or not 100%.  for instance, to the best of my knowledge, the war in IRAQ will take many years.  It could end tomorrow or the next day as I am not 100% sure.


Has anyone tried backing up any sega sports (ESPN) games with QWIX?

[the joker]

I've never made anything more complex than the 9660 libs out there - but common sence tells me that for loading times to be faster, you'd need data on the outer layer. balanced AVL trees will only affect where there is many many files in each dir - like KOTOR has.

Correct me if I'm wrong =)