xboxscene.org forums

Pages: [1] 2 3 ... 5

Author Topic: Breaking The 137 Gb Limit  (Read 631 times)

bobmckenzie

  • Archived User
  • Newbie
  • *
  • Posts: 10
Breaking The 137 Gb Limit
« on: April 19, 2003, 12:57:00 PM »

I was wondering why the XBOX is limited to 137 GB hard drives.  I know that this limit is normally caused by IDE 's 28 bit LBA addressing, but of course all drives over 137 GB support double-pumping the registers to extend to 48 bit LBAs.  This does not require any new hardware, only an update to the IDE driver.

Is the IDE driver in the BIOS? If it is located it in the BIOS it should not be too difficult to extend the IDE driver to support 48 bit addressing....

** POST MODIFIED WITH DETAILS BELOW **

48 bit addressing was purposely created to allow backward compatibility with old IDE host controllers (even with controllers older than "ATA-33"...) The actual hard drive contains the logic for handling 48 bit addressing, not the host controller!

You can refer to the ATA-6 or ATA-7 specifications (drafts can be found here) for specific details on how an IDE DRIVE handles 48 bit addresses:

http://www.t13.org/docs2002/d1410r3b.pdf

http://www.t13.org/docs2003/d1532v1r2a.pdf

You can also see that the ATA Host Adapter Standards spec does not contain any mention of 48-bit addressing (since it is handled in the drive):

http://www.t13.org/docs2002/d1510r1.pdf

** Things to keep in mind regarding ATA/IDE **

(Forgive the long post...I hope somebody finds this stuff interesting.)

* I use the ATA and IDE interchangably

* Terms such as ATA33/ATA66/ATA100/ATA133 are really just marketing terms that describe the maximum DMA transfer rate that a specific drive supports.  This is also sometimes called UDMA33/UDMA66/UDMA100/UDMA133.

* The actual DMA transfer mode used by a PC (or in this case XBOX) will the the highest common mode supported by both the drive and the PC (in the XBOX's case this will be ATA33 as most drives you buy today will support better transfer rates than ATA33).

* "Typical" PC IDE Host controller's are not very intellegent.  The only thing they really do is implement a scatter gather DMA engine.  During boot the BIOS queries the drive and configures the Host and drive for the fastest transfer mode possible.  This transfer mode is used for all future DMA commands issued to the disk drive.

* There are many features that are implemented in later ATA specs that do not require any changes to most host controllers (ATAPI, 48 bit LBA, SMART, etc.)  These are all handled in the IDE driver.

* A command is issued to an IDE disk drive in a sequence that looks something like this:

1. LBA/Size request comes to IDE driver from somewhere (FS/OS/Upper level driver/etc)

2. IDE driver generates DMA scatter gather tables and configures PC host controller DMA engine (probably enables the DMA transfer here -- the host controller waits for signals from the IDE drive to begin transfer)

3. Commands to a IDE disk drive are issued through a defined set of registers that are actually located in the disk drive (as opposed to command packets which are used for things like SCSI, and 1394).  (IDE ATAPI devices use a combination of registers and command packets.)  The IDE driver programs these drive registers with the LBA and block size of the transfer (these are 8 bit registers with a total supported LBA range of 28 bits).  With 48 bit addressing the driver essentially writes each register twice to give a total of 48 bits of addressing.

4. IDE driver writes to the command register of disk drive with the desired command (in this case a DMA read or DMA write).  The disk drive initiates transfer with host controller when it is ready to transfer data.

4a. Some host controllers try to be smart (like caching the IDE registers and only writing them to the IDE disk drive when the command register is written).  In reality this is not smart as it prevents new features from properly being implemented and can cause havok for IDE driver developers. I hope the XBOX hardware does not do this...

5. Eventually the command completes and an interrupt is generated.  The IDE driver checks for errors, cleans up, and returns status.

It seems to me that step #3 is pretty easy to fix to support bigger drives in the XBOX (assuming the XBOX IDE controller works the same as most PC controllers).  The big question is #1.  Is the filesystem/os restricted to 28 bits of LBA.  I'm hoping that it uses 32 bits (which would still allow us to use very large disk drives).

* I believe that support for 48 bit LBA was added to the Linux kernel around version 2.4.20.  Even if large disks cannot be supported for standard XBOX applications, the XBOX Linux project should still be able to use > 137 GB HDDs (assuming the XBOX IDE controller does not try to be fancy).

** FYI: I took a look at the Cromwell BIOS ide driver source code.  I don't see anything there that would lead me to believe that there is any hardware limitation in the XBOX for supporting the 48 bit LBA addressing (somebody just needs to write a little code).  However, the Cromwell BIOS is using an "int" for the "block" (LBA) parameter so that would leave us with 31 bits of usable LBA address space (that's still a terabyte which is a lot better than 137 GB).  Also, it looks like the XBOX linux kernel is using the off-the-shelf source for the IDE disk driver (it's not in their CVS repository), so that should already have support for > 137 gb HDDs.

This post has been edited by bobmckenzie: Apr 20 2003, 03:44 AM
Logged

EvilWays

  • Archived User
  • Hero Member
  • *
  • Posts: 909
Breaking The 137 Gb Limit
« Reply #1 on: April 19, 2003, 01:02:00 PM »

No no no no no! The IDE limitation is set in the MPCX 3 chip. There is NO way to bump up the max drive size without changing that chip...
Logged

bobmckenzie

  • Archived User
  • Newbie
  • *
  • Posts: 10
Breaking The 137 Gb Limit
« Reply #2 on: April 19, 2003, 01:21:00 PM »

Just out of curiosity, does anyone know the technical reason why the MPCX 3 chip limits the size of IDE devices?  Does it prevent the IDE driver from double-pumping the IDE registers some how?

Thanks for humoring me...
Logged

BenJeremy

  • Archived User
  • Hero Member
  • *
  • Posts: 5645
Breaking The 137 Gb Limit
« Reply #3 on: April 19, 2003, 02:51:00 PM »

Well, bobmckenzie feels that there is more to the story, so I'll open this thread for him to post his info.

I seriously doubt this will ever be messed with, as it would require a couple of things done (partition table still needs to be hacked out to support a larger drive, as well)

I'm still not aware of any ATA33 drives supporting the IDE addressing modes needed to support drives > 137GB, but you are free to post info here.

I would think, hoever, the best place to START would be with the Cromwell BIOS crew, to see if it can be done at all. That would be the best place to start.

This post has been edited by BenJeremy: Apr 19 2003, 09:52 PM
Logged

BenJeremy

  • Archived User
  • Hero Member
  • *
  • Posts: 5645
Breaking The 137 Gb Limit
« Reply #4 on: April 19, 2003, 02:55:00 PM »

Oh, and please make this  PRODUCTIVE thread.

If it's something you really want, post your research here, and give the BIOS developers a good base to work with. If you are a developer, you can use DUMPBIN to disassemble the xboxkrnl.exe file in Symbols of the XDK to get intimate details of what's happening in there.
Logged

bobmckenzie

  • Archived User
  • Newbie
  • *
  • Posts: 10
Breaking The 137 Gb Limit
« Reply #5 on: April 19, 2003, 02:57:00 PM »

Thanks.  I'll check with the cromwell team and post any findings here.
Logged

bobmckenzie

  • Archived User
  • Newbie
  • *
  • Posts: 10
Breaking The 137 Gb Limit
« Reply #6 on: April 20, 2003, 09:30:00 PM »

So far things are looking pretty positive.  There is no signs of any hardware limitations, and on the software side things don't look too complicated (at least when using Cromwell and Linux).

The good news is that the Cromwell BIOS uses "unsigned long" to specify the block number (aka sector/LBA) and parition tables have 32 bits allocated for start and size of a partition.  That means without any really messy hacking we should be able to address up to 32 bit LBA's (which is not as good as 48 bit, but still is much better than 28 bits).

Just an FYI. 32 bits of LBA will give us:

2^32 * 512 bytes = 2,199,023,255,552 bytes = 2.2 Terabytes (this of course is using disk drive manufacturer's "metric system" math for determining disk size)

We shouldn't hit this boundary until XBOX-2 comes out, so I'm not too worried about support larger drives for now.

I have no visability into the other BIOS's so I'm not sure how much work it will take to add this support into them.  I'll continue with Cromwell and Linux (I've already sent out a preliminary hacked up version of the ide driver to the Cromwell team) and see how things go.  If there is anyone active in BIOS development snooping around, drop me a line and I can give a dump on how to implement 48-bit LBA.

P.S. Things may go a lot faster if somebody wanted to donate a Xecuter 2 PRO for the effort (my cheapmod isn't really working out all that great for this type of development)...   :P  

This post has been edited by bobmckenzie: Apr 21 2003, 04:48 AM
Logged

razorrifh

  • Archived User
  • Sr. Member
  • *
  • Posts: 329
Breaking The 137 Gb Limit
« Reply #7 on: April 21, 2003, 10:48:00 AM »

try getting in touch with superfr0/dextrose


might be useful to you
Logged

old engineer

  • Archived User
  • Full Member
  • *
  • Posts: 155
Breaking The 137 Gb Limit
« Reply #8 on: April 21, 2003, 12:05:00 PM »

Blimey...

It will be nice when you get there, although the "411gb Project" is a far easier route if you love using a screwdriver and a soldering iron!

:ph34r:

This post has been edited by old engineer: May 2 2003, 09:23 AM
Logged

bobmckenzie

  • Archived User
  • Newbie
  • *
  • Posts: 10
Breaking The 137 Gb Limit
« Reply #9 on: April 21, 2003, 05:24:00 PM »

What is the 410 gb project?  I did a search and didn't find anything...

This post has been edited by bobmckenzie: Apr 22 2003, 12:25 AM
Logged

t_rex

  • Archived User
  • Jr. Member
  • *
  • Posts: 88
Breaking The 137 Gb Limit
« Reply #10 on: April 21, 2003, 04:53:00 PM »

smile.gif

Basicially just a hard drive switch smile.gif
Logged

EvilWays

  • Archived User
  • Hero Member
  • *
  • Posts: 909
Breaking The 137 Gb Limit
« Reply #11 on: April 21, 2003, 07:40:00 PM »

I was just thinking (damn that hurts)...

I wonder if it's possible to get the Xbox IDE interface to do ATA66. I know in Windows 2000, the kernel itself won't do ATA66 without tweaking the registry...I wonder if that part of the interface is software/driver driven as well or if that's controlled in the MCPX chip.

Anyone know what's contained in the microcode in the MCPX chip? Could the code on the chip be flashed like on the TSOP (by bridging some points together perhaps)?

I was told that the IDE interface was controlled by the MCPX chip, so that's why I said what I did...learn something new every day...

This post has been edited by EvilWays: Apr 22 2003, 02:40 AM
Logged

bobmckenzie

  • Archived User
  • Newbie
  • *
  • Posts: 10
Breaking The 137 Gb Limit
« Reply #12 on: April 23, 2003, 12:17:00 PM »

I thought I would post a little tuturoial to try and educate those working in the IDE area of the XBOX.  Here goes...

Adding support for hard drives larger than 137 GB is actually very easy.  The following guide should help those people developing in this area.

NOTE: All of the examples below can be changed to use 32-bit variables (instead of 64-bit).  Even if only 32-bits of the 48-bit LBA is supported in XBOX software, this will still allow 2.2 Tera-byte hard drives!

There are two places in the IDE driver that need to be update to add support for 48-bit LBA addressing:

1. Initialization
2. Issuing Commands

Initialization:

During initialization the drive is issued an Identify Drive command to determine the capabilities of the drive.  48-bit LBA support is specified by bit 10 in word 83.  If this bit is set, than the "real" size of the drive is given by the four words starting at word 100.

Sample code follows:

****************************************************************************

uint16_t drive_info[265];
uint64_t maxLba;

... Issue identify drive command and store in drive_info array

/* Check for drive support of 48-bit LBA */
if( drive_info[83] & 1ul<<10 ) {
   /* 48-bit LBA supported */
   maxLba = *((uint64_t*)&(drive_info[100]));
} else {
   /* Only 28-bit LBA supported */
   maxLba = *((uint32_t*)&(drive_info[60]));
}

****************************************************************************


2. There are some new commands supported by drives that support the 48-bit LBA feature.  These commands are the same as the non-EXT flavors of the commands, except these commands support "double-pumping" some of the IDE registers in order to support 48-bit LBAs.  The new commands include:

   - FLUSH CACHE EXT (0xEA)
   - READ DMA EXT (0x25)
   - READ DMA QUEUED EXT (0x26)
   - READ MULTIPLE EXT (0x29)
   - READ NATIVE MAX ADDRESS EXT (0x27)
   - READ SECTOR(S) EXT (0x24)
   - READ VERIFY SECTOR(S) (0x42)
   - SET MAX ADDRESS EXT (0x37)
   - WRITE DMA EXT (0x35)
   - WRITE DMA QUEUED EXT (0x36)
   - WRITE MULTIPLE EXT (0x39)
   - WRITE SECTOR(S) EXT (0x34)
   
Keep in mind that all of these commands work almost exactly the same as the non-EXT flavors, with the exception of having to double pump the regististers in order to communicate the 48-bit LBA.   I will focus my example on the READ EXT command.

Here is some sample code that shows how to issue a read command (using standard read command).  This code is from the Cromwell ide driver (with hacked support of 48-bit LBA).

****************************************************************************

int BootIdeReadSector(int nDriveIndex, void * pbBuffer, unsigned int block, int byte_offset, int n_bytes)
{

...

   if( block >= 0x10000000 ) { /* 48-bit LBA access required for this block (a.k.a. LBA) */

      /* This routine uses a max LBA of 32 bits (due to unsigned int data type used for block parameter) */

      tsicp.m_bCountSectorExt = 0; /* Don't support read sizes greater than 512 sectors */
      
      tsicp.m_wCylinderExt = 0; /* 47:32 */
      tsicp.m_bSectorExt = (block >> 24) & 0xff; /* 31:24 */
      tsicp.m_wCylinder = (block >> 8) & 0xffff; /* 23:8 */
      tsicp.m_bSector = block & 0xff; /* 7:0 */
      tsicp.m_bDrivehead = IDE_DH_DRIVE(nDriveIndex) | IDE_DH_LBA;
      ideReadCommand = IDE_CMD_READ_SECTOR_EXT;
   
   } else {

      tsicp.m_bSector = block & 0xff; /* lower byte of block (lba) */
      tsicp.m_wCylinder = (block >> 8) & 0xffff; /* middle 2 bytes of block (lba) */
      tsicp.m_bDrivehead = IDE_DH_DEFAULT | /* set bits that must be on */
         ((block >> 24) & 0x0f) | /* lower nibble of byte 3 of block */
         IDE_DH_DRIVE(nDriveIndex) |
         IDE_DH_LBA;
      ideReadCommand = IDE_CMD_READ_SECTOR;
   }

   BootIdeIssueAtaCommand(uIoBase, ideReadCommand, &tsicp);

   ....
}

int BootIdeIssueAtaCommand( unsigned uIoBase, ide_command_t command, tsIdeCommandParams * params )
{
   int n;

   IoInputByte(IDE_REG_STATUS(uIoBase));

   n=BootIdeWaitNotBusy(uIoBase);

   /* 48-bit LBA */
   /* this won't hurt for non 48-bit LBA commands since we re-write these registers below */
   IoOutputByte(IDE_REG_SECTOR_COUNT(uIoBase), params->m_bCountSectorExt);
   IoOutputByte(IDE_REG_SECTOR_NUMBER(uIoBase), params->m_bSectorExt);
   IoOutputByte(IDE_REG_CYLINDER_LSB(uIoBase), params->m_wCylinderExt & 0xFF);
   IoOutputByte(IDE_REG_CYLINDER_MSB(uIoBase), (params->m_wCylinderExt >> 8));

   IoOutputByte(IDE_REG_SECTOR_COUNT(uIoBase), params->m_bCountSector);
   IoOutputByte(IDE_REG_SECTOR_NUMBER(uIoBase), params->m_bSector);
   IoOutputByte(IDE_REG_CYLINDER_LSB(uIoBase), params->m_wCylinder & 0xFF);
   IoOutputByte(IDE_REG_CYLINDER_MSB(uIoBase), (params->m_wCylinder >> 8));
   IoOutputByte(IDE_REG_DRIVEHEAD(uIoBase), params->m_bDrivehead);

   IoOutputByte(IDE_REG_COMMAND(uIoBase), command);
   Delay();

   n=BootIdeWaitNotBusy(uIoBase);

   return 0;
}

****************************************************************************

The actual ATA specification that gives the details for the 48-bit LBA feature can be found here:

http://www.t13.org/docs2002/d1410r3b.pdf
Logged

desertboy

  • Archived User
  • Hero Member
  • *
  • Posts: 523
Breaking The 137 Gb Limit
« Reply #13 on: April 25, 2003, 11:07:00 AM »

I just like to say bob your 1st six posts put mine to shame.


This post has been edited by desertboy: Apr 25 2003, 06:09 PM
Logged

SupeRdUPErBlakE

  • Archived User
  • Hero Member
  • *
  • Posts: 787
Breaking The 137 Gb Limit
« Reply #14 on: April 25, 2003, 11:15:00 AM »

ph34r.gif
Logged
Pages: [1] 2 3 ... 5