PS3 HDD Contents

[HanSooloo]

From the analysis performed so far, here are some findings:

1. The HDD is encrypted with a (most probably) Sony proprietary format.
2. If Linux is setup on the machine, the HDD will contain the relevant ext2 or ext3 partitions, but it will NOT be visible to a regular OS. This is because, the HDD does NOT have a standard partition table. If one uses WinHex to scan the HDD, then the program will find the ext2/ext3/swap partitions at their respective offsets.
3. A program has been written to scan blocks of 16bytes for where contiguous data is on the HDD. This program has identified major blocks of data on a freshly formatted 60GB HDD.
4. Of major interest is that right around the 380MB marker, we start seeing blocks of 64KB data, and this repeats itself EVERY 183.72MBs. Why does a system need 64KB worth of markers every so often, is a mystery at the moment.
5. Each HDD is "individualized" the moment it is formatted on a particular PS3 unit. An individualized HDD CANNOT be used in another PS3 unit due to (in theory) a unit based signature being written to each HDD.
6. A project is underway to "individualize" 2 same make and model (Seagate Momentus 60GB 2.5" SATA) HDDs and perform a byte level diff to spot differences in the disk layouts.
7. This diff will also be analyzed by the data block scanning program mentioned in Item-3 above.

Here are 2 files of interest for the DEV community.

DISCLAIMER: these files are posted here for STRICTLY EDUCATIONAL PURPOSES. I hereby disclaim any harm that may arise from the use of the programs and the information posted here. You accept that you are doing this at your own risk.

Some information about the files:

HDD Info.xls: contains Start and End addresses of a minimum of 16-byte contiguous data blocks. Contiguous data block is defined as any 16-byte block that is not ALL ZEROES.

scan.rar: compressed source code for the "scan hdd" program that has been compiled on Linux. It currently does NOT support large HDD addresses due to 64-bit limitation on variables. It is planned for the program to have 128-bit variable support with publicly available libraries. Any suggestions are welcome.

Attached Files

	HDD Info.xls (341.5 KB, 368 views)
	scan.rar (994 Bytes, 222 views)

[HanSooloo]

Note about "scan hdd" program: this program uses the sysfs functions to read the size of the HDD. If your Linux setup does not have this, the program will most probably fail. If that is the case, you may want to replace the "filesize" variable with the actual size of your HDD in bytes.

Also, the program currently tries to read from "/dev/sda" for the analysis. So, please make sure that your OS drive is NOT "/dev/sda". If you want to scan another drive, simply replace the string on line 51 to that of your actual HDD you want to scan.

Next version will provide some parameters for which device to scan.

OK, so the next phase in the project is to start doing some comparisons between different units' first few bytes of the HDD. The next version of the program will have an option to do a data dump to a file based on a size parameter.

You can also use the "dd" command in Linux to achieve the same goal.
Based on the postings, we can start doing some research on where the differences come and if there is a common thread.

Last edited by HanSooloo; 04-05-2007 at 10:35 PM. Reason: Automerged Doublepost

[r3pek]

scan.rar: compressed source code for the "scan hdd" program that has been compiled on Linux. It currently does NOT support large HDD addresses due to 64-bit limitation on variables. It is planned for the program to have 128-bit variable support with publicly available libraries. Any suggestions are welcome.

try using "unsigned long long int" variables. They are huge!
(MAX value is 18446744073709551615 on a 32bits box)

[HanSooloo]

The scan hdd program has been updated now capable of supporting large address spaces.
Attached to this post is source code package and the "results.txt" file showing data blocks in a freshly installed 60GB HDD in the PS3.
The definition of "freshly installed" is as follows:

1. Go to System Settings and choose Restore to Factory Defaults.
2. When the "Press X to restart" prompt comes on, press and hold the power button to power off the PS3.

The idea here is that the HDD data does not contain ANY customization / personalization and is as generic as it gets.

Attached Files

	scan-0.2.1.rar (16.5 KB, 86 views)
	results.txt (26.6 KB, 186 views)

[HanSooloo]

OK, so after looking at this fresh set of analysis results, I am starting so see a (kind of random) pattern (although encrypted probably). There are different sizes of data blocks (16KB, <mostly> 64KB, 32768KB) repeating themselves throughout the HDD.

So the layout looks like this:

1. a small startup section of 4 and 8KBs
2. a 64KB section
3. a 6 and 2KB section
4. 83 repetitions of 64KB blocks, separated by 183.7MBs
5. 128KB, 6914KB and 160KB blocks
6. 50 repetitions of 64KB blocks
7. 30 and 6KB blocks
8. 64 and 80KB blocks
9. 20 repetitions of 64KB blocks
10. 12, 112, 48KB blocks
11. 2 sets of 4, 64, 208KB blocks each having a 8448 and 32832KB block follow ups respectively
12. 32 and 64KB blocks
13. 105 repetitions of 32768, 16 and 64KB blocks (sequenced like that)
14. 5616KB block
15. 36 repetitions of 64KB blocks
16. 512 bytes
17. 1.5KB block
18. 1KB block
19. 536KB block

Right now this does not make too much sense, but I would be very interested in the output from other PS3s. The key thing here is to compare apples to apples.

When you post your results, please note the type of PS3 (US, JAP, EUR, AUS, etc.), the HDD size (20 or 60GB) and the firmware version number.

Also, please make sure that you follow the initialization steps I have provided at the top of the post to make sure that the HDD has no personalization on it.

The easiest way I can see a user doing this is:

1. Get a SATA to USB/FireWire adapter for the HDD or just plug it into your PC, if you have an available internal SATA port.
2. Copy the scan hdd source code to a USB key, as well as the GNU MP library source tarball package.
3. Boot with a live cd Linux distro. The one I currently use is Knoppix 5.0.1, which is very capable in auto-detecting a large selection of hardware.
4. Extract the GNU MP tar package from the USB key to the home directory (usually ends up being something like "~/gmp-4.2.1/").
5. Follow these steps, exactly:
   1. cd ~/gmp-4.2.1
   2. ./configure
   3. make
   4. make check
   5. sudo make install
6. These steps will ensure that you have a properly configured GMP install.
7. Extract the scan hdd source from the USB key to your home (something like "~/scan/").
8. You will need to modify the HDD_DEV define to that of the device name corresponding to the PS3 SATA drive connected to your PC.
   1. Go to your /dev directory
   2. Look for block device names starting with "sd"; something like "sda" or "sdb" are the most probably options.
   3. A good indicator of the PS3 hdd is that you will NOT see numbered partitions such as "sda1" or "sda2" for the PS3 hdd. It will be a block device without any partitions, since Linux cannot automatically recognize the partition types on the hdd.
9. Follow these steps to compile scan hdd:
   1. cd ~/scan/src
   2. gcc -lgmp -o scan scan.c
10. Now, to execute the scan hdd program:
    1. sudo ./scan
11. Wait for the program to go through the entire HDD. It will take close to 10 minutes. If you want, you can change the BUFFER_SIZE define to take larger or smaller chunks of data into memory for analysis.

That's all for now. Any new ideas and analysis results are welcome.

[HanSooloo]

A minor update to the program to remove the GNU MP library and start using the uint64_t type for large variables.
I would like to thank r3pek for pointing out the obvious here (which I missed!) ;-)

No new functionality, but analysis continues.

Attached Files

scan-0.2.2.rar (21.0 KB, 52 views)

[HanSooloo]

Updated RAR image. Was missing scan.c source code. :tongue2

Thanks to savage for pointing this out.

Attached Files

scan-0.2.2.rar (24.3 KB, 58 views)

[HanSooloo]

A new version of the program is posted. From the changelog:

> Added command line processing.
-h 3 letter block device name to scan e.g.; sda
-b Number of bytes to reserve for large hdd reas e.g.; 256000000
-s Byte signature to scan in comma searated form e.g.; 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0

Hopefully this will make life easier for people, since you have the capability to customize the way scan hdd works based on your environment.

At this point, the program still relies on '/sys/block/<devname>/size' file to fetch the size of the harddisk. This could also be put in as an argument to the program in the future.

My attention is now changing towards figuring out the sector encryption (that I believe exists). savage and I have been exchanging messages about what algorythms may be in use; and he seems to have some good ideas.

At this point, we are trying different theories and trying to understand the suitability.

Hopefully there will be some more news as the days go by ;-)

Attached Files

scan-0.3.tar.gz (20.7 KB, 59 views)

[HanSooloo]

Last couple of days has not yielded too much information, other than some repeated disk imaging and layout analysis using the scan hdd program.

The only interesting find came as a result of getting my new 60 and 40GB hdd's, since I can now do independent tests.

So, without further ado, here are the findings:

1. I used a new 60GB hdd and had the PS3 initialize it. The sector distribution layout is the same as the original hdd. The first sector is the same. There are differences in a few dozen sectors here and there.
2. I also used a new 40GB hdd and had the PS3 initialize it. The sector distribution layout is EXACTLY the same as the original hdd, with a few dozen 64KB marker blocks missing. It almost fits the 2/3 ratio of a 40/60GB hdd disk size ratio we see here. :eek2
3. In BOTH size of hdds, at disk addres 0x5DD000 there is a block of data whose size is DEPENDENT on the size of the hdd. In the 60GB hdd, this block is 6KB long, in the 40GB it is 4KB long. :eek2
4. SO, are we seeing a "complete disk map" data block at this address? Most probably. Does that indicate a database-like file system that needs to have a static disk map that is set at the time PS3 initializes the hdd?
5. The NEXT block after this "complete disk map" ALWAYS starts at 0x5E1000. This is only based on a 40 and a 60GB hdd comparison. If anyone else has installed an hdd LARGER than 220GB in their PS3, PLEASE run the scan hdd program on your freshly installed hdd and send me the results. This will help us understand how the PS3 allocates disk space.
6. A freshly initialized hdd seems to have the following exact layout no matter what the hdd size is (dependent on item-5 further analysis results):
   1. 4KB block
   2. 8KB
   3. 64KB
   4. 6KB <== this one dependent on hdd size
   5. 2KB
   6. "N" number of 64KB blocks, where "N" is dependent on hdd size (196 on a 40GB hdd, 300 on a 60GB hdd)
   7. 0.5KB
   8. 1.5KB
   9. 1KB
   10. 536KB
7. There is some amount of "empty" space left at the end of the hdd IRRESPECTIVE of the hdd size. The empy space is ALWAYS 2,146,914,304 bytes.
8. When a 64MB "all zeroes" .BMP file is transferred to the hdd, the data gets split into 2 major chunks of around 30MBs and several other smaller chunks. I cannot explain the reasoning behind this. Why that arbitrary layout? Why not a big chunk, or a consistent division into smaller blocks? Does this indicate some sort of on the fly disk allocation block adjustment? Who knows...

Well, that is pretty much it now.

I REALLY would appreciate if you can run the scan hdd program on your freshly PS3 init'ted hdd and share the results. I am especially interested in those from "larger than 220GB" hdds.

Just to make sure you don't lose any data, you can always back up your save games, etc to a memory stick or any flash memory device to restore them later on.

Please contact me if you have any other information or would suggest some other approaches.

[GrandpaHomer]

First of all - thanks to PS3News for granting access to DEV forum ...

Originally Posted by HanSooloo Please contact me if you have any other information or would suggest some other approaches.

OK - I have a pile of hard drives here with which I've done a numerous tests, scans and dumps in past couple of days (actually mainly nights).

1. Can you please let me know what scans (e.g. signature) and what dumps (suppose just partial ones) you need?

I'll do it for all drives I have here.

For record - I have PAL PS3 with version 1.50.

2. Would it be possible to include another 2 command line parameters in your scan program:

A) Start position for scan
B) Bytes to scan

It really take AGES to scan 750 GB and 500 GB drives ...

And - one more additional request for the scan program:

It would be great if you can time stamp the verbose messages (like hddcntr is to help estimate remaining time of scan.

If you'd like to make it REALLY comfortable you can even display elapsed time as well as the estimated time (based on the size of the disk or scan) - not in a realtime, just with each verbose message.

Thanks.

Oh - and if you can please print numbers (hard drive sizes etc.) with thousand separators it would greatly increase the readibility of numbers with so many places.

One more additional question - some of the drives I'm using are brand new (virgin) - e.g. never used / formated / partitioned elsewhere - but some others are aready from various places thus used / containing some data on them.

As far as the HDD initialization in PS3 is pretty quick - even 750 GB drive takes like 10 to 12 seconds - it's indeed not blanking the whole drive ... Are you viping the drives before the test and if yes by what utility and to what content (e.g. all zeroes)?