PS3 HDD Contents

[GrandpaHomer]

GrandpaHomer: before your next release, can you please give me a chance to incorporate your latest changes so we don't have different versions of the code out there?
My idea is to incorporate your 0.3.2 changes into a 0.4.2 and from then on go on a synchronized release schedule.

Indeed! I don't want to run two parallel versions as well ...

That's why it was called an interim version. I have a couple more things which I forget to mention / fix / implement in last 2 interim versions:

1. The length calculation on results.txt is incorrect! (see also bellow attached results-1.txt and resultsOK.txt for comparison). I've already amended that in my 0.3.3 version here but you can do in yours as well. Just amend the following line:

length = endaddr - straddr;

to:

length = endaddr - straddr + 1;

2. As already mentioned in 0.3.2 notes I was having problems on some systems to use sprintf to access the drive using your formula with hddname. It was producing incorrect device name - instead of expected "/sys/block/sdb/size" it returned "/sys/block/sdf/sys/block//size" and indeed end up with error.

I was trying to amend that but unsuccessfully so far so I've ended with putting the fixed values back to the source on both places where the device name is "generated". Optionally - it could be sorted by using the another command line parameter to switch between the default and requested SDx device but I'd prefer to avoid that.

[GrandpaHomer]

GrandpaHomer's code is used mostly as is in this version of the code
Big thank you goes to him!

I suppose you've used the code from 0.3.1 ... :??

The seek part of code is bugged and it's now amended in above posted version 0.3.2

We should somehow merge our current latest version, test the final product and from now on try release only the "official" version. But at least I've dusted a bit my C skills and I have now better understanding of how the program works instead of being just pure consumer of somebody's products ...

OK - I've just found out that you've actually fixed already both "length-1" and seek bugs so ...

Two more quick questions:

1. I believe that it's not necessary anymore to specify -lgmp during compile?

2. I suppose you're using dd for making the images of the disk?

[HanSooloo]

I suppose you've used the code from 0.3.1 ... :??

The seek part of code is bugged and it's now amended in above posted version 0.3.2

We should somehow merge our current latest version, test the final product and from now on try release only the "official" version. But at least I've dusted a bit my C skills and I have now better understanding of how the program works instead of being just pure consumer of somebody's products ...

OK - I've just found out that you've actually fixed already both "length-1" and seek bugs so ...

Two more quick questions:

1. I believe that it's not necessary anymore to specify -lgmp during compile?

2. I suppose you're using dd for making the images of the disk?

I found the scan length bug myself and correceted in 0.4.1. It's good that we both saw it and corrected it; great minds think alike.

On the sysfs device size file, there was actually some bad char pointer arithmetic that was causing it. I had forgotten to include an extra byte for the terminating NULL character, which I changed in the 0.4.1 release. I actually came across the same problem with

/sys/block/sdf/sys/block//size

So we don't need to have hard coded device names anymore. The -h option should take care of it:

# ./scan -hsdf

Now both bugs are fixed!

Allright kids, maybe we are onto something here. I have been formatting my poor hdds with all sorts of filesystems lately and taking images of them to compare against the PS3 FS.

Well, it seems like the layout "resembles" ReiserFS. Just to show some samples here:

PS3 HDD Layout:

Start        End         Length
0x0B7CD000   0x0B7CDFFF  0x10000
0x16F85000   0x16F85FFF  0x10000

ReiserFS HDD Layout:

Start        End         Length
0x0B7CD000   0x0B7CDFFF  0x10000
0x10007E00   0x10007E0F  0x00010
0x16F85000   0x16F85FFF  0x10000
0x18007E00   0x18007E0F  0x00010

These sections are from the first occurance of a repeating section. Notice that the 0x10000 length sections start at the EXACT same address on a 60GB hdd.

Interesting thing, too, is how ReiserFS's 0x00010 section is absent in the PS3 layout. Maybe it is consolidated to the beginning of the hdd. Who knows?

The next step is to copy the same file to both PS3 and the ReiserFS formatted hdd and see where the changes are.

Keep checking back in kids...

Well, maybe after all we had a false alarm It seems like I did not properly zero the hard disk before putting the ReiserFS system on it. Prior to the ReiserFS format, there were PS3 FS signatures on the disk, that resulted in the merged layout on the disk. OH WELL...

But maybe, just maybe, they are using a HUGE block size of 64K. Anyway, I am sorry if I had people's hopes up. Will try to come up with better news later.

[GrandpaHomer]

I have some scan results here from past couple of days which I've not posted yet ... I have also (compressed - each between 2 and 2.5 MB) full images of drives in case you'd like to compare / view results. So - even if you don't want to fiddle with you PS3 drive you can still participate ...

I've used 2 of the smallest hard drive the PS3 can still handle - 2.6 GB & 3.2 GB. It's optimal due to the faster scanning and imaging and less disk space to go through when you're looking for some specific signatures / data.

I've performed various tests resulting in different images / scan results. Each disk went through 5 stages described bellow:

1. Wiped / zeroed - called Init0 (there is no scan neither image as far as it's simply full of zeroes).

2. Initialization / formating by PS3 up to the prompt "Press x to restart". At this point the PS3 was powered off by the back power switch! (Of course AFTER all the disk activity finished) - called Init1.

3. The same like in point 2 but this time the PS3 was shut down using the front panel power button. There is definitely some disk activity AFTER the power button is pressed but both default scans shows the same results so we need to compare the whole images to seek differences - called Init2.

4.The PS3 is restarted using the x as prompted and SWITCHED OFF at the moment when the XMB shows up and the start up sound finish. It needs to be BEFORE the ps3 will connect online - I've also disconnected the network cable for that - called Init3.

5. The PS3 is restarted using the x as prompted and let to connect online then switched off using the front panel power button - called Init4.

There are clear differences between Init2 and Init3 as well as between Init3 and Init4.

Also - apparently due to the limited space on the hard drive - the PS3 will actually (despite the initial formating without any errors) not start up correctly with 2.6 GB hard drive ... It will "fiddle" with the disk for a while (which results in even longer scan results in Init3 than for 3.2 GB drive) and then simply beep 3 times and shut itself down to "red flashing mode" ...

This is also the reason why there is no Init4 scan result for 2.6 GB drive.

>HanSooloo:

Another couple of additions to scan program:

1. Can you implement the -c command line from my last version?

2. Also - it would be useful to include in the results file the first 16 bytes of every reported block (especially when you're looking for NOT matching signatures).

3. And finally - as already mentioned before) - the size of the drive.

I'll finish the scan of remaining drives and post result here as well. As agreed it will be the "Init2" type.

Then I'm heading toward filling up the drives (mainly the smallest ones) with some pre-defined files and I'll be looking for them in the images.

[HanSooloo]

Thanks for the new scans. I will look at them immediately. I am in the process of finishing up Version 0.4.2 and have added almost everything you asked for.

I have a couple of questions about the implementation and would like to talk to you live. If you can drop by EFNet, msg me (HanSooloo).

[HanSooloo]

A new version of ScanHdd is posted. Following is the ChangeLog:

[FONT=Courier New] 2007-04-27 0.4.2 Han Sooloo, GranpaHomer
> Added new parameters based on feedback from GrandpaHomer
and changed a number of the old parameters
-s This parameter now tells the program on how to proceed
after it dumps all the summary information to the screen
0: Do not start, 1: Start, 2: wait for a "y" from stdin
Remember that this one used to take a CSV list of scan signature bytes
-t This parameter now takes a file name as input. This file
is a binary file with the scan signature in it.
-r File name where "results.txt" should be directed.[/FONT]

[GrandpaHomer]

A new version of ScanHdd is posted.

Just missed you over at EFNet ... I was bout to MSG you with some of the bellow details.

I'm having some issues with 0.4.2:

1. Like the help but it messed up all my scripts as far as I'm using the scan mainly without any params ...
So - I suggest to put help on "standard" --help option ...

2. For some reason - the scan now get stuck on "0s : harddisk position is : 0.00" and result.txt is completely empty, not even column's headings ... :?? (it WILL read the first batch of data to buffer and then freeze)
This only happen when default sig is used / -t is not specified!

3. Also - I'd suggest to change the default value for -s to 2.

4. It's also a bit slower now due to the more code and multiply conditions in the main loop. We may need to look for some code optimization / multipass scan as discussed ...

and some additions:

5. Can we add also the program version in the results.txt after the date / time? (Maybe also Moon phase, local weather, NASDAQ ... )

6. As discussed as well - including of first 16 bytes of new block - preferably both in hexa and ASCII (printable)

7. Counting the result lines and both mark then in the result file as well as report the number of results after scan will finish:

 420s   : Scan harddisk analysis finished.

 8 results found

and:

Nr.   Data Start Address      Data End Address      Data Length         First 16 bytes        ASCII
0001  0000000000000000

If you have any additional question let me now here or on EFNet (you might need to "knock" couple of times as I'm gone to lay down for a bit ...)

[GrandpaHomer]

Oh - there is also something funny about the hddcntr display - at least I hope it's just the formating (the buffer size is set to 111000000):

 153s   :       harddisk position is    : 999000000.00

 168s   :       harddisk position is    : 1110000000.00

 182s   :       harddisk position is    : 1220999936.00

 197s   :       harddisk position is    : 1332000000.00

 212s   :       harddisk position is    : 1443000064.00

 227s   :       harddisk position is    : 1554000000.00

 242s   :       harddisk position is    : 1664999936.00

 257s   :       harddisk position is    : 1776000000.00

 273s   :       harddisk position is    : 1887000064.00

 288s   :       harddisk position is    : 1998000000.00

 303s   :       harddisk position is    : 2108999936.00

 319s   :       harddisk position is    : 2220000000.00

 334s   :       harddisk position is    : 2331000064.00

 350s   :       harddisk position is    : 2441999872.00

 367s   :       harddisk position is    : 2552999936.00

 383s   :       harddisk position is    : 2664000000.00

 399s   :       harddisk position is    : 2775000064.00

 416s   :       harddisk position is    : 2886000128.00

 432s   :       harddisk position is    : 2996999936.00

 438s   :       harddisk position is    : 3108000000.00

See the attached full log.

But - the biggest problem - the results seems to be all messed up as well ...

As far as default sig does not work I've used the (attached) zero.bin file. See results for 0.4.1 and 0.4.2 ... (0.4.1 is the same as 3G2Init4 from my last results post).

Also - as mentioned before - the main loop is now 12.22% slower. No big deal so far ...

To all others - I suggest to use 0.4.1 for now till we'll manage to iron out the 0.4.2 version ...

[HanSooloo]

The bad hdd buffer counter bug is fixed. Floating point display issue is still there. Don't know about the performance, but everything else is working now.

2007-04-29 0.4.3 Han Sooloo
> A bug with the hdd buffer counter not being incremented has been fixed. The
variable was not being set properly during the initialization section of the
code. Thanks (again) to GrandpaHomer for pointing this out.

> There is a known issue at the moment, where when the hdd counter is being
incremented, sometimes the displayed value is in some weird floating point
format (e.g., 1119999996.00). This needs to be investigated. But regardless,
the code is counting in normal decimal form; it's just the fprintf() display
that is wrong.

[GrandpaHomer]

Thanks for the update.

I've done a couple of tests with 0.4.4 and here are the results / comments:

1. The printing of the harddisk position during scan seems to be fixed now.

2. The initial seek is still messing the whole scan up if the start position is 0. As suggested above - it needs to be performed ONLY when start position > 0.

No additions to the wish list this time (only what I've mentioned already in previous posts and chat) - I don't want to "overflow" your todo list ...