Ok, it's an hard work but we have to start in order to obtain some progress, so here's the place where to discuss/share our consideration on the PKG file structure.
I attached a picture of the pkg header of Flow USA, so we can refer to some real offsets when discussing about it.
I signed the first, easy and noticeable structure:
1) Blue (offsets 1C - 1F): Global pkg file length:
-- It's sure, total file lenght = 131.017.248 bytes so 7CF2A20
2) Green (offset 2C - 2F and F0 - F3): contained file length.
-- It's a supposition, contained file lenght should be the pkg file lenght - the pkg header lenght, so could be 131.016.832 bytes so 7CF2880
3) Red (offsets 30 - 5F): Code name of the Demo/Game:
-- It's sure, there are 12 double words reserved.
Please, share your findings here so together we could find something useful to understand those interesting files
This is much help but... I expect that the sizes for offsets mentioned at 0x01c, 0x02c, 0x0f0 are not 32 bits, but actually 64 bits. You'll notice the preleading bytes are 0s which help support this, but does not really confirm anything.
Have you had any success isolating any of the "known" information in the headers yet? Is there enough packages out there that we can determine if/where the region is stored other than the text field?
Dword 0x0c has a seemingly random 4 or 5 value. Not sure on the pattern yet. Dword 0xc8 almost looks promising. Of about a dozen packages I've seen it has a value of 2 for premium/pay content, and a value of 3 otherwise. Only two of these packages don't conform to this, but I can't verify if I've been correct on premium analysis. For instance, is GTHD to be considered premium? I'm pretty skeptical about 0xc8 but who knows, and I'm no expert.
NDT great idea making this thread (hope we don't get questions like can we make a boot loader with this)
mrbeans I very much so like your insight into this, I had intially thought that variation to simply be allowed in order to change the hash.
Would it help if we knew the files contained inside the pkg? Would it not help because it is compressed? It is in my opinion that the demo's do have eboot.bin and I can also bet that games with a "credits" section will have a text file (like fight night does) (so we can search with names). I don't know if that helps any. If I can help let me know how.
It's hard to know exactly what is in a package file, but it would certainly help. For some reason I have a gut feeling that these files may not be fully encrypted, but just compressed and signed. The signature, of course, need be some form of public/private key instead of a simple hash. I'd guess the signature to be within the 0x60-0xbf section within the header, or alternatively 0x100-0x13f. I'm prone to believe content starts at 0x140 as pointed to by the qword at 0x20.
Also, I believe you are correct and there is an eboot.bin although I am unsure if it is encrypted in the same style as the blu-ray disks, or if it is a regular signed exec. If it is in the style of blu-ray it'll need an encryption key somewhere, which I'd guess in the opposite section of the above 0x60/0x100 choices. Of course, there is some heavy guessing here.
With respect to known text (credits) will be hard to search for because of compression/encryption. I have no idea what algorithms have been applied. I haven't seen any text fragments that you would see on straight compression of a text file, which goes completely against my gut feeling above... unless it is a compression scheme using a bit stream and not a byte stream. Someone with PSP knowledge can probably give us some starting points on which compression and encryption algorithms to look at. I expect there will be some overlap.
It'd be useful if we could collect as many different versions of the same program to compare the headers to find things like region codes, for instance the differences between the same demo from the NA store and the JP store... You are welcome to check this out.
I have been thinking about hex editing values with one of the (small) working demos to figure out which regions can be changed before errors are triggered. It makes sense that some validation is stored in the file, as there appears to only be a single connection for demos (no additional credit card transactions.) Knowing what triggers the errors and what the different errors are may potentially be useful... You are more than welcome to try this too , but be warned it is tedious. The smaller the file the better.
To be honest though, I think we have better chances of attacking the encrypted/compressed licence agreement in the firmware update files. These files are tiny, and the plaintext isn't to hard to come by. It'd likely give insight to the PKG files, and the signed executables themselves.
It doesn't help that I am generally lazy, and a lot if not all of the above is really theory and guesses.
I just got the link for a pkg for the keys.bin of a psp game, it is very small and most of it is 0's, but since I also have the keys.bin file that it "extracts" to, it should help a bit to find out the structure, no?