Sorry for the delay, I made some last-minute cosmetic changes. Here are all the tools I made, and something like a dev-log:
At the beginning I wanted to make a general cpk repacker. What I noticed is that it is not necessery to compress the files, for example all the cpk files (in the archive) are stored uncompressed, like in the mapXR.cpk (which contains only cpk files). However, you still need to edit the Toc values (offsets). Since it's easier to work with smaller files, I worked with the cpks from the demo, and I managed to make a 1:1 copy of the map archives. The rootR.cpk is almost the same, except the directory value is not constant. Everything worked fine in the demo, but because the files weren't compressed, the rootR was a bit bigger.
Then I tried these with the full game files, but there was a problem I realised when testing. In the Toc, every file has a 64bit offset. Theorically, the game could use files that are over 4GB (the size of all files in rootR is about 3.4GB). However most standart tools in programming (eg, fseek, ftell) use a 32 bit signed integer (2,147,483,647 bytes max). Although you can work around this by using 64bit values, we don't know how whether the game supports it or not (since it was probably not designed for this large files). I tried the game with a rootR repacked with 64bit offsets, but it only showed the black screen, so it probably uses 32bit signed offset too (or I made something wrong ) Also you have to add padding after the files so it could be divided by 32 (AlignValue).
So if we have to keep the files under 2GB we have the following options:
compress all files: we have to reverse the uncompressing feature in utftab, I think i could be done, but the source files have a very few comments or info.
replace the original compressed file with the still compressed one from the english one: if you edit the unpacker script at around line 90, you can unpack the files, but for repacking you will need both rootR, or the sizes for the new files for setting the correct sizes in the Toc. I will try this method if we can't compress the files
append the modified files to the end of the archive and point their offset to the new places. This is probably the easiest method, but you have to be very careful to not go over 2GB. You can't replace large datafiles like the voices with this, but if you just add the texts and textures it should still be a bit under 2GB. I made a working tool for this.
And while I was writing this I just remembered that the english scrips are not in the rootR but next to it, and I forgot to include them. <goes to test it> Well, the only thing I noticed is that Press X is also in english on the title screen. By the way, this new eboot is fantastic!
So, here are the tools:
cpkforp 1.1c with gui
This is what most of you will need. It can add files to a cpk based on a file list. For this, you will need the original rootR (you can rename it if you want), the files from the english version (with keeping the original directory order), a special filelist with the files and file IDs in the archive and another one with the files you want to add. Run the program without any parameters for help. I also made a nice (?) gui for starting it for those who don't like command lines. However, the message log only updates after the program has finished. Well, better than nothing. About the lists, of course it would be better to read that directly from the cpk, but I wanted to keep it simple as possible so I made another program to do it.
It's been a long time I made something like this in C, it is not fool-proof but it works and almost fully commented so you don't have to spend hours thinking about what a line does. I post this for learning/development but you can use it as long as it is not commercial.
cpkbuild beta with tocdump
This was the original program. Tocdump dumps the header parts of a cpk and cpkbuild makes a cpk file based on these and the new files. cpkbuild's code was upgraded to cpkforp, however tocdump is actually a modified version of hcs' cpk unpacker. I broke something in tocdump when I made it work with rootR, so it only works with that file now, before it could work with the mapXR files too. Recommended only for devs (hint: turn on verbose mode)
I made these with tocdump, you will need it for cpkforp. The first file is the list of all files, in the second are only the new files. You can copy any line from the firt one to the second as long as the final size of the cpk is under 2GB (2,147,483,647 bytes).
no problem, I just added to files to the root.cpk of US version (SysString ones that are missing,) now that I do a tocdump on this new file, I can't see them in there but the file is modified and I can check the new files in there with a hex editor, added to the end. I give the files my own ID (because they were new, added highest value after the files in the original), now I'm moving it to PS3 to test if it works or not, if not I guess I have to replace stuff in the Japanese cpk.
ok, that didn't work, stuck in endless loading screen, better than black screen! now I will go and inject stuff into japanese cpk, maybe we make a undub
Last edited by Lando43; 06-21-2012 at 01:14 PMReason: Automerged Doublepost
I wanted to write so much stuff that I forgot to add some useful info When you add files to a JP rootR, the english SysString files should be also placed in the directory of the new files (as it originally should be). So it should look like /Sys/en/SysString.bin and /Sys/ja/SysString.bin. If the program finds a file that is on both list, it will use the file from the directory of the US files (where you extracted the US rootR). If not, skips the values (offsets ,sizes, ...) in the Toc. Because the name could be anything, what the program checks in the filelist is the ID, the name is used only for opening the new files.
About the IDs, there are two kinds of sorting in the cpk. In the Toc, file info is sorted randomly (or maybe based on some value) and this is how they are in the list files also. The file contents however are (originally) sorted by the file ID (which is in the Toc), so the first file after the content offset will be the file with id 0, then the one with id 1 and so on. If you want to replace a file, you should keep its original file ID, the program only modifies the size and offset values. It does not add new values to the Toc. It could be done, but probably the game would not use it anyway.
So the IDs must match in both list, sorry if I wasn't clear. There are some unused information in the file (like ContentSize), so I'm not sure whether the game uses the ID or not, but this is needed in the list for now (at least until I replace the ID check to strcmp )
You can also check the file on PC with the unpacker script from a few pages earlier. If it can extract the files without error, then it should work with the game too.
well technically it is an undub already tocdump has some bugs, but it should work for both the original rootR and the modified one
Last edited by mamu; 06-21-2012 at 01:24 PMReason: Automerged Doublepost
I wonder how the cpk files are read; if they are just the container of files and the place of files in them is not important, then we could add the big files to map1R. But the original files would be still there (just ignored by the game). Also I wonder how DLCs or patches are working. Maybe we could replace the files with it too. I will make another tool later for replacing files in compressed form (I already have an idea) but I'd like to wait for what others think about the recompressing before spending the week with programming.
Edit: ignore the part about the original files
Last edited by mamu; 06-21-2012 at 02:19 PMReason: Automerged Doublepost
that revolution translation tool for the demo could replace everything, I wonder what they did. it didn't patch files, but patched them in memory I think.
edit: those leaked cpk 1.1 tools let you compress, if you can use the dll in your program you can do it.
edit2: I couldn't get those cpks inside the cpk (menus ones) packed in because of 2GB error, but I could get all the folders except "snd" and "btl" packed. maybe I could pack in "btl" too gotta try again. it works with the new cpk and it's almost all english with the new eboot but still lots of things in the menus are gibberish/null font.
Last edited by Lando43; 06-21-2012 at 03:24 PMReason: Automerged Doublepost