How about you do the things we have asked so we can further diagnose what went wrong so we understand the situation and provide the best help possible. Worst case scenario, you will have to downgrade with a hardware flasher.
But I need to know if the vsh.self on your ps3 is the CEX or Shop version. You can this out by going into system settings and then to system information. Hold down 'L1', 'R1', 'left d-pad button', 'square' for about 3 seconds and then immediately press and hold down start. Somewhere on the first line it should either say 'CEX' or 'Shop'. Can you tell me which one it is?
Last edited by ps3hen; 02-05-2012 at 02:26 AMReason: Automerged Doublepost
Ok your VSH is 'Shop' then you can't install CEX firmware under normal circumstances. Unfortunately my PS3 recently died, so I can't replicate the scenario and give a definite answer. This is what I would do, in the order I would have done it( I am assuming you have a PS3 with a NOR flash. Don't do this If your ps3 doesn't have a NOR flash) :
1/ (this is most likely not to work, but still worth a shot) I would use PS3MFW Builder to patch an (original) OFW 4.00 pup with (and only with) the "Patch firmware for installation shop/promo unit". IF you don't have that option in PS3MFW Builder you can download the task file for that here: "https://rapidshare.com/files/2110721203/patch_shop.tcl".
Once downloaded just put it in the task folder of PS3MFW builder and patch the OFW 4.00 pup. The modified pup you created should be installable from the XMB. Don't worry if this doesn't work it'll just give you an error and nothing bad will happen. If it does work your ps3 will be CEX(retail) and on OFW 4.00.
2/ If you have two hard-drives which are compatible with the ps3. I would do a HDD switch, (again this may not work). While the ps3 is off, take out the HDD you are using the ps3 with in demo mode, and put in another one (this other HDD must be blank or come from another PS3 or only have retail FW on it, also this HDD hdd will be formatted if it has not been used on this PS3 before) and turn on the ps3. It should prompt you to insert a usb stick with 4.00 or later FW. Put a original (retail) OFW 4.00 in the PS3/UPDATE/ directory on your USB. Follow the prompts, It will ask if you want to format the HDD, say yes.
Once its done it's preparations it should then install OFW 4.00 and when it finishes you will no longer be in demo mode. If this did not work go to "3/" otherwise keep reading. Now that you have one HDD booting into retail FW that works, swap this HDD with the Demo FW HDD. Synchronise your trophies with PSN and copy all your saves to a USB stick.
Swap the HDD to the Retail FW one. Make the same user accounts that were on the other HDD copy the saves to the relevant accounts. Then sign into PSN on the accounts you just created, Synchronise your trophies with PSN on each account that has a PSN account. Your PS3 is now running OFW 4.00 with the saves and trophies from your old HDD.
3/ If "1/" and "2/" did not work I would resort to downgrading my PS3 with a hardware flasher like the E3 Flasher. If you want to keep you don't want to format the original HDD (the one you are currently using with the DEMO firmware) use another hdd during the downgrade to 3.55 process. Once you have downgraded to 3.55.
Turn off the PS3 put swap the HDD already in the PS3 with your Demo 4.00 HDD. It'll say the PS3 cannot boot properly and require a re-installation using PS3 FW 3.55 or above. You can either Install OFW 4.00 or my 4.00 HFW v3. But if you still want to have homebrew you should install Rogero's 3.55 CFW v2 (most likely the same firmware you used in the downgrade procedure). Once installed you should not be running in demo mode anymore