Normally it do not work in case of the system ios does not support DL discs (backups).There for you need a custom system ios to boot dlīs via the rescue menu.
I have readed a post in the past about that there is a modchip out who can boot DLīs via RM but i have never tested it. If you would have installed my RescueMii-Core then you now would be able to boot DLīs via Rescue Menu.
And only installing a custom ios does not do the job.You first need a custom SYSTEM ios to be able to boot Hombrewed discs via the Rescue Menu.
Because if you insert a disc the drive read and send the data to the Wii and the Systemīs IOS will check the data and verify the signing.
So at least even with a modchip there are a few things they need to be done from the consoles system ios (driver of your FW) such as check signing (as stated above) and maybe also handling of a DL disk.
If your systems driver is not hacked so that the sys knows how to handl a Backup or a Backup from a DL disk you can not boot it without a modchip.But i dont know why most modchips have a problem with booting DLīs via Rescue Menu.
Last chance is to try Original SSBB (US).