This should be a constrained enough use case that you can use the alternate character set, but you need to limit your use to ASCII characters with code up to and including 111/0x6f/"o". So upper case letters and numbers should work fine.
Using the great
iss Oric Fonter (click normal-Oric), you can see the lower line of characters is what you need to avoid using.
But looking at your picture, it looks like the garbled parts are actually pieces from your character and tile-set down in the standard character set, so what I think might possibly also be happening is that the first text line isn't set correct for using only the alternate character set (assuming that is what you want). Could it be that the routine for copying the whole standard set has written into screen memory, and there are left over character set data in that line? If so, try only copying upto the start of the text screen address.
Character set data only use the lower 6 bit in each byte, and will easily look like attributes by the ULA if is pointed to it as text. A trick if one wants to experiement with double use/overlapping memories is to set one or both of the upper bits in each byte of character bitmap so the ULA doesn't interpret them as attributes. But I don't think this is useful in your case.