Novalight - very fast tape loading

Anything related to the tools Tap2Wav, Tap2CD, Tap2Dsk, Sedoric Disc Manager, Tape Header Creator, WriteDsk, and generaly speaking tools related to the management of Oric data files and devices.
User avatar
Symoon
Archivist
Posts: 1510
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France
Contact:

Re: Novalight - very fast tape loading

Post by Symoon » Sat Mar 09, 2019 9:14 am

Chema wrote:
Mon Feb 25, 2019 11:36 pm
Well, in fact what I have is the first Atmos ROM version, which does the right thing, and it is not bugged :) It checks for errors and stops loading if it finds them. What the Oric people did, after realizing that there were loading errors still, was to remove the error checking altogether in the second version 1.1b!

That is what the small program does... sets the IRQ vector to point to a small routine that clears the error flag... good job guys! :evil:
Back to this subject.
Actually there are two modifications that were done on the "1.1b" ROM in tape routines:
1- stop checking parity errors during the synchro (bytes 16 16 16 ... 16 16 24)
2- only display "Errors Found" and don't stop the program execution

Chema, I now understand better why you say it was not bugged while I said it was: I had in mind the 1st change, you had in mind the second ;)
BTW that's quite funny to try fixing a loading problem with... A program to load :lol:
I will investigate all this ROM/patch story in detail and try to write an article about it.

Now, a poll: this 'ALC' program is useless with Novalight. Should the WAV generator keep it anyway (for historical reasons or whatever), or should it detect it and remove it from the WAV file?

User avatar
iss
Squad Leader
Posts: 775
Joined: Sat Apr 03, 2010 5:43 pm
Location: Bulgaria
Contact:

Re: Novalight - very fast tape loading

Post by iss » Sat Mar 09, 2019 5:03 pm

For the poll: If it's not too difficult - detect and discard it during WAV generation.
Here is the quick disassembly of ALC:

Code: Select all

LE7B6           := $E7B6
LE867           := $E867
LEE22           := $EE22
        php
        sei
        lda     $FFF9
        cmp     #$01
        bne     L02B2
        lda     $E4B6
        cmp     #$A2
        bne     L02A6
        ldy     #$09
L0293:  lda     L02B5,y
        sta     $0221,y
        dey
        bpl     L0293
        lda     #$21
        sta     $0245
        lda     #$02
        sta     $0246
L02A6:  jmp     LE867

        sta     ($02,x)
        .byte   $BF
        .byte   $02
        .byte   $C7
        .byte   $80
        .byte   $FF
        .byte   $FF
L02B1:  brk
L02B2:  jmp     LE7B6

L02B5:  pha
        lda     #$00
        sta     L02B1
        pla
        jmp     LEE22
It intercepts the IRQ and clears the error flag - so not useful (especially) with Novalight I think.
For historical and archive purposes we can comment the above source and upload it (for instance) on oric.org.

@Chema: I'm interested to have your ROM if possible. Do you know is it factory PROM or custom patched EPROM?
Do you know if it's in the list already here: http://oric.free.fr/ROMS/romident.txt
You can use testfbs testing utility to find ROM's crc32 which I posted here.
BTW, did your friend had chance to try the same testfbs with Jasmin ?

User avatar
Symoon
Archivist
Posts: 1510
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France
Contact:

Re: Novalight - very fast tape loading

Post by Symoon » Sat Mar 09, 2019 8:42 pm

iss wrote:
Sat Mar 09, 2019 5:03 pm
For the poll: If it's not too difficult - detect and discard it during WAV generation.
Here is the quick disassembly of ALC.
It intercepts the IRQ and clears the error flag - so not useful (especially) with Novalight I think.
This leads to another question: would it be useful to have an option to insert ALC before the "normal speed" Novalight loader? What I don't get yet is if the 1st generation of Atmos who might detect parity errors during the synchro, may in certain conditions have loading error mistakes without ALC, even with a good sound. Maybe this is why Chema has trouble loading it...

Edit: mmmh, I probably tested it on an Atmos with the 1st version of the ROM already (no idea which of my machines have this 1st ROM version), so this is probably useless.

User avatar
Symoon
Archivist
Posts: 1510
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France
Contact:

Re: Novalight - very fast tape loading

Post by Symoon » Sun Mar 10, 2019 11:10 am

1st working tests with user memory relocation!
Sadly doesn't help Psychiatric loading - it was using page 1 but I suspect it uses specific loading routines or protection.

Anyway.
Now, removing ALC and optimizing dictionary. I think I need some holidays ;)

User avatar
Symoon
Archivist
Posts: 1510
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France
Contact:

Re: Novalight - very fast tape loading

Post by Symoon » Tue Mar 12, 2019 10:57 pm

The (now useless) ALC program is now detected and skipped by Novalight!

Seems like French editors (Cobra Soft, Loriciels, Infogrames) enjoyed modifying it to store a part of their protection; in such cases of course the modified ALC program is kept.

User avatar
Symoon
Archivist
Posts: 1510
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France
Contact:

Re: Novalight - very fast tape loading

Post by Symoon » Sat Mar 16, 2019 1:54 am

After a rest, I'm back to this dictionary matter.
Reading my existing code again, I see something interesting from my thoughts of 2 years ago.

Once the code considers it has chosen rightly the 14 values to put in the dictionary (wrongly or not actually), it sorts them again in the dictionary to make it optimal in a quite obvious way: you need to re-assign each selected value according to its frequency only. The way you selected them doesn't matter anymore: if you selected 500 "A", 30 "B" and 200 "C", you will obviously minimize the newly inserted length by assigning the shortest time to "A", then "C", then "B".

If you look at your Novalogic table BTW, it seems each time you choose to invert the selection order of two values, the 2nd one actually has more occurencies (is it just by chance, I don't know! Only exception is the last "TRUE" which I actually don't understand why it says "TRUE" ;) ).
NekoNoNiaow wrote:
Sun Mar 03, 2019 3:46 am
Image
You also can see that, from the new signal length point of view, it's obvious that it would be much better to use "I" (frequency 940) from 3rd or 2nd, to 1st position, than K (frequency 860)
=> 940*4+860*6=8920, is better than K 1st and I 2nd: 940*6+860*4=9080

So I wonder if the problem solution could not actually be simplified/optimized:
1- remove the biggest frequency*gain for the first 13 values: if a choice had to be inverted, it will be selected next anyway
2- 14th value has to take the next choice into consideration to select the best one (if you don't, you may drop the best choice, just like my very first example with "A" and "B") EDIT: mmmh, no, this was another problem actually (I was not using gain, but initial length only)
3- the gain is finally optimized by sorting the 14 values by frequency

... Or am I again on a totally wrong track? :lol:
EDIT: going to bed, I'm actually a bit lost again in how to initially select the right 14 values! I tend to think we are close to perfection with your initial selection: frequency*gain (which means calculation for each dictionary entry), and with my final optimisation which is just sorting again by freqency the 14 selected items, avoiding this "do I switch my choice or not" step during the selection.

User avatar
Symoon
Archivist
Posts: 1510
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France
Contact:

Re: Novalight - very fast tape loading

Post by Symoon » Sat Mar 16, 2019 10:05 am

Symoon wrote:
Sat Mar 16, 2019 1:54 am
I tend to think we are close to perfection with your initial selection: frequency*gain (which means calculation for each dictionary entry), and with my final optimisation which is just sorting again by freqency the 14 selected items, avoiding this "do I switch my choice or not" step during the selection.
Ok, just tried this solution, and it's actually better than my previous "use an average value to compute the gain"! ;)
=> Zorgon is 0.01 second faster, choosing $CE (frequency 271), over $05 (frequency 613) with the previous method. That may sound ridiculous, but it just shows it's better, and compensates about 50% of the time lost with the new Novalight structure. I like that :mrgreen:
Maybe with other programs the gain will be better.

EDIT: arrrgh, nope, 3D Fongus is slower with this new version. Something is still wrong somewhere. I think I got it: the last value to be chosen must be compared between a gain, and a length..
For instance if I take A and leave B out of the dictionary, I will gain what I calculated for A, but I will leave for B its inital length, not gain. The last comparison is different from the previous ones.

User avatar
Symoon
Archivist
Posts: 1510
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France
Contact:

Re: Novalight - very fast tape loading

Post by Symoon » Sun Mar 17, 2019 12:47 pm

Ok, after another evening/morning working at it, I realise there is no simplified way to be optimal with the dictionary: each attempt (tried about 8 of them) gave better results on several programs, and inferior results on other. None was "always better".

The only solution IMHO would be to compute and combine all possibilities. Forget it, the gain is too small for such a complexity to deal with :lol:
So I will keep the code as it was, avoiding to complexify it too much for a random minor gain. That being said, anyone willing to give it a try as we did with NekoNoNiaow, please do ;)

User avatar
Symoon
Archivist
Posts: 1510
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France
Contact:

Re: Novalight - very fast tape loading

Post by Symoon » Sun Mar 24, 2019 2:44 pm

Novalight 1.2a just released.
https://sourceforge.net/projects/euphor ... Novalight/

Holds the following changes since previously released v1.1k:
V 1.2a (released 2019-03-24):
- auto-relocation in memory if page 1 is used by the program to load. /!\ heavy ASM/C dependency
- ALC program is now detected and skipped (it's a now useless patch for old tape players)
- public release, cumulating all previous changes from v1.1l to v1.1n.

V1.1n (not released)
!!! incompatibility with ALC program and Loriciels software using it as copy protection!
=> new version required to store the dictionary in page 1.
- stopped using page 2 to store the dictionary, it caused compatibility problems (especially with Loriciels
software). This leads to split Common Area 0 with additional 0a and 0b parts.
- this means the old loader is obsolete, so "old loader" option removed. Same for "no loader", as the old
loader was required to load a Novalight file without loader.
- "multipart booster" option allows to use a reduced Common Area 1 if the Kernel and Common Area 0 are kept
intact. Requires re-loading of Common Area 0a at the end of a part.
- optimized the dictionary by not only taking care of the time wasted by the most frequent isolated
bytes, but also by taking care of how much time they use with their new encoding in the dictionary.
(sometimes, it is worth selecting a byte that occupies a bit less time in the original signal,
but that will occupy much less time with its new encoding!)


V1.1m (not released)
- removed the $24 starting byte. This was here for historical versions and had become useless.
- the room saved by this removal (7 bytes) is now used for a new change: in an uncompressed byte,
the bit sequence '1111', which should be coded by 6 + 6 samples, is now coded by 7 samples.

!!! OLD LOADER not working anymore.

V1.1l (not released)
- removed 3 useless bytes in Common Area 2: in ROM 1.0, clearing the status line is
done by the CLOAD (it was historically required because Novalight used to display
its own "Searching", but this has been removed before the 1st public release)
- added 'multipart booster' option
- corrected a bug: oldloader was ignored if combined with F16 speed
- just for the joke, modified the options letters a bit ;) (-s, -i, -m, -o and -n)

'No Loader' is superior to all the other options: standard speed, old loader and multipart booster will be ignored.
'Old Loader' will ignore 'Multipart Booster', as the booster only works with the normal loader.

Post Reply