Novalight - very fast tape loading

Anything related to the tools Tap2Wav, Tap2CD, Tap2Dsk, Sedoric Disc Manager, Tape Header Creator, WriteDsk, and generaly speaking tools related to the management of Oric data files and devices.
User avatar
Dbug
Site Admin
Posts: 2900
Joined: Fri Jan 06, 2006 10:00 pm
Location: Oslo, Norway
Contact:

Re: Novalight - very fast tape loading

Post by Dbug » Thu Feb 21, 2019 12:15 pm

Jack_Free wrote:
Thu Feb 21, 2019 11:14 am
It does not work under WIN7 64bit.
Is there a solution to work on this system?
Thank you.

error Access Denied
Which version and command line are you using?
When I run novalight_1.1k.exe from the command line I get the list of parameters, so that tells me that at least the compiler used does work fine on Windows 7 64.

User avatar
Symoon
Archivist
Posts: 1606
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France
Contact:

Re: Novalight - very fast tape loading

Post by Symoon » Thu Feb 21, 2019 1:17 pm

IIRC, it's compiled in 32 bits (I'm not a professional of compilers so forgive the potential awkward way to explain).
32 bits is supposed to work on 32/64 bits Windows command lines, but not on real old MS-DOS.
Compiling in 16 bits would allow old MS-DOS + Win 32 compatibility, but not Win 64.

So for instance, I suppose things like DosBox must be avoided.

Jack_Free
1st Star Corporal
Posts: 11
Joined: Fri Feb 16, 2018 11:33 pm

Re: Novalight - very fast tape loading

Post by Jack_Free » Thu Feb 21, 2019 3:39 pm

So I apologize, my mistake, I downloaded another program that is on the top page.

Download Latest Version
wrtdsk23.exe (56.8 kB)
  Get Updates

That made me nervous, novalight_1.1k.exe works, of course.

User avatar
Symoon
Archivist
Posts: 1606
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France
Contact:

Re: Novalight - very fast tape loading

Post by Symoon » Thu Feb 21, 2019 3:58 pm

No need to apologize, good news ;)

User avatar
Symoon
Archivist
Posts: 1606
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France
Contact:

Re: Novalight - very fast tape loading

Post by Symoon » Thu Feb 21, 2019 10:28 pm

Symoon wrote:
Sun Feb 17, 2019 9:17 am
I found something whose decoding could fit in 6 bytes/8µs: back to one of the starting ideas of Novalight which was bit compression.
The 1111 sequence is coded by 6 + 6 samples. The idea is to use 7 samples instead of 6+6.

Ok, just got to:
- modify the signal generation and confirm if it's a positive change
- remove the 7 bytes and see if it still works
- add the 6 bytes for decoding
- test
Well, 1st working test tonight, success on emulators and real Atmos.
Still some work ("old loader" needs to be adapted) and tests (by changing the few bytes in the code, I made about 7 mistakes...)

The good news is that it does improve the loadings of big file by about 0.3s, and that the added bytes in the loader have no negative impact on small files, which is good since, on small files, making the loader a little bigger is sometimes not compensated by the little time saved on the short program to load. For instance my usual HIRES screen is loaded at the exact same speed than before (ok, actually 0.001s faster :lol: ).

User avatar
Symoon
Archivist
Posts: 1606
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France
Contact:

Re: Novalight - very fast tape loading

Post by Symoon » Sat Feb 23, 2019 1:20 pm

Are there any mathematicians aroud? I think my dictionary in Novalight can be optimized.

What I'm doing now (the RLE compression has already been performed and the concerned bytes are ignored):
1- calculate the time taken by each uncompressed byte in the Novalight signal, which is: occurencies*encoding time. For instance, if "Z" lasts 19 samples, and is present 100 times in the TAP file, its total time is 19*100 = 1900
2- the dictionary has 14 entries, so I take the 14 highest "total times" calculated. This way I'm removing a maximum of time from the uncompressed signal.
3- in the dictionary, each entry will last a different time (between 9 and 16 samples), so I'm optimizing the dictionary assignation: among the 14 bytes, the bytes with the highest occurencies, will be assigned to the shortest new encoding time. This way, I'm minimizing the new time I re-insert in the signal.

But I realize this way to go can be false!

For instance: in a TAP file, a byte "A" that lasts 22 samples is present 4 times, and another one "B" that lasts 12 samples is present 8 times.
So we get, uncompressed:

Code: Select all

A: 22 *4 = 88
B: 12*8 = 96	total 96+88= 184
Imagine there's room in the dictionary for ONE of the two, which will last 4 samples.
If I pick up "B", as it is the longest time in the signal (removing 96 samples insted of 88 if I had taken "A"), I get this in the final signal:

Code: Select all

4*8 = 32; 32 + 88 = 120 samples
But if I had taken "A", removing less time from the original signal, I get this in the end:

Code: Select all

4*4 = 16; 16 + 96 = 112 samples
Which is better!

So I save time with my method but it may not be optimal... But I really can't figure how to calculate the best result. What is the best way to do? Is there a magic formula, with 256 possibles bytes combined in 14 possible bytes of a dictionary?

User avatar
Symoon
Archivist
Posts: 1606
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France
Contact:

Re: Novalight - very fast tape loading

Post by Symoon » Sat Feb 23, 2019 4:26 pm

To sum it up, the best choice is the one where (removed_length - added_length) is maxmized. My main problem is that I don't know the added length since it depends on the choice of what I choose to remove.

User avatar
Chema
Game master
Posts: 2423
Joined: Tue Jan 17, 2006 10:55 am
Location: Gijón, SPAIN
Contact:

Re: Novalight - very fast tape loading

Post by Chema » Sat Feb 23, 2019 5:34 pm

I am quite sure I made something similar in my text compression routine. I should check the sources of my compressor in the space 1999 folder.

User avatar
Symoon
Archivist
Posts: 1606
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France
Contact:

Re: Novalight - very fast tape loading

Post by Symoon » Sat Feb 23, 2019 5:40 pm

That would be cool ;)
I tried to work with average values and there is no real difference. Maybe I'm just wasting time here.

But the problem is interesting and my statistics/mathematics lessons are way too far away for me to find the right formula.
Symoon wrote:
Sat Feb 23, 2019 4:26 pm
To sum it up, the best choice is the one where (removed_length - added_length) is maxmized. My main problem is that I don't know the added length since it depends on the choice of what I choose to remove.
Actually it would rather be: the choice I have to do depends on the whole, but the whole depends on the choice I make... Argh. Chosing to put a byte in a dictionary prevents other bytes to go there, which could have had a better positive effet. How to be sure ? Calculating 256*256*256... (14 times) combinations to sort the best won't be possible ;)

User avatar
Symoon
Archivist
Posts: 1606
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France
Contact:

Re: Novalight - very fast tape loading

Post by Symoon » Sat Feb 23, 2019 6:49 pm

Using an average value for the dictionary simplifies the problem, and gives a good-if-not-perfect solution.
It alows to set for good a variable parameter, and easily calculate the real benefit (what I remove - what I add)

User avatar
Chema
Game master
Posts: 2423
Joined: Tue Jan 17, 2006 10:55 am
Location: Gijón, SPAIN
Contact:

Re: Novalight - very fast tape loading

Post by Chema » Sat Feb 23, 2019 8:07 pm

Mmmm no, my routine goes on a similar path, but can work only with selecting the pair of letters which repeats most. In fact it is a bit more complex, as can code combinations of several letters if they repeat often enough with just one token. But your problem is the calculation of the savings, so won't be of any help.

Just for the records, I described my method at the end of this thread viewtopic.php?f=4&t=190#p845

Decompression is very fast but makes use of the stack... Nothing that could really help you, I'm afraid.

User avatar
Symoon
Archivist
Posts: 1606
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France
Contact:

Re: Novalight - very fast tape loading

Post by Symoon » Sat Feb 23, 2019 8:24 pm

Well, thanks for having looked ;)
I suspect anyway that this might not bring drastic changes :-/ I just noticed it by spotting a small difference after a few modifications.
So it's corrected with an average value to calculate the real potential benefit of the dictionary, hence the choice of bytes to use for it.

Another interesting thing could be to try and mix Novalight with Filepack. Just talked about it with Dbug a bit, as I'm a newbie in Filepack use.

User avatar
Symoon
Archivist
Posts: 1606
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France
Contact:

Re: Novalight - very fast tape loading

Post by Symoon » Mon Feb 25, 2019 10:27 pm

Remember the Atmos had a 1st ROM version that was a bit bugged with tape loading? (IIRC Chema, you got this version?).
Oric released a small program (called "ALC" on the demo tape Welcome to Oric Atmos) deisgned to fix this, and loading 1st before many Atmos programs.

I just realised that this little program loading will probably crash with Novalight if you have this ROM version or an Oric-1.
So, as Novalight doesn't use the buggy part of the Atmos ROM, I will probably change the generator so it recognizes ths ALC program and... Trash it!

User avatar
Chema
Game master
Posts: 2423
Joined: Tue Jan 17, 2006 10:55 am
Location: Gijón, SPAIN
Contact:

Re: Novalight - very fast tape loading

Post by Chema » Mon Feb 25, 2019 11:36 pm

Well, in fact what I have is the first Atmos ROM version, which does the right thing, and it is not bugged :) It checks for errors and stops loading if it finds them. What the Oric people did, after realizing that there were loading errors still, was to remove the error checking altogether in the second version 1.1b!

That is what the small program does... sets the IRQ vector to point to a small routine that clears the error flag... good job guys! :evil:

We found all that after disassembling both ROMs when testing Fabrice's tap2cd version for SkoolDaze.

User avatar
Symoon
Archivist
Posts: 1606
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France
Contact:

Re: Novalight - very fast tape loading

Post by Symoon » Mon Feb 25, 2019 11:51 pm

Yes, sorry I was writing a bit fast by saying bugged (though it could be debated if it could be considered as a bug, not a technical one, but a design one ;) )

I need to study further my solution. Removing this program is not so OK, as some editors (such as Loriciels) used a modified version of this patch to insert a little protection. So this is also a problem for Novalight, as it crashes, too.
Well, I think I made a design flaw ;) by trying to be smart and load the 14 bytes dictionary with the header (in 2B2-2BF).

I have room to store it on top of the program, which would then occupy $100-$1D6 (instead of $100-$1C8 right now)... Really doesn't leave much room for the stack!
I could try and load it elsewhere but then need to find room to code the loading, and it would probably hurt the "multipart booster" option.
S**t! :?

Post Reply