Novalight - very fast tape loading
Novalight - very fast tape loading
Hi all,
After years working on it, with your help, I'm happy to release:
Novalight, very fast tape loading
You can download it here, compiled for Windows command line:
https://sourceforge.net/projects/euphor ... Novalight/
I hope the C code can be compiled on other platforms.
The work-in-progress thread is here:
viewtopic.php?f=19&t=1360
After years working on it, with your help, I'm happy to release:
Novalight, very fast tape loading
You can download it here, compiled for Windows command line:
https://sourceforge.net/projects/euphor ... Novalight/
I hope the C code can be compiled on other platforms.
The work-in-progress thread is here:
viewtopic.php?f=19&t=1360
Re: Novalight - very fast tape loading
Here's the readme:
Novalight, very fast tape loader for Oric, 02/2019.
By Symoon, based on Fabrice's tools TAP2WAV and TAP2CD.
Version 1.1k
Novalight converts an Oric TAP file into a 44 kHz WAV file that should load very fast on most Orics, ROM 1.0 or ROM 1.1.
It must be used in MS-DOS, or Windows command line (see below).
Default version is compiled in 32 bits and should work on 32/64 bits Windows command line.
Can be compiled in 16-bits mode for old real MS-DOS, and I hope it could be compiled for Unix/Linux.
WARNING:
- requires a *perfect* WAV player (Audacity?).
- do not convert the WAV file, in any way (mp3 or whatever). Each and every sample of the WAV file is important.
In case of failure loading the file:
- keep cellphones away, switch off WiFi
- try different volume settings if loading fails, some Orics require high volume, others very low
- try rebooting your computer playing the WAV if loading fails - this happens with my PC when it's been running for a while!
- some Orics may not load this signal
Usage:
Novalight [ -option1 ] [ -option2 ] <.TAP file> <.WAV file>
Options: -s 'standard speed': use standard speed instead of F16 speed
for the loader (slower, but some Orics may not like F16?)
-o 'old loader': use the ROM 1.1-only loader instead of the
latest 1.0/1.1 loader. Old one has an advantage: it can be
loaded only once, then used with several CALL#100 to load
multiple parts, as long as it remains intact. But no ROM 1.0.
-n 'no loader': generates the Novalight file(s), without loader.
You should load an 'old Novalight' loader yourself first, then
use CALL#100 to load each of the file(s). ROM 1.1 only.
-p 'long pause': generates a 5 seconds silence between each part
of multipart programs (ERE Informatique programs, for instance,
require time to draw a loading screen).
Loading time examples:
Programs compatibility:
How does it work:
Novalight uses the TAP2CD bit encoding (2 bits per period: 00, 01, 10 and 11), but with shorter periods: 3, 4, 5 and 6 samples for 2 bits, while TAP2CD used the equivalent of 4, 6, 8 and 10 samples. Four periods are required to make a full byte.
Based on statistics made on about 1400 TAP files, showing that Oric files hold 60% of "0" and 40% of "1", Novalight encodes the zeroes on the shortest persiods ("00" is 3 samples long, while on TAP2CD it was "11").
A RLE compression has been added, repeated bytes will be encoded with the shortest possible period (3 samples, 4 samples for the last repeated one).
Two 7-bytes dictionaries are set, stored with the program header and filled with the most repeated bytes that have not been dealed by the RLE compression. Those bytes are also encoded in a shorter way.
Partiy check has been removed.
Stop bits have been merged in the start bit, its length determines the type of encoding that follows (normal byte, RLE compressed, or dictionary encoded)
Novalight's loaded has been minimised: using F16 speed (compatible with standard ROM), inverting bytes to save 10% of loading time (bits 1 coded as 0 and reversed). It loads first a minimal kernel able to load at Novalight speed without compression, then loads at fast speed small banks that do different things: VIA initialization, display, loading of banks, RLE and dictionaries decoding code, VIA restore and progam launch. This reduces the loader's loading time at about 0.6 seconds, plus the banks that follow (very, very short loading time)
Novalight is loaded in page 1, and leaves rather little room for the stack management (54 bytes). Beware if you ever call it in a sub-sub-program or in a loop.
While at it, Novalight also fixes the HIRES CLOAD bug on ROM 1.0: a HIRES file loaded on ROM 1.0 should not be damaged anymore by a black line once loaded (which was the result of the erasing of the displayed of "Loading.." while in HIRES).
Acknowledgements:
Novalight's name is a tribute to Twilighte, the great Oric coder, and Paul Woakes, co-founder of Novagen company, author of Mercenary games and Novaload fast tape loader for C64.
RIP guys, I had the greatest respect for your work.
Thanks to Fabrice for having written the Oric tapes conversion tools long ago, for TAP2CD and for his constant help, giving ideas and answering my questions about tapes, WAVs, interrupts and Oric coding.
Thanks to the Oric community, especially on forums Defence-Force (http://forum.defence-force.org) and Oric.org (http://forums.oric.org). For months, many people here spent time on various coding questions, tests, advice: FredV60, Musepat, Kenneth, Dom50, Voyageur, Oric1-Atmos, Froggy, Godzil, DrPsy, ISS, Chema, Dbug, NekoNoNiaow.
I would *never* have been able to program Novalight without this help.
Enjoy!
Novalight, very fast tape loader for Oric, 02/2019.
By Symoon, based on Fabrice's tools TAP2WAV and TAP2CD.
Version 1.1k
Novalight converts an Oric TAP file into a 44 kHz WAV file that should load very fast on most Orics, ROM 1.0 or ROM 1.1.
It must be used in MS-DOS, or Windows command line (see below).
Default version is compiled in 32 bits and should work on 32/64 bits Windows command line.
Can be compiled in 16-bits mode for old real MS-DOS, and I hope it could be compiled for Unix/Linux.
WARNING:
- requires a *perfect* WAV player (Audacity?).
- do not convert the WAV file, in any way (mp3 or whatever). Each and every sample of the WAV file is important.
In case of failure loading the file:
- keep cellphones away, switch off WiFi
- try different volume settings if loading fails, some Orics require high volume, others very low
- try rebooting your computer playing the WAV if loading fails - this happens with my PC when it's been running for a while!
- some Orics may not load this signal
Usage:
Novalight [ -option1 ] [ -option2 ] <.TAP file> <.WAV file>
Options: -s 'standard speed': use standard speed instead of F16 speed
for the loader (slower, but some Orics may not like F16?)
-o 'old loader': use the ROM 1.1-only loader instead of the
latest 1.0/1.1 loader. Old one has an advantage: it can be
loaded only once, then used with several CALL#100 to load
multiple parts, as long as it remains intact. But no ROM 1.0.
-n 'no loader': generates the Novalight file(s), without loader.
You should load an 'old Novalight' loader yourself first, then
use CALL#100 to load each of the file(s). ROM 1.1 only.
-p 'long pause': generates a 5 seconds silence between each part
of multipart programs (ERE Informatique programs, for instance,
require time to draw a loading screen).
Loading time examples:
Code: Select all
FAST standard FAST F16 TAP2CD Novalight
HIRES screen 0:56 0:34 0:08 0:04 - 0:02
TEXT screen 0:10 0:07 0:05 0:015
Zorgon 4:31 2:37 0:31 0:15
Xenon-1 4:15 2:31s 0:32 0:12
Acheron's Rage 5:12 3.12 0:40 0:12
Oricium 5:45 3:25 0:49 0:18
Lone Raider 5:08 2:59 fails
Trouble In Store 5:14 fails fails
Programs compatibility:
Code: Select all
FAST standard FAST F16 TAP2CD Novalight
Oric-1 yes yes no yes
Atmos yes yes yes yes
Emulator yes yes yes yes
Real tape yes no no no
Digital player yes 11kHz yes 44kHz yes 22kHz yes 44kHz
How does it work:
Novalight uses the TAP2CD bit encoding (2 bits per period: 00, 01, 10 and 11), but with shorter periods: 3, 4, 5 and 6 samples for 2 bits, while TAP2CD used the equivalent of 4, 6, 8 and 10 samples. Four periods are required to make a full byte.
Based on statistics made on about 1400 TAP files, showing that Oric files hold 60% of "0" and 40% of "1", Novalight encodes the zeroes on the shortest persiods ("00" is 3 samples long, while on TAP2CD it was "11").
A RLE compression has been added, repeated bytes will be encoded with the shortest possible period (3 samples, 4 samples for the last repeated one).
Two 7-bytes dictionaries are set, stored with the program header and filled with the most repeated bytes that have not been dealed by the RLE compression. Those bytes are also encoded in a shorter way.
Partiy check has been removed.
Stop bits have been merged in the start bit, its length determines the type of encoding that follows (normal byte, RLE compressed, or dictionary encoded)
Novalight's loaded has been minimised: using F16 speed (compatible with standard ROM), inverting bytes to save 10% of loading time (bits 1 coded as 0 and reversed). It loads first a minimal kernel able to load at Novalight speed without compression, then loads at fast speed small banks that do different things: VIA initialization, display, loading of banks, RLE and dictionaries decoding code, VIA restore and progam launch. This reduces the loader's loading time at about 0.6 seconds, plus the banks that follow (very, very short loading time)
Novalight is loaded in page 1, and leaves rather little room for the stack management (54 bytes). Beware if you ever call it in a sub-sub-program or in a loop.
While at it, Novalight also fixes the HIRES CLOAD bug on ROM 1.0: a HIRES file loaded on ROM 1.0 should not be damaged anymore by a black line once loaded (which was the result of the erasing of the displayed of "Loading.." while in HIRES).
Acknowledgements:
Novalight's name is a tribute to Twilighte, the great Oric coder, and Paul Woakes, co-founder of Novagen company, author of Mercenary games and Novaload fast tape loader for C64.
RIP guys, I had the greatest respect for your work.
Thanks to Fabrice for having written the Oric tapes conversion tools long ago, for TAP2CD and for his constant help, giving ideas and answering my questions about tapes, WAVs, interrupts and Oric coding.
Thanks to the Oric community, especially on forums Defence-Force (http://forum.defence-force.org) and Oric.org (http://forums.oric.org). For months, many people here spent time on various coding questions, tests, advice: FredV60, Musepat, Kenneth, Dom50, Voyageur, Oric1-Atmos, Froggy, Godzil, DrPsy, ISS, Chema, Dbug, NekoNoNiaow.
I would *never* have been able to program Novalight without this help.
Enjoy!
Re: Novalight - very fast tape loading
Absolutely incredible work! Congratulation, Symoon!
Thanks for all your effort in pushing CLOAD to the limits.
Releasing sources and the well documented details makes Novalight huge event in this small Oric world!
Thanks for all your effort in pushing CLOAD to the limits.
Releasing sources and the well documented details makes Novalight huge event in this small Oric world!
Re: Novalight - very fast tape loading
Impressive work indeed! A great achievement.
I must find a way to make audio loads again on my Atmos, because I lost the laptop which was really reliable and couldn't find a way to make turbo loads work again
I must find a way to make audio loads again on my Atmos, because I lost the laptop which was really reliable and couldn't find a way to make turbo loads work again
Re: Novalight - very fast tape loading
Thank you guys
@Chema I'm sure you'll find. I'm using an old eeepc for this, those tiny computers are still great for such a job.
And I bet ISS will add it to TapOric at some point anyway - well I'm sure he will, I'm just worried about possible 44/48kHz issues.
@Chema I'm sure you'll find. I'm using an old eeepc for this, those tiny computers are still great for such a job.
And I bet ISS will add it to TapOric at some point anyway - well I'm sure he will, I'm just worried about possible 44/48kHz issues.
Re: Novalight - very fast tape loading
Great piece of software, Symoon, I really can't imagine how many time and effort you put in Novalight but it's worth! Chapeau bas!
The source compiles fine under Linux with GCC (just 2-3 harmless warning - no problem at all) and the tool works nice.
I spotted just one thing which seams like bug: IMO the "+5" should be "+6", right?
The source compiles fine under Linux with GCC (just 2-3 harmless warning - no problem at all) and the tool works nice.
I spotted just one thing which seams like bug: IMO the "+5" should be "+6", right?
Re: Novalight - very fast tape loading
Ooooh very nice spotting, I think you're right indeed.
Thanks!
It seems I get this bug since the very beginning. Strange, how can it have not crashed when I was running the code on ROM 1.0? does the op code $13 act like a NOP ?
I'm correcting right now and will upload the corrected verion before tonight (train travelling this afternoon)
Thanks!
It seems I get this bug since the very beginning. Strange, how can it have not crashed when I was running the code on ROM 1.0? does the op code $13 act like a NOP ?
I'm correcting right now and will upload the corrected verion before tonight (train travelling this afternoon)
Re: Novalight - very fast tape loading
Thanks a lot ISS; it's corrected, tested on Oric-1, and uploaded on SourceForge!
About the time spent on it, I think something like 3 years, with large breaks due to real life. That allows ideas, thinking and so on, to optimize.
The only really boring thing was testing on real machines to find the right working WAV form, and find the right thresholds that would work on most machines.
That's great news!
Re: Novalight - very fast tape loading
I'm puzzled. I've tried a program with $13 $60, called it and it hangs.
But I've loaded again an old test file of Acherons Rage on Oric-1: no problem, though the bug is supposed to be here.
Re: Novalight - very fast tape loading
Ok I think I got it. Unbelivable luck.
$13: undocumented opcode (2 bytes):
ASO (ab),Y ;13 ab This opcode ASLs the contents of a memory location and then ORs the result with the accumulator.
$63: undocumented opcode (2 bytes):
RRA (ab,X) ;63 ab RRA RORs the contents of a memory location and then ADCs the result with the accumulator.
So 13 20 65 E5 does this:
ASO (20),Y
RRA (E5,X)
instead of calling the "clear status line" code. Which is not visible since this cleaning cleans nothing (Loding from the original CLOAD command is already cleaned, and I removed the display of "Searching" in Novalight) to replace it by "Loading .. PROGRAM NAME".
In conclusion:
- there was a bug indeed
- unbelivable luck made it invisible and not crashing
- I think I could remove this "clear line status" part!
$13: undocumented opcode (2 bytes):
ASO (ab),Y ;13 ab This opcode ASLs the contents of a memory location and then ORs the result with the accumulator.
$63: undocumented opcode (2 bytes):
RRA (ab,X) ;63 ab RRA RORs the contents of a memory location and then ADCs the result with the accumulator.
So 13 20 65 E5 does this:
ASO (20),Y
RRA (E5,X)
instead of calling the "clear status line" code. Which is not visible since this cleaning cleans nothing (Loding from the original CLOAD command is already cleaned, and I removed the display of "Searching" in Novalight) to replace it by "Loading .. PROGRAM NAME".
In conclusion:
- there was a bug indeed
- unbelivable luck made it invisible and not crashing
- I think I could remove this "clear line status" part!
Re: Novalight - very fast tape loading
By testing heavily again this evening, I found 2 other small problems.
Updated SourceForge.
1- F16 speed was disabled. It's now back so you should see the loader is now much faster to load!
2- I had reduced a pause between two parts of the loader, that worked fine with real machines and emulators, but just found that apparently, it caused problems with multipart programs in Euphoric. Tried with Ere Informatique's Karate, and the 5th or 6th part ended with a syntax error, while it didn't if I put back the original pause. So did I, even though it's a bit unclear why.
Updated SourceForge.
1- F16 speed was disabled. It's now back so you should see the loader is now much faster to load!
2- I had reduced a pause between two parts of the loader, that worked fine with real machines and emulators, but just found that apparently, it caused problems with multipart programs in Euphoric. Tried with Ere Informatique's Karate, and the 5th or 6th part ended with a syntax error, while it didn't if I put back the original pause. So did I, even though it's a bit unclear why.
- NekoNoNiaow
- Flight Lieutenant
- Posts: 272
- Joined: Sun Jan 15, 2006 10:08 pm
- Location: Montreal, Canadia
Re: Novalight - very fast tape loading
Good job! I am eager to have a look at the sources to see what magic you had to resort to.
Re: Novalight - very fast tape loading
I'm just thinking at a possible small optimization for multipart programs.
Once fully loaded, the kernel and common area 0 ($100-$18C) are not destroyed, unless the loaded program decides to use page 1.
So maybe it only needs to be loaded once, then for next parts only loading a reduced version of Common Area 1 is required, without the "load Common area 0" part. And then the other Common areas must be loaded as usual.
This would save I guess around 0.3 or 0.4 second for each additional program. But would require a new option to choose between this or having the full loader every time, which would make the options more complex...
Worth it?
Once fully loaded, the kernel and common area 0 ($100-$18C) are not destroyed, unless the loaded program decides to use page 1.
So maybe it only needs to be loaded once, then for next parts only loading a reduced version of Common Area 1 is required, without the "load Common area 0" part. And then the other Common areas must be loaded as usual.
This would save I guess around 0.3 or 0.4 second for each additional program. But would require a new option to choose between this or having the full loader every time, which would make the options more complex...
Worth it?
- NekoNoNiaow
- Flight Lieutenant
- Posts: 272
- Joined: Sun Jan 15, 2006 10:08 pm
- Location: Montreal, Canadia
Re: Novalight - very fast tape loading
How long is the loader (in bytes)?
Could you store it in a darkened part of graphic memory and then wipe it entirely once the loading is done?
Could you store it in a darkened part of graphic memory and then wipe it entirely once the loading is done?
Re: Novalight - very fast tape loading
It's a bit complicated to answer, because it's using banks that are loaded as the WAV goes on and crush each other. And they are loading at different speeds: F16 first for a part of the kernel, then a reduced Novalight speed.
So the required room is 202 bytes, but the loader is longer since it swaps data inside this room (the total size is 368 bytes).
Take a look at the XLS file in the Novalight ZIP, it explains how it is built and loaded
Page 1 (Fabrice's idea!) seems to be the less used area by programs.NekoNoNiaow wrote: ↑Tue Feb 12, 2019 1:13 am Could you store it in a darkened part of graphic memory and then wipe it entirely once the loading is done?
I had thought at choosing an area according to the programs addresses, but it's not enough: once loaded programs can use any RAM area! And there is one or two jumps that are not relative, so relocating it dynamically could be done but with a bit of rework.