Novalight - very fast tape loading

Anything related to the tools Tap2Wav, Tap2CD, Tap2Dsk, Sedoric Disc Manager, Tape Header Creator, WriteDsk, and generaly speaking tools related to the management of Oric data files and devices.
User avatar
Symoon
Archivist
Posts: 1631
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France
Contact:

Re: Novalight - very fast tape loading

Post by Symoon » Tue May 21, 2019 12:50 pm

Ouch, having LOADS of new optimization ideas... If things work as I hope (no gaurantee), it might save about 18 bytes, extend the dictionary from 18 to 21 bytes, and save time on repeats.

User avatar
Symoon
Archivist
Posts: 1631
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France
Contact:

Re: Novalight - very fast tape loading

Post by Symoon » Wed May 22, 2019 12:47 pm

One of the ideas might be, instead of a dictionary for a whole program only, to use a part of the dictionary for a sub-disctionary for each page loaded. For instance a 6-bytes long dictionary, that would be loaded after each page crossing in memory, and specific to the next page.

If you take the 1st 1024 bytes from Zorgon, that makes four pages, so four dictionaries of 6 bytes of the "most used bytes in the page".
Well among those 24 bytes (6*4 = 24), there are 17 different values. Only 7 are found in two pages or three, none in the four.
It has to be checked is if the loading time of those 6 bytes saves actually time with the gain thanks to this dictionary.

Just an idea so far, for the moment I have other issues to solve, that could help saving room, to code this new loading.

User avatar
Symoon
Archivist
Posts: 1631
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France
Contact:

Re: Novalight - very fast tape loading

Post by Symoon » Mon May 27, 2019 8:54 am

Symoon wrote:
Wed May 22, 2019 12:47 pm
One of the ideas might be, instead of a dictionary for a whole program only, to use a part of the dictionary for a sub-disctionary for each page loaded. (...)
It has to be checked is if the loading time of those 6 bytes saves actually time with the gain thanks to this dictionary.
I simulated the new WAV size with such pages dictionary, and zrogon was acually a bit slower, even before adding the bytes to load. Probably due to the fact that it makes the big dictionary less efficient. Idea dropped!
Back to technical optimization, ideas will be for later.

User avatar
NekoNoNiaow
Flight Lieutenant
Posts: 272
Joined: Sun Jan 15, 2006 10:08 pm
Location: Montreal, Canadia

Re: Novalight - very fast tape loading

Post by NekoNoNiaow » Wed Jun 12, 2019 12:41 am

So many improvements one after another that I am now officially completely lost as to what the actual progression is. :D
Maybe you should setup a page somewhere (possibly http://wiki.defence-force.org/doku.php? ... :novalight ? ;)) so the current status is clear.

Also, I still do not understand why you modify your code to also work with Oricutron, I think you should definitely not do so.
If Oricutron does not behave like the Oric's then Oricutron just needs to be updated but you should probably not modify your code since that increases the risk of it being compatible with less actual machines.

You may actually be losing some optimization opportunities by trying to make it work with Oricutron.

User avatar
Symoon
Archivist
Posts: 1631
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France
Contact:

Re: Novalight - very fast tape loading

Post by Symoon » Wed Jun 12, 2019 6:38 am

NekoNoNiaow wrote:
Wed Jun 12, 2019 12:41 am
So many improvements one after another that I am now officially completely lost as to what the actual progression is. :D
Well, I'm currently stuck in a "nothing works" warp: Orics not working, phones not working, computer not working as expected, and similar things at work, so I can't work at all on my projects :|
Also, I still do not understand why you modify your code to also work with Oricutron, I think you should definitely not do so.
That's because it's a major debugging tool for me ;)
Euphoric already doesn't work anymore, and without Oricutron I'd be blind to understand what happens in case of failure (even with it, it's not so easy :mrgreen: )

User avatar
NekoNoNiaow
Flight Lieutenant
Posts: 272
Joined: Sun Jan 15, 2006 10:08 pm
Location: Montreal, Canadia

Re: Novalight - very fast tape loading

Post by NekoNoNiaow » Thu Jun 13, 2019 3:21 am

Symoon wrote:
Wed Jun 12, 2019 6:38 am
NekoNoNiaow wrote:
Wed Jun 12, 2019 12:41 am
Also, I still do not understand why you modify your code to also work with Oricutron, I think you should definitely not do so.
That's because it's a major debugging tool for me ;)
Euphoric already doesn't work anymore, and without Oricutron I'd be blind to understand what happens in case of failure (even with it, it's not so easy :mrgreen: )
Can I strangle you? With all my love? ;)
I understand that it is useful when the hardware and emulator behave the same, but when Oricutron is not accurate then your debugging is not either. ;)

One thing that would solve it though: debugging the original hardware via a minimal hardware interface between your PC and the Oric.
On consoles (PS4, Xbox, Switch, etc.) we use devkits which allow to remote-debug and trace directly on the original hardware from our dev PCs, if the Oric expansion connector exposes the signals necessary to freeze the CPU, it should be possible to do the same with a relatively minimal hardware interface (an Arduino or a Raspberry PI would be more than enough).
And maybe Jede's current WIP expansion may be able to be used for that purpose?

Sorry for the off-topic. Dreaming out loudly. (I have been thinking about designing such an interface for my Amiga 500 for quite some time. :D )
(Someone made one for the MegaDrive/Genesis: https://hackaday.com/2014/06/18/the-seg ... e-dev-kit/.)

User avatar
Symoon
Archivist
Posts: 1631
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France
Contact:

Re: Novalight - very fast tape loading

Post by Symoon » Thu Jun 13, 2019 6:47 am

NekoNoNiaow wrote:
Thu Jun 13, 2019 3:21 am
Symoon wrote:
Wed Jun 12, 2019 6:38 am
That's because it's a major debugging tool for me ;)
Euphoric already doesn't work anymore, and without Oricutron I'd be blind to understand what happens in case of failure (even with it, it's not so easy :mrgreen: )
Can I strangle you? With all my love? ;)
I understand that it is useful when the hardware and emulator behave the same, but when Oricutron is not accurate then your debugging is not either. ;)
Ha ha, wait wait wait ;)
I'm not talking about validating new improvements. For this, of course I use real Orics, about 5 of them!
I'm talking about debugging my code in which I often make mistakes in a new version ;)
Novalight code is self-modifying, it's a real hell (for me) to follow on paper and change it being sure I'm not breaking anything, especially when I can only work on it half an hour every 8 days. So being able to reach any part of the code in a monitor and execute it step by step really saves time ;)
One thing that would solve it though: debugging the original hardware via a minimal hardware interface between your PC and the Oric.
On consoles (PS4, Xbox, Switch, etc.) we use devkits which allow to remote-debug and trace directly on the original hardware from our dev PCs, if the Oric expansion connector exposes the signals necessary to freeze the CPU, it should be possible to do the same with a relatively minimal hardware interface (an Arduino or a Raspberry PI would be more than enough).
And maybe Jede's current WIP expansion may be able to be used for that purpose?
Well if there's something that can help me seeing the VIA state and stop the incoming WAV file after each instruction (so I can measure the time after each of them), I'd buy it ;)

User avatar
Symoon
Archivist
Posts: 1631
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France
Contact:

Re: Novalight - very fast tape loading

Post by Symoon » Mon Jun 17, 2019 12:44 pm

Yesterday, I corrected a bug in the 1.3h version.
It became visible at the latest Oric meeting when I tried to convert the TAP of Retroric's game (Electroric). It seemed to load but didn't run.

The reason: when the last byte of a memory page was using a specific encoding (1 sinusoid for '1111', which is longer to decode), there was not enough time to both decode it and change de memory page address.

I added back a small delay in the tape signal at the end of each memory page (it existed before), and everything is back to normal in such cases.
It wasn't easy to find such cases because it requires to:
- be the last byte of a memory page
- not be a RLE byte
- not be a dictionary byte
- have a specific value

With later versions, it might be possible to remove this delay again. I now have all I need to test it, anyway ;)

User avatar
NekoNoNiaow
Flight Lieutenant
Posts: 272
Joined: Sun Jan 15, 2006 10:08 pm
Location: Montreal, Canadia

Re: Novalight - very fast tape loading

Post by NekoNoNiaow » Tue Jun 18, 2019 1:12 pm

Symoon wrote:
Thu Jun 13, 2019 6:47 am
Ha ha, wait wait wait ;)
I'm not talking about validating new improvements. For this, of course I use real Orics, about 5 of them!
I'm talking about debugging my code in which I often make mistakes in a new version ;)
Novalight code is self-modifying, it's a real hell (for me) to follow on paper and change it being sure I'm not breaking anything, especially when I can only work on it half an hour every 8 days. So being able to reach any part of the code in a monitor and execute it step by step really saves time ;)
Ok, I forgive you then. ;)
Symoon wrote:
Thu Jun 13, 2019 6:47 am
One thing that would solve it though: debugging the original hardware via a minimal hardware interface between your PC and the Oric.
[...]
And maybe Jede's current WIP expansion may be able to be used for that purpose?
Well if there's something that can help me seeing the VIA state and stop the incoming WAV file after each instruction (so I can measure the time after each of them), I'd buy it ;)
I am not sure I correctly understand your needs, can you elaborate a bit?
To me, it seems that what you want is:
- assuming the wav file is represented by a stream of analog values sent to the VIA's input pin
- that this stream can be frozen at the same time as the CPU and VIA so you can inspect values (current time, etc.) while step-tracing through your code

Is this correct?

User avatar
Symoon
Archivist
Posts: 1631
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France
Contact:

Re: Novalight - very fast tape loading

Post by Symoon » Thu Jun 20, 2019 9:10 am

NekoNoNiaow wrote:
Tue Jun 18, 2019 1:12 pm
Symoon wrote:
Thu Jun 13, 2019 6:47 am
Well if there's something that can help me seeing the VIA state and stop the incoming WAV file after each instruction (so I can measure the time after each of them), I'd buy it ;)
I am not sure I correctly understand your needs, can you elaborate a bit?
To me, it seems that what you want is:
- assuming the wav file is represented by a stream of analog values sent to the VIA's input pin
- that this stream can be frozen at the same time as the CPU and VIA so you can inspect values (current time, etc.) while step-tracing through your code

Is this correct?
Well I was describing what I'm doing with Oricutrion, which helps a lot: executing my new code step by step while it reads the WAV file I just produced with a beta version of Novalight.
The code can be wrong, but the WAV file too (I'm sometimes changing the signal).
I can't see how to do on real hadware, especially since the WAV file would carry on playing while I'd be watching a monitor.

Doing this helped me debugging, and optimizing, finding when I have spare execution time I could use (the goal being to minimize the time spent by the Oric "waiting" for the VIA to detect the next pulse). It also helps me understanding when I've been too optimistic on some treatments, which last too long and lead to information loss. Not forgetting that Oricutron is more optimistic than the real harware that requires more time.

User avatar
NekoNoNiaow
Flight Lieutenant
Posts: 272
Joined: Sun Jan 15, 2006 10:08 pm
Location: Montreal, Canadia

Re: Novalight - very fast tape loading

Post by NekoNoNiaow » Wed Jul 10, 2019 2:04 am

Symoon wrote:
Thu Jun 20, 2019 9:10 am
Well I was describing what I'm doing with Oricutrion, which helps a lot: executing my new code step by step while it reads the WAV file I just produced with a beta version of Novalight.
The code can be wrong, but the WAV file too (I'm sometimes changing the signal).
I can't see how to do on real hadware, especially since the WAV file would carry on playing while I'd be watching a monitor.

Doing this helped me debugging, and optimizing, finding when I have spare execution time I could use (the goal being to minimize the time spent by the Oric "waiting" for the VIA to detect the next pulse). It also helps me understanding when I've been too optimistic on some treatments, which last too long and lead to information loss. Not forgetting that Oricutron is more optimistic than the real harware that requires more time.
Oki, gotcha. Thanks for the precisions!

Is there a way on the Oric to freeze the entire machine ULA, CPU, VIA, at the same time?

Post Reply