Novalight - very fast tape loading

[Symoon]

Ouch, having LOADS of new optimization ideas... If things work as I hope (no gaurantee), it might save about 18 bytes, extend the dictionary from 18 to 21 bytes, and save time on repeats.

[Symoon]

One of the ideas might be, instead of a dictionary for a whole program only, to use a part of the dictionary for a sub-disctionary for each page loaded. For instance a 6-bytes long dictionary, that would be loaded after each page crossing in memory, and specific to the next page.

If you take the 1st 1024 bytes from Zorgon, that makes four pages, so four dictionaries of 6 bytes of the "most used bytes in the page".
Well among those 24 bytes (6*4 = 24), there are 17 different values. Only 7 are found in two pages or three, none in the four.
It has to be checked is if the loading time of those 6 bytes saves actually time with the gain thanks to this dictionary.

Just an idea so far, for the moment I have other issues to solve, that could help saving room, to code this new loading.

[Symoon]

One of the ideas might be, instead of a dictionary for a whole program only, to use a part of the dictionary for a sub-disctionary for each page loaded. (...)
It has to be checked is if the loading time of those 6 bytes saves actually time with the gain thanks to this dictionary.

I simulated the new WAV size with such pages dictionary, and zrogon was acually a bit slower, even before adding the bytes to load. Probably due to the fact that it makes the big dictionary less efficient. Idea dropped!
Back to technical optimization, ideas will be for later.

[NekoNoNiaow]

So many improvements one after another that I am now officially completely lost as to what the actual progression is.
Maybe you should setup a page somewhere (possibly http://wiki.defence-force.org/doku.php? ... :novalight ? ) so the current status is clear.

Also, I still do not understand why you modify your code to also work with Oricutron, I think you should definitely not do so.
If Oricutron does not behave like the Oric's then Oricutron just needs to be updated but you should probably not modify your code since that increases the risk of it being compatible with less actual machines.

You may actually be losing some optimization opportunities by trying to make it work with Oricutron.

[Symoon]

So many improvements one after another that I am now officially completely lost as to what the actual progression is.

Well, I'm currently stuck in a "nothing works" warp: Orics not working, phones not working, computer not working as expected, and similar things at work, so I can't work at all on my projects

Also, I still do not understand why you modify your code to also work with Oricutron, I think you should definitely not do so.

That's because it's a major debugging tool for me
Euphoric already doesn't work anymore, and without Oricutron I'd be blind to understand what happens in case of failure (even with it, it's not so easy )

[NekoNoNiaow]

Also, I still do not understand why you modify your code to also work with Oricutron, I think you should definitely not do so.

That's because it's a major debugging tool for me
Euphoric already doesn't work anymore, and without Oricutron I'd be blind to understand what happens in case of failure (even with it, it's not so easy )

Can I strangle you? With all my love?
I understand that it is useful when the hardware and emulator behave the same, but when Oricutron is not accurate then your debugging is not either.

One thing that would solve it though: debugging the original hardware via a minimal hardware interface between your PC and the Oric.
On consoles (PS4, Xbox, Switch, etc.) we use devkits which allow to remote-debug and trace directly on the original hardware from our dev PCs, if the Oric expansion connector exposes the signals necessary to freeze the CPU, it should be possible to do the same with a relatively minimal hardware interface (an Arduino or a Raspberry PI would be more than enough).
And maybe Jede's current WIP expansion may be able to be used for that purpose?

Sorry for the off-topic. Dreaming out loudly. (I have been thinking about designing such an interface for my Amiga 500 for quite some time. )
(Someone made one for the MegaDrive/Genesis: https://hackaday.com/2014/06/18/the-seg ... e-dev-kit/.)

[Symoon]

That's because it's a major debugging tool for me
Euphoric already doesn't work anymore, and without Oricutron I'd be blind to understand what happens in case of failure (even with it, it's not so easy )

Can I strangle you? With all my love?
I understand that it is useful when the hardware and emulator behave the same, but when Oricutron is not accurate then your debugging is not either.

Ha ha, wait wait wait
I'm not talking about validating new improvements. For this, of course I use real Orics, about 5 of them!
I'm talking about debugging my code in which I often make mistakes in a new version
Novalight code is self-modifying, it's a real hell (for me) to follow on paper and change it being sure I'm not breaking anything, especially when I can only work on it half an hour every 8 days. So being able to reach any part of the code in a monitor and execute it step by step really saves time

One thing that would solve it though: debugging the original hardware via a minimal hardware interface between your PC and the Oric.
On consoles (PS4, Xbox, Switch, etc.) we use devkits which allow to remote-debug and trace directly on the original hardware from our dev PCs, if the Oric expansion connector exposes the signals necessary to freeze the CPU, it should be possible to do the same with a relatively minimal hardware interface (an Arduino or a Raspberry PI would be more than enough).
And maybe Jede's current WIP expansion may be able to be used for that purpose?

Well if there's something that can help me seeing the VIA state and stop the incoming WAV file after each instruction (so I can measure the time after each of them), I'd buy it

[Symoon]

Yesterday, I corrected a bug in the 1.3h version.
It became visible at the latest Oric meeting when I tried to convert the TAP of Retroric's game (Electroric). It seemed to load but didn't run.

The reason: when the last byte of a memory page was using a specific encoding (1 sinusoid for '1111', which is longer to decode), there was not enough time to both decode it and change de memory page address.

I added back a small delay in the tape signal at the end of each memory page (it existed before), and everything is back to normal in such cases.
It wasn't easy to find such cases because it requires to:
- be the last byte of a memory page
- not be a RLE byte
- not be a dictionary byte
- have a specific value

With later versions, it might be possible to remove this delay again. I now have all I need to test it, anyway

[NekoNoNiaow]

Ha ha, wait wait wait
I'm not talking about validating new improvements. For this, of course I use real Orics, about 5 of them!
I'm talking about debugging my code in which I often make mistakes in a new version
Novalight code is self-modifying, it's a real hell (for me) to follow on paper and change it being sure I'm not breaking anything, especially when I can only work on it half an hour every 8 days. So being able to reach any part of the code in a monitor and execute it step by step really saves time

Ok, I forgive you then.

One thing that would solve it though: debugging the original hardware via a minimal hardware interface between your PC and the Oric.
[...]
And maybe Jede's current WIP expansion may be able to be used for that purpose?

Well if there's something that can help me seeing the VIA state and stop the incoming WAV file after each instruction (so I can measure the time after each of them), I'd buy it

I am not sure I correctly understand your needs, can you elaborate a bit?
To me, it seems that what you want is:
- assuming the wav file is represented by a stream of analog values sent to the VIA's input pin
- that this stream can be frozen at the same time as the CPU and VIA so you can inspect values (current time, etc.) while step-tracing through your code

Is this correct?

[Symoon]

Well if there's something that can help me seeing the VIA state and stop the incoming WAV file after each instruction (so I can measure the time after each of them), I'd buy it

I am not sure I correctly understand your needs, can you elaborate a bit?
To me, it seems that what you want is:
- assuming the wav file is represented by a stream of analog values sent to the VIA's input pin
- that this stream can be frozen at the same time as the CPU and VIA so you can inspect values (current time, etc.) while step-tracing through your code

Is this correct?

Well I was describing what I'm doing with Oricutrion, which helps a lot: executing my new code step by step while it reads the WAV file I just produced with a beta version of Novalight.
The code can be wrong, but the WAV file too (I'm sometimes changing the signal).
I can't see how to do on real hadware, especially since the WAV file would carry on playing while I'd be watching a monitor.

Doing this helped me debugging, and optimizing, finding when I have spare execution time I could use (the goal being to minimize the time spent by the Oric "waiting" for the VIA to detect the next pulse). It also helps me understanding when I've been too optimistic on some treatments, which last too long and lead to information loss. Not forgetting that Oricutron is more optimistic than the real harware that requires more time.

[NekoNoNiaow]

Well I was describing what I'm doing with Oricutrion, which helps a lot: executing my new code step by step while it reads the WAV file I just produced with a beta version of Novalight.
The code can be wrong, but the WAV file too (I'm sometimes changing the signal).
I can't see how to do on real hadware, especially since the WAV file would carry on playing while I'd be watching a monitor.

Doing this helped me debugging, and optimizing, finding when I have spare execution time I could use (the goal being to minimize the time spent by the Oric "waiting" for the VIA to detect the next pulse). It also helps me understanding when I've been too optimistic on some treatments, which last too long and lead to information loss. Not forgetting that Oricutron is more optimistic than the real harware that requires more time.

Oki, gotcha. Thanks for the precisions!

Is there a way on the Oric to freeze the entire machine ULA, CPU, VIA, at the same time?

[Symoon]

Seems something's wrong between Pulsoids TAP version and Novalight.
Loading directly the game works fine, but loading it after the loading screens doesn't. I wonder if the execution of the Twilighte's animation could have something to do with that, or if it's a completely different reason.
The good news is that it crashes both on real machine and on Oricutron, so understanding the puzzle should be easier.

I tried relocating Novalight, and discovered by the same way that my working beta version of Novalight 1.3 has some bugs with relocation parameter -ouch, more work on the TO DO list, don't hold your breath for Novalight 1.3.

[Symoon]

Seems something's wrong between Pulsoids TAP version and Novalight.

Bug solved, it probably affected all Novalight's versions: when a multipart program had one part beginning with a byte equal to the last byte of the previous part, this byte was encoded as a RLE repetition - but the value was wrong since other things were loaded in between.
So I reinitialised the "previous byte value" at the beginning of each new part.
Pulsoids works on Oricutron now

[Vyper68]

Great news Symoon thanks for such a great program, it has made my life a lot better as a tape based user

[iss]

Well, it's almost one year later but I think that the 'last word' about the max speed has not yet been spoken.

@Symoon: Can you remember the reason to use this code for sending a bit in Novalight/F16 source :

Code: Select all

// F16 is an optimised WAV signal working a bit faster, with all Oric standard ROM routines.
void emit_F16_bit(int bit)
{
    switch (speed) {
      case 44100:
        if (bit) {
          emit_level(1,48);
          emit_level(2,208);
          emit_level(2,48);
        } else {
          emit_level(1,48);
          emit_level(2,208);
          emit_level(20,48);
        }
        break;
    }
}

which semi-graphically should look like this:

Code: Select all

   ____
__|    |____ - for '1'

   ____
__|    |__... x20 ...__ - for '0'

and why not like this:

Code: Select all

 ____
|    |______ - for '1'

 ____
|    |__... x21 ...____ - for '0'

or: why you start with 1 sample low-level instead directly with high-level assuming that previous bit ended with a low-level?

btw, your code works fine I just want to find the clue .