Experimental very fast tape loading

Anything related to the tools Tap2Wav, Tap2CD, Tap2Dsk, Sedoric Disc Manager, Tape Header Creator, WriteDsk, and generaly speaking tools related to the management of Oric data files and devices.
User avatar
Symoon
Archivist
Posts: 2301
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France

Re: Experimental very fast tape loading

Post by Symoon »

Thanks for your tests ;)
iss wrote: Tue May 29, 2018 2:05 pm Quick news if I type HIRES:POKE#21F,0:CLOAD"" then voilier_std_1.1g.wav works! Of course the hires screen has 'Searching...' just like ROM v.1.0 error :). I'll check what exactly the routine 'PrintSearching' at $E57D do...
Strange! If I'm not mistaken (away from all my code here), 21F is the flag that indicates the Hires screen mode is active. I shouldn't initialize this in any way in my code, should I!
iss wrote: BTW, both loaders (f16 and std) are exactly the same, right?
Absolutely: same data, only difference is in the WAV format.
iss wrote: EDIT: The only significant difference I found in routine at $E57D is: when executed from TEXT it uses 746 cycles, but from HIRES - only 86 ... and probably this leads to kind of de-synchronization. I'm not sure this can be an emulator issue too, but definitely if printing on status line is executed then loading works - which is not bad at all. :)
Usually, unexpected behaviour ends up being a timing problem indeed.
User avatar
Chema
Game master
Posts: 3013
Joined: Tue Jan 17, 2006 10:55 am
Location: Gijón, SPAIN
Contact:

Re: Experimental very fast tape loading

Post by Chema »

Hi Symoon!

Bad news, from my side. I was not able to work out a setup to load your files. I am not sure what happened with my laptop. I am quite sure I used it before for loading, but maybe due to some windows drivers update or something, now I am not able to load anything. Not even usual wavs... The Oric does not go beyond Searching... which usually means there is not enough output power.

I have a second laptop in which I installed Win 98. I also loaded things from that one, but now the sound does not work. Win 98 complains about the soundcard drivers :/ I need to check that too, but nothing I can do in the following days.

I also loaded things from my phone. But now I can't. Not sure why, though, but this time Oricium in tap2cd format almost loaded a couple of times, but there were errors and the loading stopped near the end. I am quite mad at this, because I had a quite reliable setup for loading from tape, and now I simply can't load anything!

I tried the file you sent me with the picture of the boat and all I got was garbage, as if the picture was scrambled, but no invalid attribute errors, I think.

The files made with novalight did not went beyond Searching..

So I am now unable to load things in my Oric from tape. I have to solve this. I have yet another laptop I can check, or I might try to make the soundcard of the one running win98 work, but, as I said, nothing I can solve soon.

When I find out what happens, I will get back to you and check. Now that I am thinking about it, I removed the disk controller, but kept the amplibus card plugged... I think that should not be an issue, but who knows.
User avatar
Symoon
Archivist
Posts: 2301
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France

Re: Experimental very fast tape loading

Post by Symoon »

75% of my machines can't load a tape when a disk drive is plugged. Do try this first!
And why not checking the cable? ;)

Anyway, I hope you can solve this soon. Failing hardware is extremely frustrating.
User avatar
Symoon
Archivist
Posts: 2301
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France

Re: Experimental very fast tape loading

Post by Symoon »

Ok, tried both F16 and STD pictures from the very latest Novalight version, with Oricutron 1.2 : both load fine... Except the STD one ends up with a SYNTAX ERROR, which I think I got from Euphoric too but not from real machines!

Another strange and new mystery :?
I don't knoww how to use Oricutron's monitor yet, but that's probably the next step for me ;)
User avatar
Symoon
Archivist
Posts: 2301
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France

Re: Experimental very fast tape loading

Post by Symoon »

I thought I had posted but my message seems to have vanished... So posting again !

I checked quickly the latest version with Oricutron 1.2, and I got problems I don't experience on a real machine. The last RTS goes back right in the middle of the code. So either a problem with Oricutron (bug, or timing issues with my program), or a bug in my program and me having been very lucky with real machines.
I thought it was the stack pointer but I was wrong ;)

Time for me to fix this ROM 1.0 hires display bug for good, and I'll release new WAV files (at least)
User avatar
Symoon
Archivist
Posts: 2301
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France

Re: Experimental very fast tape loading

Post by Symoon »

I think I got the "bug".
I used what some could call a "riksy" programming. Let me explain: in the way the WAV is coded for Novalight, it works like this for each byte:
1st a sunisoid whose length (in samples) is telling the format that follows:
3 = nothing, loop waiting (used for synchro or timing gaps).
4 = unused.
5= "normal" byte.
6= "byte to search in the 1st dictionnary".
7= "byte to search in the 2nd dictionnary".
8= RLE repetition sequence following.
9 = end of program.

2nd follow other sinusoids, according to the format.

As you can see, it's not the program that counts the bytes to stop the loading. The end is marked in the WAV by a special sinusoid (9 samples).
So when 9 samples are encountered, the program jumps to the end. But as this occurs in the middle of the byte-reading subroutine, it also removes the two "return address" bytes from the stack to end properly.

Now, in my latest structure modifications, there is a short sequence when the byte-reading subroutine is called... In a subroutine, to load a few bytes.
In this sequence, it should NEVER encounter the "9 samples end of program" sinusoid (by design of the WAV file). If it did, it would jump out of the sub-subroutine, going to a bad place, and only removing from the stack ONE return address.

I begin to think that this works fine on real machines, but for a reason on Oricutron an equivalent of a 9 samples length is detected.
I can see a problem with an unexpected return address in the stack.
Hence the difference according to the WAV format (STD or F16), though the data are similar: it's a WAV interpretation that goes wrong.

I haven't put the finger on the very exact moment when this happens though, so it has to be confirmed...
User avatar
Symoon
Archivist
Posts: 2301
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France

Re: Experimental very fast tape loading

Post by Symoon »

Oricutron and Novalight problem explained (I think)

The problem is that apparently, when loading a WAV file, Oricutron stops reading the WAV right at the end of the last byte... But doesn't read all the last stop bits that follow this last byte.
So, when it starts reading again the WAV, Oricutron begins by reading apparently the last stop bit, before reading the next program.
So when Novalight begins to load after the standard loader, Oricutron reads the remaining "standard fast speed" stop bit, and it means something else for Novalight, hence the strange behaviour.

This does not occur with F16 format, probably since F16 is optimised and only has two stop bits instead of three in the loader.
On real machines this doesn't occur since the WAV is not stopped, it keeps playing so the stop bits are far away when Novalight starts reading the WAV.

I'm not sure how to define Oricutron's behaviour in WAV loading that would work in any situation...
User avatar
Symoon
Archivist
Posts: 2301
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France

Re: Experimental very fast tape loading

Post by Symoon »

This being set, I think I'll do a few measures with Oricutron, to see how thresolds are set with it.
Then I'll probably release the final conversion tools.

Sorry but I'll probably won't clean my source code much - once again (a bit fed up / lazy / and the code being modified by successive loadings, I don't really know how to display it correctly).
User avatar
NekoNoNiaow
Flight Lieutenant
Posts: 272
Joined: Sun Jan 15, 2006 10:08 pm
Location: Montreal, Canadia

Re: Experimental very fast tape loading

Post by NekoNoNiaow »

Symoon wrote: Thu May 31, 2018 1:38 pm 9 = end of program.
[...]
In this sequence, it should NEVER encounter the "9 samples end of program" sinusoid (by design of the WAV file). If it did, it would jump out of the sub-subroutine, going to a bad place, and only removing from the stack ONE return address.

I begin to think that this works fine on real machines, but for a reason on Oricutron an equivalent of a 9 samples length is detected.
Given the possibility of bad readings from the Oric it might be a good idea to not just interpret "end of program" too lightly and use a more complex sequence to confirm that this was not a misinterpretation.

After all, you would not want the loading to crash just because there is a bit of noise on the tape line which causes an "end of program" to be received.
I guess 9 sinusoids followed by 4 then by 9 again would be a more robust sequence. And if a 9 appears without being followed by that sequence then you would know that something went wrong and could exit by restoring the stack safely.

Also, since that sequence would appear only once in the audio, it would not lengthen the loading time at all.
User avatar
Symoon
Archivist
Posts: 2301
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France

Re: Experimental very fast tape loading

Post by Symoon »

NekoNoNiaow wrote: Sun Jun 03, 2018 3:02 am
Symoon wrote: Thu May 31, 2018 1:38 pm 9 = end of program.
[...]
In this sequence, it should NEVER encounter the "9 samples end of program" sinusoid (by design of the WAV file). If it did, it would jump out of the sub-subroutine, going to a bad place, and only removing from the stack ONE return address.

I begin to think that this works fine on real machines, but for a reason on Oricutron an equivalent of a 9 samples length is detected.
Given the possibility of bad readings from the Oric it might be a good idea to not just interpret "end of program" too lightly and use a more complex sequence to confirm that this was not a misinterpretation.
Actually all the rest of the data loading is done with similar risk, that's the price for high speed ;)
There is nothing to do in case of misinterpretation: no possible re-read of the signal, and no time for self-correction (like CRC or parity) before next data has to be read. The loading would crash anyway if any value is mis-read.

Not counting the (lack of) available memory left in page 1 when loading the main program!
Data validation and correction is for slow-loaders, pfah! :mrgreen:
User avatar
Symoon
Archivist
Posts: 2301
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France

Re: Experimental very fast tape loading

Post by Symoon »

Symoon wrote: Sat Jun 02, 2018 11:04 am Oricutron and Novalight problem explained (I think)

The problem is that apparently, when loading a WAV file, Oricutron stops reading the WAV right at the end of the last byte... But doesn't read all the last stop bits that follow this last byte.
So, when it starts reading again the WAV, Oricutron begins by reading apparently the last stop bit, before reading the next program.
So when Novalight begins to load after the standard loader, Oricutron reads the remaining "standard fast speed" stop bit, and it means something else for Novalight, hence the strange behaviour.

This does not occur with F16 format, probably since F16 is optimised and only has two stop bits instead of three in the loader.
Okay, I thought Novalight didn't work anymore with Euphoric (that seemed strange to me), but WAV with F16 speed seem to work fine.
So I did dig a little and it appears that the standard speed problem that occurs with Oricutron also occurs with Euphoric, but one stop bit before.

In the end I guess I'm a bit lucky that the F16 speed works with the emulators. But it works and that's fine ;)
User avatar
NekoNoNiaow
Flight Lieutenant
Posts: 272
Joined: Sun Jan 15, 2006 10:08 pm
Location: Montreal, Canadia

Re: Experimental very fast tape loading

Post by NekoNoNiaow »

Symoon wrote: Sun Jun 03, 2018 5:20 am Actually all the rest of the data loading is done with similar risk, that's the price for high speed ;)
There is nothing to do in case of misinterpretation: no possible re-read of the signal, and no time for self-correction (like CRC or parity) before next data has to be read. The loading would crash anyway if any value is mis-read.

Not counting the (lack of) available memory left in page 1 when loading the main program!
Data validation and correction is for slow-loaders, pfah! :mrgreen:
Putting aside the notion of validation and correction for a moment, I agree that your reasoning is correct for non-end-of-audio signals

However, for the end of audio signal it does not cost anything to do a validation since the signal is supposed to be finished anyway.
The only test needed is only activated after a 9 is detected and it does not need to run in any other case so it would not slow regular signal interpretation. Now, that would lengthen the code a bit for sure and that indeed could be a reason to decide not to implement it, but technically this does not slow loading and makes user experience much better in case of bad audio (a "bad signal" message is slightly better than a crash ;) ).

This said, maybe there is a way to both not add this "9 validation" system and to avoid crashing. ;)
Imagine that you can predict or slightly control which values are on the stack when this error happen, then you could use that to force the program to crash in predictable ways (like trigger a BRK, divide by zero, etc.) which could then be detected. That would allow the program to redirect the code to an RTS-correction routine of some kind which would then exit cleanly.
Possibly a bit tricky, but if it is doable, this is a nice bonus for users. ;)

Now, going back to validation, it might be possible to add some form of validation if you have a few cycles and bytes to spare here or there.
But this said, I agree that the priority should be to get this working efficiently on as many machines as possible.

Once that is done, maybe there will still be some room for validation. ;)

In any case, I want to restate that I think you are doing an excellent job and that I have no intention to tell you what to do. I am just a fairy whispering ideas in your ear. :lol:
User avatar
Symoon
Archivist
Posts: 2301
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France

Re: Experimental very fast tape loading

Post by Symoon »

NekoNoNiaow wrote: Wed Jun 06, 2018 3:53 am Now, going back to validation, it might be possible to add some form of validation if you have a few cycles and bytes to spare here or there.
But this said, I agree that the priority should be to get this working efficiently on as many machines as possible.

Once that is done, maybe there will still be some room for validation. ;)

In any case, I want to restate that I think you are doing an excellent job and that I have no intention to tell you what to do. I am just a fairy whispering ideas in your ear. :lol:
Ha ha, no problem with discussing and new ideas ;) I hope I'm not too depressing myself since I tend to reject everything because I have spent so much time reducing execution time and bytes, and I feel like I'm at the end of a several years journey now - I see the release date getting closer and I'm now focused at releasing the 1st version :D

I may be wrong but I honestly think there is no more execution time available - in some cases I only have I think 1 or 2 cycles left.
I also tried to minimise the amount of bytes in the "loader kernel" (or whatever one calls it) so the loader is at minimum length (something like 0.64 seconds to be loaded).

But of course this might be clearer when you get the lastest version of the source code. Well I hope it will be clearer since it's in complete mess now ;)
And I'm sure there will be room for improvement, whatever it is. I'm not that good in ASM programming, and I think I've done about 20 or 30 different versions of the program, which shows indeed that ideas and improvments are always possible - and welcome :)
User avatar
Symoon
Archivist
Posts: 2301
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France

Re: Experimental very fast tape loading

Post by Symoon »

Ok, back to it. Currently working at a version that saves 4 bytes in the stack (it's the 10th release candidate...).
After this, unless another new idea springs to mind, I'll release it and leave it to others to find if anything can be optimized ;)

(Last time I promised a release, I got my computer stolen, got a bit sick afterwards, went to holidays, had to deal with damages at my home while on holidays, and later spent much more time than expected working at a Christmas gift... Hoping I can stay focused this time :D)
User avatar
Chema
Game master
Posts: 3013
Joined: Tue Jan 17, 2006 10:55 am
Location: Gijón, SPAIN
Contact:

Re: Experimental very fast tape loading

Post by Chema »

hehe... I hope nothing goes wrong this time and we can all enjoy the release :)
Post Reply