New tape fileformat (.ort)

Anything related to the tools Tap2Wav, Tap2CD, Tap2Dsk, Sedoric Disc Manager, Tape Header Creator, WriteDsk, and generaly speaking tools related to the management of Oric data files and devices.
User avatar
Xeron
Emulation expert
Posts: 426
Joined: Sat Mar 07, 2009 5:18 pm
Contact:

New tape fileformat (.ort)

Post by Xeron »

I have created a new format for Oric tape files.

Motivation for creating this format

At the moment, you can store Oric tapes in TAP or WAV format.

TAP files contain just the data from the tape, after decoding. This makes for small, easy to read files, but the downside is that TAP files can only represent data that was stored in standard Oric tape format.

WAV files are just audio samples of the tape output, so any data can be stored in them, including games using non-standard loaders (for speed or better error checking). The downside is that they are quite large because they store 8 or 16 bytes per sample.

ORT (Oric Raw Tape) files can contain raw square wave data for the tape input, as well as decoded bytes, and it can interleave the two.

For example, you could have a bit at the start of an ORT file containing a small loader, which could be in decoded form (like a tap file), the rest of the ORT file might contain a game stored in a fastloader format.

Oricutron in the SVN can create and load ORT files, and when the next version is released, this will make it useful for developing fastloaders etc.

I will also make a converter that can read .tap/.wav/.ort and save out .tap/.wav/.ort (any combination).



Format description
Now I will describe the format.

At the start of the file is a 4-byte header. The first three bytes are "ORT", and the 4th is the version of the format (currently must be a zero byte).

The next byte describes the initial state of the tape waveform (0 or 1).

The rest of the file is decoded like this:

* Get the next byte
* If the byte is smaller than 0xfc, multiply it by 2, wait that many cycles, toggle the tape data bit
* If the byte is 0xfc, take the byte after it, multiply it by 2, wait that many cycles, toggle the tape data bit
* If the byte is 0xfd, read the 2 bytes after it as a big endian value, multiply it by 2, wait that many cycles, toggle the tape data bit
* If the byte is 0xff, read the next 2 bytes after it as a big endian value (N). For the next N bytes, read the data as you would from a .TAP file.

Loop to the end of the file.



Implementation in Oricutron

In Oricutron, you can at any point start capturing the tape output in .ORT format, just like AVI capturing, but instead of recording an AVI, it saves the tape output in .ORT format. If turbotape is enabled, and you do a CSAVE, the CSAVE'd data will be added to the capture file in an 0xff block, otherwise it'll just record the waveform as normal.

If you do a CSAVE while NOT recording to an ort file, it'll just save it as a normal .TAP file.



Questions/Comments?
User avatar
Xeron
Emulation expert
Posts: 426
Joined: Sat Mar 07, 2009 5:18 pm
Contact:

Post by Xeron »

Additional info:

After an 0xff block, I will probably put another byte indicating the tape bit state before returning to normal decoding. I haven't yet, but I can see that it might be useful.

When implementing this, I found that the Oric ROM routines always output 1 more byte than is necessary. Saving a BASIC program saves a byte off the end of the BASIC program in memory. Likewise, doing:

CSAVE "",A#C000,E#C010

actually saves 17 bytes.

This extra byte is not (as far as I can tell) loaded back in, but I have updated the CSAVE patch to also write this extra byte in Oricutron, just to keep the same behaviour as the Oric ROM.

Edit: This byte actually is loaded! This explains why I always used to have to append a byte to the end of tap files to stop the Oric ROM from waiting for more data at the end when turbo tape was turned off!
User avatar
Symoon
Archivist
Posts: 2301
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France

Post by Symoon »

I think the .4K8 format that once was used by Euphoric was close to what you want to achieve, but Fabrice dropped it for (bigger) WAV because people didn't undertand what 4k8 was.
CSAVE "",A#C000,E#C010

actually saves 17 bytes.
Which seems absolutely normal, am I wrong?
#C000 to #C010 means you want to save from 0 to 16, which means 17 bytes.
User avatar
Xeron
Emulation expert
Posts: 426
Joined: Sat Mar 07, 2009 5:18 pm
Contact:

Post by Xeron »

Well, certainly CSAVE saves an extra garbage byte at the end of basic programs. Also most tap files from the internet are 1 byte shorter that the Rom routines try and read.
User avatar
Symoon
Archivist
Posts: 2301
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France

Post by Symoon »

Xeron wrote:Well, certainly CSAVE saves an extra garbage byte at the end of basic programs. Also most tap files from the internet are 1 byte shorter that the Rom routines try and read.
I don't get it - if it was so, these files would not load on Euphoric or real Orics. Where did you get them from?

That being said, I just made a quick test, it seems indeed that CSAVE"" (so, basic files) saves an extra byte, which is included in the loading addresses from the header. So if one removes this extra byte, the game won't load unless the end address is modified in the header.
I also seem to recall having noticed once, after having transfered the same basic program twice, that it had a different ending byte.
User avatar
Xeron
Emulation expert
Posts: 426
Joined: Sat Mar 07, 2009 5:18 pm
Contact:

Post by Xeron »

OK.. the extra byte stuff is just a sidenote... I might be wrong about the missing byte in .tap files, it seems my routine to convert .TAP to a squarewave for the ROM routine when turbo tape is turned off had a timing mistake, which could account for that.
User avatar
Xeron
Emulation expert
Posts: 426
Joined: Sat Mar 07, 2009 5:18 pm
Contact:

Post by Xeron »

Back on topic, one thing to bear in mind about the .ort format is that oricutron won't force it on you.

You can CSAVE directly to .TAP in Oricutron, and you'll be able to load .tap, .wav or .ort without having to do any conversion.

Its just that if you capture the raw tape output, you'll get a ".ort" file.
User avatar
Symoon
Archivist
Posts: 2301
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France

Re: New tape fileformat (.ort)

Post by Symoon »

BTW I must confess I haven't really understood the decoding process (what's happening if there's a data byte on the original program whose value is FF?), but if you ever need some conversion testing or exotic WAV files from protected tapes, I have quite a collection, just let me know when you'll be beta testing ;-)
User avatar
Xeron
Emulation expert
Posts: 426
Joined: Sat Mar 07, 2009 5:18 pm
Contact:

Re: New tape fileformat (.ort)

Post by Xeron »

Shimon wrote:whats happening if there's a data byte on the original program whose value is FF?
nothing. Read it more carefully :)
User avatar
Symoon
Archivist
Posts: 2301
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France

Re: New tape fileformat (.ort)

Post by Symoon »

Xeron wrote:
Shimon wrote:whats happening if there's a data byte on the original program whose value is FF?
nothing. Read it more carefully :)
Ok, I did and just understood that, by reading a (PC) byte, the value it holds defines a (Oric) bit :wink:
So you are using the higest values to do something else (I'm writing live as I'm thinking ;-))
What I still don't fully understand is:
- Why multiply by 2 some values? to match some 16-bit definition read by Oricutron?
- What's the purpose of FC and FD commands? (reading FC and the next byte is the same thing than reading a value smaller than FC, am I wrong?)
- the cycles waiting seems to depend on the byte value? I might begin to understand: with 1 byte, you defeine the bit value (0 or 1 according to < or > 128 I suppose), and its length (from 1 to 128), so FC and FD are used for longer lasting levels?
User avatar
Xeron
Emulation expert
Posts: 426
Joined: Sat Mar 07, 2009 5:18 pm
Contact:

Re: New tape fileformat (.ort)

Post by Xeron »

OK... heres how it works.

The format describes a squarewave as a list of timing values. Each value is just the number of cycles until the next time the signal flips.

If the value is less than 0xfc, we can just store it as a byte.

We divide the actual cycle count by 2, because this way the standard Oric timings always fit into one byte. It just makes for smaller files without too much loss of accuracy.

If you need to encode a longer value, you use one of four "special" codes. 0xFC enables you to encode values 0xFC, 0xFD, 0xFE or 0xFF and only costs you one extra byte.

0xFD allows you to store values from 0x100 to 0xFFFF.

0xFE is reserved for future use.

0xFF is a marker that says "the following bytes are encoded as standard Oric loader format", so you store the actual bytes rather than the waveform, to save space.

This approach takes a bit more space than Euphorics old 1 bit per 208 cycles format, but it should take less space than a .wav. It also should compress really well for most loader schemes.

The other advantage over Euphorics's old 1 bit ber 208 cycles format is that it can describe any arbitary waveform at any useful frequency.
User avatar
Symoon
Archivist
Posts: 2301
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France

Re: New tape fileformat (.ort)

Post by Symoon »

Thanks for the details, I now finally got it ;-)
That's intersting !
Still willing to help testing when you need it :wink:

PS: to complete the discussion about nowadays WAV format: it compresses in ZIP in a perfect way, since the signal is cleaned and the values used are always the same. Being able to read a WAV from a ZIP file would be a benefit for storage space saving, but that's another story!
User avatar
Dbug
Site Admin
Posts: 4437
Joined: Fri Jan 06, 2006 10:00 pm
Location: Oslo, Norway
Contact:

Re: New tape fileformat (.ort)

Post by Dbug »

Symoon wrote:Thanks for the details, I now finally got it ;-)
That's intersting !
Still willing to help testing when you need it :wink:

PS: to complete the discussion about nowadays WAV format: it compresses in ZIP in a perfect way, since the signal is cleaned and the values used are always the same. Being able to read a WAV from a ZIP file would be a benefit for storage space saving, but that's another story!
Well, what would be a benefit would also if the code that read WAV would support all the variants of WAV, being mono, stereo, 8/16/24 bit per chanel, compressed or not. Because basically right now you can't just record with whatever tool you happen to have and save to whatever WAV format the tool uses by default: In my experience it's not going to work, meaning that archiving oric games is a science requiring years of experience and fiddling :p
User avatar
Symoon
Archivist
Posts: 2301
Joined: Sat Jan 14, 2006 12:44 am
Location: Paris, France

Re: New tape fileformat (.ort)

Post by Symoon »

Dbug wrote:Well, what would be a benefit would also if the code that read WAV would support all the variants of WAV, being mono, stereo, 8/16/24 bit per chanel, compressed or not. Because basically right now you can't just record with whatever tool you happen to have and save to whatever WAV format the tool uses by default: In my experience it's not going to work, meaning that archiving oric games is a science requiring years of experience and fiddling :p
I'm afraid this would be very hard to achieve. IIRC Fabrice chose 44kHz because at a lower rate, he wasn't able to decode 100% correctly the signal. The signal from the tape is so distorded and the tape players are using different speed, that it's really complicated sometimes to know how long each level lasts. For instance, that's one of the things that prevented me from finishing my slow speed decoder: I don't recall in detail but it worked with slow WAV files produced by Euphoric, but not with slow files from tapes (which makes it quite useless :? )
Stereo: another challenge, which channel will you use to read the WAV? Some tapes only have one channel recorded, the 2nd one being silent or so low that it can't be read. And belive me: some tapes use the left channel and, of course, other use the right channel :twisted:
User avatar
Dbug
Site Admin
Posts: 4437
Joined: Fri Jan 06, 2006 10:00 pm
Location: Oslo, Norway
Contact:

Re: New tape fileformat (.ort)

Post by Dbug »

I must be a total retard, but in oricutron 0.9 I tried to do a normal CSAVE"FILE" and did not get a file in my oricutron folders.
The ORT export worked, but I wanted a TAP file, and failed exporting it.
Post Reply