Loci - my Oric storage emulation project

[Sodiumlightbaby]

Hello readdir! A bit slower progress of late with limited playtime, but we have as of late yesterday basic opendir/readdir/closedir functions implemented across the barrier from Oric to CUmini and the USB storage device! Doesn't look like much, oboy that felt good to get working

cumini_rom_readdir_call.jpg

[Dbug]

Will that work only with the local USB storage, or can that also work with the CUmini connected to a PC, and be used for example to load stuff from the PC, debug the Oric from a PC, etc...?

[ibisum]

Amazing progress!

For my part, now that you'vd got readdir/etc. working and probably have a synchronization point in place, I wonder if a Wifi SD Card can now be used with CUmini, so that indeed, the Oric storage subsystem could be remotely accessible/administered.

Something like:

https://www.amazon.de/MASUNN-Ezshare-Ad ... 7FW17TZ4/
Fun times ahead!

[Sodiumlightbaby]

Will that work only with the local USB storage, or can that also work with the CUmini connected to a PC, and be used for example to load stuff from the PC, debug the Oric from a PC, etc...?

In this setup CUmini works as a USB host device, so only local storage directly. For connecting to a PC it would need to e.g. be by a USB serial adapter on the CUmini side and connection via real RS232 to the PC. Ok if you have an old PC or don't mind plugging two USB serial adapters together

A more modern userfriendly option for developers or pro users I have had in mind is having a USB device based version of the firmware. You lose the option of connecting storage and HID devices, but it would allow control from the PC to Oric by only using the USB cable (implementing CDC device on CUmini side).

It's possible to get the same functionality both ways, it just requires more interfacing HW when both CUmini and the PC are USB hosts. The TinyUSB stack doesn't look like its easy to have both device and host implemented at the same time, allowing run or boot-time switching.

[Sodiumlightbaby]

Amazing progress!

For my part, now that you'vd got readdir/etc. working and probably have a synchronization point in place, I wonder if a Wifi SD Card can now be used with CUmini, so that indeed, the Oric storage subsystem could be remotely accessible/administered.

Something like:

https://www.amazon.de/MASUNN-Ezshare-Ad ... 7FW17TZ4/
Fun times ahead!

Interesting. I'll have to have a look at how that works. So far, my remote storage/control/comms thinking has been centered around USB CDC serial, and possibly RNDIS as there's support for host RNDIS mode in TinyUSB and I've seen a project to make a PicoW into a driverless RNDIS wifi dongle.
https://github.com/sidd-kishan/PicoPiFi

[Bieno]

I didn't know about this. It's fantastic!

[Sodiumlightbaby]

Ugh, the host network support (CDC-RNDIS-HOST) in TinyUSB is very much WIP. I tried to enable that module in the USB stack and it fails to compile cleanly. It may be due to platform assumptions but I'm not going to spend time on that now. I think perhaps a better solution also resource wise will be to take the PicoPiFi idea but change it to look more like a serial/modem on the USB side, i.e. change the USB profile on the Pico W to a simple CDC serial profile. Sounds like something that should already be out there, if not - it shouldn't be too hard to do.

[Sodiumlightbaby]

I didn't know about this. It's fantastic!

Assuming 'this' is CUmini, glad you like it It will be a little longer before it is ready and probably a good while before it is polished, but I really hope I'm able to give something back to this community.

[Sodiumlightbaby]

First mount! So that's proof of life using rudimetary browsing on Oric of the USB storage device connected to the CUmini expansion board, selecting a dsk file, and having it virtually mounted to the Microdisc FDC emulator on the CUmini. As I'm don't have the "exit to Oric" functionality in place, this is as far as it goes, but remember we've done all the actual (basic) FDC emulation already.

Edit: Changing dsks works as expected as well.

While doing the mount and unmount functions I had some second thoughts on the config approach, so I'll need to have a think before I progress further on the ROM and investigate more some the config approaches of RP6502.

[Sodiumlightbaby]

Status: Slow but steady progress.

Software
Cleaned up the directory sorting(case insensitive as Dbug requested ) and navigation a bit making it ready for stepping through larger directories, ignoring unix like hidden files (.*), added navigated path on the border and we're now showing directories with '/' prefix. Not sure I've landed on the best solution for switching back to the Oric, but for now I've implemented a "boot" operation on the Oric to CUmini API and after a little debugging we have the first sequence of
1. Boot to CUmini ROM
2. Navigate to a tap file
3. Mount tap file in the emulated cassette drive
4. Escape from CUmini and boot to Oric Basic
5. CLOAD
6. Running application

cumini_rom_tap_dir.jpg

cumini_rom_tap_selected.jpg

cumini_rom_tap_running.jpg

Why tap file instead of dsk? Well, I need to switch emulated ROM banks for the Microdisc device ROM to boot instead of the Basic ROM and I realised I need to move some of that code on the CUmini firmware to make it dynamic with the new boot operation. Not difficult, but that means the tap file gets the first win

One question for the back-end directory listing. What has others done when it comes to allocating memory for the listing? How large is good enought? Right now I'm trying to keep things very clean with just standard readdir type interface into CUmini, which means slurping up all the filenames for a directory before sorting and presenting. I have a plan/hope to strictly limit the amount of RAM the ROM uses so save and restore can work. Right now I'm using a fixed buffer filling up with filenames from the head and pointers to the names from the tail. I can make the buffer reasonable large, but if the directory is too large, either the listing becomes slow (sorting a window of the full directory) or confusing (each directory listing chunk is sorted separately).

Still loads to do on the ROM side like device navigation, configuration management etc, but once dsk loading is in place we actually have the first working (if ever so barely ) end-to-end tap and dsk loading CUmini prototype.

Hardware
In parallel, the next hardware prototype is getting closer and I think I have a candidate solution for making the power connection safe but low voltage drop. It's possible the diode is switched to a modern power distribution switch. It's similar to an ideal diode circuit (managed FET basically) but with some reverse current protection and current limiting features. I've not used these before, but they look promising and a very close to the right tool for the job. Removing the voltage drop on the 5V means feel good about going ahead with the CUmini (minimal + low cost) concept for now.

The I2C IO expander is still giving me some uncertainty. The safe route is to upscale to a capable 1.7MHz part, but that comes with a cost. Probing the RP2040 it is clear it is not actually running at the requested I2C clock rate. It's still close to 1MHz so clearly out of spec, but I need to do some more testing here to check what it did when requested I2C clock was set to 400kHz before I swap that part.

[Dbug]

One question for the back-end directory listing. What has others done when it comes to allocating memory for the listing? How large is good enought? Right now I'm trying to keep things very clean with just standard readdir type interface into CUmini, which means slurping up all the filenames for a directory before sorting and presenting. I have a plan/hope to strictly limit the amount of RAM the ROM uses so save and restore can work. Right now I'm using a fixed buffer filling up with filenames from the head and pointers to the names from the tail. I can make the buffer reasonable large, but if the directory is too large, either the listing becomes slow (sorting a window of the full directory) or confusing (each directory listing chunk is sorted separately).

I suppose that's for when using the file system API to use the Cumini as a type of large storage device?
If yes, I've never used anything like that on the Oric, so not sure I've relevant proposals to do.

Regarding readdir, I suppose you mean running this type of code on the Oric to list a complete folder content?

Code: Select all

DIR *dir;
struct dirent *ent;
if ((dir = opendir ("c:\\src\\")) != NULL) {
  /* print all the files and directories within directory */
  while ((ent = readdir (dir)) != NULL) {
    printf ("%s\n", ent->d_name);
  }
  closedir (dir);
} else {
  /* could not open directory */
  perror ("");
  return EXIT_FAILURE;
}

I guess having "standard libraries" is nice, but if it's unusable because it uses all the memory, it's indeed kind of useless if all the memory is used.

I'm assuming all the data is available on the Cumuni, and technically you can just return the next batch of data when "readdir" is called, meaning we can keep a single directory entry if we want, but indeed that makes sorting a problem.

If API compatibility is not an issue, a possibility could be to extend the opendir function, or add some global context feature (a bit like "locale") to indicate the behavior we want, like "please return the directory sorted alphabetically, or by date inverted".

My take on all these things is that pragmatism and practicality wins over standard and correctness (as long as everything that works differently than from the expected is documented of course).

Sorry for not being more useful

[Sodiumlightbaby]

I'm assuming all the data is available on the Cumuni, and technically you can just return the next batch of data when "readdir" is called, meaning we can keep a single directory entry if we want, but indeed that makes sorting a problem.

Yes, readdir returns one file entry from the USB file system each time you call it so the issue is really only when you want to present everything nice and sorted. On the CUmini side the processor is much faster but RAM is still limited so the problem would be very close to the same on that side.

Here's the current directory listing code in the CUmini ROM.

Code: Select all

/* Fill the directory buffer with filenames from the bottom
   and pointers from the top
*/
void dir_fill(char* dname){
    DIR* dir;
    struct dirent* fil;
    uint16_t tail;     //Filename buffer tail
    dir = opendir(dname);
    
    strcpy(dir_buf,"/..");
    dir_ptr_list[-1] = dir_buf;
    tail = 4; //strlen("/..")+1
    dir_entries = 1;


    while(tail < DIR_BUF_SIZE){             //Safeguard
        do {
            fil = readdir(dir);
        }while(fil->d_name[0]=='.');        //Skip hidden files
        if(fil->d_name[0] == 0){            //End of directory listing
            break;
        }
        dir_ptr_list[-(++dir_entries)] = &dir_buf[tail];  
        if(fil->d_attrib & DIR_ATTR_DIR){
            dir_buf[tail++] = '/';
        }else{
            dir_buf[tail++] = ' ';
        }
        strncpy(&dir_buf[tail], fil->d_name, DIR_BUF_SIZE-tail-(dir_entries*sizeof(char*)));
        tail += strlen(fil->d_name) + 1;
    }

    closedir(dir);
    qsort(&dir_ptr_list[-(dir_entries)], dir_entries, sizeof(char*), dir_cmp);
}

But having slept on it, I now think it's probably going to be a miserable user experience anyway trying to browse through endless of files and I've started to think a limited buffer (e.g. 4kB directory buffer for a few hundred files), feedback if buffer runs full, plus a user accessible filter function that narrows the number of files we want to present, is a much better and friendly solution. It could also default to *.dsk when you try to mount something to an emulated floppy, and *.tap when trying to mount something to the cassette.

Sorry for not being more useful

On the contrary, I find the technical comments and questions here help me a lot looking at things from angles I've not considered and nudge me on to better directions. It is much apprechiated.

[Dbug]

Enforcing limits is definitely not a bad idea, we have a limited screen real estate and limited amount of memory, so it would be better for people to use multiple folders with a limited number of file entries per folder than allowing them to have thousands of files in a single directory: On a machine like the Oric that's simply unmanageable.

[Sodiumlightbaby]

Fully agree so limit it is then!
You get the warning which you can dismiss and continue to select a file if you see it, but the directory view will be incomplete. This test is with a 1kB buffer and 256 files in a directory, but as noted before the buffer size is easy to tune and I'm likely going with 4kB later.

cumini_rom_dir_max_files.jpg

No filter function implemented yet though, so currently the message lies And paginated browsing of the directory listing (the files that actually are in the buffer) is also missing. So loads of TODOs.

[ibisum]

Hey, leave some of the fun work for some of us to contribute!