Cumulus Firware Compilation

This is the right place to discuss on how to implement hardware vsync, adding a VIA or AY chipset, puting multiple roms, or how to design a new flash expansion card.
Godzil
Squad Leader
Posts: 774
Joined: Sat May 21, 2011 7:21 pm
Location: Between UK and France
Contact:

Re: Cumulus Firware Compilation

Post by Godzil »

Chema: Have you verified that the "card_type" variable is correctly set?
Last edited by Godzil on Fri Jan 23, 2015 1:46 pm, edited 1 time in total.
User avatar
Chema
Game master
Posts: 3013
Joined: Tue Jan 17, 2006 10:55 am
Location: Gijón, SPAIN
Contact:

Re: Cumulus Firware Compilation

Post by Chema »

Yes. At least when the card is an SD it is detected as such (I plot this info whenever an error occurs). ¿Any hints?
User avatar
Chema
Game master
Posts: 3013
Joined: Tue Jan 17, 2006 10:55 am
Location: Gijón, SPAIN
Contact:

Re: Cumulus Firware Compilation

Post by Chema »

Just FYI, I traced the error again back to the card read code, this time at the routine that reads a sector. It is failing. The thing is that I am not sure if this is a somewhat intermitent error or what.

For those interested the code is:

Code: Select all

/* Reads one sector of 512 bytes from the card */
uint8_t card_read(uint32_t sector, uint8_t* buffer) 
{
	uint16_t i;
	uint8_t result;
	uint16_t timeout_cntr;
	uint8_t card_response; 
	
	uint32_t address = (card_type == CARD_TYPE_SDHC) ? (sector) : (sector << 9);		
		
	/* Lower Chip Select */
	PORTCbits.RC2 = 0;

	/* Send CMD17 */
	timeout_cntr = 0;
	result = card_command(0x51, (uint16_t) (address >> 16), (uint16_t) address, 0xFF);	
				
	while (result != 0)
	{
		result = SPI(0xFF);
		timeout_cntr ++;
		if (timeout_cntr > 48000)
                {
                    gErrorCode=20;
			return 0;
                }
	}
			
	/* Wait until 0xFE is received */
	timeout_cntr = 0;
	while ((card_response = SPI(0xFF)) == (uint8_t) 0xFF)
	{		
		timeout_cntr ++;
		if (timeout_cntr > 48000)
                {
                        gErrorCode=21;
			return 0;
                }
	}
		
	if (card_response != 0xFE)
        {
            gErrorCode=22;
            return 0;
        }
	
	/* Read data */
	for(i = 0; i < 512; i ++) 
		buffer[i] = SPI(0xFF);  
			
	/* Receive CRC */
	SPI(0xFF); 
	SPI(0xFF);
		
	/* Let the card finish */
	SPI(0xFF); 
			
	return 1;	
}
You see I added a global variable with the error code for debugging. One of the cards (the small one) fails with a code 21 and the second with a code 22.

The code is correct, I think. Except the fact that the timeouts are not "timed" but done through loops. The card is answering correctly to most comands when they are small, but seems to fail in this routine which reads a lot of data.

Is it possible that the Cumulus is using a clock which could be too fast for old SD cards, so that receiving bytes becomes error prone?
Godzil
Squad Leader
Posts: 774
Joined: Sat May 21, 2011 7:21 pm
Location: Between UK and France
Contact:

Re: Cumulus Firware Compilation

Post by Godzil »

Tilt!

Could you give me the frequency on the quartz of the cumulus?
User avatar
Chema
Game master
Posts: 3013
Joined: Tue Jan 17, 2006 10:55 am
Location: Gijón, SPAIN
Contact:

Re: Cumulus Firware Compilation

Post by Chema »

I don't know...
Godzil
Squad Leader
Posts: 774
Joined: Sat May 21, 2011 7:21 pm
Location: Between UK and France
Contact:

Re: Cumulus Firware Compilation

Post by Godzil »

No problem I found it in the BOM (32Mhz)


I'm not sure the frequency is a real problem, but let's try:


Test with this in the code after the "Speed up the clock" comment:

Code: Select all

    /* Speed up the clock */
    if (card_type == CARD_TYPE_SDHC)
    {
    	SSPCON1 = 0x10; // CKP High, SPI Master, clock = Fosc/4
    }
    else
    {
    	SSPCON1 = 0x11; // CKP High, SPI Master, clock = Fosc/16
    }
Remove the old "SSPCON1 = " lines and replace them by the one I've but on top.

If it still does not work, try to set SSPCON1 to 0x12 in the else, it's worth the try.
User avatar
Chema
Game master
Posts: 3013
Joined: Tue Jan 17, 2006 10:55 am
Location: Gijón, SPAIN
Contact:

Re: Cumulus Firware Compilation

Post by Chema »

No luck. Same symptoms. However I was able to produce a dump of the sector read just before the I/O Error (the first bytes). Everything seems to work and the sector from the boot sector of the first partition is correctly loaded in memory. For both cards. Which means the MBR is decoded correctly.

This is a partial output from a dump done on a Windows box (it is a valid boot sector from the 1st partition of my 1GB card)

Code: Select all

EB 00 90 20 20 20 20 20 20 20 20 00 02 20 01 00 02 00 02 00 00 F8 ED 00 3F 00 20 00 85 00 00 00 7B 97 1D 00 80 00 29 E5 50 F7 FF 4E 4F 20 4E 41 4D 45 20 20 20 20 46 41 54 31 36 20 20 20 00 00
Decoded, this means:

Code: Select all

FAT16 filesystem found from partition 0
  Jump code: EB:00:90
  OEM code: [        ]
  sector_size: 512
  sectors_per_cluster: 32
  reserved_sectors: 1
  number_of_fats: 2
  root_dir_entries: 512
  total_sectors_short: 0
  media_descriptor: 0xF8
  fat_size_sectors: 237
  sectors_per_track: 63
  number_of_heads: 32
  hidden_sectors: 133
  total_sectors_long: 1939323
  drive_number: 0x80
  current_head: 0x00
  boot_signature: 0x29
  volume_id: 0xFFF750E5
  Volume label: [NO NAME    ]
  Filesystem type: [FAT16   ]
  Boot sector signature: 0xAA55
It corresponds to the data read by Cumulus. What puzzles me is that if data is correct then it should be interpreted correctly too, so I don't know why there are any errors...

When you send a CMD17 to read a sector, the card answers 0 and then it should send a token 0xFE before the actual data. The 0xFE is not received correctly ant the read is not performed. Then a I/O Error is issued.

So maybe the second time it tries to access the disk to get the root directory is when it all fails?

I am quite sure that it also failed once because of the sectors per cluster being less than 16 (which is not possible from the real dumps); but maybe I did something wrong...
Godzil
Squad Leader
Posts: 774
Joined: Sat May 21, 2011 7:21 pm
Location: Between UK and France
Contact:

Re: Cumulus Firware Compilation

Post by Godzil »

By the way, the "timeout" code is I think invalid, retromaster code seems to have been really inspired by the Arduino SD library (not exactly the same, but there are some really similar logic) the Arduino use the same number for timeout, but the number is in milisec. Here it is a number of loop, which is quite different.

If the clock change does not give results, I propose that the work on the UI refactor will trigger also some changes in the SD management, We may need to do more proper things, and I suspect that adding timer here for a timeout will be just a nightmare to add and test.

Do you know the number of the sector that fail?

What you can do, if you fear to loose everything on your SDHC card, is by using the rawwrite tool make an image of your 1GB SD card and write it to your SDHC card, they will be proposing the exact same values for the first 1GB and as the partition map tell that there is only a 1GB partition and read sectors will be the same for all the Filesystem structures.

Also for the specific timeout that fail, multiply the number by 8 or 10 just to see.
User avatar
Chema
Game master
Posts: 3013
Joined: Tue Jan 17, 2006 10:55 am
Location: Gijón, SPAIN
Contact:

Re: Cumulus Firware Compilation

Post by Chema »

And what if we are facing a combination of factors? The first patch I did with the loop waiting for a response in the initialization must be there, but may not be all that is needed.

One of my SD cards was formatted as FAT32 from Windows. When I formatted my working SDHC card from windows it stopped working. Maybe it is the same problem here.

The second one is formatted in FAT16, the SD Formatter utility forces this for non HC cards. Cumulus code is thought to support FAT32, with or without MBR. What if it does not work with FAT16?

The root sector address is at a fixed location in FAT16, but not in FAT32, where four bytes in the boot sector indicates the location. If the handles this incorrectly it may end up looking for an invalid sector number, so an I/O error will occur either due to non-existent sector, or because the data is not a real root directory data. That could be the cause of errors in the fat routines where I saw them!

I think I will have a look and see if I can patch the code so it handles fat16....

Then I will try to figure out why windows formatting cards are not working either...
Godzil
Squad Leader
Posts: 774
Joined: Sat May 21, 2011 7:21 pm
Location: Between UK and France
Contact:

Re: Cumulus Firware Compilation

Post by Godzil »

Big warning when toying with firmware and test on the cumulus!!!

As always PIC errata list are as a long arm, there is one which is really annoying and should take with huge concideration, the endurance of the programm flash is really really low: about 1000 erase cycles. This is 100x time less than what the datasheet still say ( http://ww1.microchip.com/downloads/en/D ... 39564c.pdf )

So every firmware programming will reduce the life span of your cumulus. 10 test a day == life of 100 days, which is really short. You should be careful not testing things just because you want to see how it behave.

For the curious, the current errata list is here: http://ww1.microchip.com/downloads/en/D ... 00404H.pdf

I really don't want to, but we may need to move to another uC on a reworked cumulus...
User avatar
Chema
Game master
Posts: 3013
Joined: Tue Jan 17, 2006 10:55 am
Location: Gijón, SPAIN
Contact:

Re: Cumulus Firware Compilation

Post by Chema »

Alas! I did not know that! So I am drastically reducing the life time span of my Cumulus???? I have updated the firmware probably 30 times by now!!!

And once the PIC dies, replacing it won't be easy (it is not socketed)... oh my!

I might do a couple tests more and leave it then. It is a pity because I think I am quite close. I went as far as to load the directory of a FAT16 SD card on Cumulus, though it did not correctly read the disk contents (could not mount the image on a drive). As I suspected the code only supports FAT32 so the thing was:
- Official SD formatting utility forces FAT16 for SD cards and FAT32 for SDHC cards.
- Windows allows formatting them in FAT32, but then again windows formatted cards are not recognized by cumulus (don't know why: it is not a simple thing about having an MBR or not).

I hacked the code to support FAT16 to a certain extent. I ran into many points where it is not compatible and solved most of them. In fact I have a code that runs on my Windows Box and which is now able to correctly read the files on images of SD/FAT16 cards as well as SDHC/FAT32. I only have to test these modifications in my cumulus to see if they work.

But even if they do, the code is just a quick and dirty hack which might not work in all the situations and still the problem of Windows-formatted cards is there. But I won't risk my cumulus trying to find the problem.

So I am a bit stuck. :(

Oh, and I even dreamed with an extension for commands so the Oric could talk to the Cumulus directly which would open the possibility not only of mounting/managing disk images from the Oric, but also to write a whole new OS and make Cumulus work as a kind of Hard Disk drive unit with a FAT filesystem on the SD card.

Dammit!
Godzil
Squad Leader
Posts: 774
Joined: Sat May 21, 2011 7:21 pm
Location: Between UK and France
Contact:

Re: Cumulus Firware Compilation

Post by Godzil »

Chema wrote:Alas! I did not know that! So I am drastically reducing the life time span of my Cumulus???? I have updated the firmware probably 30 times by now!!!

And once the PIC dies, replacing it won't be easy (it is not socketed)... oh my!

I might do a couple tests more and leave it then. It is a pity because I think I am quite close. I went as far as to load the directory of a FAT16 SD card on Cumulus, though it did not correctly read the disk contents (could not mount the image on a drive). As I suspected the code only supports FAT32 so the thing was:
- Official SD formatting utility forces FAT16 for SD cards and FAT32 for SDHC cards.
- Windows allows formatting them in FAT32, but then again windows formatted cards are not recognized by cumulus (don't know why: it is not a simple thing about having an MBR or not).

I hacked the code to support FAT16 to a certain extent. I ran into many points where it is not compatible and solved most of them. In fact I have a code that runs on my Windows Box and which is now able to correctly read the files on images of SD/FAT16 cards as well as SDHC/FAT32. I only have to test these modifications in my cumulus to see if they work.

But even if they do, the code is just a quick and dirty hack which might not work in all the situations and still the problem of Windows-formatted cards is there. But I won't risk my cumulus trying to find the problem.

So I am a bit stuck. :(

Oh, and I even dreamed with an extension for commands so the Oric could talk to the Cumulus directly which would open the possibility not only of mounting/managing disk images from the Oric, but also to write a whole new OS and make Cumulus work as a kind of Hard Disk drive unit with a FAT filesystem on the SD card.

Dammit!

30 is not enough to "kill" the chip, you can do more, but yes, even if it not exactly true that 1K erase are the real limit, it could be a bit more, shouldn't be less. I wasn't aware also about this defect, the chip is rated in his datasheet for 100K erase, you still have available "retry" to test your stuff.

I really fear that we will need a real hardware revision (changing the uC to something "better")...
User avatar
iss
Wing Commander
Posts: 1637
Joined: Sat Apr 03, 2010 5:43 pm
Location: Bulgaria
Contact:

Re: Cumulus Firware Compilation

Post by iss »

Nice finding, Godzil.
But guys, don't be despondent - http://hypnocube.com/2014/11/flash-endurance-testing/
IMO, the case is not so dramatic.;)
User avatar
Chema
Game master
Posts: 3013
Joined: Tue Jan 17, 2006 10:55 am
Location: Gijón, SPAIN
Contact:

Re: Cumulus Firware Compilation

Post by Chema »

Made a test... not complete success :( Now my Cumulus mounts the SEDORIC.DSK disk image from my SD card and starts booting, but it hangs when reading sector 2 (as displayed by Cumulus) with a 'Record not found' message.

It seems my patch worked for the first sector of 512 bytes in the disk, but some bug is lurking around preventing the correct load of the second....

Small steps...

BTW, for a total rewrite I'd suggest using a decent fat library. There are sources out there with everything to support many different formats, they are mantained and include routines for fread/fwrite individual files of any kind...
Godzil
Squad Leader
Posts: 774
Joined: Sat May 21, 2011 7:21 pm
Location: Between UK and France
Contact:

Re: Cumulus Firware Compilation

Post by Godzil »

The problem is, the chip maker rate for a number of MINIMAL cycle from a gaussian point of view. 80-90% of the manufactured chip will follow what they say. If the errata says 1K erase cycle, you should not expect more, even if some chip support more.

And we can't afford to have a bad block in the flash of the uC as it is pure code and not a filesystem like SD card or usb drive stick where we can do tricks like wear leveling.

Anyway such a test does not proove anything and if microchip change the specs from 100K to 1K there is a reason: the flash inside this chip is not reliable after a certain amount of erase cycle. (and we definitely can't stand for a bad block on the device)
Post Reply