SampleTweaker

Questions, bug reports, features requests, ... about the Oric Software Development Kit. Please indicate clearly in the title the related element (OSDK for generic questions, PictConv, FilePack, XA, Euphoric, etc...) to make it easy to locate messages.

User avatar
jbperin
Flight Lieutenant
Posts: 480
Joined: Wed Nov 06, 2019 11:00 am
Location: Valence, France

SampleTweaker

Post by jbperin »

Hello

I can't find the documentation of the SampleTweaker program that comes with OSDK.

Is there any chance that this tool works with 8 bits samples dispatched on two channels?


Thanks
User avatar
Dbug
Site Admin
Posts: 4444
Joined: Fri Jan 06, 2006 10:00 pm
Location: Oslo, Norway
Contact:

Re: SampleTweaker

Post by Dbug »

It's mostly that it's a tool that's pretty much unusable by itself, each time I used it I modified it to test new stuff, it only load raw files (so can't load wav, etc...), I would not recommend using it as is.
User avatar
jbperin
Flight Lieutenant
Posts: 480
Joined: Wed Nov 06, 2019 11:00 am
Location: Valence, France

Re: SampleTweaker

Post by jbperin »

Isn't this tool the only way to prepare samples to be used with the digiplayer.s ?

I think I can do something similar in Python .. and add a few options such as support of ogg, wav, mp3 ..

And as it is possible to make exe from Python script, perhaps I can prepare an alternative to SampleTweaker .

But it would add some dependencies in OSDK build process (Python plus some extra libs)
User avatar
Dbug
Site Admin
Posts: 4444
Joined: Fri Jan 06, 2006 10:00 pm
Location: Oslo, Norway
Contact:

Re: SampleTweaker

Post by Dbug »

The digiplayer is not a part of the OSDK, thus the tools is not officially part of it either.
The easiest way to do the wav to raw is to use something like SoX (http://sox.sourceforge.net)

Digiplayer.s is pretty much different in all the versions of my demos, some replay in 8bit using conversion tables over the three channels (and for that you don't need SampleTweaker at all, you can just use the RAW 8bit unsigned sample), some replay are in 4bit using various types of encoding, with different linear adaptations to compensate for the log volume.

Just look at the code, it's shit:
https://osdn.net/projects/oricsdk/scm/s ... r/main.cpp

Half of it is commented out, there are some ifdef, some stuff is for Oric some is for Atari.

Feel free to take snippets of it and make your own tool, that one is not worth investing any time to make usable by most people.
User avatar
Dbug
Site Admin
Posts: 4444
Joined: Fri Jan 06, 2006 10:00 pm
Location: Oslo, Norway
Contact:

Re: SampleTweaker

Post by Dbug »

I guess if I want to have a audio sample replay routine in the OSDK sample code, I would need to get the converter updated so it works out of the box.

Adding things like FLAC, MP3, AIFF or OGG would be complicated, but I guess I could add something like
Dr Wav which seem to have a license we can deal with, so at least that would provide native loading code for WAV files.

Not sure what the feature set should be, but I guess:
- Resampling to a specific replay frequency
- Number of bits (2 bits, 4 bits, 8 bits, ...)
- Other?
User avatar
jbperin
Flight Lieutenant
Posts: 480
Joined: Wed Nov 06, 2019 11:00 am
Location: Valence, France

Re: SampleTweaker

Post by jbperin »

If you want to read any kind of audio file and remain in C you can have a look at libsndfile:

http://www.mega-nerd.com/libsndfile/

But really I could not give you a better advice than considering using python with pysoundfile and numpy. Python can generate exe file just as C compiler can do and trust me python is far more convenient than C for what you want to do.

Opening a sound file (whatever the format, WAV, MP3, OGG, FLAC .. ) = 1 line of code in python
Resampling the audio = 2 lines of code in python
Adapting signal level = 1 line of code
Generate buffer in C or ASM from samples = 10 lines of code
Dealing with command line argument parsing = 20 lines of code.

If you want I can prototype a small sample tweaker in python for you so that you can check the performance that can be reached and see how simple the code can be.

In my opinion, the only bad aspect of using python for this task could potentially be the size of the resulting exe file. But I am very confident for aspects such as sustainability, performance and feature power.
User avatar
Dbug
Site Admin
Posts: 4444
Joined: Fri Jan 06, 2006 10:00 pm
Location: Oslo, Norway
Contact:

Re: SampleTweaker

Post by Dbug »

jbperin wrote: Tue Aug 10, 2021 9:46 pm If you want to read any kind of audio file and remain in C you can have a look at libsndfile:
http://www.mega-nerd.com/libsndfile/
Unfortunately does not support MP3.
jbperin wrote: Tue Aug 10, 2021 9:46 pm But really I could not give you a better advice than considering using python with pysoundfile and numpy. Python can generate exe file just as C compiler can do and trust me python is far more convenient than C for what you want to do.
I'm not using C, I'm using C++, which support pretty much everything you have in python (dictionaries, strings, etc...) so really using Python's only advantage is that you can use it without compiling, which in the case of the OSDK is a non issue.

And as you mentioned, using python, you have the definitive risk of having much larger executables, and what you did not mention: An order of magnitude slower (you are the one who said your Viterbi python script required 1 hour crunching for 4 seconds of audio).

I'm fine with python as a prototyping of scripting language, not as a development language.

Also I try to keep the OSDK as independent of external dependencies as possible, I'm not going to add support for Python on top of what we already has.
User avatar
jbperin
Flight Lieutenant
Posts: 480
Joined: Wed Nov 06, 2019 11:00 am
Location: Valence, France

Re: SampleTweaker

Post by jbperin »

Dbug wrote: Wed Aug 11, 2021 8:58 am I'm not using C, I'm using C++, which support pretty much everything you have in python (dictionaries, strings, etc...) so really using Python's only advantage is that you can use it without compiling, which in the case of the OSDK is a non issue.
Even though C++ offers some of the language facilities that Python has, the python language is far more expressive and powerful than C++.
Its power also relies on the huge number of library that exist and the ease of use of these libraries.
A python script can be run as a script OR be compiled into an executable file.
When an OSDK user download the OSDK, he gets a set of executable file. He doesn't care if this executable was programmed in C++ or Python.
It's none of its business. He just want the functionality.

Dbug wrote: Wed Aug 11, 2021 8:58 am And as you mentioned, using python, you have the definitive risk of having much larger executables, and what you did not mention: An order of magnitude slower (you are the one who said your Viterbi python script required 1 hour crunching for 4 seconds of audio).
I did mention the fact that I was confident in the performance level of Python for the task of processing audio.
To help clarify things, what's slow in the Viterbi python script is NOT the python script, it's the Viterbi algorithm. Do not confuse things.
Dbug wrote: Wed Aug 11, 2021 8:58 am I'm fine with python as a prototyping of scripting language, not as a development language.
Also I try to keep the OSDK as independent of external dependencies as possible, I'm not going to add support for Python on top of what we already has.
As I said above, if you deliver an executable .. the OSDK's user has nothing to do with Python and Python will only be an external dependency for the one that builds the OSDK. Not for the one that use the OSDK.
I've been a developer for 20 years now. I used dozens of language in my career. And when I'm given the choice of the language for a new development, my first choice clearly goes to Python (except in some very rare exception of: critical performance required, web apps or android apps).
User avatar
Dbug
Site Admin
Posts: 4444
Joined: Fri Jan 06, 2006 10:00 pm
Location: Oslo, Norway
Contact:

Re: SampleTweaker

Post by Dbug »

Ok, let's stop that right here, because the choice of language is not up for debate.
- You are entitled to think that Python is awesome and want to you use it for your own projects
- I am the one maintaining the OSDK and I am entitled to use what I think is best for the project, and what I am confortable with

As benevolent dictator of the OSDK, only C (for legacy stuff) and C++ and allowed, but the codebase is available and you can fork it if you want, the fact the user does not care about the language has nothing to do with the fact that I AM the one who has to deal with the language choice.

Now regarding Python vs C++:
- C++ has most probably at least as many libraries than Python
- Python 2 to 3 upgrade has been an ugly process, also means you can't just use old Python libraries out of the box, while we are still able to compile the OSDK C Compiler code base which is many decades old.
- Give me some examples of expressiveness and power of Python over C++
- In my experience, compiled Python has always been much slower than C++, sometimes by one order of magnitude

Regarding the final statement: There is a difference between developing for you, and developing for others.

My moral compass tell me that when I release something to the world, it has to be as efficient as it can possibly be, which is why I statically link everything so there are no runtime dependencies requirement for the users), all is compiled in 32bit instead of 64 so it runs fine on every single version of Windows down to Windows 95 (at least it did last time I checked) and Wine as well.

(By the way "been developer for X years" has never mattered, many of us have started programming when we were children, so by that count I've been a developer for 37 years.)
User avatar
jbperin
Flight Lieutenant
Posts: 480
Joined: Wed Nov 06, 2019 11:00 am
Location: Valence, France

Re: SampleTweaker

Post by jbperin »

Dbug wrote: Wed Aug 11, 2021 3:49 pm Ok, let's stop that right here, because the choice of language is not up for debate.
- You are entitled to think that Python is awesome and want to you use it for your own projects
- I am the one maintaining the OSDK and I am entitled to use what I think is best for the project, and what I am confortable with

As benevolent dictator of the OSDK, only C (for legacy stuff) and C++ and allowed, but the codebase is available and you can fork it if you want, the fact the user does not care about the language has nothing to do with the fact that I AM the one who has to deal with the language choice.
OK I totally agree with these statements.
C and C++ are used for OSDK because YOU like this language and YOU are the one who maintains OSDK.
YOU are a dictator. Bear in mind that being a dictator make you decide but it doesn't make you necessarily tell truth.
I am not a dictator, and I am not the one who maintains OSDK.
That's the reason why I only ADVISED to use Python (while being perfectly aware that it doesn't only have good aspect)
I will clearly not impose Python (I can't), neither insist to have it used in OSDK.
But I strongly disagree with the fact that Python is only for prototyping and is reluctantly slow.
I feel able to struggle against these affirmations .. even if I'm facing a dictator who tries to hide behind wrong arguments.
Dbug wrote: Wed Aug 11, 2021 3:49 pm - Give me some examples of expressiveness and power of Python over C++
First example : THIS SCRIPT which:
  • takes less than 200 lines
  • was written by a non-talented developer (me) in ONE SINGLE day
  • do the same as SampleTweaker + Bin2Txt
This script is not noticeably slower than using SampleTweaker + Bin2Txt (even faster because only one system call) but this script :
  • accept tons of audio format/encoding as input
  • can extract audio part from numerous kind of audio/video format (provided ffmpeg is installed on the machine)
  • automatically detect file format, encoding and sampling rate
  • mix stereo to mono if required
  • resample signal to whatever replaying frequency is asked (downsampling or oversampling with interpolation if required)
  • can generate buffer in ASM or C format
  • offers a command line interface for programmatic usage
Here's the command help:

Code: Select all

usage: SampleTweaker [-h] [--rfreq REPLAYFREQUENCY] [--outrad OUTPUTFILENAMERADICAL]
                     [--outvar OUTPUTVARIABLENAME] [--language {c,asm}] [--nchan {1,2}]
                     [--samplerate SAMPLERATE]
                     [--subtype {PCM_S8,PCM_16,PCM_24,PCM_32,PCM_U8,FLOAT,DOUBLE,ULAW,ALAW,GSM610,DWVW_12,DWVW_16,DWVW_24,VOX_ADPCM}]
                     soundfile

Sample tweaker for Oric replaying

positional arguments:
  soundfile             audio file to convert (VAV, MP3, OGG, FLAC ...)

optional arguments:
  -h, --help            show this help message and exit
  --rfreq REPLAYFREQUENCY
                        replay frequency
  --outrad OUTPUTFILENAMERADICAL
                        radical of the output file name
  --outvar OUTPUTVARIABLENAME
                        name of the variable to be created
  --language {c,asm}    type of generated file [c | asm]
  --nchan {1,2}         number of channel in audio file (only for RAW audio file)
  --samplerate SAMPLERATE
                        sampling frequency of audio file (only for RAW audio file)
  --subtype {PCM_S8,PCM_16,PCM_24,PCM_32,PCM_U8,FLOAT,DOUBLE,ULAW,ALAW,GSM610,DWVW_12,DWVW_16,DWVW_24,VOX_ADPCM}
                        sampling frequency of audio file (only for RAW audio file)

I challenge you (that I consider as a talented developer) to do the same with your "development" language.
One day, less than 200 lines of code, ISO-functionnality.

Then you will see what I mean by power and expressiveness of a language.


being developer for 20 years and having used wide range of language only means that I know a little bit about what I'm talking about when it comes to compare between some languages I know. Afer 20 years, I'm still incompetent but I can compare
Last edited by jbperin on Wed Aug 11, 2021 6:19 pm, edited 1 time in total.
User avatar
jbperin
Flight Lieutenant
Posts: 480
Joined: Wed Nov 06, 2019 11:00 am
Location: Valence, France

Re: SampleTweaker

Post by jbperin »

Dbug wrote: Wed Aug 11, 2021 8:58 am
jbperin wrote: Tue Aug 10, 2021 9:46 pm If you want to read any kind of audio file and remain in C you can have a look at libsndfile:
http://www.mega-nerd.com/libsndfile/
Unfortunately does not support MP3.
ffmpeg is your friend.
User avatar
Dbug
Site Admin
Posts: 4444
Joined: Fri Jan 06, 2006 10:00 pm
Location: Oslo, Norway
Contact:

Re: SampleTweaker

Post by Dbug »

jbperin wrote: Wed Aug 11, 2021 5:47 pm OK I totally agree with these statements.
C and C++ are used for OSDK because YOU like this language and YOU are the one who maintains OSDK.
YOU are a dictator. Bear in mind that being a dictator make you decide but it doesn't make you necessarily tell truth.
I never claimed to have the truth, I just don't think your arguments are worth doing the change you suggest.

Also, I did not choose C and C++, it just happens that all the existing code on which the OSDK is based on, both from me (the original PCX converter that became PictConv) and the original cross compiler from Fabrice, Alexios and Vagelis were written in C. Rewriting in something else would not have given any benefit, and some of the stuff was converted to C++ because it's a painless operation since you can easily compile C with a C++ compiler with just a few tweaks.

And yes, I'm the dictator: I never pretended that Defence Force was a democracy, if you think it was, you are mistaken.
It just happens I tolerate most people, I validate personally every single person that joins the forum, and now and then I get the moderation tool out (but so far only banned one person in a very long time, who kept spamming despite the warnings).
jbperin wrote: Wed Aug 11, 2021 5:47 pm
Dbug wrote: Wed Aug 11, 2021 3:49 pm - Give me some examples of expressiveness and power of Python over C++
First example : THIS SCRIPT which:
  • takes less than 200 lines
  • was written by a non-talented developer (me) in ONE SINGLE day
  • do the same as SampleTweaker + Bin2Txt
Well, number of lines is kind of useless metric, personally I prefer code that can be understood and debugged.

In your script you have that:

Code: Select all

def encodeSample(x):
    'sample x is between -1.0 and 1.0 included'
    return 0 if (x==0) else max(0,min(15,round(15 - math.log2(1.0/(x/2 + 0.5)**2))))
which seem more or less like the equivalent of

Code: Select all

unsigned char SampleToVolume(unsigned char sample)
{
  if (!sample)
  {
    // To avoid a divide by 0
    return 0;
  }
  else
  {
    double y=(double)sample;
    y=y/255.0;

    double x;
    x = 15.0-2.0*(log10(1.0/y)/log10(2.0));

    if (x<0.0)
    {
      x=0.0;
    }
    else
      if (x>15.0)
      {
        x=15.0;
      }
      return (unsigned char)x;
  }
}
and believe it or not, but C++ has min, max, ternary operator, and the entire function could have been written as one single line as well, but I don't believe that makes it maintainable, and the encodeSample above would not have been accepted where I work.

jbperin wrote: Wed Aug 11, 2021 5:47 pm This script is not noticeably slower than using SampleTweaker + Bin2Txt (even faster because only one system call) but this script :
Well yes, of course.

Python is not slow at calling libraries, and looping over a set of values to convert them is not going to make it sweat, heck, you can do that in BASIC on the Oric with an acceptable runtime.

What's slow in Python is things that do significant processing, and are not part of an existing library (which generally are written in C).

Your list of features is just the list of what the audio library you are using is supporting, that does not say much about Python.

By the way, if you remove the command line handling part, this is the entire source code of Bin2Txt:

Code: Select all

	std::string NameSrc(cArgumentParser.GetParameter(0));
	std::string NameDst(cArgumentParser.GetParameter(1));
	cTextFileGenerator.SetLabel(cArgumentParser.GetParameter(2));

	printf("Converting <%s> to <%s> with label <%s>\n",NameSrc.c_str(),NameDst.c_str(),cTextFileGenerator.GetLabel().c_str());

	void* ptr_buffer_void;
	size_t file_size;
	if (!LoadFile(NameSrc.c_str(),ptr_buffer_void,file_size))
	{
		ShowError("Unable to load the source file");
	}
	printf("Size Read:%d\n",(signed int) file_size);
	unsigned char *BigBuffer=(unsigned char*)ptr_buffer_void;

	std::string cDestString(cTextFileGenerator.ConvertData(BigBuffer,file_size));

	if (!SaveFile(NameDst.c_str(),cDestString.c_str(),cDestString.size()))
	{
		ShowError("Unable to save the destination file");
	}

	free(BigBuffer);
but it's true it could be more compact:

Code: Select all

    std::vector<char> data;
	if (!LoadFile(argumentParser.GetParameter(0),data))
	{
		ShowError("Unable to load the source file");
	}
	TextFileGenerator textFileGenerator.SetLabel(argumentParser.GetParameter(2));
	if (!SaveFile(argumentParser.GetParameter(1),textFileGenerator.ConvertData(data)))
	{
		ShowError("Unable to save the destination file");
	}
Also wants to mention that Bin2Txt also supports different data size (1, 2 and 4 bytes), endianess, and can express the values in decimal or hexadecimal, it also supports BASIC, and the number of entries per line.

In short, your script does not do the same thing.

jbperin wrote: Wed Aug 11, 2021 5:47 pm I challenge you (that I consider as a talented developer) to do the same with your "development" language.
One day, less than 200 lines of code, ISO-functionnality.

Then you will see what I mean by power and expressiveness of a language.
You are aware that I never code fast?

Please, don't challenge me on things that make no sense to me (no idea what "ISO-functionnality" means in the context).

It's not because you like to release non finished things two days after you started writing the first line that I should do the same.

I said that I was considering doing a proper version of an audio tool for the OSDK, it will take the time it will take, it will have the number of lines that I judge proper, it will be open source like everything else, and if you want to make a Python version, please do, I don't care.

I'm certainly not going to spend one day writing something for the sake of writing something.

And to go back to the original discussion, this library seems interesting:
http://sol.gfxile.net/soloud/quickstart.html

Code: Select all

	SoLoud::Soloud gSoloud; // SoLoud engine
	SoLoud::Wav gWave;      // One wave file

	gSoloud.init(); // Initialize SoLoud

	gWave.load("some_audio_file.wav"); // Supports ogg, wav, mp3, ...

	gSoloud.play(gWave); // Play the wave

	(wait)

	gSoloud.deinit(); // Clean up!
User avatar
jbperin
Flight Lieutenant
Posts: 480
Joined: Wed Nov 06, 2019 11:00 am
Location: Valence, France

Re: SampleTweaker

Post by jbperin »

Development is creating new things or enhancing existing ones.

Programming language is the mean by which the developer tells the machine how to realise the things he wants it to do.

If a cheap developer like me needs 8 hours to create the feature he needs in Python, while a talented developer need years to do the same in C++, I consider it is a proof that Python is a more powerful language than C++.

Tons of powerful libraries, light and concise syntax, easy environment setup, multiplatform, and so on ..

I would never use python to write a compiler (which is intensively used during development process) .. but for a tool that is going to be used once or twice in a project workflow .. it totally makes sense.

To go back to the original discussion, perhaps you would find interesting stuffs in the following archive :
pcmenc.zip
(82.83 KiB) Downloaded 163 times
It is the C++ version of the viterbi encoder for PSG AY that could be found here.

They read data from wav file and use a "real development" language to perform the encoding.
So perhaps it can be added to SampleTweaker ..
Dbug wrote: Tue Aug 10, 2021 11:37 am Not sure what the feature set should be, but I guess:
- Resampling to a specific replay frequency
- Number of bits (2 bits, 4 bits, 8 bits, ...)
- Other?
Other ? .. viterbi encoding .. ADPCM :-)
User avatar
iss
Wing Commander
Posts: 1641
Joined: Sat Apr 03, 2010 5:43 pm
Location: Bulgaria
Contact:

Re: SampleTweaker

Post by iss »

jbperin wrote: Thu Aug 12, 2021 10:08 amOther ? ADPCM
Indeed, everything PWM-like is interesting!
I hope to have some free time this weekend to check more deeply the viterbi things...
User avatar
Dbug
Site Admin
Posts: 4444
Joined: Fri Jan 06, 2006 10:00 pm
Location: Oslo, Norway
Contact:

Re: SampleTweaker

Post by Dbug »

jbperin wrote: Thu Aug 12, 2021 10:08 am If a cheap developer like me needs 8 hours to create the feature he needs in Python, while a talented developer need years to do the same in C++, I consider it is a proof that Python is a more powerful language than C++.
If anybody can build a shed in one afternoon using whatever they found around, while a professional would take a week build one, it's definitely a proof of what exactly?
jbperin wrote: Thu Aug 12, 2021 10:08 am To go back to the original discussion, perhaps you would find interesting stuffs in the following archive :
pcmenc.zip

It is the C++ version of the viterbi encoder for PSG AY that could be found here.

They read data from wav file and use a "real development" language to perform the encoding.
So perhaps it can be added to SampleTweaker ..
Two questions:
- Have you checked that the output of the pcmenc was identical to what your python script did
- Have you reached the conclusion that viterbi encoding was worth having on the oric?

Basically, I integrate things in the OSDK when they have been validated as things ready to use, I don't add stuff that will not be used.
jbperin wrote: Thu Aug 12, 2021 10:08 am Other ? .. viterbi encoding .. ADPCM :-)
Only if we get a decoder with it.
Adding features for the sake adding features is pointless, and there are plenty of tools that can be used to test these things without needing additional tools (Sox for example).

If we come up with a proper use of ADPCM, which is either more compact, faster, or has better sound quality than the existing stuff, sure.

I wrote 2/4/8 bits, because we already have players for these, which have been used.
If Viterbi, in it's current implementation, is not better than any of the current versions on any of the criteria, and also requires hours of data crunching to achieve the same result, it's not worth investing time in it.
Post Reply