learn more DSP or electronics?

[ferki]

Hello everyone,

I am delving into the world of plugin development. To be honest, I have a rough idea but I'm not an expert in this field.

Currently, I am reading "Designing Audio Effect Plugins in C++: For AAX, AU, and VST3 with DSP Theory" and I believe it's a good starting point.

From what I've seen, the approaches are divided into black box, white box, or a mix of both (gray box). What I realize is that if you want to focus on the most precise "black box" model, you need to know DSP (Digital Signal Processing) and be good at electronics, which makes it complicated to excel in both areas.

Is it better to focus on one? For example, should I concentrate on DSP and leave electronics a bit aside?

[mystran]

For "black box" you would probably just want to study DSP and mathematical modelling techniques, where as it's "white box" where you often want to understand something about electronics 'cos the "box" is often some analog piece of electronics.. but what you really want is not boxes, but rather solutions to problems.

The theoretical side of electronics (ie. not worrying too much about practicalities of components, but more on how the circuits actually function) and digital signal processing are not necessarily that far from each other. The main difference really is that in one (DSP) we're working in discrete time and in the other (analog) we are working in continuous time... yet because we can approximate continuous processes by discrete time processes, many things carry over. Discrete time introduces it's own problems, most obviously bandwidth limitations and aliasing considerations, but especially on the high-level the principles of how you process audio are exactly the same.

I would not worry about learning everything at once, 'cos that's just impossible (there's just too much stuff for anyone to learn it all), but I would also not limit yourself strictly to a single subject, because often you need a bit of this and a bit of that. In my opinion, it's better to start working on problems that interest you, study the topics that are relevant for what you want to do, whether that's DSP, analog electronics, something inbetween, perhaps pure mathematics.. and over time you will build a better understanding of them all.

Pick a project, then start learning what you need to learn to complete that project.

[pekbro]

Most folks dont get that you can do math in analog HW, much as you do in code.

[mystran]

Most folks dont get that you can do math in analog HW, much as you do in code.

Don't know about "most folks" but like.. you have a point: analog signal processing, especially a lot of effects used for musical purposes are just as much "computation" as any signal processing code.. and probably the most important part to learn for DSP is "what are we computing?"

[ferki]

For "black box" you would probably just want to study DSP and mathematical modelling techniques, where as it's "white box" where you often want to understand something about electronics 'cos the "box" is often some analog piece of electronics.. but what you really want is not boxes, but rather solutions to problems.

The theoretical side of electronics (ie. not worrying too much about practicalities of components, but more on how the circuits actually function) and digital signal processing are not necessarily that far from each other. The main difference really is that in one (DSP) we're working in discrete time and in the other (analog) we are working in continuous time... yet because we can approximate continuous processes by discrete time processes, many things carry over. Discrete time introduces it's own problems, most obviously bandwidth limitations and aliasing considerations, but especially on the high-level the principles of how you process audio are exactly the same.

I would not worry about learning everything at once, 'cos that's just impossible (there's just too much stuff for anyone to learn it all), but I would also not limit yourself strictly to a single subject, because often you need a bit of this and a bit of that. In my opinion, it's better to start working on problems that interest you, study the topics that are relevant for what you want to do, whether that's DSP, analog electronics, something inbetween, perhaps pure mathematics.. and over time you will build a better understanding of them all.

Pick a project, then start learning what you need to learn to complete that project.

it's true that it's such a broad topic that it seems challenging. I imagine that the best way to learn is to take on projects and learn little by little.

A bit of a noob question, can you do a good job with just DSP? I think I'm going to start by learning DSP well and then get into analog electronics.

[mystran]

A bit of a noob question, can you do a good job with just DSP? I think I'm going to start by learning DSP well and then get into analog electronics.

I would not think about it that way, because the lines of what is DSP and what is something else are entirely drawn in water. At the end of the day, it's all just mathematics really. Perhaps a bit of physics for physical signal processing, but even that's something we model with mathematics.

Pick a project, then figure out what you need for it (this forum can help with that). Don't worry which field the information comes from, it might even be control theory or information theory or CS fundamentals or .. whatever.

If there is one "fundamental" thing that is worth learning, that would be getting a good hang of sampling theory, how Nyquist-Shannon sampling represents band-limited signals by sampling, what perfect reconstruction means and how it works, how the band-limited nature of Nyquist sampling results in aliasing when non-linear operations are applied, how this applies to impulse responses of filters (eg. impulse invariant) just as much as regular signals. That's the fundamental "DSP" aspect of signal processing really, everything else (in a sense) is just the same whether your domain is discrete-time or continuous-time... and often it is easier to design things in continuous-time and then implement them in discrete-time than it is to work purely in digital, 'cos the continuous-time mathematics tends to be easier.

But really... pick a project, preferably something that seems fairly simple (it's probably more complex than you think and even simple projects have a tendency of getting more complex as you think of cool features). When you have a specific project, other can help you guide as to what exactly it is that you would need for that project. If it still seems too daunting, perhaps simplify or pick an easier project.

[Fender19]

I learned electronics first but that's because I'm an old guy and audio DSP wasn't in common use in those days. You don't need to know electronics to understand DSP but in my case I tend to "visualize" the electronics in DSP code.

One of the nice things about DSP is it often behaves as almost "perfect" electronics. For example, a "ganged" potentiometer can easily be made with any number of channels with perfect tracking. "Gain" is exactly what you set it to be and is not affected by tube/transistor beta, temperature, component tolerances, etc. We actually have to work to make DSP non-linear to create things like tube and transformer distortion, etc.

At the risk of sounding too "simpleton", it is entirely possible to develop useful DSP plugins without much math at all. Most of the basic building blocks (EQ, delay, FFTs, etc.) are all well-established code in public domain. Put the blocks together to form some desired function.

Once you get the general hang of DSP then you can dive deeper into the math and other details; create your own EQ algorithms, etc. In the meantime, just get started!

[S0lo]

From what I've seen, the approaches are divided into black box, white box, or a mix of both (gray box).

There is also "no box". i.e, invent something completely new.

[ferki]

A bit of a noob question, can you do a good job with just DSP? I think I'm going to start by learning DSP well and then get into analog electronics.

I would not think about it that way, because the lines of what is DSP and what is something else are entirely drawn in water. At the end of the day, it's all just mathematics really. Perhaps a bit of physics for physical signal processing, but even that's something we model with mathematics.

Pick a project, then figure out what you need for it (this forum can help with that). Don't worry which field the information comes from, it might even be control theory or information theory or CS fundamentals or .. whatever.

If there is one "fundamental" thing that is worth learning, that would be getting a good hang of sampling theory, how Nyquist-Shannon sampling represents band-limited signals by sampling, what perfect reconstruction means and how it works, how the band-limited nature of Nyquist sampling results in aliasing when non-linear operations are applied, how this applies to impulse responses of filters (eg. impulse invariant) just as much as regular signals. That's the fundamental "DSP" aspect of signal processing really, everything else (in a sense) is just the same whether your domain is discrete-time or continuous-time... and often it is easier to design things in continuous-time and then implement them in discrete-time than it is to work purely in digital, 'cos the continuous-time mathematics tends to be easier.

But really... pick a project, preferably something that seems fairly simple (it's probably more complex than you think and even simple projects have a tendency of getting more complex as you think of cool features). When you have a specific project, other can help you guide as to what exactly it is that you would need for that project. If it still seems too daunting, perhaps simplify or pick an easier project.

I was thinking of a BPM detector or something like that, but really, thank you for taking the time to answer. These tips are very helpful for someone who is just starting out.

[mystran]

I was thinking of a BPM detector or something like that, but really, thank you for taking the time to answer. These tips are very helpful for someone who is just starting out.

I'm not sure what would be the best approach to BPM detection, but I'd probably explore something like auto-correllation (efficient with FFT even over long periods of time) of an envelope (so you don't correllate pitch, only dynamics). For the envelope, I'd probably try an RMS estimate with exponentially moving average (often around 200ms time constant is a sensible starting point) to avoid being too sensitive to peak levels and instead work from the short-term "energy." You could perhaps weight that further with filters, or detect with multiple bands.

The idea with auto-correllation is that we detect correllation (similarity) with a signal and a shifted copies of the signal (which can be computed efficiently for all shifts at once with FFT) and then we get a measure of how well the signal correllates with itself at different time offsets. In case of BPM detection, you would expect various levels of correllation peaks at musical time intervals, from which you can then try to guess what is a beat and what is a bar.

Someone with more experience with this particular topic might have better ideas of what can be done.. but at least that's some potential pointers that you could explore if you're entirely lost.

[soundmodel]

I am not experienced to answer this holistically, but my suggestion would be to pick one and then find someone to do the other.

Doing too many domains is not efficient, because you cannot possibly keep up with everything.

But I would probably learn enough to know, when your problem fits better to another implementation than to the one you're trying.

Yes, as it has been implied, something like pitch detection is large enough to spend a lifetime in.

It's also very unlikely that you would reach an economically viable level at different skills. It'd be more economical to do your own thing and then pay for someone who is fast with electronics or something.

---

What's your chosen delivery medium?

If you only use computers, then all is DSP, and the electronics aspect is only for "analog-modelling", which is still DSP. If you want to attempt realistic models, then do more electronics, because the rest is just about discretizing that. It's like 90% about knowing what the circuit is and what functions can represent it, the rest is just about expressing that in a form that's computable in real-time.

[ferki]

I was thinking of a BPM detector or something like that, but really, thank you for taking the time to answer. These tips are very helpful for someone who is just starting out.

I'm not sure what would be the best approach to BPM detection, but I'd probably explore something like auto-correllation (efficient with FFT even over long periods of time) of an envelope (so you don't correllate pitch, only dynamics). For the envelope, I'd probably try an RMS estimate with exponentially moving average (often around 200ms time constant is a sensible starting point) to avoid being too sensitive to peak levels and instead work from the short-term "energy." You could perhaps weight that further with filters, or detect with multiple bands.

The idea with auto-correllation is that we detect correllation (similarity) with a signal and a shifted copies of the signal (which can be computed efficiently for all shifts at once with FFT) and then we get a measure of how well the signal correllates with itself at different time offsets. In case of BPM detection, you would expect various levels of correllation peaks at musical time intervals, from which you can then try to guess what is a beat and what is a bar.

Someone with more experience with this particular topic might have better ideas of what can be done.. but at least that's some potential pointers that you could explore if you're entirely lost.

I plan to post my progress on this forum as I work on the project, so I can receive feedback. I realize that I will need to study the topic in depth to fully understand the concepts and techniques involved in BPM detection

[ferki]

I am not experienced to answer this holistically, but my suggestion would be to pick one and then find someone to do the other.

Doing too many domains is not efficient, because you cannot possibly keep up with everything.

But I would probably learn enough to know, when your problem fits better to another implementation than to the one you're trying.

Yes, as it has been implied, something like pitch detection is large enough to spend a lifetime in.

It's also very unlikely that you would reach an economically viable level at different skills. It'd be more economical to do your own thing and then pay for someone who is fast with electronics or something.

---

What's your chosen delivery medium?

If you only use computers, then all is DSP, and the electronics aspect is only for "analog-modelling", which is still DSP. If you want to attempt realistic models, then do more electronics, because the rest is just about discretizing that. It's like 90% about knowing what the circuit is and what functions can represent it, the rest is just about expressing that in a form that's computable in real-time.

The idea is that, both things seem necessary, but for example, many times if you don't have the schematic it is better to go for purely DSP? or very complex electronic schematic where it is difficult to emulate them with epice, I don't know how the companies that are dedicated to this or people who know well do it.

[Uncle E]

But really... pick a project, preferably something that seems fairly simple (it's probably more complex than you think and even simple projects have a tendency of getting more complex as you think of cool features). When you have a specific project, other can help you guide as to what exactly it is that you would need for that project. If it still seems too daunting, perhaps simplify or pick an easier project.

This. I know of at least one developer we all revere who got into it by delving into a single project very, very deeply. 20 years later and all of his products are still based on that first starting point.

[soundmodel]

The idea is that, both things seem necessary, but for example, many times if you don't have the schematic it is better to go for purely DSP? or very complex electronic schematic where it is difficult to emulate them with epice, I don't know how the companies that are dedicated to this or people who know well do it.

When you develop a product, then e.g. your costs will usually determine your platforms. Computers are very popular, because programs are much cheaper to produce than physical electronics. But in many cases the results are not similar.

Many plug-in companies do both. They do a hardware version and a software version.

"Better" depends on what you need to achieve.