The real-time low latency multi channel audio streaming needed for musicians is awfully similar to the real time low latency multi channel audio streaming required for telephony.
Yet somehow the two industries have pretty much entirely different tech stacks and don't seem to talk to one another.
This is a very interesting thought. I'm not super experienced with low level audio and basically completely ignorant of telephony.
I feel like most people doing audio in music are not working at the low level. Even if they are creating their own plugins, they are probably not integrating with the audio interface. The point of JACK or Pipewire is to basically abstract all of that away so people can focus on the instrument.
The latency in music is a much, much bigger issue than in voice, so any latency spike would render network audio completely unusable.
I know Zoom has a "real time audio for musicians" feature, but outside of a few Zoom demos during lockdown, I'm not sure anybody uses this.
Pipewire supports audio channels over network, but again I'm not entirely sure what this is for. Certainly it's useful for streaming music from device A to device B, but I'm not sure anybody uses it in a production setting.
I could see something like a "live coding symphony", where people have their own livecoding setups and the audio is generated on a central server. This is not too different than what, say, Animal Collective did. But while live coding is a beautiful medium on its own, it does lack the muscle memory and tactile feedback you get from playing an instrument.
I would love to see, as you said, these fields collaborate, but these, to me, are the immediate blockers which make it less practical.
Telephony is significantly less latency sensitive than real time audio processing, it’s also significantly less taxing since you’re dealing with a single channel.
The level of compression and audio resolution required are significantly different too. You can tune codecs for voice specifically, but you don’t want compression when recording audio and can’t bias towards specific inputs.
They’re only similar in that they handle audio. But that’s like saying the needs of a unicycle and the needs of an F1 car are inherently the same because they have wheels.
Just an fyi to anyone making or thinking of making one of these:
Turning a knob with a mouse is the worst interface I can think of.
I don't know why audio apps/DAWs fall so hard on skeuomorphism here when the interface just doesn't make sense in the context.
It allows for dense controls and everyone's used to them. I don't find them to be a problem, they aren't intuitive in that you might think you're supposed to grab the knob and "turn" it with a circular cursor motion or something, but once you learn to drag linearly, they're an easy to use and consistent interface. And as giancarlostoro mentioned, you can map them to a MIDI device if you want to twiddle knobs while playing/recording live.
I'll add in addition - the skeumorphism here is generally pretty functional, you touched on this when you said "everyone is used to them"
But the layout of these buttons, while certainly not standard, is generally familiar across various filters, etc. So if you are dealing with a complex interface the skeumorphism absolutely helps to make the input more familiar and easily accessible.
This is what skeumorphism is for and this is a great place to use it.
Imagine if the symbols for "play" "pause" and "stop" were changed simply because it no longer made sense to follow the conventions of a VCR, then multiply that by an order of magnitude.
most daws allow you to map hardware to the dials so u dont need to tweak by mouse. that being said, good automations are a fair replacement depending on your style of music. lfos, adsrs and pattern tools for automation lanes aswell as ability to record automations (to keep em consistent, modify manually etc ), and ofc humanization algorithms that u can apply to automation lanes.
i never use 'hardware', totally happy doin what i do. (thats music i think. enjoying your craft). most ppl i know using similar tools do have midi controllers to have more of an instrumental interface. theres tons of options. no need to discourage anyone...
and most interfaces have a condition watching for CTRL or SHIFT to ++/-- values slower or faster depending on the modifier held... that allows one to turn a knob with much greater precision than a physical interface!
double-clicking usually lets one type the value... really good interfaces let one scroll seamless independent of screen borders; the perfect pair with a trackball or a long surface/desk for sliding the mouse
If not using hardware, you just click and move horizontally or vertically; not sure what a better interface would be? Though I do like it when the numeric value shows when changing. I really don't know what other UI would work well here. Usually there are so many knobs it makes sense to be compact. Though really it makes sense as well to match the visualization of the knobs on my midi controller anyway.
No. MIDI controllers have their place, but many people work without one, or only use one for live performances. There are often also way more knobs in the various FX chains in a DAW than you would reasonably want to map to a controller, but still want to touch at least a few times while making a song.
Knobs are confusing when converted to a mouse paradigm because there can be a few strategies to control them (click+drag up/down, click+drag right/left, weird rotational things, etc), and you have to guess since each FX studio and software may implement it just a little different.
it's an add for apps that cost as much as a box of decent used pedals and rack mount gear.
though "linux musicians" does appear to be a thing, and the bot used to check if you are human, is amusing and fully automated.
The real-time low latency multi channel audio streaming needed for musicians is awfully similar to the real time low latency multi channel audio streaming required for telephony.
Yet somehow the two industries have pretty much entirely different tech stacks and don't seem to talk to one another.
This is a very interesting thought. I'm not super experienced with low level audio and basically completely ignorant of telephony.
I feel like most people doing audio in music are not working at the low level. Even if they are creating their own plugins, they are probably not integrating with the audio interface. The point of JACK or Pipewire is to basically abstract all of that away so people can focus on the instrument.
The latency in music is a much, much bigger issue than in voice, so any latency spike would render network audio completely unusable. I know Zoom has a "real time audio for musicians" feature, but outside of a few Zoom demos during lockdown, I'm not sure anybody uses this.
Pipewire supports audio channels over network, but again I'm not entirely sure what this is for. Certainly it's useful for streaming music from device A to device B, but I'm not sure anybody uses it in a production setting.
I could see something like a "live coding symphony", where people have their own livecoding setups and the audio is generated on a central server. This is not too different than what, say, Animal Collective did. But while live coding is a beautiful medium on its own, it does lack the muscle memory and tactile feedback you get from playing an instrument.
I would love to see, as you said, these fields collaborate, but these, to me, are the immediate blockers which make it less practical.
This is very much not true.
Telephony is significantly less latency sensitive than real time audio processing, it’s also significantly less taxing since you’re dealing with a single channel.
The level of compression and audio resolution required are significantly different too. You can tune codecs for voice specifically, but you don’t want compression when recording audio and can’t bias towards specific inputs.
They’re only similar in that they handle audio. But that’s like saying the needs of a unicycle and the needs of an F1 car are inherently the same because they have wheels.
irony amplified by the nature of the tech stacks xD surely they can figure out some channel to communicate over clearly haha
Great demo of the JUNE - Classic Analog Polysynth JUNO-60 Plugin - AudioThing
https://youtu.be/GMsUqsyy62Q?t=46
Just an fyi to anyone making or thinking of making one of these:
Turning a knob with a mouse is the worst interface I can think of. I don't know why audio apps/DAWs fall so hard on skeuomorphism here when the interface just doesn't make sense in the context.
It allows for dense controls and everyone's used to them. I don't find them to be a problem, they aren't intuitive in that you might think you're supposed to grab the knob and "turn" it with a circular cursor motion or something, but once you learn to drag linearly, they're an easy to use and consistent interface. And as giancarlostoro mentioned, you can map them to a MIDI device if you want to twiddle knobs while playing/recording live.
I'll add in addition - the skeumorphism here is generally pretty functional, you touched on this when you said "everyone is used to them"
But the layout of these buttons, while certainly not standard, is generally familiar across various filters, etc. So if you are dealing with a complex interface the skeumorphism absolutely helps to make the input more familiar and easily accessible.
This is what skeumorphism is for and this is a great place to use it.
Imagine if the symbols for "play" "pause" and "stop" were changed simply because it no longer made sense to follow the conventions of a VCR, then multiply that by an order of magnitude.
most daws allow you to map hardware to the dials so u dont need to tweak by mouse. that being said, good automations are a fair replacement depending on your style of music. lfos, adsrs and pattern tools for automation lanes aswell as ability to record automations (to keep em consistent, modify manually etc ), and ofc humanization algorithms that u can apply to automation lanes.
i never use 'hardware', totally happy doin what i do. (thats music i think. enjoying your craft). most ppl i know using similar tools do have midi controllers to have more of an instrumental interface. theres tons of options. no need to discourage anyone...
and most interfaces have a condition watching for CTRL or SHIFT to ++/-- values slower or faster depending on the modifier held... that allows one to turn a knob with much greater precision than a physical interface!
double-clicking usually lets one type the value... really good interfaces let one scroll seamless independent of screen borders; the perfect pair with a trackball or a long surface/desk for sliding the mouse
If not using hardware, you just click and move horizontally or vertically; not sure what a better interface would be? Though I do like it when the numeric value shows when changing. I really don't know what other UI would work well here. Usually there are so many knobs it makes sense to be compact. Though really it makes sense as well to match the visualization of the knobs on my midi controller anyway.
Isn't the entire idea that you hook it up to physical hardware?
No. MIDI controllers have their place, but many people work without one, or only use one for live performances. There are often also way more knobs in the various FX chains in a DAW than you would reasonably want to map to a controller, but still want to touch at least a few times while making a song.
Knobs are confusing when converted to a mouse paradigm because there can be a few strategies to control them (click+drag up/down, click+drag right/left, weird rotational things, etc), and you have to guess since each FX studio and software may implement it just a little different.
it's an add for apps that cost as much as a box of decent used pedals and rack mount gear. though "linux musicians" does appear to be a thing, and the bot used to check if you are human, is amusing and fully automated.
https://linuxmusicians.com/
Is there a way I can see which would run on a raspberry pi?