Add Plugin Pin Connector #7459

messmerd · 2024-08-17T23:25:48Z

This PR is the first step towards audio routing in LMMS, adding a new "Plugin Pin Connector" feature based on REAPER's feature of the same name.

Supercedes "L/R routing" from #7247.

The Plugin Pin Connector window provides automatable matrix widgets that provide users with full control over how audio is routed in and out of a plugin. It can be accessed from the plugin parameters window of VST plugins.

Use cases

Upmixing the output of mono plugins to stereo (Fixes Mono VST plugins output only to the left channel #6558)
Using plugins with multiple outputs or multiple inputs
- For example, CrossOver by Robin Schmidt, a band splitter with 4 stereo output channels
- (Unfortunately since you cannot add more track channels yet, it cannot be used to its full potential for sidechaining or band splitting. Support for this can be added later.)
Swapping left and right channels
Combining two mono plugins into a full stereo plugin without upmixing
...

Track channel explanation

The term "track channel" is borrowed from REAPER. A track channel is a mono audio pipeline. For Instrument Tracks, track channels run from the instrument output, through each effect plugin in the effects chain, then are routed to a mixer channel. For Mixer Channels, there are also track channels, but the audio source is the mixer input instead of an instrument.

Currently LMMS only has 2 track channels - the main unnamed L/R pair. In the future, I hope LMMS will support user-created track channels beyond the main pair similar to REAPER. They are a prerequisite for audio routing.

Pin connector rules

Reading from a track channel does not have any effect on the track channel's audio data
Writing to a track channel overwrites whatever audio data was already in that track channel
If a plugin does not write to a track channel, it passes through the plugin unmodified (bypass)
If multiple inputs are routed to a single output channel, they are averaged together

Changes

PluginPinConnector: Pin connector model + audio routing methods
PluginPinConnectorView: Pin connector window
Replaced the useless "Close" buttons in the parameters windows with buttons that open the Plugin Pin Connector
lmms::Span: A simple stand-in for C++20's std::span
lmms::unreachable: A stand-in for C++23's std::unreachable
A few audio data aliases that describe the layout of audio data ("split" or "interleaved") and support track channels
Pin connector integration for VST instruments and effects
The Remote Plugin code was using both "interleaved" and "split" audio data layouts but reinterpret-casting data to SampleFrame* parameters (which is interleaved data), so I replaced those with float*, which can be any data layout
...

Future

This PR only adds pin connector support to VST plugins, but there are already plans to extended support to Lv2 and CLAP after this PR is merged. For CLAP, the pin connector could also provide a dropdown for selecting the audio port configuration.

Eventually we would want all instruments and effects to provide a pin connector, including native LMMS plugins and LADSPA, but this would require more difficult UI integration and it hasn't been discussed among devs yet.

Before full audio routing functionality is introduced, we may want to provide support for adding new track channels to Instrument Tracks and Mixer Channels.
This would enable full use of band splitter plugins and drum machine plugins with multiple output channels without the pain involved in implementing full track-channel-to-track-channel audio routing. The Pin Connector is already designed to work with track channels >= 2, but since LMMS only has a fixed number of 2 track channels currently, we have only seen a mere 2% of its power.

JohannesLorenz · 2024-08-18T18:19:39Z

While still doing a review, I wonder if the SampleFrame redesign can go into a separate PR? However, then would we have 2 PRs based on another (alternatively, simply squashing this PR and splitting it up into 2 atomic commits might also make the review more easy, I guess).

messmerd · 2024-08-18T18:51:01Z

@JohannesLorenz The changes I made to SampleFrame don't affect any of its parameter or return value types. InterleavedAudioDataPtr<sample_t> is just an alias for sample_t*, so that class wasn't redesigned in any way - just clarified. (Interleaved means the data is "LRLRLRLR" instead of "LLLLRRRR")

JohannesLorenz

Incomplete review. Some questions might be a bit dumb because I am not through yet 😆

include/PluginPinConnector.h

src/core/PluginPinConnector.cpp

JohannesLorenz · 2024-08-20T21:50:45Z

src/core/PluginPinConnector.cpp

+				{
+					for (f_cnt_t frame = 0; frame < frames; ++frame)
+					{
+						bufferOut[frame] += inPtr[frame];


Very minor comment: I think you save an addition per cycle if you use 2 pointers instead of 1 index? Wouldn't this be an application to use Span? As often as this occurs, one could even define a way to increase two Span iterators at once (like an operator++/operator< for pair<Span>, if these do not even exist already)?

include/RemotePluginClient.h

JohannesLorenz

First functional code review complete.

After this review, follow-ups would be possible:

optional second functional code review
style review
test review

For testing, I would at least test the following:

Both VST mono and stereo effects and instruments should be tested
The "pin connector rules" (see the original post on top of this thread) shall be tested with these different types of instruments/effects
Loading/Saving must be tested carefully, especially with automated and controlled pins

JohannesLorenz · 2024-08-26T19:32:45Z

include/PluginPinConnector.h

+	 * `in`     : track channels from LMMS core (currently just the main track channel pair)
+	 * `out`    : plugin input channels in Split form
+	 */
+	void routeToPlugin(f_cnt_t frames, CoreAudioData in, SplitAudioData<sample_t> out);


This might work for VST, which wants split channels, but e.g. the LMMS-native Amplifier wants its audio data interleaved. Are you planning to add routines for that case?

plugins/VstBase/VstPlugin.cpp

src/core/PluginPinConnector.cpp

JohannesLorenz · 2024-09-03T19:37:46Z

src/gui/PluginPinConnectorView.cpp

+	return calculateSize();
+}
+
+void PluginPinConnectorView::MatrixView::paintEvent(QPaintEvent*)


What is the reason why you prefer paintEvent over QGridView? I would have probably not used paintEvent for both classes, and instead just would have MatrixView let contain both the images and the text.

In general, I think paintEvent always uses a lot of constants (like e.g. 4 or 2 above), requires implementing sizeHints, and can get difficult if we want to change the size of the widgets. Though I understand paintEvent sometimes cannot be avoided.

If I made each individual cell a QWidget that I add to a grid layout, it might be difficult to update the view when the number of plugin channels or track channels changes. I might have to remove all the widgets from the layout and then add them all back in to get it to work.

JohannesLorenz · 2024-09-03T19:48:02Z

src/gui/PluginPinConnectorView.cpp

+{
+
+constexpr auto CenterMargin = QSize{48, 0};
+constexpr auto WindowMarginTop = QSize{0, 96};


On my 24'' monitor with 1920x1080 resolution, this is almost 1 inch of "void" above the matrices. Is this done for design reasons?

I just didn't want it to be right up against the edge of the window. This is the first time I've done anything with UIs in Qt, so I don't really know what I'm doing.

plugins/Vestige/Vestige.cpp

include/PluginPinConnector.h

- Make new sample types strong typedefs - Rename some types in AudioData.h - Provide `audio_cast` methods for converting between raw audio data pointers and pointers to the new sample types - Append "Ref" to names of `SampleFrame` methods that return a reference, and stop using `const float&` instead of `float` - Improve `lmms::Span`

It can be added back in later if desired, but it is not necessary here for the pin connector

messmerd added 28 commits May 8, 2024 03:14

Add PluginPortConfig class

93884a2

Avoid misleading types and weird casts

f1f1010

Use PluginPortConfig for mono VSTs

c78c17f

Refactor; Fix automation clip tooltip

4da60af

Begin pin connector implementation (WIP)

798e088

Loading/saving (WIP)

af4d6b3

More efficient loading

aafe41c

Use new routing methods (WIP)

360eca6

Progress (WIP)

7b7f674

Merge branch 'master' into pin-connector

1568d3a

Fix up post-merge (WIP)

5f31c04

Fix crash

853d80c

Begin implementing PluginPinConnectorView

7935718

Progress (WIP)

b3ad72a

Fix pin icon while dragging

b55bdec

More progress

bda93bc

Draw plugin channel text

15f395e

Fix Linux build

8d2acad

Some refactoring

a00cdba

Fix build

c98353f

Introduce MatrixView class

774e3b6

Use layouts

e768a58

Make PluginPinConnector::Matrix a struct; Add tooltips

2ee2c08

Set model display names

8445f2c

Fix a display bug

ecc6318

Remove some unused code

a317e16

Fix crash when reloading project with pin connector open

2571547

Fix miscalculation in mousePressEvent

ada15d3

JohannesLorenz self-requested a review August 18, 2024 16:38

messmerd added 2 commits August 25, 2024 01:43

Better pin connector button text; Better naming and comments

a308725

Use pointing hand cursor when hovering over cells

09eb59f

JohannesLorenz reviewed Aug 25, 2024

View reviewed changes

JohannesLorenz mentioned this pull request Aug 26, 2024

Lv2: Do not duplicate mono plugins #7387

Open

JohannesLorenz reviewed Sep 4, 2024

View reviewed changes

messmerd added 7 commits September 13, 2024 21:06

Make routing methods const

f6910c6

Improve routing normalization

ba3ffc6

Use bitwise OR

af5c16b

Move some pin connector methods into Matrix

32557d4

Use const float* for process's in buffer

1716186

Rename m_trackChannelsUsed to m_trackChannelsUpperBound

23fd48b

Merge branch 'master' into pin-connector

089ffb4

messmerd mentioned this pull request Sep 22, 2024

Layout issues after SlicerT UI update #7510

Open

1 task

messmerd added 4 commits September 22, 2024 23:51

Use MAXIMUM_BUFFER_SIZE

b7883f1

Revert strongly typed sample types

bd404df

It can be added back in later if desired, but it is not necessary here for the pin connector

Minor optimization in mixInputs lambda

dcf1fc9

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Plugin Pin Connector #7459

Add Plugin Pin Connector #7459

messmerd commented Aug 17, 2024 •

edited

Loading

JohannesLorenz commented Aug 18, 2024

messmerd commented Aug 18, 2024

JohannesLorenz left a comment

JohannesLorenz Aug 20, 2024

JohannesLorenz left a comment

JohannesLorenz Aug 26, 2024

JohannesLorenz Sep 3, 2024

messmerd Sep 14, 2024

JohannesLorenz Sep 3, 2024

messmerd Sep 14, 2024

Add Plugin Pin Connector #7459

Are you sure you want to change the base?

Add Plugin Pin Connector #7459

Conversation

messmerd commented Aug 17, 2024 • edited Loading

Use cases

Pin connector rules

Changes

Future

JohannesLorenz commented Aug 18, 2024

messmerd commented Aug 18, 2024

JohannesLorenz left a comment

Choose a reason for hiding this comment

JohannesLorenz Aug 20, 2024

Choose a reason for hiding this comment

JohannesLorenz left a comment

Choose a reason for hiding this comment

JohannesLorenz Aug 26, 2024

Choose a reason for hiding this comment

JohannesLorenz Sep 3, 2024

Choose a reason for hiding this comment

messmerd Sep 14, 2024

Choose a reason for hiding this comment

JohannesLorenz Sep 3, 2024

Choose a reason for hiding this comment

messmerd Sep 14, 2024

Choose a reason for hiding this comment

messmerd commented Aug 17, 2024 •

edited

Loading