Extend NPlug for host functionality?

[cuikp]

I just want to ask if you have any plans to extend NPlug for a host-side cross-platform library, for a DAW host to be able to use VST3s from C#?

I would like to tackle it, but at this point lack the skills. (I had previously dabbled in a C++/CLI solution based on the VST3 SDK and got a semi-working interop that can be accessed from C# to read in and send audio samples to VST3s through Process(). However, even that's only for Windows, and something based on NPlug would allow crossplatform usage and would be much better.

If you don't have the time to extend NPlug, perhaps you could point me or someone in the direction to try to do something like that, based on NPlug? (I guess much of your code can be reused since you've already done the heavy lifting of creating managed versions of the SDK and creating the ComObjectHandle, etc. for crossplatform.)

[xoofx]

I just want to ask if you have any plans to extend NPlug for a host-side cross-platform library, for a DAW host to be able to use VST3s from C#?

Personally, I don't have a plan. I developed mainly this library to create VST plugins (that I still haven't used it for 😅)

If you don't have the time to extend NPlug, perhaps you could point me or someone in the direction to try to do something like that, based on NPlug?

So it is definitely possible, but it is quite some work (e.g several weeks of work full time):

• The interop codegen generated in https://github.com/xoofx/NPlug/tree/main/src/NPlug/Interop by the CodeGenerator.cs would have to generate the reverse proxy (Calling from C# to C++), I think it does probably already but would have to be double checked.
• Then for each plugin interface exposed by NPlug (e.g IAudioControllerParameterFunctionName) a bridge implementation would have to be implemented to call the generated interop proxy above.
• Then it would require to implement the logic of loading and unloading plugins and the entire protocol of a host to communicate to these plugins (e.g you need to call this interface before calling the others. Don't think that everything has to be implemented as this can be left to the implementer, but there are some bootstrap basics to setup).
• As some items have been heavily abstracted in NPlug (the sync of plugin parameters with the controller and the processor), it might require more changes here and there.

[cuikp]

Definitely sounds like a major undertaking. I do have a bit of time (though sorely lacking knowledge in C/C++), so I'll have to take baby steps :)

From inside C# of course, it's an easy enough matter to simply get the pointer for the PluginFactory of a Vst3 file:

( public void LoadPlugin(string filePath)
 {     ...
     var pluginDllHandle = NativeLibrary.Load(filePath);
    if (pluginDllHandle != IntPtr.Zero)
     {
         _nativeProxyHandle = pluginDllHandle;
         var getPluginFactoryPtr = NativeLibrary.GetExport(pluginDllHandle, "GetPluginFactory");
         if (getPluginFactoryPtr == IntPtr.Zero) throw new InvalidOperationException("Missing GetPluginFactory");
         PluginFactoryPtr = Marshal.GetDelegateForFunctionPointer<GetPluginFactoryDelegate>(getPluginFactoryPtr)();...

But then where to go with the pointer...

Regarding going the other way or "reversal" to convert C objects such as the PluginFactory2 and PClassInfo into C# usable objects, after building and running the NPlug.CodeGen project, it generated pretty much the same files already on the NPlug github page, but it seems to be aimed at creating (not reading), C objects. For example, the file generated for IPluginFactory2 looks like this:

  public partial struct IPluginFactory2
  {
      private static IAudioPluginFactory Get(IPluginFactory2* self) => ((ComObjectHandle*)self)->As<IAudioPluginFactory>();
      
      private static partial ComResult getClassInfo2_ToManaged(IPluginFactory2* self, int index, PClassInfo2* info)
      {
          var pluginClassInfo = Get(self).GetPluginClassInfo(index);
          info->cid = pluginClassInfo.ClassId.ConvertToPlatform();
          info->cardinality = pluginClassInfo.Cardinality;
          //public fixed byte category[32];
          CopyStringToUTF8(GetPluginCategory(pluginClassInfo), info->category, 32);
          ...
          return true;
      }
  }

whereas (based on the SDK) a usable C# interface should look something like this:

internal interface IPluginFactory2
{
    int GetFactoryInfo(out PFactoryInfo info);
    internal int CountClasses();
    internal void GetClassInfo(int index, out PClassInfo2 info);
    int CreateInstance([MarshalAs(UnmanagedType.LPStr)] string cid, [MarshalAs(UnmanagedType.LPStr)] string iid, out IntPtr obj);
}

So, I'm a bit confused regarding the generated code and how to use it.

Since it may be more complicated for you to explain than to do it yourself, I'll try to continue studying NPlug a bit more and see what I come up with!

[xoofx]

Let me try to clarify a bit the interop used in NPlug.

In order to interop with VST from C#, we need to generate 2 kind of managed wrapper, and in the case of a C# plugin perspective, it means:

• C++ to C#: This is the default case of implementing a plugin in C# (For example IPluginFactory) The C++ host is going to call us through this.
• C# to C++: This is the case of interacting from C# with the host, and from the perspective of the C# plugin, it's only used to interact with host interface (For example IAttributes). The C# plugin is going to call the host through this.

As you discovered, NPlug has been designed so far to only cover the case of developing a plugin, and so the requirement for a host plugin would be the inverse of what is described above.

What does that mean? It means that in the end, the code generator would have to generate wrapper in both directions instead of generating a wrapper in mostly one direction (e.g the usecase of developing a plugin in C#)

So, then, how do we detect which wrapper to generate today? It's done here and what it does is that it is inferring the usage of the interface from the comment describing the C++ class. For example, for IPluginFactory the comment is here whether it contains the string [plugin imp] or [host imp].

Based on this information, we are going to generate only one kind of wrapper for the use case of developing a C# plugin (as per the InterfaceKind)

So, in order to support developing a Host in C#, ultimately, we would have to generate both wrappers for any kind of interface. That's the starting point.

That's why today in the generated code you will see that some classes are only generated for one kind of wrapper while other are generated with both (there are cases in the VST API where they say that an interface can be implemented both by the host and the plugin).

Then, it's all about plumbing these and this is the part that is going to be a bit more laborious than generating the wrapper (because that part is mostly automated). In the case of the C# plugin, these are all the classes in the folder Interop where I had to then manually do the last bit of C++ to C# interop or the opposite.

Hope it clarifies things a bit more?

[cuikp]

Oh, I didn't notice the [... imp] qualifiers in the SDK.
It is a bit clearer, yes. When you say that for the Host we need to generate both wrappers, (for all of the interfaces, not just the ones marked [host imp], then I would need to modify the CodeGen (or the SDK) to do that?
For example, since IPluginFactory is marked [plug imp], I need add [host imp] so that it generates both wrappers?

[xoofx]

For example, since IPluginFactory is marked [plug imp], I need add [host imp] so that it generates both wrappers?

You don't need to add host imp but change the CodeGen to force using InterfaceKind.Both always.

But keep in mind that it's the easy part. Most of the work will require a significant amount of classes to be implemented and wrapped in managed. It should be as involving (if not more) as doing the plugin version. Not saying this to discourage you, but that you realize where you are going 🙂

[cuikp]

Thanks for the further explanation. I amended the CodeGenerator.cs as follows:

    else if (_pluginOnly.Contains(name))
    {
        //kind = InterfaceKind.Plugin;
        kind = InterfaceKind.Both;
    }

And generated it again. Of course the individual managed structs, such as LibVst.IPluginFactory.cs are unimplemented.

Actually at this point I'm just trying to conceptually understand NPlug and what's going on, so I think a simple goal would be to return the integer countClasses of IPluginFactory by passing its pointer from a Vst3 file.

Once I understand how that works (that is, how one actually lays the "plumbing" in NPlug), then maybe I can begin the hard work of actually doing the plumbing lol. (It should correspond in structure to how I did it in C++/CLI, where I successfully got my C# host to load in VST3s, show their editors and Process floats, although not much more lol.)

So, IPluginFactory. The newly generated code seems to be the same as NPlug's original:

  public unsafe partial struct IPluginFactory : INativeGuid, INativeVtbl
   {
       public static Guid* NativeGuid => (Guid*)Unsafe.AsPointer(ref Unsafe.AsRef(in IId));
       
       public static int VtblCount => 7;
       
       [MethodImpl(MethodImplOptions.AggressiveInlining)]
       public static void InitializeVtbl(void** vtbl)
       {
           FUnknown.InitializeVtbl(vtbl);
           vtbl[3] = (delegate*unmanaged[MemberFunction]<IPluginFactory*, LibVst.PFactoryInfo*, int>)&getFactoryInfo_Wrapper;
           vtbl[4] = (delegate*unmanaged[MemberFunction]<IPluginFactory*, int>)&countClasses_Wrapper;
           vtbl[5] = (delegate*unmanaged[MemberFunction]<IPluginFactory*, int, LibVst.PClassInfo*, int>)&getClassInfo_Wrapper;
           vtbl[6] = (delegate*unmanaged[MemberFunction]<IPluginFactory*, byte*, byte*, void**, int>)&createInstance_Wrapper;
       }
...

and the method I'm interested in here would be:

  private static partial int countClasses_ToManaged(IPluginFactory* self);
  
  [UnmanagedCallersOnly(CallConvs = new Type[] { typeof(CallConvMemberFunction)})]
  private static int countClasses_Wrapper(IPluginFactory* self)
  {
      if (InteropHelper.IsTracerEnabled)
      {
          var __evt__ = new NativeToManagedEvent((IntPtr)self, nameof(IPluginFactory), "countClasses");
          try { return countClasses_ToManaged(self); } catch (Exception ex) { __evt__.Exception = ex; return default; } finally { __evt__.Dispose(); }
      }
      else
      {
          try { return countClasses_ToManaged(self); } catch { return default; }
      }
  }

with the yet-to-be-implemented LibVst.IPluginFactory.cs being:

internal static unsafe partial class LibVst
{
    public partial struct IPluginFactory
    {
...
        private static partial int countClasses_ToManaged(IPluginFactory* self)
        {
            //how to use "self" and return the native countClasses output...
        }

Does it require a conversion of "self" using the ComObjectManager? or is that also intended only for PlugIn creation?

[xoofx]

Thanks for the further explanation. I amended the CodeGenerator.cs as follows:

    else if (_pluginOnly.Contains(name))
    {
        //kind = InterfaceKind.Plugin;
        kind = InterfaceKind.Both;
    }

Not exactly, this is covering only one part. What I meant is to replace entirely

NPlug/src/NPlug.CodeGen/CodeGenerator.cs

Lines 745 to 752 in 4048b85

	var kind = InterfaceKind.None;
	if (comment != null)
	{
	var isPluginOnly = comment.Contains("[plug imp]");
	var isHostOnly = comment.Contains("[host imp]");
	if (isPluginOnly) kind |= InterfaceKind.Plugin;
	if (isHostOnly) kind |= InterfaceKind.Host;
	}

and

NPlug/src/NPlug.CodeGen/CodeGenerator.cs

Lines 766 to 778 in 4048b85

	if (_hostOnly.Contains(name))
	{
	kind = InterfaceKind.Host;
	}
	else if (_pluginOnly.Contains(name))
	{
	kind = InterfaceKind.Plugin;
	}
	if (kind == InterfaceKind.None)
	{
	kind = InterfaceKind.Both;
	}

with just:

    var kind = InterfaceKind.Both;

[xoofx]

I have pushed the commit b07104f that moves the repo to net8.0 and generates all bi-directional proxies in all cases.

I will respond to your other questions later, have limited time right now.

[cuikp]

I changed all to kind = INterfaceKind.Both as per your instruction and got usable code, thx.
for example, countClasses() comes out like this:

[MethodImpl(MethodImplOptions.AggressiveInlining)]
 public int countClasses()
 {
     if (InteropHelper.IsTracerEnabled)
     {
         var __self__ = (LibVst.IPluginFactory*)Unsafe.AsPointer(ref this);
         var __evt__ = new ManagedToNativeEvent((IntPtr)__self__, nameof(IPluginFactory), "countClasses");
         var __result__ = ((delegate*unmanaged[MemberFunction]<LibVst.IPluginFactory*, int>)Vtbl[4])(__self__);
         __evt__.Dispose();
         return __result__;
     }
     else
     {
         return ((delegate*unmanaged[MemberFunction]<LibVst.IPluginFactory*, int>)Vtbl[4])((LibVst.IPluginFactory*)Unsafe.AsPointer(ref this));
     }
 }

and while I don't quite understand all of what's going on in there (and I'm not sure I need to), I drew up a quick-and-dirty wrapper

 public class PluginFactoryWrapper
 {
     private IntPtr _factoryPtr;

     public PluginFactoryWrapper(IntPtr factoryPtr)
     {
         _factoryPtr = factoryPtr;
     }

     public int CountClasses()
     {
         unsafe
         {
             var factory = (IPluginFactory*)_factoryPtr.ToPointer();
             return factory->countClasses();
         }
    }
}

Which I will test tomorrow. Hopefully, if I create the appropriate wrappers around the generated methods and structs, I shouldn't need to totally understand what's going on under the hood of the generated code.

I'll post again if and when this makes any significant progress, thanks!

[xoofx]

Yes, most of the wrappers should be similar in the end.

One aspect to be careful is about using structs whenever possible, as NPlug has been developed with the goal of minimizing managed allocations (especially during audio processing). A host that would allocate managed objects for interacting with plugins could cause GC pause, and that's something to avoid for a VST Host.

The second aspect is that you will have to be careful at trying to reuse some of the existing interface to interact from a Host with plugins. For example, the interface IAudioPluginFactory is abstracting all the IPluginFactory, IPluginFactory2, IPluginFactory3 and you could for example integrate with it like this:

using System;
using static NPlug.Interop.LibVst;

namespace NPlug;

/// <summary>
/// A factory to create <see cref="IAudioPluginObject"/> instances.
/// </summary>
public interface IAudioPluginFactory
{
    /// <summary>
    /// Gets information about this factory.
    /// </summary>
    AudioPluginFactoryInfo FactoryInfo { get; }

    /// <summary>
    /// Gets the number of plugins this factory supports.
    /// </summary>
    int PluginClassInfoCount { get; }

    /// <summary>
    /// Gets the information about a plugin class at the specified index.
    /// </summary>
    /// <param name="index">The index of the plugin.</param>
    AudioPluginClassInfo GetPluginClassInfo(int index);

    /// <summary>
    /// Creates a new instance of the plugin with the specified id.
    /// </summary>
    /// <param name="pluginId">The id of the plugin.</param>
    /// <returns>A new instance of the plugin; otherwise null if this factory does not support this id.</returns>
    IAudioPluginObject? CreateInstance(Guid pluginId);

    public static AudioPluginFactoryClient Get(IntPtr factoryPtr) => new(factoryPtr);
}

public readonly unsafe struct AudioPluginFactoryClient : IAudioPluginFactory
{
    private readonly IPluginFactory* _factory;

    public AudioPluginFactoryClient(IntPtr factoryPtr) => _factory = (IPluginFactory*)factoryPtr;

    public int PluginClassInfoCount => _factory->countClasses();

    public AudioPluginFactoryInfo FactoryInfo => throw new NotImplementedException();

    public AudioPluginClassInfo GetPluginClassInfo(int index)
    {
        throw new NotImplementedException();
    }

    public IAudioPluginObject? CreateInstance(Guid pluginId)
    {
        throw new NotImplementedException();
    }
}

[cuikp]

No rush, please respond when convenient. Just trying to get my feet on the ground here.

structs, not classes, got it.

So far I'm able to retrieve all of the classinfo, but still taking baby steps here as not fully understanding the NPlug architecture.
Now inside AudioPluginFactoryClient I'm attempting to just probe the VST3 to check and retrieve info about IComponent, similar to how I have done with PInvoke or C++/CLI, using the native factory.

Here, however, even though createInstance() is returning a True or IsSuccess ComResult, the next step of setIoMode() returns an "unknown" result, rather than the expected "not implemented" (for the particular VST3 I'm testing with). I know it should return "not implemented" (0x80004001L) because that's the value returned when using PInvoke.
Also, trying to call queryInterface() produces a False return value (the object retrieved by createInstance is apparently not an IComponent object).
So I imagine my iid or cid parameters are not being correctly converted for use in the NPlug environment?

 public string ProbePlugin()
 {
     PClassInfo2 pinfo2;
     _factory-> getClassInfo2(0, &pinfo2);  //test VST3 plugin has one class
     
     FIDString cidFS = ConvertGuidToFIDString(pinfo2.cid);
     FIDString componentFS = ConvertGuidToFIDString(IComponent.IId);

     void* pObj = null;
     ComResult retval = _factory->createInstance(cidFS, componentFS, &pObj);
     //ComResult queryRes = _factory->queryInterface(IComponent.NativeGuid, &pObj);

     IComponent* icomp = (IComponent*)pObj;
     ComResult setIoResult = icomp->setIoMode((int)IoModes.kAdvanced);

the conversion method is this:

 static FIDString ConvertGuidToFIDString(Guid guid)
 {
     byte[] guidBytes = guid.ToByteArray();
     IntPtr unmanagedPointer = Marshal.AllocHGlobal(guidBytes.Length);
     Marshal.Copy(guidBytes, 0, unmanagedPointer, guidBytes.Length);
     return new FIDString() { Value = (byte*)unmanagedPointer.ToPointer() };
 }

(NPlug seems to have its own LibVst.FIDString definition, which I'm probably not using properly)