Home

How to package a managed dll with its native dependencies?

JohnMJohnM

I'm trying to create a nuget package wrapping some native dll:s and a managed lib using those dll:s. Basically Scenario 3 from https://github.com/NuGet/docs.microsoft.com-nuget/issues/2070

As I understand it the structure should be as follows for my scenario

lib\net5.0\managed.dll
runtimes\win-x64\native\native*.dll

Yet in my query (v6 as well as v7) I get " An attempt was made to load a program with an incorrect format."
If I manually copy the files from runtimes\win-x64\native\ to the temp shadow folder things work.
Checking "Copy all nuget assemblies to local folder" seems to only result in the managed lib being copied

Any suggestions?

· Share on FacebookShare on Twitter

Comments

  • JoeAlbahariJoeAlbahari

    Did you create a NuGet package and then add the NuGet package reference to LINQPad (i.e., as NuGet reference rather than a DLL reference)? If so, it should work correctly as long as your package is valid. (If it doesn't work, try referencing that package in Visual Studio - it's unlikely to work there, either). Note that you must change the version number every time you update your NuGet package, otherwise parts of the old version may still be cached on your machine.

    If you're referencing it in LINQPad as a DLL reference rather than a NuGet package reference, LINQPad will not probe the runtimes folder unless a .deps.json file is present.

    · Share on FacebookShare on Twitter
  • JohnMJohnM

    Yes, NuGet package was referenced (using an Azure Artifacts feed) and versions have been updated between iterations.

    The package is created using a csproj more or less like so, just bundling the files in question.

    <Project Sdk="Microsoft.Build.NoTargets/3.2.9">
<!-- [...] -->
  <ItemGroup>
    <Content Include="$(Managed)bin\$(AssemblyName).dll" Link="lib\$(TargetFramework)\$(AssemblyName).dll">
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
      <Pack>true</Pack>
      <PackagePath>lib\$(TargetFramework)\$(AssemblyName).dll</PackagePath>
    </Content>
    <Content Include="@(Native)" Link="runtimes\$(RuntimeIdentifier)\native\%(Filename)%(Extension)">
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
      <Pack>true</Pack>
      <PackagePath>runtimes\$(RuntimeIdentifier)\native\%(Filename)%(Extension)</PackagePath>      
    </Content>
  </ItemGroup>
</Project>

    I just tried recreating the query with a small project in visual studio, which runs fine. (two projects actually, a cli.csproj referencing a lib.csproj referencing the nuget package)

    · Share on FacebookShare on Twitter
  • JoeAlbahariJoeAlbahari

    From LINQPad, go to Query Properties (F4), Advanced, and click "Show Assembly Resolution Log". Can you post the output? Also please post the .nuspec file from the extracted package (rename the package file to .zip) and a full directory listing of its contents.

    · Share on FacebookShare on Twitter
  • JohnMJohnM
    edited April 2022

    The nuspec is basically empty

    <?xml version="1.0" encoding="utf-8"?>
<package xmlns="http://schemas.microsoft.com/packaging/2012/06/nuspec.xsd">
  <metadata>
    <id>...</id>
    <version>...</version>
    <authors>...</authors>
    <description>...</description>
    <dependencies>
      <group targetFramework="net5.0" />
    </dependencies>
  </metadata>
</package>

    the resolution log seems to list all the dll:s

    LOG:

Expanding NuGet references for packageId packageVersion(pre):
  packageId packageVersion
Main NuGet assemblies:
  ...\.nuget\packages\packageId\packageVersion\lib\net5.0\packageId.dll version
Main NuGet native files:
  ...\.nuget\packages\packageId\packageVersion\runtimes\win-x64\native\nativelibs*.dll version
LINQPad.Runtime:
  ...\AppData\Local\LINQPad\Updates6.AnyCPU\6.15.12\LINQPad.Runtime.dll 1.0.0.0, 6.15.12
LINQPad.Runtime.UI:
  ...\AppData\Local\LINQPad\6.15.12\LINQPad.Runtime.UI.dll 1.0.0.0, 6.15.12
LINQPad.Windows.Forms.DataVisualization.dll:
  ...\AppData\Local\LINQPad\6.15.12\LINQPad.Windows.Forms.DataVisualization.dll 1.0.0.0, 1.0.0.0

    directory listing:

    ./packageId.nuspec
./lib/net5.0/packageId.dll
./package/services/metadata/core-properties/someguid.psmdcp
./runtimes/win-x64/native/nativelibs*.dll
./[Content_Types].xml
./_rels/.rels
    · Share on FacebookShare on Twitter
  • JohnMJohnM

    I gave up on the runtimes folder. Instead put the dll:s under contentFiles/any/any and added this to the nuspec

        <contentFiles>
      <files include="any/any/resources/**" buildAction="None" copyToOutput="true" />
      <files include="any/any/*.dll" buildAction="None" copyToOutput="true" />
    </contentFiles>

    Initially tried a subfolder for the dll:s as well with flatten=true, but either linqpad ignored the flatten attribute, or I got something wrong regarding the semantics, because the subfolder was copied over into the shadowfolder

    · Share on FacebookShare on Twitter
  • JohnMJohnM
    edited May 2022

    Actually contentFiles creates its own set of problems

    They'll be added to the solution explorer in visual studio. Which is annoying but can be changed with Visible=false
    Thet don't flow transitively, one must override the implicit exclusion of contentFiles from dependencies

    Since putting files in runtimes/win-x64/native/ does work for Visual Studio it would be helpful if linqpad also supported this.

    For this I belive linqpad needs to change in two ways

    Recognize non-dll files in the runtimes/ folder structure.

    Copy all recognized files to he shadow folder (manged dlls, native dlls, non-dlls), not only managed dlls

    · Share on FacebookShare on Twitter
  • JoeAlbahariJoeAlbahari

    When you get the error "An attempt was made to load a program with an incorrect format.", is there any indication of the full location of the library that it's trying to load? And if so, does it match the value seen in LINQPad's assembly resolution log?

    Also, how does the managed library actually load the native library? Does it apply the DllImport attribute, or does it use the NativeLibrary.Load method, and if the latter, which overload does it call?

    · Share on FacebookShare on Twitter
  • JohnMJohnM

    I don't have the source code for the managed dll:s so don't actually know what it is doing.

    Latest attempt now, avoiding a dependency graph, packaging everything in a single nuget. This package has one net6.0 wrapper, and one net48 respectively in lib/net6.0/net6wrapper.dll, and lib/net48/net48wrapper.dll, and the remaining native.dlls and resource files in runtimes/win-x64/native/. (this package works in visual studio)

    Since this is net6.0 now I'm using linqpad 7 to test this.

    One curious difference is that linqpad 6 will list managed and native dll:s in the assembly resolution log, while linqpad 7 only lists the managed one. Another curiosity is that even after copying all files from the visual studio bin dir to the linqpad shadow dir it still throws.

    The error I get in linqpad 7 for this one is a ReflectionTypeLoadException wrapping a FileLoadException: "Could not load file or assembly 'net6wrapper,...'. Could not find or load a specific file. (0x80131621)". No path information inside the exception as such I'm affraid.

    · Share on FacebookShare on Twitter
  • JohnMJohnM

    There seems to be some code involving NativeLibrary.SetDllImportResolver with a callback looping over NativeLibrary.TryLoad if that helps

    · Share on FacebookShare on Twitter
  • JoeAlbahariJoeAlbahari

    Sounds like it could be bypassing .NET Core's standard native library resolution system. You could try the workaround described in this thread.

    You might need to experiment with the output folder - maybe add "runtimes\win-x64\native" to the target folder.

    · Share on FacebookShare on Twitter
  • JohnMJohnM

    Since things blow up before the script has a chance to run, (even before OnInit()) there is little oppertunity to add any file copying to the script.

    That's why I was hoping on "Copy all nuget assemblies to local folder" to get things in place

    · Share on FacebookShare on Twitter
  • JoeAlbahariJoeAlbahari

    Try the 7.4.3 beta. It now has an option in Query Properties > Advanced to create a local runtimes\ folder.

    · Share on FacebookShare on Twitter
  • JohnMJohnM

    It works!

    I had to configure the lib to look in Path.Combine(Environment.CurrentDirectory, @"runtimes\win-x64\native") for its dlls and resources (it defaults to looking next to the managed dll) but from there it seems to work nicely :smile:

    · Share on FacebookShare on Twitter
  • ImpPhil1ImpPhil1

    Hi, thanks for the info, checking the options in Query Properties > Advanced to create a local runtimes\ folder fixed my issue.

    I had worked out how to create a nuget package with native dependencies, all I needed was:

      <ItemGroup>
    <Content Include="runtimes\win-x64\native\*"
             CopyToOutputDirectory="Always"
             Pack="true"
             PackagePath="runtimes\win-x64\native" />

    <Content Include="runtimes\win-x86\native\*"
             CopyToOutputDirectory="Always"
             Pack="true"
             PackagePath="runtimes\win-x86\native" />
  </ItemGroup>

    which copies everything from 'runtimes' to the bin folder for local project refs and also adds 'runtimes' to the nuget package. All worked fine except when testing in LinqPad

    · Share on FacebookShare on Twitter
Sign In or Register to comment.