System.IO.InvalidDataException for .Net 6 - 6.0.413

[sacOO7]

Unhandled exception. Fake.Core.BuildFailedException: Target 'NetStandard - Build' failed.
---> System.AggregateException: One or more errors occurred. (String record number 30030 is invalid: string index -8 is not within 8488.)
---> System.IO.InvalidDataException: String record number 30030 is invalid: string index -8 is not within 8488.
at Microsoft.Build.Logging.StructuredLogger.BuildEventArgsReader.GetStringFromRecord(Int32 index) in C:\MSBuildStructuredLog\src\StructuredLogger\BinaryLogger\BuildEventArgsReader.cs:line 1228
at Microsoft.Build.Logging.StructuredLogger.BuildEventArgsReader.ReadDeduplicatedString() in C:\MSBuildStructuredLog\src\StructuredLogger\BinaryLogger\BuildEventArgsReader.cs:line 1203
at Microsoft.Build.Logging.StructuredLogger.BuildEventArgsReader.ReadOptionalString() in C:\MSBuildStructuredLog\src\StructuredLogger\BinaryLogger\BuildEventArgsReader.cs:line 1192
at Microsoft.Build.Logging.StructuredLogger.BuildEventArgsReader.ReadTargetSkippedEventArgs() in C:\MSBuildStructuredLog\src\StructuredLogger\BinaryLogger\BuildEventArgsReader.cs:line 293
at Microsoft.Build.Logging.StructuredLogger.BuildEventArgsReader.Read() in C:\MSBuildStructuredLog\src\StructuredLogger\BinaryLogger\BuildEventArgsReader.cs:line 167
at Microsoft.Build.Logging.StructuredLogger.BinLogReader.ReadRecordsFromDecompressedStream(Stream decompressedStream)+MoveNext() in C:\MSBuildStructuredLog\src\StructuredLogger\BinaryLogger\BinLogReader.cs:line 275

This started happening recently with latest versions of .Net 6 and .Net 7. For old versions, build works just fine.
more information related to error log -> https://github.com/ably/ably-dotnet/actions/runs/6133243415/job/16644925573

[baronfel]

FAKE uses a library to collect and read MSBuild Binary logs (https://www.nuget.org/packages/MSBuild.StructuredLogger), but the version of the library it required is now too old to parse the binlogs created by newer versions of MSBuild shipped with the .NET SDK.

To fix this, either FAKE needs to update it's dependency version, or your build project can manually bump this version. This is what I've done with all of the FAKE-using tools and libraries that I maintain.

[sacOO7]

@baronfel thanks for the response. For the time being, I have lowered by dependency versions for dotnet 6 and dotnet 7, and the builds seem to be working fine. Can you tell me exactly what you did to fix the issue? So, issue shouldn't occur for latest versions of dotnet6 and dotnet7

[sacOO7]

You can see, my F# build project is here -> https://github.com/ably/ably-dotnet/tree/main/build-script. As per logs, it import

Tool 'paket' (version '7.2.0') was restored. Available commands: paket
Tool 'fake-cli' (version '5.23.1') was restored. Available commands: fake
Tool 'fsdocs-tool' (version '17.2.0') was restored. Available commands: fsdocs
Tool 'fantomas' (version '5.1.5') was restored. Available commands: fantomas

before starting build

[baronfel]

Yep! Add https://www.nuget.org/packages/MSBuild.StructuredLogger to your paket.dependencies file (to make it as a top-level dependency) and then run 'paket update MSBuild.StructuredLogger' and you should be good to go.

[sacOO7]

Wow, Actually I had a thought of doing this, but never tried it! ( Kept updating Microsoft.Build instead ). I will surely update this.
Btw, as you can see my paket.dependencies file is ->

nuget Fake.Core.Target
nuget Fake.DotNet.AssemblyInfoFile
nuget Fake.DotNet.Cli
nuget Fake.DotNet.Testing.XUnit2
nuget Microsoft.Build 17.3.2
nuget Microsoft.Build.Tasks.Core 17.3.2

If I try to update Microsoft.Build 17.3.2 to just Microsoft.Build, it fails with a different error. I mean it does update Microsoft.Build to latest version but fails with

Unhandled exception. Fake.Core.BuildFailedException: Target 'Xamarin - Build' failed.
 ---> System.AggregateException: One or more errors occurred. (Could not load file or assembly 'Microsoft.Build.Framework, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.
)
 ---> System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Build.Framework, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.

File name: 'Microsoft.Build.Framework, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'
   at Fake.DotNet.MSBuildBinLog.getErrorsAndWarnings(String binLogFilePath)
   at Fake.DotNet.MSBuild.handleAfterRun(String command, FSharpOption`1 binLogPath, Int32 exitCode, String project) in D:\a\FAKE\FAKE\src\app\Fake.DotNet.MSBuild\MSBuild.fs:line 957
   at Fake.DotNet.MSBuild.build(FSharpFunc`2 setParams, String project) in D:\a\FAKE\FAKE\src\app\Fake.DotNet.MSBuild\MSBuild.fs:line 1110
   at [email protected](TargetParameter _arg6) in /Users/runner/work/ably-dotnet/ably-dotnet/build-script/build.fs:line 165
   at Fake.Core.TargetModule.runSimpleInternal(TargetContext context, Target target) in D:\a\FAKE\FAKE\src\app\Fake.Core.Target\Target.fs:line 371
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   at Fake.Core.TargetModule.raiseIfError(OptionalTargetContext context) in D:\a\FAKE\FAKE\src\app\Fake.Core.Target\Target.fs:line [1319](https://github.com/ably/ably-dotnet/actions/runs/6133187077/job/16644821393#step:6:1320)
   at Fake.Core.TargetModule.runOrDefaultWithArguments(String defaultTarget) in D:\a\FAKE\FAKE\src\app\Fake.Core.Target\Target.fs:line 1542
   at Build.main(String[] argv) in /Users/runner/work/ably-dotnet/ably-dotnet/build-script/build.fs:line 557

[sacOO7]

@baronfel if would be great, if you can help with this one 👍

[baronfel]

you're using the incorrect packages here - remove Microsoft.Build and Microsoft.Build.Tasks.Core completely (you don't need them for your build script and they will just cause problems) and add MSBuild.StructuredLogger (which is the actual package that has the binlog parser that FAKE needs).

[sacOO7]

@baronfel with above changes, seems I am still getting the same error here : (
https://github.com/ably/ably-dotnet/actions/runs/6139039467/job/16656528250?pr=1199
Is fake tool not accepting the latest version of nuget MSBuild.StructuredLogger ?
Paket dependencies looks like -> https://github.com/ably/ably-dotnet/blob/feature/integration-2.0/build-script/paket.dependencies
Paket references -> https://github.com/ably/ably-dotnet/blob/feature/integration-2.0/build-script/paket.references
Also updated paket.lock accordingly

It's the same error

Unhandled exception. Fake.Core.BuildFailedException: Target 'Xamarin - Build' failed.
 ---> System.AggregateException: One or more errors occurred. (Could not load file or assembly 'Microsoft.Build.Framework, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.
)
 ---> System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Build.Framework, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.

File name: 'Microsoft.Build.Framework, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'
   at Fake.DotNet.MSBuildBinLog.getErrorsAndWarnings(String binLogFilePath)
   at Fake.DotNet.MSBuild.handleAfterRun(String command, FSharpOption`1 binLogPath, Int32 exitCode, String project) in D:\a\FAKE\FAKE\src\app\Fake.DotNet.MSBuild\MSBuild.fs:line 957
   at Fake.DotNet.MSBuild.build(FSharpFunc`2 setParams, String project) in D:\a\FAKE\FAKE\src\app\Fake.DotNet.MSBuild\MSBuild.fs:line 1110
   at [email protected](TargetParameter _arg6) in /Users/runner/work/ably-dotnet/ably-dotnet/build-script/build.fs:line 165
   at Fake.Core.TargetModule.runSimpleInternal(TargetContext context, Target target) in D:\a\FAKE\FAKE\src\app\Fake.Core.Target\Target.fs:line 371
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   at Fake.Core.TargetModule.raiseIfError(OptionalTargetContext context) in D:\a\FAKE\FAKE\src\app\Fake.Core.Target\Target.fs:line [1319](https://github.com/ably/ably-dotnet/actions/runs/6133187077/job/16644821393#step:6:1320)
   at Fake.Core.TargetModule.runOrDefaultWithArguments(String defaultTarget) in D:\a\FAKE\FAKE\src\app\Fake.Core.Target\Target.fs:line 1542
   at Build.main(String[] argv) in /Users/runner/work/ably-dotnet/ably-dotnet/build-script/build.fs:line 557

[sacOO7]

@baronfel any help related to this would be useful. Seems removing

nuget Microsoft.Build 17.3.2
nuget Microsoft.Build.Tasks.Core 17.3.2

Gives above error : (

[sacOO7]

Also, adding https://www.nuget.org/packages/MSBuild.StructuredLogger doesn't seem to be fixing the issue : (
Can you point me to the repo/example code where we have implemented this?

[sacOO7]

@baronfel did you get time to look into this?

[KirillOsenkov]

What is your TargetFramework? Can you try and target net7.0?

[sacOO7]

@KirillOsenkov I have targets for both net6.0 and net7.0, and it fails for both : (

[sacOO7]

You can check failing CI here ably/ably-dotnet#1199.
I reverted dotnet versions for net6.0 and net7.0, builds still seem to be failing

[sacOO7]

https://github.com/ably/ably-dotnet/actions/runs/6227702948/job/16902899345?pr=1199

[KirillOsenkov]

You do need the latest version of MSBuild.StructuredLogger NuGet package.

You will then run into another problem, because if you're targeting net6.0, then Microsoft.Build.Framework.dll doesn't get copied to the output directory, and your code fails when it tries to load it. I think if you target net7.0 this should be fixed. But you need to double-check. I'd check whether Microsoft.Build.Framework.dll is present in the output directory, and if not, build with /bl and inspect the resulting msbuild.binlog as to why the Microsoft.Build.Framework.dll isn't getting copied.

[sacOO7]

@KirillOsenkov you can clone the repo and try to reproduce issue on the CI if needed 👍

[KirillOsenkov]

Which branch/commit should I try? where is the commit where you update the binlog to latest?

[KirillOsenkov]

I found this commit: ably/ably-dotnet@e6531a8

The problem is that I think net7.0 will succeed, but net6.0 will fail.

[KirillOsenkov]

I'm running the tests locally with build Test.NetStandard.Unit.WithRetry -f net7.0 and I can't reproduce it locally.

All I'm seeing is an unrelated timezone failure:

For this error:

Unhandled exception. Fake.Core.BuildFailedException: Target 'Xamarin - Build' failed.
 ---> System.AggregateException: One or more errors occurred. (Could not load file or assembly 'Microsoft.Build.Framework, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.
)
 ---> System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Build.Framework, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.

File name: 'Microsoft.Build.Framework, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'
   at Fake.DotNet.MSBuildBinLog.getErrorsAndWarnings(String binLogFilePath)
   at Fake.DotNet.MSBuild.handleAfterRun(String command, FSharpOption`1 binLogPath, Int32 exitCode, String project) in D:\a\FAKE\FAKE\src\app\Fake.DotNet.MSBuild\MSBuild.fs:line 957
   at Fake.DotNet.MSBuild.build(FSharpFunc`2 setParams, String project) in D:\a\FAKE\FAKE\src\app\Fake.DotNet.MSBuild\MSBuild.fs:line 1110

You need to make sure Microsoft.Build.Framework.dll is in the same directory as MSBuild.StructuredLogger.dll

In this commit:
ably/ably-dotnet@e6531a8

You should keep
nuget Microsoft.Build
nuget Microsoft.Build.Tasks.Core

Just remove the versions or use 17.5.0

[marcpopMSFT]

Old issue triage: baronfel says this is likely solved at this point because of better compat of binlogs. Closing.