[dotnet-runtime-perf][wasm] `Perf_File.CopyToOverwrite` benchmark failing with `System.IO.IOException: File exists` #74104

radical · 2022-08-17T17:47:51Z

With wasm/aot (build 20220817.1):

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> System.IO.IOException: File exists
   at BenchmarkDotNet.Autogenerated.Runnable_48.Run(IHost host, String benchmarkName)
   at System.Reflection.MethodInvoker.InterpretedInvoke(Object , Span`1 , BindingFlags )
   --- End of inner exception stack trace ---
   at BenchmarkDotNet.Autogenerated.UniqueProgramName.AfterAssemblyLoadingAttached(String[] args)

Full run:

// Found 1 benchmarks:
//   Perf_File.CopyToOverwrite: Job-AFOGNL(PowerPlanMode=00000000-0000-0000-0000-000000000000, Runtime=Wasm, Toolchain=Wasm, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1) [size=104857600]

// **************************
// Benchmark: Perf_File.CopyToOverwrite: Job-AFOGNL(PowerPlanMode=00000000-0000-0000-0000-000000000000, Runtime=Wasm, Toolchain=Wasm, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1) [size=104857600]
// *** Execute ***
// Launch: 1 / 1
// Execute: /home/helixbot/.jsvu/v8 --expose_wasm test-main.js -- --run 372a2e50-2c5c-4b96-b4d4-9fdc4d0208d3.dll --benchmarkName "System.IO.Tests.Perf_File.CopyToOverwrite(size: 104857600)" --job "PowerPlanMode=00000000-0000-0000-0000-000000000000, Runtime=Wasm, Toolchain=Wasm, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1" --benchmarkId 48  in /home/helixbot/work/AD350993/w/A75B092C/e/performance/artifacts/bin/for-running/MicroBenchmarks/372a2e50-2c5c-4b96-b4d4-9fdc4d0208d3/bin/net7.0/browser-wasm/AppBundle
Failed to set up high priority. Make sure you have the right permissions. Message: Permission denied
Incoming arguments: --run 372a2e50-2c5c-4b96-b4d4-9fdc4d0208d3.dll --benchmarkName System.IO.Tests.Perf_File.CopyToOverwrite(size: 104857600) --job PowerPlanMode=00000000-0000-0000-0000-000000000000, Runtime=Wasm, Toolchain=Wasm, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1 --benchmarkId 48
Application arguments: --run 372a2e50-2c5c-4b96-b4d4-9fdc4d0208d3.dll --benchmarkName System.IO.Tests.Perf_File.CopyToOverwrite(size: 104857600) --job PowerPlanMode=00000000-0000-0000-0000-000000000000, Runtime=Wasm, Toolchain=Wasm, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1 --benchmarkId 48
console.debug: mono_wasm_runtime_ready fe00e07a-5519-4dfe-b35a-f867dbaf2e28
console.info: Initializing.....
// BeforeAnythingElse

// Benchmark Process Environment Information:
// Runtime=.NET Core (Mono) 7.0.0-ci, Wasm NativeAOT
// GC=Non-concurrent Workstation
// Job: Job-SSXSXT(PowerPlanMode=00000000-0000-0000-0000-000000000000, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1)


System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> System.IO.IOException: File exists
   at BenchmarkDotNet.Autogenerated.Runnable_48.Run(IHost host, String benchmarkName)
   at System.Reflection.MethodInvoker.InterpretedInvoke(Object , Span`1 , BindingFlags )
   --- End of inner exception stack trace ---
   at BenchmarkDotNet.Autogenerated.UniqueProgramName.AfterAssemblyLoadingAttached(String[] args)
// AfterAll
{"name":"ExitStatus","message":"Program terminated with exit(-1)","status":-1}
No Workload Results were obtained from the run.
// Benchmark Process 7089 has exited with code 255.

cc @adamsitnik @LoopedBard3

The text was updated successfully, but these errors were encountered:

adamsitnik · 2022-08-17T21:22:56Z

@radical This particular benchmark should overwrite any existing files:

https://github.com/dotnet/performance/blob/3edf4c3149f5c903777f13af9b08b562b1672302/src/benchmarks/micro/libraries/System.IO.FileSystem/Perf.File.cs#L272

So it looks like a product bug?

radical · 2022-08-17T21:26:15Z

How do I run a single benchmark locally? I know how to run the whole thing or partitions.

adamsitnik · 2022-08-17T21:32:08Z

How do I run a single benchmark locally? I know how to run the whole thing or partitions.

You need to specify filter, which is a glob pattern applied to namespace.typeName.methodName. In your case the simplest one will be:

--filter '*CopyToOverwrite'

(remember to escape asterisk on Unixes)

dotnet-issue-labeler · 2022-08-17T21:33:56Z

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

ghost · 2022-08-17T21:35:55Z

Tagging subscribers to this area: @dotnet/area-system-io
See info in area-owners.md if you want to be subscribed.

Issue Details

With wasm/aot (build 20220817.1):

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> System.IO.IOException: File exists
   at BenchmarkDotNet.Autogenerated.Runnable_48.Run(IHost host, String benchmarkName)
   at System.Reflection.MethodInvoker.InterpretedInvoke(Object , Span`1 , BindingFlags )
   --- End of inner exception stack trace ---
   at BenchmarkDotNet.Autogenerated.UniqueProgramName.AfterAssemblyLoadingAttached(String[] args)

Full run:

// Found 1 benchmarks:
//   Perf_File.CopyToOverwrite: Job-AFOGNL(PowerPlanMode=00000000-0000-0000-0000-000000000000, Runtime=Wasm, Toolchain=Wasm, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1) [size=104857600]

// **************************
// Benchmark: Perf_File.CopyToOverwrite: Job-AFOGNL(PowerPlanMode=00000000-0000-0000-0000-000000000000, Runtime=Wasm, Toolchain=Wasm, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1) [size=104857600]
// *** Execute ***
// Launch: 1 / 1
// Execute: /home/helixbot/.jsvu/v8 --expose_wasm test-main.js -- --run 372a2e50-2c5c-4b96-b4d4-9fdc4d0208d3.dll --benchmarkName "System.IO.Tests.Perf_File.CopyToOverwrite(size: 104857600)" --job "PowerPlanMode=00000000-0000-0000-0000-000000000000, Runtime=Wasm, Toolchain=Wasm, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1" --benchmarkId 48  in /home/helixbot/work/AD350993/w/A75B092C/e/performance/artifacts/bin/for-running/MicroBenchmarks/372a2e50-2c5c-4b96-b4d4-9fdc4d0208d3/bin/net7.0/browser-wasm/AppBundle
Failed to set up high priority. Make sure you have the right permissions. Message: Permission denied
Incoming arguments: --run 372a2e50-2c5c-4b96-b4d4-9fdc4d0208d3.dll --benchmarkName System.IO.Tests.Perf_File.CopyToOverwrite(size: 104857600) --job PowerPlanMode=00000000-0000-0000-0000-000000000000, Runtime=Wasm, Toolchain=Wasm, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1 --benchmarkId 48
Application arguments: --run 372a2e50-2c5c-4b96-b4d4-9fdc4d0208d3.dll --benchmarkName System.IO.Tests.Perf_File.CopyToOverwrite(size: 104857600) --job PowerPlanMode=00000000-0000-0000-0000-000000000000, Runtime=Wasm, Toolchain=Wasm, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1 --benchmarkId 48
console.debug: mono_wasm_runtime_ready fe00e07a-5519-4dfe-b35a-f867dbaf2e28
console.info: Initializing.....
// BeforeAnythingElse

// Benchmark Process Environment Information:
// Runtime=.NET Core (Mono) 7.0.0-ci, Wasm NativeAOT
// GC=Non-concurrent Workstation
// Job: Job-SSXSXT(PowerPlanMode=00000000-0000-0000-0000-000000000000, IterationTime=250.0000 ms, MaxIterationCount=20, MinIterationCount=15, WarmupCount=1)


System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> System.IO.IOException: File exists
   at BenchmarkDotNet.Autogenerated.Runnable_48.Run(IHost host, String benchmarkName)
   at System.Reflection.MethodInvoker.InterpretedInvoke(Object , Span`1 , BindingFlags )
   --- End of inner exception stack trace ---
   at BenchmarkDotNet.Autogenerated.UniqueProgramName.AfterAssemblyLoadingAttached(String[] args)
// AfterAll
{"name":"ExitStatus","message":"Program terminated with exit(-1)","status":-1}
No Workload Results were obtained from the run.
// Benchmark Process 7089 has exited with code 255.

cc @adamsitnik @LoopedBard3

Author:	radical
Assignees:	-
Labels:	`area-System.IO`, `untriaged`, `blocking-clean-ci-optional`
Milestone:	-

jozkee · 2022-08-19T04:21:19Z

@adamsitnik could this be a regression in 7.0?

adamsitnik · 2022-08-19T07:09:15Z

could this be a regression in 7.0?

@jozkee It could, but this benchmark runs on CLR on Unix pretty fine, so I am sure that it's wasm-specific issue.

@radical is someone from the WASM Team going to investigate it?

radical · 2022-09-07T01:42:34Z

I'm not able to reproduce this locally.
it happens for CopyTo, CopyToOverride, and ReadAllBytes, but not always
it happens only with wasm/AOT
I think the exception is thrown from the SetupReadAllBytes method:

https://github.com/dotnet/performance/blob/5f63e8dcf81a0f9ddf5a80bd791d1fbaf0babd61/src/benchmarks/micro/libraries/System.IO.FileSystem/Perf.File.cs#L92-L113

.. and all the failing benchmarks are using this setup method:

GlobalSetup(Targets = new[] { nameof(ReadAllBytes), "ReadAllBytesAsync", nameof(CopyTo), nameof(CopyToOverwrite) })]

The same issue on main shows the file name /tmp/:

---> System.IO.IOException: The file '/tmp/' already exists.
  at BenchmarkDotNet.Autogenerated.Runnable_48.Run(IHost host, String benchmarkName)
  at System.Reflection.MethodInvoker.InterpretedInvoke(Object , Span`1 , BindingFlags )
  --- End of inner exception stack trace ---
  at BenchmarkDotNet.Autogenerated.UniqueProgramName.AfterAssemblyLoadingAttached(String[] args)

I will debug the issue on CI.

danmoseley · 2022-09-07T04:09:38Z

@radical Dupe of #73721 ?

radical · 2022-09-07T04:24:12Z

@radical Dupe of #73721 ?

Possibly, though here it runs with v8 instead of nodejs.
The code here just needs a random name for the file, but Path.GetTempFileName will try to create the file, and is probably running into this issue. I'll open a PR to update that.
Thanks @danmoseley!

On wasm/aot, `System.IO.FileSystem/Perf.File`'s `CopyTo`, `CopyToOverride`, and `ReadAllBytes` benchmarks fail randomly with: ``` ---> System.IO.IOException: The file '/tmp/' already exists. at BenchmarkDotNet.Autogenerated.Runnable_48.Run(IHost host, String benchmarkName) at System.Reflection.MethodInvoker.InterpretedInvoke(Object , Span`1 , BindingFlags ) --- End of inner exception stack trace --- at BenchmarkDotNet.Autogenerated.UniqueProgramName.AfterAssemblyLoadingAttached(String[] args) ``` .. due to `Path.GetTempFileName()` being buggy - dotnet/runtime#73721. In `SetupReadAllBytes`, we are building the test file path as: ```csharp _testFilePath = Path.Combine(baseDir, Path.GetTempFileName()) ``` .. but the `Path.GetTempFileName()` is not appropriate here, as it will return a full path, and it will create the file. Instead we can use `Path.GetRandomFileName()` which should avoid the underlying issue completely. Fixes dotnet/runtime#74104 .

radical added the pipeline blocker label Aug 17, 2022

radical mentioned this issue Aug 17, 2022

perf pipeline: Switch to node 14.x #74017

Merged

radical transferred this issue from dotnet/performance Aug 17, 2022

ghost added the untriaged New issue has not been triaged by the area owner label Aug 17, 2022

radical added blocking-clean-ci-optional Blocking optional rolling runs area-System.IO and removed pipeline blocker labels Aug 17, 2022

radical added the arch-wasm WebAssembly architecture label Aug 17, 2022

runfoapp bot mentioned this issue Aug 17, 2022

Infrastructure - Status/Health #702

Closed

adamsitnik added this to the 8.0.0 milestone Aug 19, 2022

ghost removed the untriaged New issue has not been triaged by the area owner label Aug 19, 2022

radical self-assigned this Sep 6, 2022

radical mentioned this issue Sep 7, 2022

Fix some randomly failing Perf.File tests on wasm/aot dotnet/performance#2601

Merged

radical added the in-pr There is an active PR which will close this issue when it is merged label Sep 7, 2022

adamsitnik closed this as completed in dotnet/performance#2601 Sep 8, 2022

ghost locked as resolved and limited conversation to collaborators Oct 8, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[dotnet-runtime-perf][wasm] `Perf_File.CopyToOverwrite` benchmark failing with `System.IO.IOException: File exists` #74104

[dotnet-runtime-perf][wasm] `Perf_File.CopyToOverwrite` benchmark failing with `System.IO.IOException: File exists` #74104

radical commented Aug 17, 2022 •

edited

Loading

adamsitnik commented Aug 17, 2022

radical commented Aug 17, 2022

adamsitnik commented Aug 17, 2022

dotnet-issue-labeler bot commented Aug 17, 2022

ghost commented Aug 17, 2022

jozkee commented Aug 19, 2022

adamsitnik commented Aug 19, 2022

radical commented Sep 7, 2022

danmoseley commented Sep 7, 2022

radical commented Sep 7, 2022

[dotnet-runtime-perf][wasm] Perf_File.CopyToOverwrite benchmark failing with System.IO.IOException: File exists #74104

[dotnet-runtime-perf][wasm] Perf_File.CopyToOverwrite benchmark failing with System.IO.IOException: File exists #74104

Comments

radical commented Aug 17, 2022 • edited Loading

adamsitnik commented Aug 17, 2022

radical commented Aug 17, 2022

adamsitnik commented Aug 17, 2022

dotnet-issue-labeler bot commented Aug 17, 2022

ghost commented Aug 17, 2022

jozkee commented Aug 19, 2022

adamsitnik commented Aug 19, 2022

radical commented Sep 7, 2022

danmoseley commented Sep 7, 2022

radical commented Sep 7, 2022

[dotnet-runtime-perf][wasm] `Perf_File.CopyToOverwrite` benchmark failing with `System.IO.IOException: File exists` #74104

[dotnet-runtime-perf][wasm] `Perf_File.CopyToOverwrite` benchmark failing with `System.IO.IOException: File exists` #74104

radical commented Aug 17, 2022 •

edited

Loading