Use macOS clonefile(2) for copy-on-write file cloning in Copy task#13671
Draft
jankratochvilcz wants to merge 1 commit into
Draft
Use macOS clonefile(2) for copy-on-write file cloning in Copy task#13671jankratochvilcz wants to merge 1 commit into
jankratochvilcz wants to merge 1 commit into
Conversation
Add an experimental optimization that uses macOS clonefile(2) syscall for O(1) copy-on-write file cloning on APFS, gated behind the MSBUILD_EXPERIMENTAL_COPY=1 environment variable. - Add clonefile P/Invoke to Tasks/NativeMethods.cs - Integrate TryCloneFile into Copy.cs with silent fallback to File.Copy - Skip clone attempt when CopyWithoutDelete escape hatch is active - Add 6 macOS-only unit tests: happy path, copy-on-write divergence, read-only handling, env var fallback, incremental build invariant, and performance load test (1000 files, 1KB-1MB: ~2.6x speedup) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Member
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add an experimental optimization that uses macOS
clonefile(2)syscall for O(1) copy-on-write file cloning on APFS, gated behind theMSBUILD_EXPERIMENTAL_COPY=1environment variable.What is clonefile(2)?
clonefile(2)is a macOS syscall (Sierra 10.12+) that creates a copy-on-write clone of a file on APFS. It only copies metadata; data blocks are shared until either side is modified, then diverge transparently. Cost is O(1) — typically 50–200 µs regardless of file size.Why not hardlinks?
Hardlinks share an inode:
chmod/chownon the destination affects the source, and any tool that opens the destination for writing withoutO_TRUNCmutates the source. Build outputs that get post-processed (codesign, ilrepack) can't safely use hardlinks. Clonefile gives the speed of a hardlink with the semantics of a copy.Changes
src/Tasks/NativeMethods.cs: Addedclonefile(2)P/Invoke andTryCloneFile()helpersrc/Tasks/Copy.cs: Integrated clone attempt beforeFile.Copyfallback, gated behindMSBUILD_EXPERIMENTAL_COPY=1env varsrc/Tasks.UnitTests/Copy_Tests.cs: 6 macOS-only tests ([MacOSOnlyFact])Performance (macOS APFS, Apple Silicon)
Design decisions
MSBUILD_EXPERIMENTAL_COPY=1— no behavioral change without opt-inFile.Copyon anyclonefileerror (non-APFS, cross-device, Docker, etc.)CopyWithoutDeleteescape hatch is active and dest existsclonefilepreserves mtime/attributes → incremental builds work correctly