Race condition in EnsureFolderAsync can cause "folder already exists" exception under concurrent requests

[aramB]

Category

• Bug

Describe the bug

We are intermittently seeing a race condition in EnsureFolderAsync when multiple concurrent requests attempt to create the same folder structure.

The issue appears to occur in the following logic inside Folder.EnsureFolderAsync:

try
{
    currentFolder = await currentFolder.PnPContext.Web
        .GetFolderByServerRelativeUrlAsync(currentUrl, expressions)
        .ConfigureAwait(false);
}
catch (SharePointRestServiceException)
{
    currentFolder = await currentFolder
        .AddFolderAsync(folderName)
        .ConfigureAwait(false);

    currentFolderWasCreated = true;
}

Under concurrent execution, two threads/processes can both receive a 404 from GetFolderByServerRelativeUrlAsync, after which both attempt AddFolderAsync.

One succeeds, while the second fails with:

HttpResponseCode: 400
Code: Microsoft.SharePoint.SPException
Message: A file or folder with the name ... already exists.

Observed sequence:

Thread A: GET folder -> 404
Thread B: GET folder -> 404

Thread A: POST AddFolder -> success
Thread B: POST AddFolder -> 400 already exists

This makes EnsureFolderAsync non-idempotent under concurrent access even though the final desired state ("folder exists") is actually achieved.

The issue is intermittent and difficult to reproduce consistently, but logs strongly indicate a race condition between the GET and POST operations.

Steps to reproduce

1. Execute multiple concurrent calls to EnsureFolderAsync targeting the same folder path.
2. Example code:

public async Task<string?> EnsureFolderAsync(string siteUrl, string listTitle, string folderPath)
{
    using PnPContext pnpContext = await _pnpContextFactory.CreateAsync(siteUrl);

    IList list = await GetList(pnpContext, listTitle);

    IFolder? folder = await list.RootFolder.EnsureFolderAsync(folderPath);

    return folder?.ServerRelativeUrl;
}

3. Under concurrent load, intermittent failures occur where:

   • GetFolderByServerRelativeUrlAsync returns 404
   • AddFolderAsync subsequently throws 400 already exists

Observed exception:

HttpResponseCode: 400
Code: Microsoft.SharePoint.SPException
Message: A file or folder with the name sites/C7/SE/C7/#3705-1/4379 already exists.
ClientRequestId: 90a5e1a1-801c-e000-887f-4fc30128cff1
SPClientServiceRequestDuration: 157
X-SharePointHealthScore: 3
X-SP-SERVERSTATE: ReadOnly=0

Expected behavior

EnsureFolderAsync should behave safely under concurrent execution.

If folder creation fails because another concurrent request already created the folder, the implementation could re-fetch the folder instead of propagating the exception.

Conceptually:

GET -> 404
POST -> "already exists"
=> re-fetch folder and continue

Environment details (development & target environment)

• SDK version: 1.14

• OS: Windows

• SDK used in: ASP.NET Web App

• Framework: .NET 8

• Browser(s): N/A

• Tooling: Visual Studio 2022

• Additional details:

  • Issue occurs intermittently under concurrent requests.
  • No deterministic reproduction yet.
  • Discussion context suggested this may be a race condition between GetFolderByServerRelativeUrlAsync and AddFolderAsync.

Additional context

Based on discussion in this thread: #1710, a possible mitigation would be handling the specific "already exists" exception from AddFolderAsync and then re-fetching the folder instead of failing the operation.

[Adam-it]

Thanks @aramB for openning up the bug.
Since it is pretty well definited do you wanna get assigned to it and start working on it and open a PR?
Or you rather leave it for someone else from the community to pick it up?

[aramB]

@Adam-it I would love to give it a try 🙂

I’m not very familiar with the source code/internal flow yet, so I might need some guidance or feedback along the way, but I’m happy to investigate it and see if I can put together a proper fix and PR.

[Adam-it]

@aramB all yours.
Don't hesitate to reach out if you need any help.
You may use our contribution guide https://pnp.github.io/pnpcore/contributing/readme.html to get you up and running
if you use any of the AI agents you may use context7 MCP and get grounded context for PnP Core to get better results https://context7.com/pnp/pnpcore
We also support deepwiki https://deepwiki.com/pnp/pnpcore for in depth technical content if you wann really go into details.

[aramB]

@Adam-it I have some changes including some unit tests in place now. But I am having some trouble with running the tests since I haven't managed to setup my test environment.
Tried to use https://github.com/pnp/pnpcore/blob/dev/src/sdk/PnP.Core.Test/setuptestenv.ps1 but it seems to use the old auth method? Can you help me? Does it need to be updated?

[Adam-it]

@Adam-it I have some changes including some unit tests in place now. But I am having some trouble with running the tests since I haven't managed to setup my test environment.
Tried to use https://github.com/pnp/pnpcore/blob/dev/src/sdk/PnP.Core.Test/setuptestenv.ps1 but it seems to use the old auth method? Can you help me? Does it need to be updated?

Yes it is out of date 😢. Sorry for that. I have the issue to refresh the docs on me for some time now but there a a couple of other high priority stuff I need to catch up until I do that.
I've done that in the past so it's been a while since I went over the process. Will try to prep something over the night for ya

[Adam-it]

@aramB sorry for the late response. I updated the setup test env script and you may see the change in this PR #1768. It supports the new interactive login method and has verification steps for each point of the test setup.
Give it a try and let me know if the script did the trick. I will update the docs as well as soon as I find time for it

[aramB]

Thanks a lot, @Adam-it! I tested the script and in order to make it work for me I needed to make 2 small changes:

1- Added -Interactive to all Connect-PnPOnline otherwise I got prompted with user/password in the terminal.
2- Needed to change this (I guess bcs Email was not filled for my principal in my test Entra tenant?):

    if ([String]::IsNullOrEmpty($UPN)) {
      $UPN = (Get-PnPWeb -Connection $tenantContext -Includes CurrentUser).CurrentUser.Email
    }

to this:

    if ([String]::IsNullOrEmpty($UPN)) {
        $UPN = (Get-PnPWeb -Connection $tenantContext -Includes CurrentUser).CurrentUser.LoginName

        # LoginName comes back as i:0#.f|membership|my-upn@contoso.onmicrosoft.com so strip the prefix
        $UPN = $UPN -replace "i:0#\.f\|membership\|", ""
    }

I also have a suggestion, what do you think about adding $ErrorActionPreference = 'Stop' to the script? Since there are dependencies between different steps. For instance if we fail to assign $pnpTestSite when creating a site then both connecting to it and creating a subsite will fail, right? Do we want the script continue executing anyway?

[Adam-it]

@aramB glad it got you unblocked.
Thanks for the tips. The $ErrorActionPreference = 'Stop' we should for sure add. I even had it but I removed it to test the script out for different scenarios and I forgot to add it 🤦‍♂️.
Other tips are also good advice and I will add them to my script. I think the interactive mode is default in my case but definitely worth adding it