Directus's S3 assets become unavailable after a burst of HEAD requests

Summary

There's some tools that use Directus to sync content and assets. Some of those tools use HEAD method, like Shopify, to check the existence of files. Although, when making many HEAD requests at once, at some point, all assets are being served as 403.

Details

When I was investigating this issue, I have found that after the burst of HEAD requests, the amount of sockets held on Agent on NodeHttpHandler was always equal to STORAGE_CLOUD_MAX_SOCKETS making it impossible to have new connections causing assets to be inaccessible.

After looking into this issue on AWS SDK I found that if the stream is requested, it needs to be consumed otherwise will hang forever. And as can be seen here the stream is not consumed.

The timeouts set here had no noticeable effect on tests made.

PoC

This can be easily reproduced with the following steps:

• setup AWS S3 storage
• set STORAGE_CLOUD_MAX_SOCKETS: "50" (this value is lower than default for easier reproduction)
• upload a file to your project
• run this file (Replace the the file ID with the one you just uploaded):

import axios from "axios";

async function start() {
  Array.from({ length: 400 }, (_, i) => {
    axios
      .head(
        "http://localhost:8055/assets/e536aa35-3a81-4fa9-b856-3780584d38d8"
      )
      .then(() => console.log("✅"))
      .catch((e) =>
        console.log("⛔", e.response?.status || e.code || e.message)
      );
  });
}

start();

Here's an example:

https://github.com/user-attachments/assets/29d65bf0-5637-478f-a215-083c2ded3753

Impact

This causes denial of assets for all policies of Directus, including Admin and Public.