SslConnection does not release ByteBuffers in case of error code paths.
For example, TLS handshakes that require client-auth with clients that send expired certificates will trigger a TLS handshake errors and the ByteBuffers used to process the TLS handshake will be leaked.
Configure explicitly a RetainableByteBufferPool with max[Heap|Direct]Memory to limit the amount of memory that is leaked.
Eventually the pool will be full of "active" entries (the leaked ones) and will provide ByteBuffers that will be GCed normally.
With embedded-jetty
int maxBucketSize = 1000;
long maxHeapMemory = 128 * 1024L * 1024L; // 128 MB
long maxDirectMemory = 128 * 1024L * 1024L; // 128 MB
RetainableByteBufferPool rbbp = new ArrayRetainableByteBufferPool(0, -1, -1, maxBucketSize, maxHeapMemory, maxDirectMemory);
server.addBean(rbbp); // make sure the ArrayRetainableByteBufferPool is added before the server is started
server.start();
With jetty-home/jetty-base
Create a ${jetty.base}/etc/retainable-byte-buffer-config.xml
<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">
<Configure id="Server" class="org.eclipse.jetty.server.Server">
  <Call name="addBean">
    <Arg>
      <New class="org.eclipse.jetty.io.ArrayRetainableByteBufferPool">
        <Arg type="int"><Property name="jetty.byteBufferPool.minCapacity" default="0"/></Arg>
        <Arg type="int"><Property name="jetty.byteBufferPool.factor" default="-1"/></Arg>
        <Arg type="int"><Property name="jetty.byteBufferPool.maxCapacity" default="-1"/></Arg>
        <Arg type="int"><Property name="jetty.byteBufferPool.maxBucketSize" default="1000"/></Arg>
        <Arg type="long"><Property name="jetty.byteBufferPool.maxHeapMemory" default="128000000"/></Arg>
        <Arg type="long"><Property name="jetty.byteBufferPool.maxDirectMemory" default="128000000"/></Arg>
      </New>
    </Arg>
  </Call>
</Configure>
And then reference it in ${jetty.base}/start.d/retainable-byte-buffer-config.ini
etc/retainable-byte-buffer-config.xml
https://github.com/eclipse/jetty.project/issues/8161
{
    "nvd_published_at": "2022-07-07T21:15:00Z",
    "github_reviewed_at": "2022-07-07T20:55:37Z",
    "cwe_ids": [
        "CWE-404"
    ],
    "severity": "HIGH",
    "github_reviewed": true
}