Vert.x has a DoS via unbounded server-side SNI SslContext cache growth

Potential unbounded server-side SNI SslContext cache growth in Vert.x TLS handling, with = resource-exhaustion / DoS impact. On affected versions, matching server-side SNI names are cached via computeIfAbsent(serverName, ...) in a serverName-keyed SslContext cache.

The implementation differs slightly by branch, but the same sink appears to be present in released versions 4.3.4 through 5.0.11:

• 4.3.x: SSLHelper
• 4.4.x / 4.5.x: SslChannelProvider
• 5.0.x and current master: SslContextProvider

When server-side SNI is enabled and wildcard or otherwise broad hostname mappings are used, an unauthenticated client can send many distinct matching SNI names and cause the server to retain increasing numbers of SslContext entries over time, leading to increasing memory consumption and possible DoS conditions.

Steps to reproduce

1. Configure a Vert.x server with setSsl(true) and setSni(true).
2. Use a keystore or mapping where many distinct SNI names match a wildcard or similarly broad rule.
3. Send repeated connections with distinct matching SNI values.
4. Observe that the SNI cache size grows with the number of unique matching names.

What are the affected versions?

Affected released versions confirmed on origin:

• 4.3.4 through 4.3.8
• 4.4.0 through 4.4.9
• 4.5.0 through 4.5.26
• 5.0.0 through 5.0.11

Not affected by the same sink:

• 4.0.x through 4.2.x
• 4.3.0 through 4.3.3