Netty

An engine built on Netty: one event loop runs per CPU core, accepting and serving connections independently. It is the counterpart of GenHTTP’s ioxide engine - the request parser, the request and response types and the serialization are the ones of the internal engine, what differs is how connections are run.

import org.genhttp.engine.netty.Host
import org.genhttp.modules.functional.Inline

suspend fun main() {
    Host.create()
        .handler(Inline.create().get { "Hello World!" })
        .run()
}

How It Works

• One event loop per core. On Linux, every loop binds its own socket to the port of an endpoint (SO_REUSEPORT), so the kernel spreads new connections over the loops, and a connection stays on the loop that accepted it for its whole life. Sockets are registered with epoll once, a request costs a read and a write.
• Coroutines on the loop. Every connection is served by a coroutine on its loop. The coroutine suspends while it waits for data and is resumed by the loop as soon as the data arrives, so handlers run on the loop as well. Timers (delay, withTimeout) are scheduled on the loop too.
• Shared parsing. Requests are parsed by glyph11-kt with the same limits as on the internal engine. Responses are written straight into the pooled buffers of Netty, the responses to pipelined requests are sent with a single flush. The state of a connection (its request, buffers and writer) is reused per loop, so short-lived connections do not allocate it again.
• Backpressure. Writing never blocks the loop. Instead, a connection whose client does not read its responses (or websocket messages) stops being read until the client catches up, so the client is slowed down by TCP rather than filling the memory of the server.
• Buffered request bodies. The body of a request is received completely before your handler is invoked, as RequestBody.asStream() is a blocking InputStream that cannot wait for data on the loop that has to deliver it. Bodies are buffered up to 512 MB, larger ones are rejected with 413 Payload Too Large.
• Websockets on the loop. The websockets module reads upgraded connections by suspending as well, so websockets do not leave the loop either - their handlers must not block, just like request handlers.

Serving Asynchronous Payloads

The Netty engine achieves its performance by keeping a request on a single CPU core from start to finish. Suspending code continues to work: the HTTP client of the JDK (sendAsync(...).await()), coroutine based drivers and every other library that suspends instead of blocking can be used without modification. When such a library resumes your handler on another thread, the engine hands it back to the event loop of the connection. This is correct, but not free: under load, every hop reduces the very advantage the engine was chosen for.

A library that blocks - JDBC, for example - does not belong on the loop. Move its calls to a dispatcher made for blocking work, the handler continues on its loop as soon as they return:

import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext

import org.genhttp.modules.webservices.ResourceMethod

import javax.sql.DataSource

class Orders(private val dataSource: DataSource) {

    @ResourceMethod
    suspend fun count(): Int = withContext(Dispatchers.IO) {
        dataSource.connection.use { connection ->
            connection.createStatement().use { statement ->
                statement.executeQuery("SELECT COUNT(*) FROM orders").use { result ->
                    result.next()
                    result.getInt(1)
                }
            }
        }
    }

}

Tuning the Event Loops

The defaults - one event loop per CPU core on the native transport - work for most deployments. If you need finer control, Host.create(...) takes EngineOptions, grouped by the part of the engine each setting belongs to: the event loops (eventLoop) and the TCP transport with its TLS (tcp).

import org.genhttp.engine.netty.EngineOptions
import org.genhttp.engine.netty.EventLoopOptions
import org.genhttp.engine.netty.Host
import org.genhttp.engine.netty.TcpTransportOptions
import org.genhttp.engine.netty.Transport
import org.genhttp.modules.functional.Inline

import kotlin.time.Duration.Companion.seconds

suspend fun main() {
    val options = EngineOptions(
        eventLoop = EventLoopOptions(count = 16, transport = Transport.IoUring),
        tcp = TcpTransportOptions(idleTimeout = 120.seconds, sendTimeout = 30.seconds)
    )

    Host.create(options = options)
        .handler(Inline.create().get { "Hello World!" })
        .run()
}

• transport selects Epoll, IoUring or Nio. The default (Auto) uses epoll if it is available and NIO otherwise. io_uring needs a recent kernel, and its system calls may be blocked by the seccomp profile of a container runtime, in which case the server does not start.
• idleTimeout closes connections that neither sent nor received anything for this long, sendTimeout closes connections whose responses could not be sent for this long (a client that stops reading). Both are disabled by default: an idle websocket is a healthy one, and a large download to a slow client takes time. Set them if you serve no upgrades, or comfortably above the ping interval of your websockets.
• handshakeTimeout (10 seconds) bounds the TLS handshake, listenBacklog (1024) the queue of connections waiting to be accepted.

Host.create(...) also takes a hook that runs once per event loop, on the loop itself, before the server accepts connections - useful to set up services that live on a loop:

import org.genhttp.engine.netty.Host
import org.genhttp.modules.functional.Inline

suspend fun main() {
    Host.create(onEventLoopStart = { loop -> println("Started $loop") })
        .handler(Inline.create().get { "Hello World!" })
        .run()
}

TLS

TLS is configured through the bind(...) overload that takes a certificate provider, exactly as on the internal engine. It is terminated by OpenSSL (BoringSSL, shipped with the engine for Linux, macOS and Windows), with the SSLEngine of the JDK as the fallback where it cannot be loaded. The endpoint logs which of them it uses when the server starts.

import org.genhttp.api.infrastructure.CertificateProvider
import org.genhttp.engine.netty.Host
import org.genhttp.modules.functional.Inline

import java.security.KeyStore

suspend fun serve(certificate: KeyStore.PrivateKeyEntry) {
    Host.create()
        .handler(Inline.create().get { "Hello World!" })
        .bind(null, 8443, CertificateProvider.from(certificate))
        .run()
}

Providers and Cipher Suites

The TLS implementation and the offered cipher suites are part of the TCP options:

import org.genhttp.api.infrastructure.CertificateProvider
import org.genhttp.engine.netty.EngineOptions
import org.genhttp.engine.netty.Host
import org.genhttp.engine.netty.TcpTransportOptions
import org.genhttp.engine.netty.TlsProvider
import org.genhttp.modules.functional.Inline

import java.security.KeyStore

suspend fun serve(certificate: KeyStore.PrivateKeyEntry) {
    val options = EngineOptions(
        tcp = TcpTransportOptions(
            tlsProvider = TlsProvider.OpenSsl,
            cipherList = "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256"
        )
    )

    Host.create(options = options)
        .handler(Inline.create().get { "Hello World!" })
        .bind(null, 8443, CertificateProvider.from(certificate))
        .run()
}

• tlsProvider selects OpenSsl (the server does not start if it cannot be loaded), Jdk or Auto (the default: OpenSSL if available, the JDK otherwise).
• cipherList restricts the cipher suites of TLS 1.2, cipherSuites those of TLS 1.3. Both take a list separated by colons, in OpenSSL notation (ECDHE-RSA-AES128-GCM-SHA256) or in the notation of the JDK (TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256). A suite the provider does not support stops the server at startup. BoringSSL does not allow to restrict the suites of TLS 1.3, so cipherSuites only applies to the JDK provider - the server logs a warning if it is set for OpenSSL.

Server Name Indication

If you would like to serve several host names from one port, pass a CertificateProvider that selects the certificate by the name the client sends during the handshake, as described in Security. The provider is asked for every handshake, the TLS context built for a certificate is reused for every handshake answered with it.

Client Certificates

A CertificateValidator passed to bind(...) decides whether a client certificate is required and whether it is accepted, as on the internal engine. Any validator works - unlike ioxide, the engine does not need the trust anchors up front.

Certificate Rotation

A renewed certificate does not need a restart: as the provider is asked for every handshake, returning the renewed certificate is all it takes. New connections are answered with it, connections already established keep the certificate they authenticated with.

Limitations

• The engine speaks HTTP/1.0 and HTTP/1.1. HTTP/2 and HTTP/3, which ioxide serves, are not supported yet.
• Responses are not paced while they are written: the body of a large response is held in memory until the client has received it, as with ioxide. For large downloads to many slow clients, the internal engine needs less memory.
• Content of upgraded connections other than websockets (such as the websocket tunnel of the reverse proxy) is served on a virtual thread, as it reads the connection blocking.
• Files are served by the portable implementation of the files handler; there is no counterpart of ioxide’s native file serving yet.