Patterns in application-layer protocol design

There are a lot of different application-layer protocols, and it seems they all end up evolving solutions to many of the same problems. This article seeks to identify the aspects of functionality commonly found in many application-layer protocol, and serves as a dictionary of those functions.

Authentication. Most commonly to authenticate one side of the association, such as the client.

Channel security. A protocol may provide channel security (that is, confidentiality and integrity), most commonly by delegation to a lower-layer protocol such as TLS, DTLS or QUIC.

Framing. Where an application-layer protocol desires to exchange frames but runs on a byte-oriented lower-layer protocol (bytepipe), it can be adapted to provide a frame interface (framepipe) via a framing protocol. At its simplest, such a mechanism may consist of a simple length prefix before each frame's payload.

Frame typing. Where an application layer exchanges frames with multiple kinds of meaning, it needs a way to distinguish them. This is often facilitated via some prefixed type information at the start of each frame. Sometimes this functionality is tightly coupled together with framing functionality, for example in the TLV construction, where a type field precedes a length field; by using the LTV ordering, logically constructed as LV(TV), this coupling can be avoided.

Extensibility framework. Application-layer protocols frequently find the need to evolve a highly generalised framework for future extensibility. Most commonly this is facilitated via further use of LTVs within a frame, possibly nested in turn. Note that though terms such as TLV usually imply a binary format, we consider things such as HTTP headers to be effectively LTVs in kind for our purposes here. LTVs may have a binary or textual representation.

Version negotiation. As the application protocol may receive new major versions, a mechanism for version negotiation and upgrade is almost always desired. This must facilitate both forwards and backwards compatibility. Most simply this may be in the form of a simple version integer.

Capability negotiation. If the requirements incumbent on a protocol implementation only ever increase to a superset of the previous set, a simple version negotiation scheme suffices. Where the number of implementers of a protocol is large, or the feature set supported by the protocol is substantial so that different implementers may implement different subsets, more sophisticated feature negotiation may be necessary in the form of capability negotiation, in which one or all of the parties to an association may negotiate zero or more optional and zero or more required features known as capabilities. The initiator of an association might declare its capabilities with the responder listing the subset of those capabilities it supports; or it might refuse an association if it sees a capability it does not support; or the responder may declare its capabilities first, with the initiator choosing its supported capabilities from the list given; or the two may interact with one another in an interactive process, enumerating one another's capabilities until the process of capability negotiation is complete. These capabilities have identifiers; see type identifiers for discussion on identifier design.

Implementation identifiers. Sometimes application layer protocols will have implementations offer a freeform string identifying the implementation being used. These do not affect the behaviour of the protocol and do not provide intelligible information to machines. Occasionally, where incompatibilities in protocol implementations have occurred due to bugs, or underspecification of the protocol, known implementation identifiers have been used to vary processing to work around such an incompatibility. This is not considered a good practice and should only be engaged in where absolutely necessary. It is also only possible if the protocol uses an implementation identifier.

Messaging paradigms. The bulk of the meaning of the application is in its handling of messages, though terminology may differ from protocol to protocol. Messages are frequently linked or related to other messages or used with one another in sequence. The following patterns of messaging are commonly found:

Request framework. Various patterns aid in the implementation of the set of messaging paradigms an application-layer protocol chooses to support. Requests will generally have some kind of type identifier.

Keepalives. An application layer protocol might implement application-layer keepalives. Although this functionality can often be delegated to lower-layer protocols such as TCP which support optional keepalive functionality, ensuring that this functionality is implemented at the application layer ensures that the application layer code is functioning correctly to respond to keepalive messages. (See for example the IRC PING command.)

Multiplexing. An application layer protocol might support the multiplexing of multiple logical message streams on a single lower-layer connection. What a message or stream refers to is application-specific, but the pattern is common. Such multiplexing functionality might be integrated directly into a framing mechanism discussed above, or might be separate without tight coupling to framing mechanisms.

Transport agnosticism. An application-layer protocol might be transport agnostic. For example, it might be defined to operate over either TCP, TCP with TLS, or SCTP. Frequently this will involve the definition of the core of the application-layer protocol in a transport-agnostic way, and then supplementation of this specification with a transport-layer binding specification. Since some transports, such as SCTP, provide framing, framing facilities might not be needed for some transport bindings; therefore framing facilities might be specified as part of a transport binding and not the core protocol.

Routing: service identification. An application-layer protocol may also require the submission of a service identifier before providing service, to allow multiple services offered over the same transport (e.g. TCP port) to be disambiguated. Examples include the HTTP Host header.

Routing: resource identifiers. Within a given application, the subrouting of requests is often necessary; thus application-layer protocols may often include identifiers or addresses of specific network entities, resources or objects. Object-oriented protocols will require all objects to bear identifiers which can be used to refer to them; such identifiers might be a standard field common to all requests. Examples include URLs in HTTP.

Routing: application-layer networking. An application-layer protocol may allow requests to include internal routing information which allows the addressing of other protocol machines addressable via a given peer, possibly on different machines; this essentially resembles the headers of a Layer 3 networking protocol such as IP. For example, an application-layer protocol for an MMORPG might enable requests to be directed to an internal realm server, internal player information server or internal chat server based on addressing information in a request. Other examples include SMTP or XMPP, the purposes of which are to allow messaging beyond the scope of the parties to the transport-layer association. Some scheme of addressing must be created to address the entities which can be reached.

Routing: load balancing. An application-layer protocol may allow requests received by a role to be delegated to one of several third parties for the purposes of load balancing. The request is tracked to ensure that any responses are returned to the original submitter of the request. Various load balancing algorithms may be used to choose a third party from the set available.

Roles. An application-layer protocol will typically define different roles for different peers.

Quit message. An application-layer protocol might support a quit message, to be responded to with a response merely confirming the quit message, or confirming it and terminating the transport-layer association. The advantage of this over simply tearing down the transport-layer association is that it ensures all previous requests have been handled by application-layer code. Since the completion of a request is made obvious by delivery of its response, this is only of significant use for protocols making substantial use of requests which do not receive responses. Transport-layer delivery guarantees such as TCP ACKs cannot be used for confirmation of handling as these only indicate data has been delivered to an OS kernel's TCP buffer and not to an application.

Application-layer flow control. An application-layer protocol may include facilities for flow control beyond those of the lower-layer transport protocol. For example, it may support the rate limiting of specific commands, specific categories of commands, or all commands.

Who sends first? Some application-layer protocols have the initiator of a lower-level transport association sending first, with others having the responder send first. Protocols where the responder sends first, such as for example SSH, force the responder to reveal what protocol it implements first. By contrast, protocols where the initiator sends first (e.g. HTTP) are more flexible, and there is the potential to multiplex multiple application-layer protocols on the same lower-level transport endpoint (e.g. TCP port number).

Speculative push. This is a rare function only really known for adoption in HTTP/2.0; a response is issued to a synthetic request which was never actually issued by the peer, in the expectation that the peer may soon need the response. The peer handles the response under the premise it issued the synthetic request for it.

Control/data plane split. Application-layer protocols may sometimes split their control and data planes. They may, for example, use multiple lower-level transport layer associations for a single application-layer association, with one such lower-level association being used for control, and the others for data. Examples include FTP and, when combined with SIP or H.323, RTP.

Envelope/payload split. Application-layer protocols may split requests into envelope and payload data, and relay the payload blindly as opaque data (application-layer networking) while using the envelope data for routing and other purposes. For example SMTP, which treats envelope addresses and email bodies separately. Even though those bodies themselves will usually contain headers repeating the envelope addresses, they are treated separately.

Referrals. Application-layer protocols may incorporate the ability to respond to requests with referrals to other services better able to process the request; for example, HTTP redirects.

Verbs. Some application-protocols may employ a reduced or limited set of request types intended to be used generically on a large number of resources identified via resource identifiers which are of varied type, where each kind of resource has its own implementation of these standard verbs. A given resource may or may not implement any given verb, but each verb will have some set of universal semantics which must be met by all resources which implement it. This requires use of a single resource identifier namespace shared by all resource types. Examples include HTTP and SNMP.

Enumeration. Some application-layer protocols may provide facilities for enumerating the resources or objects which can be addressed via a peer. For example, FTP provides directory listings and LDAP allows enumeration of directory objects. HTTP is notably deficient here, neglecting to provide directory listings even where its predecessor, Gopher, does so.

Publish/subscribe. Some application-layer protocols may provide facilities for peers to subscribe to notifications of some given description, and subsequently transmit any notifications of that description to that peer. A peer can cancel its previous subscription using an unsubscribe request. Note that this is a higher-level pattern which can be built on top of aforementioned messaging paradigms.

Application-layer retransmission. Application-layer protocols may enable some messages, or higher-level constructs built on top of the protocol's messages, to be made reliable by requiring acknowledgement of their delivery and retransmitting messages which were transmitted over a previous association but not acknowledged. This is a higher level construct not to be confused with reliability of an underlying transport-layer protocol. Examples include SMTP, which provides for reliable email transmission.

Transactions. The notion of atomic transactions, which apply totally or not at all, and possibly which can be rolled back is often built on top of an application-layer protocol. This is a higher-level pattern which can be built on top of aforementioned messaging paradigms.

Note on layering. The fact that all these patterns are listed together does not imply that they can not be decomposed into layered subprotocols, and in fact an analysis deducing such layering should be made. For example, an application-layer protocol which provides framing can consider this to be the lowest layer of itself, with other subprotocols of the application-layer protocol layered on top of it. Some functionality discussed herein, such as publish/subscribe, transactions or reliable queueing, is particularly high level and can be viewed as sitting above, for example, a general request/response/notification messaging layer, itself sitting on a framing layer. The full analysis of an application-layer protocol's subprotocols is inevitably specific to the protocol, so it is not discussed further here.