TCP Message Framing: Why One recv Call Is Not One Message

Treating the result of one recv() call as one complete message can break a TCP server even when the connection is working correctly. TCP is a byte stream without message boundaries: the application must define framing and repeatedly read the required header and payload lengths. This article uses a four-byte length prefix and a Python example to handle short reads, consecutive messages, and connection closure.

Concept illustration of separating a continuous byte stream into application messages. It is not a diagram of actual packets or a protocol layout.

send and recv call counts do not match

Calling send(A) and send(B) separately does not guarantee that the receiver reads exactly A and then B in two calls. It may first read only part of A, or read bytes belonging to multiple messages at once. Linux tcp(7) explicitly states that TCP does not preserve record boundaries.

The 4096 in recv(4096) is the maximum number of bytes for that call. It does not request one application message or require waiting until 4096 bytes accumulate. As recv(2) explains, ordinary receive operations can return currently available data up to the requested size.

Do not use packet counts or delays between calls as message boundaries. A custom TCP protocol exchanging JSON requests and PCM data with a TTS server follows the same principle.

Example frame: a four-byte length followed by the payload

This example first sends an unsigned 32-bit big-endian integer containing the payload byte length, followed by the payload. The length excludes the header. Do not confuse UTF-8 byte length with character count.

In Python's struct, !I specifies a four-byte unsigned integer in network byte order. Avoid implicitly making native byte order or padding part of a network protocol. Official struct documentation

The example caps each payload at 1 MiB. It checks the limit immediately after reading the header, before continuing to receive an oversized payload. This cap does not limit total memory across concurrent connections or processing time.

A runnable sender and receiver

Save the following code as framing_demo.py. It creates an IPv4 loopback TCP connection and sends two consecutive frames without external packages. OneByteReads is a test adapter limiting each receive call to at most one byte. It does not claim that TCP creates one-byte packets.

A successful run prints:

The sender uses Python's sendall() to continue until the supplied data has been sent or an error occurs. This is not an acknowledgment that the receiving application processed the message. Design an application-level response if one is required. Official socket documentation

recv_frame() gathers the four-byte header, decodes the length, and gathers that many payload bytes. Each call requests only the remaining required size, so it does not consume bytes from the next frame as part of the current payload.

Distinguish an empty payload from end of stream

b"": the valid payload of a zero-length frame.

None: this example's convention for an orderly connection close before a new header begins, meaning no more frames are available.

EOFError: the connection ended after part of a header or payload was received.

ValueError: the declared payload exceeds this example's size limit.

For this example, closing the connection is a safe default after a length error or a truncated frame. Do not skip arbitrary bytes hoping to find the next message without a separate resynchronization rule. Even Linux's MSG_WAITALL can return a short result after a disconnect or error, so return values still need checking. recv(2)

Validation scope and production considerations

Seven checks passed over local TCP connections on Python 3.12.14 / Linux x86_64: one-byte reads, two consecutive frames, an empty payload, clean EOF between frames, a truncated header, a truncated payload, and an oversized length header. The two-line output from the runnable example was also confirmed. These results do not measure packet segmentation or throughput across the public internet.

A production server also needs a deadline for completing an entire frame, concurrent connection and accumulated-buffer limits, authentication and TLS, protocol versioning, and error handling. The example's two-second socket timeout alone does not guarantee completion of a whole frame within two seconds. Serialize entire frame writes when multiple senders share a connection.

TCP_NODELAY does not restore message boundaries. Treat Nagle configuration and application framing as separate concerns. An existing Korean-language article covers the former:

TCP_NODELAY는 항상 켜는 게 좋을까?