Data transmitted over the internet doesn’t just travel freely—it moves through four structured layers, each playing a crucial role in ensuring efficient and reliable communication. You’ve likely come across terms like HTTP, TCP, IP, and TLS, or heard discussions about latency and bandwidth. But what do these concepts really mean, and how do they work together?

If some of these terms feel unclear, don’t worry, you’re not alone. This article will break them down in a way that makes sense, giving you a clear, complete picture of how data flows across the internet.

In the 1970s, the U.S. Department of Defense developed a networking model known as TCP/IP to describe how data moves across networks. Initially designed for ARPANET, a pioneering research project in computer networking, this model laid the foundation for what would later evolve into the modern internet.

In the TCP/IP model, transmitted data is broken down into smaller units called packets. Each packet contains several bytes of data, along with additional header information that helps with routing and delivery. This approach helps reduce network congestion and allows multiple transmissions to different destinations to occur simultaneously. Additionally, if an error occurs—and errors do happen—only the affected packet(s) need to be resent, rather than retransmitting the entire dataset. This makes data transfer more efficient and reliable, ensuring smooth communication across networks.

Each packet, then, goes through a 4-layer procedure during the sending process and the receiving process:

1. Application Layer: Defines the data (HTTP & TLS).
2. Transport Layer: Manages data delivery (TCP/UDP).
3. Internet Layer: Handles routing and addressing (IP).
4. Network Access: Prepares data for physical transmission using electrical signals (for copper wires), radio waves (for Wi-Fi), or light pulses (for fiber optics).

Imagine a browser communicating with a web application. The request body is split into multiple packets, and each packet is encrypted using the TLS protocol for security. Next, the transport layer assigns each packet a sequence number and adds the destination port (port 443 for secure HTTPS communication). The internet layer then attaches the destination IP address, and the network layer transmits the packets through physical devices like routers and switches.

On the receiving end, the packets are received through the network layer. The internet layer directs them to the correct server using the destination IP address. The transport layer then reassembles and arranges the packets before passing the data to the application layer, where it is decrypted and processed as an HTTP request.

As a web developer, you might feel that the network layer is too low-level to worry about—after all, countless networking professionals work around the clock to keep the internet running smoothly worldwide. And you wouldn’t be wrong. However, understanding how data transmission works is crucial because it directly impacts your web application's performance.

The speed at which data travels, determining whether your application feels fast or slow, is influenced by two key factors:

1. Latency: The time it takes for a packet to travel from the sender to the receiver.
2. Bandwidth: The maximum amount of data that can be transmitted over a network connection in a given time.

Latency is dictated by the physical distance and the medium through which the signal travels, while bandwidth refers to the maximum amount of data that can be transmitted at once, determined by the network’s capacity and infrastructure.

Imagine a user in Dubai browsing your website, while your server is located in London. The data travels through fiber optic cables, which offer high-speed transmission but still introduce some delay due to the physical distance between the two locations. Your server provider (e.g., AWS, DigitalOcean, Akamai) limits the inbound and outbound bandwidth to 10Gbps. This means that even if there is more demand, your server cannot exceed this data transfer rate. If multiple users are accessing your site simultaneously, the available bandwidth must be shared, which can slow down data transfer for each user.

To optimize network performance, you need to focus on two key factors:

1. Host your application closer to the end user.
2. Ensure your server has adequate bandwidth limits.

For the first point, you may have heard of edge computing, where applications are replicated across multiple servers in different geographical locations. This significantly reduces latency, as data doesn’t have to travel as far. However, deploying and managing edge computing solutions can be complex and expensive.

For static websites, where the same content is served to all users, this process is much simpler. Content Delivery Networks (CDNs) like Cloudflare can cache your website’s responses and store them on multiple servers worldwide. When a user requests data, the CDN delivers it from the nearest server, reducing latency and improving load times.

However, for dynamic websites, data is typically retrieved from multiple sources and assembled at the application layer before being transmitted over the network.

Take, for example, a web application that fetches dynamic content from a database. If you want to replicate this application across multiple geographically distributed servers, you'll also need to replicate the database while keeping the data in each database server in sync. Otherwise, if a server close to the user has to communicate with a distant database, the added latency could negate the performance benefits of your multi-server setup.

Every device participating in network communication is assigned a unique IP address (Internet Protocol address). This includes your website visitor's computer as well as your server.

Inside this layer, each packet is prepared with a source and destination IP address. The source highlights the sender's address and the destination highlights where the packet should be delivered to.

Once a packet has been addressed, the Internet Layer must determine the path to deliver it to its destination. It does that by consulting a local routing table.

Imagine that user in Dubai browsing your web application hosted in london. The Internet Layer assigns an address to each packet and checks the local routing table, which determines that the destination IP is not local (i.e., it belongs to a different network). The user's router then forwards the packet to their Internet Service Provider (ISP).

As the packet travels across the internet, it moves from router to router, with each router consulting its routing table to determine the next best hop toward the destination. This process continues until the packet reaches your server’s local network in London.

Once there, the final router examines the destination IP address, recognizes it as belonging to your server, and forwards the packet accordingly. The server then receives the packet, confirms the destination IP matches its own, and processes the request by moving it through the next layer (the transport layer).

In addition to addressing and routing packets, the Internet Layer is also responsible for fragmenting packets when necessary. This process occurs when one part of the network has a smaller Maximum Transmission Unit (MTU) than the size of the packet being sent. The MTU represents the largest packet size that a network segment can handle. If a packet exceeds this limit, the network will return a message to the source device, requesting that the packet be fragmented into smaller pieces.

During fragmentation, the Internet Layer breaks the original packet into smaller fragments, each with its own header. Each fragment is given a unique identification number to ensure that they can be correctly reassembled later. These fragments are then sent individually through the network, with each one traveling independently to the destination.

When the fragments arrive at the destination, the Internet Layer on that side collects the fragments, reads their identification numbers, and reassembles them into the original packet. Once the fragments are put back together, the Internet Layer passes the complete packet to the next layer for further processing. This ensures that large packets can successfully traverse networks with varying MTU sizes, preventing issues that might otherwise arise due to packet size limitations.

As a web developer, while you typically don’t have direct control over the Internet Layer, there's one aspect you can control, which is the Maximum Transmission Unit (MTU) of your server's network interface.

For example, AWS EC2 instances typically have an MTU of 9001 bytes. This larger MTU helps to reduce the number of packets needed to transmit the same amount of data within their Virtual Private Cloud (VPC) network. With fewer packets to send, the network can be more efficient and potentially experience less overhead.

However, there's a catch: The standard MTU for Ethernet networks (the most common type of network) is typically 1500 bytes. If you're running an application inside a VPC with an MTU of 9001 bytes and sending packets to the internet, these packets will likely exceed the 1500-byte limit of the external networks and will often need to be fragmented.

Fragmentation leads to additional overhead on both the network devices and your server, as each fragment needs to be processed and reassembled. This can also introduce extra latency, reduce network throughput, and potentially impact the performance of your application.

Optimizing MTU is a useful strategy to ensure smooth data transmission, reduce fragmentation, and ultimately improve the overall network performance of your web application.

In web applications, every bit of request and response data is important. A single missing bit can render the payload corrupt and unreadable. But since data is broken down into packets in the TCP/IP model, how do we ensure that all packets arrive accurately, in the correct order, and without loss?

This is where Transmission Control Protocol (TCP) comes in. TCP is an abstraction layer designed to ensure reliable data transmission over an inherently unreliable network. The process begins with an exchange between the client and server during connection establishment, known as the three-way handshake. This handshake serves a crucial purpose: allowing both parties to establish initial sequence numbers for the bytes of data they send, which helps maintain proper packet ordering and detect any packet loss.

Here's an example of this interaction:

$CLIENT: SYN, SEQ=1000
$SERVER: SYN-ACK, SEQ=2000, ACK=1001
$CLIENT: ACK, SEQ=1001, ACK=2001

1. The client initiates the connection by sending a SYN message in the first packet, including its initial sequence number (1000).
2. Upon receiving the packet, the server generates its own initial sequence number and responds with a SYN-ACK message. This message includes the server's sequence number ( 2000) and acknowledges the next expected sequence number from the client (1001).
3. The client then receives the SYN-ACK packet and completes the handshake by sending an ACK message. This acknowledgment confirms the sequence number expected by the server (1001) and also indicates the next expected sequence number for the server’s packets (2001).

After this initial handshake, each time a party sends data, it increments its own sequence number by the number of bytes sent. The receiver acknowledges received packets by sending an ACK number, which is the next expected sequence number.

$CLIENT: SEQ=1001 // 500 bytes sent
$SERVER: ACK=1501 // 500 bytes received
$SERVER: SEQ=2001 // 300 bytes sent
$CLIENT: ACK=2301 // 300 bytes received

When a packet is lost, the sending party can detect that by reading the receiving party's ACK messages and retransmit the lost sequence:

$CLIENT: SEQ=1001 // 1st packet: 500 bytes sent
$CLIENT: SEQ=1501 // 2nd packet: 500 bytes sent
$SERVER: ACK=1501 // 500 bytes received
$SERVER: ACK=1501 // duplicate...
$SERVER: ACK=1501 // duplicate...
$CLIENT: SEQ=1501 // re-transmit the second packet
$SERVER: ACK=2001 // received

When a packet arrives out of order, the receiving party keeps the out-of-order packet in an internal buffer and keeps sending duplicate ACKs until the sending party retransmits the missing one:

$CLIENT: SEQ=1001 // 1st packet: 500 bytes sent
$CLIENT: SEQ=1501 // 2nd packet: 500 bytes sent
$CLIENT: SEQ=2001 // 3rd packet: 500 bytes sent
$SERVER: ACK=1501 // 500 bytes received, SEQ=1001
$SERVER: ACK=1501 // 500 bytes received, SEQ=2001
$SERVER: ACK=1501 // duplicate...
$CLIENT: SEQ=1501 // re-transmit the second packet
$SERVER: ACK=2501 // received

In this example, the client kept SEQ=2001 in the buffer until SEQ=1501 was received. Then, it reassembles the received packets in the correct order before passing them to the next network layer.

Through the SEQ-ACK interactions, both clients and servers ensure reliable data transfer across the network by maintaining proper sequencing, detecting lost packets, and requesting retransmissions when necessary. This mechanism forms the backbone of TCP’s reliability.

When a new TCP connection begins, the sender has no knowledge of the available bandwidth or network conditions. If it immediately transmits a large number of packets, the network might become congested, leading to packet loss and retransmissions.

To prevent this, TCP starts slowly and increases the transmission rate exponentially until it finds the network’s capacity:

// Initial handshake
$CLIENT: SYN, SEQ=1000
$SERVER: SYN-ACK, SEQ=2000, ACK=1001
$CLIENT: ACK, SEQ=1001, ACK=2001

// Data transmission
$CLIENT: 10 packets
$SERVER: 10 ACKs    // ok
$CLIENT: 20 packets
$SERVER: 20 ACKs    // ok
$CLIENT: 40 packets
$SERVER: 35 ACKs    // packet loss
$CLIENT: 20 packets // rate reduction
$SERVER: 20 ACKs    // ok
$CLIENT: 21 packets
$SERVER: 21 ACKs    // ok

If packet loss is detected, as shown in the example above, TCP interprets it as a sign of network congestion and responds by reducing the transmission rate. Instead of rapidly increasing it as before, TCP adopts a more conservative approach, gradually raising the rate in a linear fashion rather than exponentially. This adaptive process continues throughout the lifetime of the connection, allowing TCP to dynamically adjust the transmission rate based on changing network conditions, ensuring both efficiency and stability.

As you may have noticed, the initial three-way handshake introduces latency, making the establishment of new TCP connections relatively expensive. Before any actual data transmission can begin, multiple packets must be exchanged, adding delay and computational overhead to the process. For applications that frequently initiate new connections, this can significantly impact performance.

Additionally, as explained in the Slow-Start section, each new connection begins by sending a limited number of packets, gradually increasing the transmission rate over time. This means that an already-established connection can transmit data much faster than a newly initiated one, as it has already ramped up its sending capacity.

Because of these factors, connection reuse becomes a critical optimization for any application running over TCP. Techniques like persistent connections allow multiple requests to be sent over a single TCP connection, reducing handshake delays, minimizing overhead, and improving overall efficiency.

This does not only apply to browser-to-server or server-to-api communication. It also applies to TCP connections with databases and cache instances. As a web developer looking to optimize your application's performance, TCP connection re-use can help a lot.

This layer is the closest to the web applications you develop. It's where HTTP requests are handled, TLS encryption is setup, and DNS lookups are performed.

Let's start with DNS lookups:

DNS is yet another network protocol, much like IP and TCP, designed to translate human-readable hostnames (e.g., example.com) into IP addresses (e.g., 93.84.36.128). This translation is essential because computers communicate using IP addresses, while humans find domain names easier to remember.

Before a client can send a request to a server, it must first know the server's IP address. The Internet Layer requires this IP address for proper routing and delivery of packets. To achieve this, it performs a DNS query to resolve the domain name to an IP address. This query happens before the actual request is sent. The DNS query follows a hierarchical system, hitting multiple servers until it receives a valid response. The response can come from one of three sources:

1. Local Cache: The client's operating system or browser may have a locally cached IP address from a previous lookup, allowing instant resolution.
2. Intermediate DNS Server: ISPs or public DNS providers (like Google DNS or Cloudflare DNS) often cache DNS records to reduce latency and network load.
3. Authoritative DNS Server: If the record isn't found in any cache, the query continues up the hierarchy to an authoritative server responsible for that domain. These servers hold the definitive records.

When you purchase a domain, your domain provider may offer a built-in DNS management service, using its own authoritative DNS servers to handle queries for your domain. Alternatively, you can choose a third-party service like Cloudflare to manage your DNS. In this case, you'll need to update the nameservers in your domain provider's dashboard to point to Cloudflare's authoritative DNS servers. By doing so, you're essentially informing any intermediate DNS server that the definitive answers can be found on Cloudflare's DNS servers.

DNS lookups are sent using UDP by default. While UDP is faster than TCP due to its lack of an initial handshake, it offers fewer guarantees when it comes to reliable delivery. To compensate for this, the DNS protocol includes built-in mechanisms to ensure reliable communication or to inform the client to switch to TCP if reliability cannot be guaranteed. This switch may also happen if the query or response is too large to fit within the typical 512-byte UDP limit, as TCP can handle larger packet sizes more efficiently.

Whether using TCP or UDP, DNS lookups are an overhead that adds latency to requests. While you have little influence over this aspect when it comes to users querying your web application, you have some degree of control over your servers communicating with other servers. This can be summarized into:

1. Configuring local DNS caches to reduce the number of DNS queries leaving your machine and reaching the slower DNS servers over the network.
2. Utilize connection pooling to re-use pre-established TCP connections in which the DNS lookup has already happened.
3. Use private IP addresses for communicating with resources inside your private network (databases, cache instances, and other servers).

As mentioned earlier, data transferred over TCP is broken down into packets that travel from hop to hop until they reach their destination. Along this journey, a malicious server could potentially listen in, record sensitive data, or even alter it. While this wasn’t a major concern in the early days of the internet, the rise of eCommerce made secure data transmission essential. The need to encrypt data in transit became critical to protect sensitive information like credit card details and personal data.

In response to this growing need, Netscape, once the most popular web browser company, developed a protocol called SSL (Secure Sockets Layer) in the mid-1990s. This protocol aimed to secure internet communication by encrypting data, ensuring that even if packets were intercepted, the information would remain unreadable.

In 1999, encrypted internet traffic was standardized with the introduction of the TLS protocol, which evolved from SSL. TLS is now the backbone of secure communication on the web, ensuring that data transmitted between clients and servers remains private and trustworthy. It achieves this by providing three critical functions:

1. Data encryption.
2. Authenticity check.
3. Integrity check.

TLS uses encryption algorithms to scramble data, making it unreadable to anyone who might intercept it. This ensures that sensitive information (passwords, credit card details, ...) cannot be viewed by unauthorized parties. Even if a malicious actor captures the data, it appears as meaningless gibberish without the decryption key.

TLS also verifies the identity of the parties involved in the communication using digital certificates issued by trusted Certificate Authorities (CAs). This prevents impersonation attacks, where a malicious actor could pretend to be a legitimate server.

In addition to encrypting and authenticating data, TLS ensures that the information is not altered or tampered with during transmission. This is achieved through message authentication codes (MACs) or hashing algorithms, which detect any changes made to the data. If even a single bit is altered, the recipient is alerted, and the data is rejected.

To perform the three checks, an additional handshake is performed between the server and the client right after the TCP handshake. It goes like this:

$CLIENT: HELLO, VERSIONS=1.2,1.3, CIPHER_SUITES=X,Y,Z, RAND=XXXX
$SERVER: HELLO, VERSION=1.3, CIPHER_SUITE=X, RAND=XXXX, CERTIFICATE=XXX
$CLIENT: CERTIFICATE=VALID, SECRET=YYY
$SERVER: FINISHED
$CLIENT: FINISHED

• The client initiates the TLS handshake by sending a ClientHello message. This message includes a list of supported TLS protocol versions and encryption algorithms (cipher suites), along with a random 32-byte value (Client Random) to contribute to key generation.
• The server responds with a ServerHello message, selecting the TLS version and cipher suite it prefers. It also sends its own random 32-byte value (Server Random) and its digital certificate, which contains its public key.
• The client verifies the server’s certificate to ensure it is:
  1. Valid (not expired or revoked).
  2. Matches the server’s domain name.
  3. Signed by a trusted Certificate Authority (CA).
• Once verified, the client generates a Secret, a critical value that will be used to derive the final symmetric encryption key. This Secret is encrypted using the server's public key and sent back to the server.
• The server decrypts the Secret using its private key. Both the client and server then combine the Secret with the Client Random and Server Random to derive the final Master Secret, which is used for encrypting all subsequent data.
• Finally, the server sends a "Finished" message, confirming that it is ready for secure communication. The client also sends its own "Finished" message. Once both sides acknowledge receipt, the encrypted session begins.

The purpose of the random numbers generated by the server and the client is to ensure that each TLS session is unique and secure. If an attacker captures old handshake messages and tries to replay them, the random values will be different for each connection, making the replay useless.

A certificate authority is an entity trusted by the client to validate the identity of websites. Every operating system and most browsers ship with a list of well-known certificate authorities. They allow clients to verify the signature of a given certificate by comparing it against this trusted list.

One of the most popular and free certificate authorities is Let's Encrypt. It allows you to issue a certificate valid for three months by verifying that you own a specific domain. Once ownership is confirmed, you can install the certificate on your server, confident that all major browsers and operating systems will recognize and trust it.

With the TCP connection and a secure channel in place, the HTTP (Hypertext Transfer Protocol) protocol takes over. This protocol defines how data is formatted and transmitted over the internet.

In the simplest form, an HTTP request, or response, is composed of three parts:

1. Request line
2. Headers
3. Body

As a web developer, you're likely already familiar with these three components. With that foundation in place, let's take a brief journey through the evolution of the HTTP protocol, exploring versions 1 and 2:

HTTP/1 was a relatively simple protocol that relied on a single open TCP connection to transfer HTTP messages over the internet. However, each connection could handle only one message at a time (either sending a request to the server or receiving a response from it). This sequential nature often led to inefficiencies, such as head-of-line blocking, where a single slow request could delay all subsequent requests on the same connection. For a browser, this meant that in order to speed things up, it has to open multiple connections to the same server from the same client to download resources in parallel.

HTTP/2, on the other hand, enables multiplexing. That is allowing multiple requests and responses to travel through the same TCP connection simultaneously. The way it does that is by breaking a message into smaller units called frames. Each frame is tagged with a stream identifier, indicating which request or response it belongs to. This enables the interleaving of multiple streams without blocking one another:

TCP Connection
┌─────────────────────────────────────────────────┐
│ Stream 1: ->[Frame n]---[Frame 2]----[Frame 1]  │
│           <-[Frame 1]---[Frame 2]----[Frame n]  │
|─────────────────────────────────────────────────|
│ Stream 2: ->[Frame n]---[Frame 2]----[Frame 1]  │
│           <-[Frame 1]---[Frame 2]----[Frame n]  │
└─────────────────────────────────────────────────┘

The figure above shows two streams actively using a single TCP connection. If we look at it from the connection's point of view, we see this:

[Frame 1.1] [Frame 2.1] [Frame 1.2] [Frame 2.2] [Frame 1.n] [Frame 2.n]

By breaking messages into smaller frames, HTTP/2 allows multiple streams to share the same TCP connection concurrently. If frames from one stream are delayed (for example, due to slow request processing) frames from other streams can continue to flow uninterrupted. This ensures that one slow request doesn't block the entire connection, improving overall efficiency and reducing latency.

The frames of each stream are binary encoded to optimize performance in multiple ways:

1. Smaller Size in Transit: Binary encoding significantly reduces the size of the data being transmitted.
2. Lower Computational Overhead: Parsing plain text is slower and more error-prone than processing binary data.

Smaller frame sizes result in smaller TCP packets, which offer several performance benefits. By reducing the size of each packet, bandwidth usage is minimized, making it easier for networks with limited capacity or high congestion to handle traffic efficiently. Smaller packets also enable faster transmission since they require less time to travel over the network and experience shorter queuing delays, ultimately lowering overall latency.

Additionally, smaller packets improve error recovery; if a packet is lost or corrupted, only a small portion of the data needs to be retransmitted, which is far more efficient than resending larger packets. This combination of reduced bandwidth usage, faster transmission, and better error handling contributes to more reliable and responsive network performance.

The evolution of the internet since its early days is truly remarkable. Advances in both physical mediums, from copper cables to fiber optics, and software protocols have significantly contributed to a faster, more reliable, and more efficient internet.

While the combination of HTTP/2, TLS, TCP, and IP represents a significant advancement over previous decades, evolution hasn't stopped there. In 2022, the HTTP/3 protocol was officially published, introducing several key benefits through substantial changes to how data is transmitted over the internet.

HTTP/3 is built on top of the QUIC protocol, which uses UDP instead of TCP. This eliminates the need for TCP’s slow, sequential handshake process, resulting in faster connection establishment. Additionally, HTTP/3 improves reliability by handling packet loss more efficiently, allowing unaffected streams to continue transmitting without waiting for lost packets to be retransmitted. Finally, HTTP/3 streamlines the process by combining the encryption handshake (TLS) and transport handshake into a single step, significantly speeding up the establishment of new connections.

Although HTTP/3 is not yet widely supported, it's essential for us as web developers to understand how it works and implement it whenever possible, preparing for the day it becomes the standard. I’ll cover this in more detail in a future article.