How to Enable TCP Fast Open on Nginx

Do you want to learn how to enable TCP Fast Open on Nginx OS? Feel free to read through this guide so you’ll find out everything you need to know. 

Transmission Control Protocol (TCP) is one of the most widely used transport layer protocols. It works underneath the most popular application layer protocols, such as HTTP, SSH, FTP, and NFS

The job of the TCP is to create a dedicated and reliable connection between the client and server. It delivers reliable and in-order data/TCP segments to the application layer.

In this article, we will cover the steps you must undergo to enable TCP Fast Open (TFO) on Nginx. First, we’ll start with a brief overview of TCP and TFO, and then we will discuss its configuration.

The TCP Fast Open protocol (TFO)

TCP provides a three-way handshake protocol between sender and receiver, where the first sender sends an SYN packet. After which, the receiver acknowledges the SYN packet. Finally, the sender sends the acknowledgment. 

This three-way handshake is sensitive to the distance between sender and receiver. In other words, the round trip delay is very high. The TCP Fast Open protocol solves this problem.

TCP Fast Open is an extension of the classical TCP protocol. It allows the connection to start during the handshake. This is possible by allowing the data in the payload of the TCP SYN packet. 

You have to take note that TFO is only possible after a normal handshake is performed beforehand. Usually, you can do this when you’ve performed a handshake first and have it recognized based on the TFO cookie.

How to Enable TCP Fast Open on Nginx

We will now discuss how to enable TCP fast open on Nginx. These are three steps you need to do:

  • Update the kernel settings. This will enable support for TFO.
  • Compile the particular version of Nginx from the source with TFO support.
  • Modify Nginx configuration. This will allow accepting TFO connections.

Update the Kernel settings

Before compiling Nginx from the source, we will update a few kernel settings as follows.

Checking the version of the Linux kernel 

It is possible that you may have already done the process to enable TCP fast open on Nginx, as the client and server support for TFO is present from the Linux kernel mainline as of version 3.7. You can check which version you are running with the following command:

$uname -r

If you don’t have it enabled, you can follow the steps below.

Create tcp-fast-open.conf file

As a root user, create /etc/sysctl.d/tcp-fast-open.conf and open the file in your favorite editor, such as nano, as follows:

#nano /etc/sysctl.d/tcp-fast-open.conf

Now, enter the following content in this file:

net.ipv4.tcp_fastopen = 3

Restart sysctl

Restart sysctl using the following command:

# systemctl restart systemd-sysctl

Verify the settings

Finally, you can verify the settings with the help of the following command:

# cat /proc/sys/net/ipv4/tcp_fastopen

Compiling Nginx with TFO support

We will now compile Nginx with TFO support. Follow the steps below.

Download the Nginx source

Download the Nginx source using the following command:

$sudo yum install wget –y

Extract the archive as follows:

$tar -xvf Nginx-1.9.7.tar.gz

Switch the directory as follows:

$cd Nginx-1.9.7

Install the Fedora EPL repository

Install the Fedora EPL repository using the following command:

$sudo yum install -y epel-release

Then install the prerequisite packages as follows:

$sudo yum install -y gcc zlib-devel libatomic_ops-devel pcre-devel openssl-devel libxml2-devel libxslt-devel gd-devel GeoIP-devel gperftools-devel

Configure the build

Configure the build with the following with -DTCP_FASTOPEN=23. Type the following command on your Terminal:

$ ./configure \ --prefix=/usr/share/Nginx \ --conf-path=/usr/share/Nginx/etc/Nginx/Nginx.conf \ --error-log-path=/usr/share/Nginx/var/log/Nginx/error.log \ --http-log-path=/usr/share/Nginx/var/log/Nginx/access.log \ --http-client-body-temp-path=/usr/share/Nginx/var/lib/Nginx/tmp/client_body \ --http-proxy-temp-path=/usr/share/Nginx/var/lib/Nginx/tmp/proxy \ --http-fastcgi-temp-path=/usr/share/Nginx/var/lib/Nginx/tmp/fastcgi \ --http-uwsgi-temp-path=/usr/share/Nginx/var/lib/Nginx/tmp/uwsgi \ --http-scgi-temp-path=/usr/share/Nginx/var/lib/Nginx/tmp/scgi \ --user=Nginx \ --group=Nginx \ --build="TFO custom build" \ --with-threads \ --with-file-aio \ --with-ipv6 \ \ --with-http_ssl_module \ --with-http_v2_module \ \ --with-http_realip_module \ --with-http_addition_module \ --with-http_xslt_module \ --with-http_image_filter_module \ --with-http_geoip_module \ --with-http_sub_module \ --with-http_dav_module \ --with-http_flv_module \ --with-http_mp4_module \ --with-http_gunzip_module \ --with-http_gzip_static_module \ --with-http_auth_request_module \ --with-http_random_index_module \ --with-http_secure_link_module \ --with-http_degradation_module \ --with-http_stub_status_module \ \ --with-mail \ --with-mail_ssl_module \ --with-stream \ --with-stream_ssl_module \ --with-google_perftools_module \ \ --with-pcre \ --with-pcre-jit \ --with-google_perftools_module \ --with-debug \ --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -DTCP_FASTOPEN=23' \ --with-ld-opt='-Wl,-z,relro -Wl,-E'

Compile Nginx

Finally, compile the Nginx using the following command:

$make -j4

Verifying the build

After the compilation, you can verify the build using the following command:

Verifying the build

Install Nginx

Now, we will install the Nginx in the prefix base directory using the following command:

$sudo make install

Create a Nginx group/user and a temporary directory

Now create the Nginx group as follows:

$ sudo groupadd -r Nginx

Add the user to the group as follows:

$ sudo useradd -r -d /usr/share/Nginx/var/lib/Nginx -g Nginx -s /sbin/nologin -c "Nginx web server" Nginx

Create a temporary directory as follows:

$ sudo mkdir -p /usr/share/Nginx/var/lib/Nginx/tmp

Change the owner of this directory to the created user as follows:

$ sudo chown -R Nginx.wheel /usr/share/Nginx/var/{log,lib}/Nginx

Nginx configuration for TFO

The final step is the Nginx configuration for TFO. Using your favorite editor, open the file /usr/share/Nginx/etc/Nginx/Nginx.conf for editing as follows:

$nano /usr/share/Nginx/etc/Nginx/Nginx.conf

Modify the listen directive as follows:

listen 80 fastopen=256

You can also specify other settings, such as:

listen rcvbuf=64000 sndbuf=200000 backlog=2048 fastopen=500;
listen rcvbuf=64000 sndbuf=200000 backlog=2048 ssl http2 fastopen=500;

Now you can check the Nginx with the following command:

$Nginx –t

This article demonstrates how you can configure and enable TCP Fast Open on Nginx. What you need to do is update the kernel settings, compile the Nginx from the source, and configure Nginx for TFO, respectively. We have discussed all these steps in detail, and we hope you learned something new from reading this article.

If this guide helped you, please share it. 

Related Posts