NGINX Accelerated including Reverse Proxy, Redis, CDN, Lets Encrypt and much more!

This is a Docker image creates a high performance, optimized image for NGINX. Deliver sites and applications with performance, reliability, security, and scale. This NGINX server offers advanced performance, web and mobile acceleration, security controls, application monitoring, and management.

Features

The image includes configuration enhancements for;

• Reverse Proxy
• SEO optimizations
• Customizable configurations
• SSL with support for Lets Encrypt SSL certificates
• Mime-type based caching
• Redis LRU cache
• Fastcgi cache
• Proxy cache
• tmpfs file cache
• Brotli and Gzip compression
• Redirects for moved content
• Security & Bot Protection
• Monitoring processes, ports, permissions... with Monit
• Standardized UID/GID and Permissions (www-data)
• Support GeoIP
• Rate limited connections to slow down attackers
• CDN support
• Cache purge
• Pair with high performance PHP-FPM container for blazing fast Wordpress installs

There are many, many other benefits to this system. Give it a try!

Getting Started

Install

The first step is to build or pull the image:

Build

docker build --build-arg "NGINX_VERSION=1.15.5" -t openbridge/nginx .

Replace NGINX_VERSION=1.17.0 with your preferred version. You can also simply pull the images. See below.

Pull

docker pull openbridge/nginx:latest

You can also use a different version of NGINX simply by pulling a build with the NGINX version you want. For example;

docker pull openbridge/nginx:latest
docker pull openbridge/nginx:1.15.4
docker pull openbridge/nginx:1.15.3

To see the available versions vist https://hub.docker.com/r/openbridge/nginx/tags/

Running

Via Docker compose

docker-compose up -d

Using a named yml file:

docker-compose -f ./compose/html.yml up -d --remove-orphans

There are a sample HTML compose file at ./compose/html.yml as well as PHP one ./compose/php.yml to get you started.

Understanding Configurations

Please note that the config files inside the /conf/* directory are opinionated. They are the working examples of a specific implementation of preferences and needs.

We provided two pre-built configurations. The first one is for html based sites within /conf/html/* and another one is for php sites within /conf/php/*. The application will look for a config directory to use that. This is done with the NGINX_CONFIG ENV variable. For example, if you are running a html-based site and want to use the /conf/html/* configuration, then set NGINX_CONFIG=html. If you are running php and want to use the /conf/php/* configuration, then set NGINX_CONFIG=php. If you have a custom config set like /conf/<my-custom-config>/* then set NGINX_CONFIG=my-custom-config.

The nginx default setup located inside /conf/basic/*. Basic allows you to run nginx in a bare metal setup. Just set NGINX_CONFIG=basic

Digging Into The ENV File

The following lines are the core variables for the ENV config inside /conf

• NGINX_DOCROOT sets the default www directory. If you do not set this the images defaults to /usr/share/nginx/html
• NGINX_SERVER_NAME sets the default server name in nginx.conf. If you do not set this it will default to localhost.

NOTE: NGINX_SERVER_NAME is the address of your server. Hence, use the localhost if you are doing local development or using openbridge.com or apple.com or mydomainname.com. Also, you should set this to the root domain. For example, use acme.com vs www.acme.com. This will keep your Nginx server_name directive clean. If you don't understand how NGINX uses that, read their docs.

• NGINX_CONFIG sets the default configuration director for your image. See the /conf directory to review a html and php configuration.
• NGINX_PROXY_UPSTREAM sets the upstream server(s) for the reverse proxy to connect with. Since the proxy is local to the container, you should use something like localhost.com:8080. If this is NOT set, it will default to localhost:8080
• REDIS_UPSTREAM sets the upstream Redis cache server(s) to connect with. If you are using compose, you might reference the redis container server redis01:6379;server redis02:6378;. You might also set it by IP server 1.2.3.4:6379; server 4.3.2.1:6379;. If this is NOT set, it will default to server localhost:6379;.

If you are using PHP, you will want to set the endpoint for PHP-FPM:

• PHP_FPM_UPSTREAM sets the upstream server(s) to connect with. If you are using compose you might reference the php-fpm01 container server php-fpm01:9000;server php-fpm01:9001;. You might also set it by IP server 1.2.3.4:9000; server 4.3.2.1:9001;. If this is NOT set, it will default to server localhost:9000;

You can set a collection of dummy files and certs for local testing:

• NGINX_DEV_INSTALL Set to true if you want self-signed SSL certs installed and "hello world" HTML and PHP pages installed. This is useful for testing.

NOTE: Self-signed SSL certificates are always installed if the system does not detect them within the default cert location: /etc/letsencrypt/live/${NGINX_SERVER_NAME}/

Check our the /env for more examples

Mounting Your Web App or Site Content

Following is the convention we will be using for sites.

• /etc/nginx/sites-available/ – nginx configuration for different sites will be available
• /usr/share/nginx/html – Your root site content/apps
• /usr/share/nginx/html/example.com – (Optional) Domain specific content/apps

To mount your web app or html files, you will need to mount the volume on the host that contains your files. Make sure you are setting the NGINX_DOCROOT in your run or docker-compose.yml file. If you do not set it the default is /usr/share/nginx/html

-v /your/webapp/path:{{NGINX_DOCROOT}}:ro

You can also set the cache directory to leverage in-memory cache like tmpfs:

-v /tmpfs:{{CACHE_PREFIX}}:ro

You can do the same to config the files, if you want to use versions of what we have provided. Just make sure you are mapping locations correctly as NGINX and PHP expect files to be in certain locations.

NGINX /conf/ Configuration File Organization

The following represents the structure of the configs used in this image. Please note the use of the nginx map settings for browser content caching and supporting content redirects. The content cache is done according to mime type.

NGINX Configuration Files:

• /etc/nginx/ – all nginx related configuration will be in this folder
• /etc/nginx/bots.d/* – Bot and spam configuration file
• /etc/nginx/conf.d/* – Core configuration files like PHP, security, SEO...
• /etc/nginx/fastcgi.d/* – The base fastcgi configuration
• /etc/nginx/geo.d/* – Configuration files for IP allow/deny
• /etc/nginx/header.d/* – The httpd, proxy and fastcgi headers
• /etc/nginx/map.d/* – Configuration files for caching, access rules, redirects....
• /etc/nginx/redis.d/* – Configuration files for Redis caching
• /etc/nginx/site-available/* – The vhost configurations
• /etc/nginx/status.d/* – Configuration to allow access to nginx server stats
• /etc/nginx/upstream.d/* – The upstream server(s)
• /etc/nginx/nginx.conf – The parent nginx configuration file

Here is a general layout for the configs though this may change over time:

/etc/nginx/
├── nginx.conf
├── conf.d/
│   ├── gzip.conf
│   ├── ssl.conf
│   ├── secure.conf
│   ├── location.conf
│   ├── seo.conf
│   └── nocache.conf
├── map.d
│   ├── cache/
│   │   ├── expires.map
│   └── redirects/
│       ├── example.map
│       ├── pages.map
├── bot.d
├── fastcgi.d
├── header.d
├── geo.d
├── redis.d
├── status.d
├── upstream.d
├── sites-available

.nginx.conf Configuration

This is base contents of the configuration file (see actual nginx.conf for current config).

conf.d and map.d Configuration Files

It is common practice to partition config files into a conf.d directory. This is no different. However, what is different is the use of the ngx_http_map_module for configuration. There are a few different map use cases represented. The use of map is preferred over if conditional statements and follows Nginx best practices.

Example HTML configuration conf/html/conf.d/location.conf

Below is an example of an advanced configuration that is designed to support single page applications (Angular, React...) based on a set of rules for bots and crawlers. If you are not using a prerender service, comment out those details for serving basic html:

location / {
   try_files                   $uri /$uri;
   aio                         threads;
   include                     /etc/nginx/redis.d/cache.conf;
}

Access (map.d/access/*)

Using map we set access policies based on address

0 $binary_remote_addr;
1 "";

Browser Cache (map.d/cache/*)

Using map we set the browser cache expires settings by mime-type. For example, here are some mime mappings with the expires policy:

image/gif max;
image/jpeg max;
image/png max;
image/svg+xml max;

Headers (map.d/header/*)

Use map to set conditional headers based on sets of rules which is useful for SEO, schemes and other purposes

Logs (map.d/logs/*)

Sets a conditional so we are not logging internal traffic. For example, we don't want Monit health check requests showing up in the logs.

No Cache (map.d/nocache/*)

Sets conditions for resources we do not want to have cached. For example, /wp-admin/ for a Wordpress blog.

Cache Purge (map.d/purge/*)

Supports requests for conditional cache purges using ngx_cache_purge.

Redirects (map.d/purge/*)

The second is using map for redirects for pages that are missing or have moved. For example, here is a map for moving the blog location:

/blog/ https://blog.openbridge.com;

Referrers (map.d/referrer/*)

Set conditional behaviors based on referrer, including detecting bots.

srcache (map.d/srcache/*)

The ngx_srcache is used for Redis LRU cache. It provides <i>transparent subrequest-based caching layout for arbitrary nginx locations</i>.

PHP-FPM

These are the primary php-fpm configurations in /conf/php/nginx:

/conf.d/location.conf
/fastcgi.d/fastcgi.conf

Please note that PHP caching is active and using TCP (vs sockets)

Check out the high performance docker PHP service we published here: openbridge/ob_php-fpm

Server locations are define via upstream:

upstream php-fpm {include /etc/nginx/upstream.d/php-fpm.conf;}

While you can pair this with any php-fpm service, it is recommended that you use this highly optimized PHP-FPM Docker Service. Most of our docker-compose files will reference this.

Redis Cache

The image is setup to use Redis as a reverse proxy LRU cache. The current cache settings reflect a balance of performance, optimization and striving for efficiencies with little resource consumption as possible. There are three main configs for Redis:

/redis.d/location.conf
/redis.d/cache.conf
/upstream.d/redis.conf

What is a Redis LRU cache?

When an http client requests a web page Nginx looks for the corresponding cached object in Redis. If the object is in redis, nginx serves it. If the object is not in Redis, Nginx requests a backend that generates the page and gives it back to Nginx. Then, Nginx put it in Redis and serves it. All cached objects in Redis have a configurable TTL with a default of 15s.

SSL

To keep things organized we default to using letsencrypt for SSL certificates/keys, paths and naming conventions. Your are free to use something other than letsencrypt, just follow the path and naming conventions set forth below. In keeping with the letsencrypt conventions make sure your certs are using the same naming scheme:

/etc/letsencrypt/live/${NGINX_SERVER_NAME}/;
├── server
│   ├── cert.pem
│   ├── chain.pem
│   ├── fullchain.pem
│   └── privkey.pem

The default locations for the SSL certs are in /conf.d/ssl.conf:

ssl_certificate /etc/letsencrypt/live/{{NGINX_SERVER_NAME}}/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/{{NGINX_SERVER_NAME}}/privkey.pem;
ssl_trusted_certificate /etc/letsencrypt/live/{{NGINX_SERVER_NAME}}/chain.pem;

Mount Your SSL Certs

You mount your certs directory on the host to the certs: /etc/letsencrypt/live/${NGINX_SERVER_NAME}.

-v /your/certs/path:/etc/letsencrypt/live/{{NGINX_SERVER_NAME}}:ro

In the event you are not using letsencrypt, mount your local SSL files in the same manner:

- /path/to/ssl/www.domain.com/fullchain.pem:/etc/letsencrypt/live/www.domain.com/fullchain.pem
- /path/to/ssl/www.domain.com/privkey.pem:/etc/letsencrypt/live/www.domain.com/privkey.pem
- /path/to/ssl/www.domain.com/chain.pem:/etc/letsencrypt/live/www.domain.com/chain.pem

The following is an example docker compose file that shows how to mount SSL certs from the host into your container with the correct pathing:

version: '3.1'
services:
  nginx:
    image: openbridge/nginx:latest
    container_name: nginx
    depends_on:
      - redis
    ports:
      - 80:80
      - 443:443
    tty: true
    restart: always
    tmpfs: /var/cache
    volumes:
      - /path/user/html:/usr/share/nginx/html
      - /etc/letsencrypt/live/www.domain.com/fullchain.pem:/etc/letsencrypt/live/www.domain.com/fullchain.pem
      - /etc/letsencrypt/live/www.domain.com/privkey.pem:/etc/letsencrypt/live/www.domain.com/privkey.pem
      - /etc/letsencrypt/live/www.domain.com/chain.pem:/etc/letsencrypt/live/www.domain.com/chain.pem
    ulimits:
      nproc: 65535
      nofile:
          soft: 49999
          hard: 99999
    env_file:
        - ./env/prod.env
  redis:
    image: redis:alpine
    container_name: redis
    restart: always
  volumes:
    site:

Installing certbot for letsencrypt SSL certs on your host

On your host, not in the Docker image, install certbot:

• Download certbot: curl -O https://dl.eff.org/certbot-auto
• Set permissions: chmod +x certbot-auto
• Move the executable: mv certbot-auto /usr/local/bin/certbot-auto
• Generate your certificate: /usr/local/bin/certbot-auto certonly -n --debug --agree-tos --email bob@gmail.com --standalone -d *.openbridge.com

If your run into an errors with certbot, trying running these commands:

rm -rf ~/.local/share/letsencrypt
rm -rf /opt/eff.org/*
pip install -U certbot
#try this
certbot certonly -n --debug --agree-tos --pre-hook="docker stop nginx" --post-hook="docker start nginx" --standalone -d {{yourdomain.com}} > /dev/null
# or this
certbot renew --debug

Certbot seems to be sensitive to OS and python updates and removing these files has helped clear up issues in the past.

You will need to setup a renewal process. The docs say check twice a day for changes. Lets add the renewal process to cron:

cat << EOF > /tmp/crontab.conf
55 4,16 * * * /opt/eff.org/certbot/venv/local/bin/pip install --upgrade certbot
59 4,16 * * * /usr/local/bin/certbot certonly -n --debug --agree-tos --pre-hook="docker stop nginx" --post-hook="docker start nginx" --standalone -d *.openbridge.com