A journey static assets web server: Part I
The main idea build this stuffs it’s just for want to learn how Caching works on Nginx because I was read bunch of great articles that’s inspired me like Why Netflix Chose NGINX as the Heart of Its CDN and Serving 100 Gbps from an Open Connect Appliance then suddenly my curious thoughts told me to browse deep about that and I was found cool The 5-hour CDN and this repository .
Okay, because currently only has 1 server and my idea is so naive. I just usually thinking treat it as a nginx –> webserver that server static server like below.
Beside this one for fun learning purpose usage, I was wondering what’s I would use for web server of static files ? Do i need used nginx or apache again and treat them as container (like I did usually ?) and yeah the answer is NO!. I need to start find something Awesome web server that can serve static files the uncommon ways.
First it’s like I just want to create it with golang then serve and done, but after that I found article The smallest Docker image to serve static websites and mention about thttpd then let’s try this one!
Enter thttpd
Lets build the container first, prepare the Dockerfile like below, TBH i copycat-ing from this repository .
# For complete version please see:
# https://github.com/emsixteeen/thttpd-container/blob/master/Dockerfile
FROM alpine:3.15 AS builder
ARG THTTPD_VERSION=2.29
# Install all dependencies required for compiling thttpd
RUN apk add gcc musl-dev make
# Download thttpd sources
RUN wget http://www.acme.com/software/thttpd/thttpd-${THTTPD_VERSION}.tar.gz \
&& tar xzf thttpd-${THTTPD_VERSION}.tar.gz \
&& mv /thttpd-${THTTPD_VERSION} /thttpd
# Compile thttpd to a static binary which we can copy around
...
# Switch to the scratch image
...
STOPSIGNAL SIGTERM
# Run thttpd
CMD ["/thttpd", "-D", "-h", "0.0.0.0", "-p", "3000", "-d", "/home/static", "-u", "static", "-l", "-", "-M", "384"]
After build it, I got nice sized container image /lgtm && /approve
...
cedeen_webz latest cf5902e2ddb7 10 months ago 188kB
...
Just usual ways to run it and mount the volume that contain your images/static files, here is my example:
datz/
├── images
│ ├── 1.webp
│ ├── ... .webp
│ └── ... .webp
├── index.html
└── robots.txt # IYKYK 🙄
docker run -p 3000:3000 -v $HOME/datz:/home/static ...
Nginx cached mode
In this part it was interesting, probably it’s time to play with caching so let’s dive into it with spawn your vhost conf file and here is my example.
# Tell nginx to create cache data storage with key_zone name "cdn"
# with max cache size 50M
proxy_cache_path /tmp/nginx-cdn levels=1:2 keys_zone=cdn:10m max_size=50M inactive=60m use_temp_path=off;
server {
...
# Show header cache status
add_header X-Cache-Status $upstream_cache_status;
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always;
...
# Regex: all file with extension .webp/jpg/jpeg in all path
location ~* ^.+\.(png|jpe?g)$ {
# Tell this location to use key_zone
proxy_cache cdn;
# Enables revalidation of expired cache
proxy_cache_revalidate on;
proxy_cache_min_uses 3;
proxy_cache_background_update on;
proxy_cache_lock on;
# valid for this http status code with x minutes time
proxy_cache_valid 200 301 302 10m;
proxy_cache_valid 404 1m;
# disable this header for cached
proxy_ignore_headers Cache-Control;
proxy_cache_use_stale error timeout http_500 http_502 http_503 http_504;
}
}
For better visbility any other module please refer Module ngx_http_proxy_module .
Testing
For testing purpose, try request it with curl and make sure to print the header section.
$ curl -LIs https://localhost:3000/images/static-site-ideas.webp
....
content-security-policy: frame-ancestors 'self'
x-dotwell: 384kbps
x-frame-options: SAMEORIGIN
x-cache-status: HIT # Got hitted
x-xss-protection: 1; mode=block
x-content-type-options: nosniff
referrer-policy: strict-origin
cache-control: public, max-age=31536000
...
PS: I try around 3-4 request and got x-cache-status: HIT this triggered by module proxy_cache_min_uses 3;.
Closing thought
- Probably I just want to create a simple CDN (Content Delivery Network) but I thought this will not become purely delivery distributed only for fun and learning purposes.
- There is another plan to use platform/tools like Minio for better access list and GUI for uploading the assets. But yeah I only have smol server :)).
- Need to rework how best practices or Unix Way to handle uploading the assets (currently I just use
scp). - If you are dealing with dynamic assets, read carefully how cache revalidate works because Caching is hard .