Nginx Extras and Ubuntu 20.04

It’s been a while since my last post, so I haven’t written anything on my blog. So here we go. Just want to share about issues on my VPS that were around 2 month before.

Suddenly I checked my junk mail and there was a bunch of inbox from Uptime Robot and notified that my blog and some dummy services is down. Yea, around 1 or 2 month before it’s busy day and much work task, so just don’t care about what happen on my VPS 🤣 (read: i hope my service can do autoheal like the blues one).

Next few days then, at the night i had spare times and dealing “Okay let’s check this machine” and doing some Ops job, like check nginx.service, /nginx/error.log and etc. Alright nothing makes me curious just do systemctl restart nginx.service and boom it’s up again then i was typing exit to my ssh.

But it did not solve the problem in the next few hours, it kept spamming my uptime check and I remembered there was quite interesting logs that I found.

Journalctl logs

[4440232.039884] systemd[1]: Starting Journal Service...
[4440232.113943] systemd[1]: Started Journal Service.
[4440232.127044] systemd-journald[234404]: Received client request to flush runtime journal.
[4468549.245241] nginx[86509]: segfault at 10 ip 00007ff08b28c593 sp 00007fffd37c82c0 error 4 in libperl.so.5.30.0[7ff08b224000+166000]
[4468549.245248] Code: 48 89 43 10 48 83 c4 18 5b 5d 41 5c 41 5d 41 5e 41 5f c3 0f 1f 40 00 0f b6 7f 30 48 c1 e8 03 48 29 f8 48 89 c3 74 89 48 8b 02 <4c> 8b 68 10 4d 85 ed 0f 84 28 01 00 00 0f b6 40 30 49 c1 ed 03 49

nginx/error.log

2021/07/22 13:59:46 [info] 247166#247166: Using 32768KiB of shared memory for nchan in /etc/nginx/nginx.conf:75
2021/07/22 13:59:49 [notice] 247171#247171: signal process started
2021/07/22 13:59:52 [notice] 247174#247174: signal process started
2021/07/22 13:59:55 [notice] 247177#247177: signal process started
2021/07/22 13:59:55 [error] 247177#247177: open() "/run/nginx.pid" failed (2: No such file or directory)

Phew, much byte number. Then I lookup the error logs (both of them) to search engine like:

The first i notice it’s issue with buggy Perl v5.30.0 that’s shiping default with Ubuntu 20.04 then make it crash with nchan module.

  Built under linux
  Compiled at Oct 19 2020 10:56:54
  @INC:
    /etc/perl
    /usr/local/lib/x86_64-linux-gnu/perl/5.30.0
    /usr/local/share/perl/5.30.0
    /usr/lib/x86_64-linux-gnu/perl5/5.30
    /usr/share/perl5
    /usr/lib/x86_64-linux-gnu/perl/5.30
    /usr/share/perl/5.30
    /usr/local/lib/site_perl
    /usr/lib/x86_64-linux-gnu/perl-base

But, what is nchan ? Alright i just browse that nchan handling Pub/Sub communication, websocket, and related that run top of Nginx.

Secondly, I noticed when dealing with my local Vagrant then played with some Ubuntu boxes, and check that package nginx-extras come with default nchan installed.

vagrant@vagrant:~$ apt show nginx-extras | grep 'nchan'

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Depends: libnginx-mod-http-auth-pam (= 1.18.0-0ubuntu1.2), libnginx-mod-http-cache-purge (= 1.18.0-0ubuntu1.2), libnginx-mod-http-dav-ext (= 1.18.0-0ubuntu1.2), libnginx-mod-http-echo (= 1.18.0-0ubuntu1.2), libnginx-mod-http-fancyindex (= 1.18.0-0ubuntu1.2), libnginx-mod-http-geoip (= 1.18.0-0ubuntu1.2), libnginx-mod-http-geoip2 (= 1.18.0-0ubuntu1.2), libnginx-mod-http-headers-more-filter (= 1.18.0-0ubuntu1.2), libnginx-mod-http-image-filter (= 1.18.0-0ubuntu1.2), libnginx-mod-http-lua (= 1.18.0-0ubuntu1.2), libnginx-mod-http-perl (= 1.18.0-0ubuntu1.2), libnginx-mod-http-subs-filter (= 1.18.0-0ubuntu1.2), libnginx-mod-http-uploadprogress (= 1.18.0-0ubuntu1.2), libnginx-mod-http-upstream-fair (= 1.18.0-0ubuntu1.2), libnginx-mod-http-xslt-filter (= 1.18.0-0ubuntu1.2), libnginx-mod-mail (= 1.18.0-0ubuntu1.2), libnginx-mod-nchan (= 1.18.0-0ubuntu1.2), libnginx-mod-stream (= 1.18.0-0ubuntu1.2), nginx-common (= 1.18.0-0ubuntu1.2), iproute2, libc6 (>= 2.28), libcrypt1 (>= 1:4.1.0), libpcre3, libssl1.1 (>= 1.1.1), zlib1g (>= 1:1.1.4)

Summary

Thanks.