Application setup and administration

Linux setup instructions

Setup from scratch

Install minimum tools

apt-get install vim
apt-get install tofrodos   # turn the newlines of shell scripts into proper format
apt-get install ntp        # hosting machine's system clock should be synchronized with NTP servers.

# Activate ntpd
systemctl enable ntpd
systemctl start  ntpd

Create the Linux application user

sudo adduser gcuser
sudo passwd  gcuser

# as user gcuser
echo 'PATH="$PATH:/etc/init.d"' >> ~/.profile

Prepare app dir

sudo mkdir               /opt/go-countdown
sudo mkdir               /opt/go-countdown/certs
sudo chmod 755           /opt/go-countdown
sudo chown gcuser:gcuser /opt/go-countdown

Now copy the app files

The below sections Golang installation and Download and compile this application can be executed either on your work machine, or on the server.

Whichever you chose, copy the application files from ~/go/src/github.com/zew/go-countdown to your application directory:

cp go-countdown   /opt/go-countdown
cp app-bucket     /opt/go-countdown
cp static         /opt/go-countdown
cp templates      /opt/go-countdown

cd ./app-bucket
mv config-example.json    config.json
mv logins-example.json    logins.json

Edit and adapt config.json and logins.json

See config.go for settings explanation

cd ..

# without lets encrypt
cp server.key     /opt/go-countdown
cp server.pem     /opt/go-countdown

sudo chown -R gcuser:gcuser /opt/go-countdown/*
sudo chmod -R 644   /opt/go-countdown/*
sudo chmod -R 755   /opt/go-countdown/go-countdown # make it executable
sudo chmod -R 755   /opt/go-countdown/app-bucket
sudo chmod -R 755   /opt/go-countdown/static
sudo chmod -R 755   /opt/go-countdown/templates
sudo chmod -R 755   /opt/go-countdown/certs

Enable ports 80 and 443 for non-root

Needs redo after each compilation.
Possibly not needed for systemd CapabilityBoundingSet... settings.

sudo setcap cap_net_bind_service=+eip /opt/go-countdown/go-countdown
# 'e', 'i', and 'p' flags specify the (e)ffective, (i)nheritable and (p)ermitted sets.

Prepare log file

sudo touch /var/log/go-countdown.log
sudo chown gcuser:gcuser /var/log/go-countdown.log

# reset log
truncate   --size=0  /var/log/go-countdown.log

Prepare pid file

sudo mkdir /var/run/go-countdown
sudo chown gcuser:gcuser /var/run/go-countdown/

sudo touch               /var/run/go-countdown/prog.pid
sudo chown gcuser:gcuser /var/run/go-countdown/prog.pid
sudo rm                  /var/run/go-countdown/prog.pid

Make script reboot-hard - and accessible to all

Put the script go-countdownctl to /etc/init.d

go-countdownctl is a start-stop script. The source is in the same directory as this file.

sudo mv ./go-countdownctl  /etc/init.d/go-countdownctl
sudo chmod 755 /etc/init.d/go-countdownctl
fromdos /etc/init.d/go-countdownctl   # remove windows newlines

chkconfig

Under debian, we do not need chkconfig - just put the script to init.d

chkconfig: 2345 85 15
description:
        2,3,4,5      runlevel
        85           starting.
        15           stopping.
[root@host ~]# chkconfig --add go-countdownctl
[root@host ~]# chkconfig --list | grep -i bspc

Manage app with systemd

Put systemd config file to sudo vim /etc/systemd/system/go-countdown.service.

Make it boot hard with sudo systemctl enable go-countdown.service

Combining syslog with standard log file, stackoverflow.
But it is not working. We have to contend looking into the syslog.

# not working - skip to next section
# create a file in /etc/rsyslog.d/go-countdown.conf with the following content:
if $programname == 'go-countdown' then /var/log/go-countdown.log
& stop
# make it writeable to syslog
sudo chown syslog:gquser /var/log/go-countdown.log

This is working:
Looking into the syslog.

sudo journalctl -b               # since reboot
sudo journalctl --since -10m
sudo journalctl --since "2020-04-08 13:46:00"  --until "2020-12-08 13:46:00"
sudo journalctl -u go-countdown.service --since "2020-04-08 13:46:00"
sudo journalctl -u go-countdown.service --since today
sudo journalctl -u go-countdown.service --since -2m

sudo systemctl daemon-reload

sudo systemctl restart  go-countdown.service
sudo systemctl start    go-countdown.service
sudo systemctl status   go-countdown.service

sudo systemctl enable   go-countdown.service

# put together
sudo systemctl daemon-reload && sudo systemctl restart  go-countdown.service
sudo journalctl -u go-countdown.service --since -2m

Manage app with service commands

go-countdownctl status
go-countdownctl stop
go-countdownctl start

Manage app manually

cd /opt/go-countdown/
./go-countdown > /var/log/go-countdown.log 2>&1 &
tail /var/log/go-countdown.log
ps aux | grep go-countdown
pkill go-countdown

Denial of service consideration

• A request memory blocks too frequent requests on application level.
  Each IP address has up to ten requests per rolling window of one minute.

• Nevertheless, the hosting machine should be behind a firewall
  preventing denial-of-service attacks on network level.

Golang installation

• Install and setup golang

• By default your golang installation will end up here:

/usr/local/go/bin/go  # under Linux
c:\Go                 # under Windows
# Otherwise set $GOROOT to your different path

• By default your source files are assumed to be here:

%USERPROFILE%\go
~\Go
# Otherwise set $GOPATH to your source file directory

• Your compiled go programs end up here

%USERPROFILE%\go\bin
~\go\bin
# To have them always available:
export PATH=$PATH:~/go/bin

• For details, refer the golang install docs

Download and compile this application

• Source code is hosted at https://github.com/zew/go-countdown.

• Thus the source code should go to

# mkdir ...
cd ~/go/src/github.com/zew
cd ..
git clone https://github.com/zew/go-countdown
cd go-countdown

# Fetch all required libraries with  
go get ./...

# Compile the application
go build

Compiling from another machine

• You may cross compile under windows for linux using crosscomp.bat.

• Copy the new executable to yourhost.com using sftp.
  The new copy should retain execution privileges.

Minimal directories

• Copy directories static/... and templates/...

• If you want to run https,
  then put your server.key and server.pem files into the app dir.

Create a configuration file

• Whenever you start the application,
  a file config-example.json is created.

• Derive your settings and save it as config.json
  into the app dir.

• Each new executable needs to be configured again
  to allow to use ports 80 and 443.
  See section Enable ports 80 and 443 .

Config reload during runtime

• Login as admin and call /config-reload

Changes to HTML templates during runtime

• Login as admin and call /templates-reload

Manage files and users

Adding an application user

• Whenever you start the application,
  a file logins-example.json is created.

• A new user must be entered into logins.json.

  • user => Username, all lowercase, a-z, 0-9

  • pass_initial => Set an one-time password

  • is_init_password => Set to true

  • email => Optional email. So far unused.

  • Optional roles

    "roles": {  
    "admin": "yes"  
    },
    

  • Login as admin call /logins-reload

  • Or restart application

  • You may call /logins-save
    to fill empty pass_initial fields with autogenerated passwords.

Adding files to download

• Linux user for the application is gcuser

• Connect via sftp as gcuser@yourhost.com
  cd /opt/go-countdown/app-bucket/download

• Copy files into any of the user directories.

• Directory public is available for everyone without login.

• You can create subdirectories inside the user dirs as you like.

• Any directory of following format is treated as a time-protected directory
  It’s files names can be viewed, but it’s file contents are only available,
  after the indicated date-time has passed
  yyyy-mm-dd_hhmm # Notice the hyphens vs. the underscore

• You may test your directory structure with the ‘public’ user.
  However omit the confidential file contents.

Getting a trusted SSL certificate

Consider the golang acme stuff for integration with letsEncrypt. Acme is even making provisions for automatic cert renewal.

Some Golang SSL Info

Activation of https via config setting "tls": true,

Let’s Encrypt

Add this to the config

    "lets_encrypt": true,
    "host_name": "fmtdownload.zew.de",

Create your own certificate

Generate private key for algorithm “RSA” ≥ 2048-bit

openssl genrsa -out server.key 2048

Key considerations for algorithm “ECDSA” ≥ secp384r1

List ECDSA supported curves: openssl ecparam -list_curves

openssl ecparam -genkey -name secp384r1 -out server.key

Generation of self-signed (x509) public key based on the private key. PEM-encodings .pem|.crt

openssl req -new -x509 -sha256 -key server.key -out server.pem -days 3650

pem is a Privacy Enhanced Mail Certificate file

Checking the modulus

openssl x509 -noout -modulus -in server.pem
openssl rsa -check -noout -modulus -in server.key

Use apache to run multiple instances under port 80

We must allow apache to use the network in order to proxy requests

sestatus -b | grep httpd_can
setsebool -P httpd_can_network_connect=1

Put the app behind an apache virtual host.

Edit httpd.conf:

# cache nothing ever
# serverfault.com/questions/4729/
<Location / >
   ExpiresActive On
   ExpiresDefault "now"
</Location>

# default virtual VirtualHost
<VirtualHost *:80>
    DocumentRoot "C:/xampp/htdocs"
</VirtualHost>

# enable mod_proxy_html.so
<VirtualHost *:80>
    ServerName go-countdown.myorg.net
    ProxyPreserveHost On
    ProxyPass        "/"   "http://127.0.0.1:8080/"
    ProxyPassReverse "/"   "http://127.0.0.1:8080/"
</VirtualHost>

another example with multiple virtual hosts
and multiple instances of go-countdown

# cache nothing ever
# serverfault.com/questions/4729/
<Location / >
   ExpiresActive On
   ExpiresDefault "now"
</Location>

<VirtualHost *:80>
    ServerName some-other.myorg.net
    DocumentRoot "/var/www/some-other.myorg.net"
    <Directory /var/www/some-other.myorg.net/>
        Options Indexes FollowSymLinks MultiViews
        AllowOverride None
        Order allow,deny
        allow from all
    </Directory>
</VirtualHost>


# enable mod_proxy_html.so
<VirtualHost *:80>
    ServerName go-countdown.myorg.net
    # doc root is ignored
    DocumentRoot "/var/www/go-countdown.myorg.net"
    <Directory /var/www/exceldb.myorg.net/>
        Options Indexes FollowSymLinks MultiViews
        AllowOverride None
        Order allow,deny
        allow from all
    </Directory>
    ProxyPreserveHost On
    ProxyPass        "/app1"   "http://127.0.0.1:8080/app1"
    ProxyPass        "/app2"   "http://127.0.0.1:8081/app2"
    ProxyPassReverse "/app1"   "http://127.0.0.1:8080/app1"
    ProxyPassReverse "/app2"   "http://127.0.0.1:8081/app2"
</VirtualHost>

Back to documentation root

Back to app

Rendered by russross/blackfriday