Tested in Ubuntu 16.04 desktop Gnome+Unity in Vmware.
OBS: pure server will automatically start X in reboot and will lack of something, e.g. Default Terminal etc.

FIREWALL

TCP port 5900+N (N is monitorId, starts from 1.)

TIGHT.VNC AS SERVER

TightVnc allows us to start from a server without X11 and do everything in command line, remotely.

install (either already with or w/o desktop)

sudo su: use root can avoid weird problems, see "ps".

sudo apt-get update && \
sudo apt-get install -y --no-install-recommends ubuntu-desktop gnome-terminal nautilus tightvncserver autocutsel vim && \
sudo apt-get install -y gnome-session-flashback gnome-panel gnome-settings-daemon && \
mkdir ~/.vnc/; mkdir -p ~/.config/nautilus

Note: possible extra parameter: --no-install-recommends.
Then run: vncserver :1 (// to kill: vncserver -kill :1)
For grey screen problem, modify ~/.vnc/xstartup to:

# Uncomment the following two lines for normal desktop:
# unset SESSION_MANAGER
# exec /etc/X11/xinit/xinitrc

[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup
[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources
xsetroot -solid grey
x-terminal-emulator -geometry 80x24+10+10 -ls -title "$VNCDESKTOP Desktop" &
x-window-manager &
autocutsel -s CLIPBOARD -fork # to share clipboard
gnome-panel &
gnome-settings-daemon &
metacity &
nautilus &
gnome-terminal &

Change mod:

chmod -R +x ~/.vnc/ # lacking of "x" permission will also cause grey screen.

To have auto-start after rebooting, add @reboot /usr/bin/vncserver :1 to the bottom of crontab file by: crontab -e
ref1: linode (evernote backup). ref2.

problem

Local physical GUI login loop, maybe with error "lvmetad is not active yet using direct activation during sysinit" in black screen/terminal.
Reason: wrong ownership of ~/.Xauthority (same for .ICEauthority and .gvfs).
Solution: chown <user>:<group> .Xauthority; ref
To kill server: vncserver -kill :N

TEAM.VIEWER ON NIX SERVER

(not really vnc, but needed for MATLAB in Nix. MATLAB canNOT run with remote java, except -X forwarding)

  1. download official installer (.deb).
  2. install with sudo dpkg -i abc.deb (with dependency problems)
  3. resolve problem with sudo apt-get -f install which installs the lacking dependencies.
  4. run teamviewer (no root; remains available even after auto time-out logout (teamviewer v.12 within Ubuntu16)) Tip: Teamviewer server can have several permanent personal passwords: It should be find to use teamviewer GUI to setup autorun. If not, search all apps for "startup", and add teamviewerd as a new startup.

REAL.VNC

(max 5 computers) (not recommended)
Warn: both server and clients need to login the same "RealVnc account" to use.

  1. download official installer (.deb).
  2. install with sudo dpkg -i viewer.deb server.deb (no dependency problem).
  3. run sudo vncserver-x11-serviced.
  4. click running logo and resolve the licence warning.

To uninstall RealVnc:
sudo apt-get purge realvnc-vnc-server
sudo apt-get purge realvnc-vnc-viewer

CLIENT

TightVnc is preferred as open software.
RealVnc is also ok.
Usage: serverIp:N (N is monitorId)

ps:
To start vnc using a non-root user, proper files' ownership and rights might be (it worked for sunny by a chance / sometimes):

(the password file is not pure password)

Light Weighted Desktop Alternatives

Desktop Size on disk Download & Install time (1~2MB net) To start desktop Note
xorg + openbox (+nautilus) 0.3G (+65M) 3 min startx ugly if not installing extra themes.
ubuntu-desktop (linode way) 0.7G . Auto used --no-install-recommends, has problems for physical monitor (at least in vmware). vnc ok. for physical monitor, gnome-session-flashback (with recommends) needs extra 200M (metacity is better than compiz).
ubuntu-desktop 2.3G 1 hour Auto exactly the same as desk version. should be much faster, US repo is slow.
lubuntu-desktop (lxde) 1.3G . . used --no-install-recommends
lubuntu-desktop (lxde) 1.9G . . .
xubuntu-desktop (xfce) 0.5G 5 min Auto used --no-install-recommends, otherwise 2.3G. physical monitor works fine, but vnc has different shortcuts from physical monitor.

Note: the size can be varying depends on what has been installed.
Tip: libreoffice needs 350M ~ 450M (depends on recommends) disk.

official doc for XServers in server version.