Full server side GIS deployment on cheap hardware

I'm a big fan of GNU/Linux, QGIS platform, PostgreSQL and more often than not the FOSS deployments I encountered were on powerful hardware.

These last few weeks were different in that I got to make a full GIS deployment on a fan-less low power system. For production.

The hardware (cost 200 EUR):

  • PC Engines APU2. Quick specs:
    • GX-412TC Quad Core
    • Kingston mS200 60GB
    • 4 GB ECC RAM
    • 3 Gigabit Ethernet channels (Intel i211AT)

The Software (server side):

  • PostgreSQL 9.6
  • QGIS Server 2.14 served by NGINX
  • NGINX as reverse proxy
  • CentOS 7 + FirewallD + BIND + DHCPD
  • Debian Stretch on KVM (on 3 cores of the APU2)
  • QWC2

To satisfy your curiosity on how we ended up installing on the APU2, it was due to a series of unforeseen events:

  • The server that were supposed to host the deployment exhibited hardware problems from the start and it just so happened that it was out of warranty.
  • We discovered that the backup ( a workstation ) also had hardware problems
  • Management - No "buying a new server" budget.
  • Time constraints - We needed to go into production ASAP - finish the contract in time.
  • As the APU2 fits really nice as a gateway/reverse proxy due to its Intel Gigabit channels and AES-NI enabled CPU, it was in the books from the start and was already doing our bidding for those specific purposes.

The QGIS Server + QWC2 parts were installed on a Virtual Machine (Debian Stretch) so that we can quickly move it to any future server.

NGINX is installed on Debian Stretch so that we can access the QGIS Server fcgi but also on the CentOS part as to act as a reverse proxy and encrypt (HTTPS) the connections.

I knew beforehand that PostgreSQL would work really well as:

  • the whole database is around 200M so it all fits intro RAM (after tuning the default settings of course)
  • the SSD fits the bill from storage perspective + there aren't lots of editors at the same time (4-5). I know the number of editors doesn't provide the whole picture but it's all that I needed for now.
  • CPU is the last bottleneck (check out the hardware selection guide)

To my surprise, QGIS Server and consequently the WEB experience worked far better than I expected as it the mean response time for GetMap requests was around 1.2 seconds (+ around 100ms transfer time).
So, the experience is good enough for now (unless they have a Retina/HIDPI display and hit this bug) as the setup can hold 3-4 WEB users at the same time if they aren't zooming/panning like crazy.

Honestly, I would have expected the GetMap requests to take at least 3-4 seconds, clearly not good enough. This is because the qgs project had lots of layers and is filled with styling and labeling rules.

The customer is:

  • pleased that the QGIS Desktop experience is better than the 2 years old ESRI solution it replaces (the Administrator is also very pleased how how OSGeo4W Network Installer works and how he can easily update QGIS to the latest versions).
  • amazed that the QGIS Server/QWC2 performance is the same as the ESRI solution it replaces. QGIS Server sits on the APU (3 core virtual cores) while ArcGIS Server sits on double socket 8 core Xeons (2 years old), so 16 Cores. QWC2 looks are also better. Management actually asked twice if they understood correctly that the whole server part runs on the 200Euros hardware I recommended. Yes siree!

I can't help but feel proud on how everything works so good considering the price of the hardware, proud that the software deployed is Open Source, proud of the communities that support it.

Meanwhile, as the project manager informed me that they found a way to spend up to 1300 EUR on hardware this year. Searching the Internet I found a real good deal: a HP ProLiant ML10 Gen9 Servers at 550 EUR after taxes. The basics:

  • E3-1225v5 Xeon - Quad Core SkyLake, 8 MB cache, 3.30 GHz base (3.7 Turbo). Integrated GPU
  • 8GB RAM (max. 64)
  • two 2TB HDDs, Intel RST SATA RAID

Really nice! The performance per thread sits on the very high end but there aren't as many cores as other very expensive CPUs.

Having two of these babies means that you can do load balancing with NGINX and have 8 of the fastest threads in the world serving QWC2 and PostgreSQL. It doesn't have hyper-threading but it will more or less perform the same. Depending on the workload hyper-threading can yield advantages but there are also cases when disabling hyper-threading improves the performance. So, in the worst case, we won't loose a lot.

Four SATA Enterprise SSDs that were supposed to be deployed on the server that failed are also waiting to be deployed.

Cool, eh?