Difference between revisions of "Infrastructure:Bastille"

From the Linux and Unix Users Group at Virginia Teck Wiki
Jump to: navigation, search
m
Line 71: Line 71:
  
 
The frontend networks, uses DHCP with [[Infrastructure:Shellshock]], while the backend network is statically routed.
 
The frontend networks, uses DHCP with [[Infrastructure:Shellshock]], while the backend network is statically routed.
 +
 +
The blades have completely ephemeral disks -- the k3s node passwords and join tokens are shipped via agenix, targeting an ssh host key that is sealed by TPM1.2 and kept in the netboot image.
  
 
You can spy their config here [https://github.com/vtluug/construct/blob/main/hosts/bastille/blade.nix]  
 
You can spy their config here [https://github.com/vtluug/construct/blob/main/hosts/bastille/blade.nix]  

Revision as of 04:26, 14 September 2026

Bastille is our Lenovo Nextscale chassis donated by jpo from his VT Surplus spoils.

It contains twelve like 'blades', with:

  • 2x Intel(R) Xeon(R) E5-2680 v3 (48) @ 3.30 GHz, 12 cores 24 threads each
  • 256 GiB DDR4
  • a fully-featured remote management interface, with a webUI kvm
  • two gigabit eth NICs, and one Intel X520 2x10G SFP+
  • notably, no peripheral ports other than a multiplexed one that uses a proprietary adapter to VGA, Serial, 2x USB

These blades are appropriately named: 

[
  "backbiter"
  "damocles"
  "durendal"
  "eyelander"
  "excalibur"
  "gram"
  "gryffindor"
  "kusanagi"
  "narsil"
  "oathbringer"
  "riptide"
  "sting"
]

They are at the moment netbooted from Infrastructure:Vesuvius as NixOS k3s nodes.

Hostname Frontend IP4 Backend IP4 Global IP6 Notes
vesuvius 10.98.0.23 10.98.3.1 2607:b400:6:cc80:21e:67ff:feca:9c12 k3s control plane; PXE boot image server
backbiter 10.98.0.25 10.98.3.2 2607:b400:6:cc80:42f2:e9ff:fec6:655f
damocles 10.98.0.26 10.98.3.3 2607:b400:6:cc80:42f2:e9ff:fec6:67db
durendal 10.98.0.27 10.98.3.4 2607:b400:6:cc80:42f2:e9ff:fec6:6895
excalibur 10.98.0.29 10.98.3.5 2607:b400:6:cc80:42f2:e9ff:fec6:6967
eyelander 10.98.0.28 10.98.3.6 2607:b400:6:cc80:42f2:e9ff:fec6:6943
gram 10.98.0.30 10.98.3.7 2607:b400:6:cc80:42f2:e9ff:fec6:7459
prospit - - - User:Enzoisotton took this out of the server room to test pxe booting, yell at him until it's back
gryffindor 10.98.0.31 10.98.3.8 2607:b400:6:cc80:42f2:e9ff:fec6:74d7
kusanagi 10.98.0.32 - 2607:b400:6:cc80:42f2:e9ff:fec6:7597 the SFP+ NIC appears broken. may work if re-seated.
narsil 10.98.0.33 10.98.3.10 2607:b400:6:cc80:42f2:e9ff:fec6:759d
oathbringer 10.98.0.34 10.98.3.11 2607:b400:6:cc80:42f2:e9ff:fec6:75f1
riptide 10.98.0.35 10.98.3.13 2607:b400:6:cc80:42f2:e9ff:fec6:7621
sting 10.98.0.36 10.98.3.14 2607:b400:6:cc80:42f2:e9ff:fec6:7723

Each machine can be remotely accessed on port 2222 from it's global IP6 address, or by DNS at <name>.vtluug.org.

Each machine has a management interface on 10.98.2.0/24, with the same last octet as its backend IP -- i.e. riptide can be managed from 10.98.2.13.

Guide

Technical

Each listed blade has four connections: two 8p8c NICs attached to Infrastructure:Magma, one for the frontend network, and one for its management interface, then two SFP+ NICs bonded together and connected to Infrastructure:Pahoehoe via QSFP+ breakout cable. The 20G bonded interface is used for the cluster's backend interconnect.

The frontend networks, uses DHCP with Infrastructure:Shellshock, while the backend network is statically routed.

The blades have completely ephemeral disks -- the k3s node passwords and join tokens are shipped via agenix, targeting an ssh host key that is sealed by TPM1.2 and kept in the netboot image.

You can spy their config here [1]

Operational

All users with a VTLUUG SSO account (read: FreeIPA/acidburn) can SSH to Infrastructure:Vesuvius on the VTLUUG LAN.

From there, all users with sudo access can use `kubectl` to manage the cluster, or proxy/jump to other nodes through the backend fabric.

Things To Host On All This (Post Suggestions!!!)

  • hentai at home (rsk)
  • tor relay (rsk)
    • both of the above possibly better as a slim nixos VM on something like spectre?
  • nix build server (clj)
  • build/parameter optimization rust compiler (clj)
  • folding@home (ap, eri)
  • port sczi over (mkl)
    • slskd/slskd:latest
    • quay.io/oauth2-proxy/oauth2-proxy:latest
    • luuul/4get:latest ported
    • linuxserver/qbittorrent:latest
    • qmcgaw/gluetun
    • quay.io/oauth2-proxy/oauth2-proxy:latest
    • mariadb:10.4.32
    • vtluug/mediawiki-vtluug:1.1 ported
    • linuxserver/swag
    • vectorim/element-web:latest
    • quay.io/dexidp/dex:latest
    • vtluug/pyqdb ported
    • thelounge/thelounge:latest
    • vtluug-site-vtluug-site ported
    • vaultwarden/server:latest
    • gitea/gitea:nightly
    • t4skforce/syncthing-relay-discovery:latest
  • Tigerbeetle (clj, meme suggestion)
  • Something like Sagemath but in a way that doesn't give people free ACE (clj)
  • map stuff (mhazinsk)
    • OSM tile server
    • async LIDAR processing - laz2ept/sinkholes, etc
  • coder ephemeral dev environments for members (mhazinsk)
  • OCCRP Aleph for VT FOIA requests as a hokieprivacy project (mhazinsk)
  • OONI probe (mhazinsk)
  • browsertrix (pew)
  • grafana (ARC and many others have done this and it looks cool) (clj)
Retrieved from "https://vtluug.org/w/index.php?title=Infrastructure:Bastille&oldid=8325"