261
edits
Changes
m
no edit summary
[[Infrastructure:Bastille|Bastille]] is our Lenovo Nextscale chassis donated by [[User:Timelord|jpo]] from his VT Surplus spoils.
It contains twelve like 'blades', with:
* 2x Intel(R) Xeon(R) E5-2680 v3 (48) @ 3.30 GHz, 12 cores 24 threads each
* 256 GiB DDR4
* a fully-featured remote management interface, with a webUI kvm
* two gigabit eth NICs, and one Intel X520 2x10G SFP+
* notably, no peripheral ports other than a multiplexed one that uses a proprietary adapter to VGA, Serial, 2x USB
These blades are appropriately named:
<pre>
[papatux@excalibur:~]$ fastfetch -l nonepapatux@excalibur----------------- "backbiter"OS: NixOS 25.11 (Xantusia) x86_64 "damocles"Host: Lenovo NeXtScale nx360 M5: -[5465AC1]- (06) "durendal"Kernel: Linux 6.12.45 "eyelander"Uptime: 1 day, 20 hours, 6 minsPackages: 391 (nix-system)Shell: bash 5.3.3Display (VGA-1): 1024x768 [External] "excalibur"Terminal: /dev/pts/0 "gram"CPU: 2 x Intel(R) Xeon(R) E5-2680 v3 (48) @ 3.30 GHz "gryffindor"GPU: Matrox Electronics Systems Ltd. G200eR2 "kusanagi"Memory: 3.88 GiB / 251.19 GiB (2%) "narsil"Swap: Disabled "oathbringer"Disk (/): 329.43 MiB / 125.59 GiB (0%) - tmpfs "riptide"Local IP (eno1): 10.98.1.36/16 "sting"Locale: en_US.UTF-8[papatux@excalibur:~]$
</pre>
They are at the moment netbooted from [[Infrastructure:Vesuvius]] as NixOS k3s nodes.
{| class="wikitable sortable"
! Hostname !! Frontend IP4 !! Backend IP4 !! Global IP6 !! Notes
|-
| [[Infrastructure:Vesuvius|vesuvius]] || 10.98.0.23 || 10.98.3.1 || 2607:b400:6:cc80:21e:67ff:feca:9c12 || k3s control plane; PXE boot image server
|-
| backbiter || 10.98.0.25 || 10.98.3.2 || 2607:b400:6:cc80:42f2:e9ff:fec6:655f ||
|-
| damocles || 10.98.0.26 || 10.98.3.3 || 2607:b400:6:cc80:42f2:e9ff:fec6:67db ||
|-
| durendal || 10.98.0.27 || 10.98.3.4 || 2607:b400:6:cc80:42f2:e9ff:fec6:6895 ||
|-
| excalibur || 10.98.0.29 || 10.98.3.5 || 2607:b400:6:cc80:42f2:e9ff:fec6:6967 ||
|-
| eyelander || 10.98.0.28 || 10.98.3.6 || 2607:b400:6:cc80:42f2:e9ff:fec6:6943 ||
|-
| gram || 10.98.0.30 || 10.98.3.7 || 2607:b400:6:cc80:42f2:e9ff:fec6:7459 ||
|-
| [[Infrastructure:Prospit|prospit]] || - || - || - || [[User:Enzoisotton]] took this out of the server room to test pxe booting, yell at him until it's back
|-
| gryffindor || 10.98.0.31 || 10.98.3.8 || 2607:b400:6:cc80:42f2:e9ff:fec6:74d7 ||
|-
| kusanagi || 10.98.0.32 || - || 2607:b400:6:cc80:42f2:e9ff:fec6:7597 || the SFP+ NIC appears broken. may work if re-seated.
|-
| narsil || 10.98.0.33 || 10.98.3.10 || 2607:b400:6:cc80:42f2:e9ff:fec6:759d ||
|-
| oathbringer || 10.98.0.34 || 10.98.3.11 || 2607:b400:6:cc80:42f2:e9ff:fec6:75f1 ||
|-
| riptide || 10.98.0.35 || 10.98.3.13 || 2607:b400:6:cc80:42f2:e9ff:fec6:7621 ||
|-
| sting || 10.98.0.36 || 10.98.3.14 || 2607:b400:6:cc80:42f2:e9ff:fec6:7723 ||
|}
Each machine can be remotely accessed on port 2222 from it's global IP6 address, or by DNS at <name>.vtluug.org.
Each machine has a management interface on 10.98.2.0/24, with the same last octet as its backend IP -- i.e. riptide can be managed from 10.98.2.13.
== Guide ==
=== Technical ===
Each listed blade has four connections: two 8p8c NICs attached to [[Infrastructure:Magma]], one for the frontend network, and one for its management interface, then two SFP+ NICs bonded together and connected to [[Infrastructure:Pahoehoe]] via QSFP+ breakout cable. The 20G bonded interface is used for the cluster's backend interconnect.
The frontend networks, uses DHCP with [[Infrastructure:Shellshock]], while the backend network is statically routed.
The blades have completely ephemeral disks -- the k3s node passwords and join tokens are shipped via agenix, targeting an ssh host key that is sealed by TPM1.2 and kept in the netboot image.
You can spy their config here [https://github.com/vtluug/construct/blob/main/hosts/bastille/blade.nix]
=== Operational ===
All users with a VTLUUG SSO account (read: FreeIPA/acidburn) can SSH to [[Infrastructure:Vesuvius]] from the VTLUUG LAN or the public IPv6 address.
From there, all users with sudo access can use `kubectl` to manage the cluster, or proxy/jump to other nodes through the backend fabric.
Non-admin tenant namespaces coming Soon^{tm}
== Things To Host On All This (Post Suggestions!!!) ==
* hentai at home (rsk)
* tor relay (rsk)
** both of the above possibly better as a slim nixos VM on something like spectre?
* nix build server (clj)
* build/parameter optimization rust compiler (clj)
* folding@home (ap, eri)
* port sczi over (mkl)
** slskd/slskd:latest
** quay.io/oauth2-proxy/oauth2-proxy:latest
** <s>luuul/4get:latest</s> ported
** linuxserver/qbittorrent:latest
** qmcgaw/gluetun
** quay.io/oauth2-proxy/oauth2-proxy:latest
** mariadb:10.4.32
** <s>vtluug/mediawiki-vtluug:1.1</s> ported
** linuxserver/swag
** vectorim/element-web:latest
** quay.io/dexidp/dex:latest
** <s>vtluug/pyqdb</s> ported
** thelounge/thelounge:latest
** <s>vtluug-site-vtluug-site</s> ported
** vaultwarden/server:latest
** gitea/gitea:nightly
** t4skforce/syncthing-relay-discovery:latest
* Tigerbeetle (clj, meme suggestion)
* Something like Sagemath but in a way that doesn't give people free ACE (clj)
* map stuff (mhazinsk)
** OSM tile server
** async LIDAR processing - laz2ept/[https://caves.science/ sinkholes], etc
* [https://github.com/coder/coder coder] ephemeral dev environments for members (mhazinsk)
* [https://aleph.occrp.org/ OCCRP Aleph] for VT FOIA requests as a hokieprivacy project (mhazinsk)
* [https://ooni.org/install/cli OONI probe] (mhazinsk)
* browsertrix (pew)
* grafana (ARC and many others have done this and it looks cool) (clj)
[[Category:Infrastructure]]
[[Category:Hosts]]
[[Category:Bastille]]
