Infrastructure:Bastille: Difference between revisions
No edit summary |
mNo edit summary |
||
| (25 intermediate revisions by 4 users not shown) | |||
| Line 6: | Line 6: | ||
* 256 GiB DDR4 | * 256 GiB DDR4 | ||
* a fully-featured remote management interface, with a webUI kvm | * a fully-featured remote management interface, with a webUI kvm | ||
* two gigabit eth NICs, and one | * two gigabit eth NICs, and one Intel X520 2x10G SFP+ | ||
* notably, no peripheral | * notably, no peripheral ports other than a multiplexed one that uses a proprietary adapter to VGA, Serial, 2x USB | ||
These blades are appropriately named: | These blades are appropriately named: | ||
| Line 27: | Line 27: | ||
</pre> | </pre> | ||
They are at the moment netbooted from Vesuvius as NixOS k3s nodes. | They are at the moment netbooted from [[Infrastructure:Vesuvius]] as NixOS k3s nodes. | ||
{| class="wikitable sortable" | |||
! Hostname !! Frontend IP4 !! Backend IP4 !! Global IP6 !! Notes | |||
|- | |||
| [[Infrastructure:Vesuvius|vesuvius]] || 10.98.0.23 || 10.98.3.1 || 2607:b400:6:cc80:21e:67ff:feca:9c12 || k3s control plane; PXE boot image server | |||
|- | |||
| backbiter || 10.98.0.25 || 10.98.3.2 || 2607:b400:6:cc80:42f2:e9ff:fec6:655f || | |||
|- | |||
| damocles || 10.98.0.26 || 10.98.3.3 || 2607:b400:6:cc80:42f2:e9ff:fec6:67db || | |||
|- | |||
| durendal || 10.98.0.27 || 10.98.3.4 || 2607:b400:6:cc80:42f2:e9ff:fec6:6895 || | |||
|- | |||
| excalibur || 10.98.0.29 || 10.98.3.5 || 2607:b400:6:cc80:42f2:e9ff:fec6:6967 || | |||
|- | |||
| eyelander || 10.98.0.28 || 10.98.3.6 || 2607:b400:6:cc80:42f2:e9ff:fec6:6943 || | |||
|- | |||
| gram || 10.98.0.30 || 10.98.3.7 || 2607:b400:6:cc80:42f2:e9ff:fec6:7459 || | |||
|- | |||
| [[Infrastructure:Prospit|prospit]] || - || - || - || [[User:Enzoisotton]] took this out of the server room to test pxe booting, yell at him until it's back | |||
|- | |||
| gryffindor || 10.98.0.31 || 10.98.3.8 || 2607:b400:6:cc80:42f2:e9ff:fec6:74d7 || | |||
|- | |||
| kusanagi || 10.98.0.32 || - || 2607:b400:6:cc80:42f2:e9ff:fec6:7597 || the SFP+ NIC appears broken. may work if re-seated. | |||
|- | |||
| narsil || 10.98.0.33 || 10.98.3.10 || 2607:b400:6:cc80:42f2:e9ff:fec6:759d || | |||
|- | |||
| oathbringer || 10.98.0.34 || 10.98.3.11 || 2607:b400:6:cc80:42f2:e9ff:fec6:75f1 || | |||
|- | |||
| riptide || 10.98.0.35 || 10.98.3.13 || 2607:b400:6:cc80:42f2:e9ff:fec6:7621 || | |||
|- | |||
| sting || 10.98.0.36 || 10.98.3.14 || 2607:b400:6:cc80:42f2:e9ff:fec6:7723 || | |||
|} | |||
== Things To Host On All This | Each machine can be remotely accessed on port 2222 from it's global IP6 address, or by DNS at <name>.vtluug.org. | ||
Each machine has a management interface on 10.98.2.0/24, with the same last octet as its backend IP -- i.e. riptide can be managed from 10.98.2.13. | |||
== Guide == | |||
=== Technical === | |||
Each listed blade has four connections: two 8p8c NICs attached to [[Infrastructure:Magma]], one for the frontend network, and one for its management interface, then two SFP+ NICs bonded together and connected to [[Infrastructure:Pahoehoe]] via QSFP+ breakout cable. The 20G bonded interface is used for the cluster's backend interconnect. | |||
The frontend networks, uses DHCP with [[Infrastructure:Shellshock]], while the backend network is statically routed. | |||
The blades have completely ephemeral disks -- the k3s node passwords and join tokens are shipped via agenix, targeting an ssh host key that is sealed by TPM1.2 and kept in the netboot image. | |||
You can spy their config here [https://github.com/vtluug/construct/blob/main/hosts/bastille/blade.nix] | |||
=== Operational === | |||
All users with a VTLUUG SSO account (read: FreeIPA/acidburn) can SSH to [[Infrastructure:Vesuvius]] from the VTLUUG LAN or the public IPv6 address. | |||
From there, all users with sudo access can use `kubectl` to manage the cluster, or proxy/jump to other nodes through the backend fabric. | |||
Non-admin tenant namespaces coming Soon^{tm} | |||
== Things To Host On All This (Post Suggestions!!!) == | |||
* hentai at home (rsk) | * hentai at home (rsk) | ||
* tor relay (rsk) | |||
** both of the above possibly better as a slim nixos VM on something like spectre? | |||
* nix build server (clj) | |||
* build/parameter optimization rust compiler (clj) | |||
* folding@home (ap, eri) | |||
* port sczi over (mkl) | |||
** slskd/slskd:latest | |||
** quay.io/oauth2-proxy/oauth2-proxy:latest | |||
** <s>luuul/4get:latest</s> ported | |||
** linuxserver/qbittorrent:latest | |||
** qmcgaw/gluetun | |||
** quay.io/oauth2-proxy/oauth2-proxy:latest | |||
** mariadb:10.4.32 | |||
** <s>vtluug/mediawiki-vtluug:1.1</s> ported | |||
** linuxserver/swag | |||
** vectorim/element-web:latest | |||
** quay.io/dexidp/dex:latest | |||
** <s>vtluug/pyqdb</s> ported | |||
** thelounge/thelounge:latest | |||
** <s>vtluug-site-vtluug-site</s> ported | |||
** vaultwarden/server:latest | |||
** gitea/gitea:nightly | |||
** t4skforce/syncthing-relay-discovery:latest | |||
* Tigerbeetle (clj, meme suggestion) | |||
* Something like Sagemath but in a way that doesn't give people free ACE (clj) | |||
* map stuff (mhazinsk) | |||
** OSM tile server | |||
** async LIDAR processing - laz2ept/[https://caves.science/ sinkholes], etc | |||
* [https://github.com/coder/coder coder] ephemeral dev environments for members (mhazinsk) | |||
* [https://aleph.occrp.org/ OCCRP Aleph] for VT FOIA requests as a hokieprivacy project (mhazinsk) | |||
* [https://ooni.org/install/cli OONI probe] (mhazinsk) | |||
* browsertrix (pew) | |||
* grafana (ARC and many others have done this and it looks cool) (clj) | |||
[[Category:Infrastructure]] | |||
[[Category:Hosts]] | |||
[[Category:Bastille]] | |||