Difference between revisions of "Infrastructure:Sysadmin Handbook"

From the Linux and Unix Users Group at Virginia Teck Wiki
Jump to: navigation, search
imported>Pew
 
(12 intermediate revisions by 2 users not shown)
Line 1: Line 1:
−
'''Refer to [[Infrastructure:Host Deployment Guide]] until we fix the wiki deletion issue'''
+
This page describes how to manage the infra. See [https://vtluug.github.io/rtfm.txt rtfm.txt] for a guide to build it from scratch.
  
−
This page describes how to build the infrastructure from scratch, as well manage it in general.
+
This covers setup of a VM on [[Infrastructure:Meltdown|meltdown]] or [[Infrastructure:Spectre|spectre]] depending on if the service is critical or not.
  
 +
== Networks ==
 +
''Further information: [[Infrastructure:Network|Network]]
  
−
== Networking ==
+
We ''should'' have the following networks in place:
−
* Set up physical boxes based on the [[Infrastructure:Diagram|Diagram]]
 
−
* Determine the ip addresses based on [[Infrastructure:Network|Network]]
 
−
=== Router ===
 
−
Configure /etc/network/interfaces:
 
  
−
<nowiki>
+
* [[Infrastructure:Meltdown|meltdown]] and [[Infrastructure:Spectre|spectre]] br0 on eno1 <--> enp4s0 on [[Infrastructure:Joey|joey]]. This is the main LUUG network.
−
# v6
+
** 10.98.0.0/16 for VTLUUG NAT
−
iface $EXTERNAL_IF inet6 auto
+
** IPv6 via prefix delegation on 607:b400:6:cc80/64
−
iface $INTERNAL_IF inet6 static
+
** Global IPv4s via ARP proxying (See https://github.com/vtluug/scripts). Gateway is 128.173.88.1/22.
−
    address $INTERNAL_IPv6
+
* Static hosts are on 10.98.0.0/24, and DHCP is enabled on 10.98.1.0/24. This is mainly just useful for organization and quickly finding new hosts or other hosts on the network.
−
    netmask 128
+
** Static host IPs are assigned via static DHCP leases for IPv4.
−
    # Enable internal network to access router's external v6 address
+
** Since we can't do this with IPv6, physical host IPs are determined upon first boot and VMs are assigned a specific MAC to pre-determine the SLAAC IP.
−
    pre-up ip route add $EXTERNAL_IPv6 via $INTERNAL_IPv6
+
* "Internet" (a CNS portal) <--> enp2s0 on [[Infrastructure:Joey|joey]]. LUUG only has one of these, and port security is probably enabled.
−
    # Enable NDP Proxying so internal boxes get SLAAC
 
−
    pre-up echo 1 > /proc/sys/net/ipv6/conf/all/forwarding
 
−
    pre-up echo 2 > /proc/sys/net/ipv6/conf/all/accept_ra
 
  
−
# VTLUUG Private Network v4
+
'''DNS/DHCP:'''
−
iface $INTERNAL_IF inet static
+
* All DNS entries for services run by VTLUUG are hosted on [https://gandi.net Gandi]. Ask an officer if you want to change something.
−
    address $INTERNAL_IPv4
+
* jkh and Roddy own ece.vt.edu. DNS updates don't happen. echarlie can add IPv6-only records if needed to wuvt.vt.edu so we have PTRs.
−
    netmask 255.255.255.0
+
* [[Infrastructure:Joey|joey]] runs DHCP via dnsmasq on enp4s0 (that is, 10.98.0.0/16). To change anything, modify it on https://github.com/vtluug/scripts first then pull that into root's homedir on [[Infrastructure:Joey|joey]]. Please don't just update it on a machine without pushing your updates.
 +
* By default, hosts are accessible via SSH on ports 22 and 2222.
  
−
# Additional IPs
+
== Adding a VTLUUG Service VM ==
−
iface $EXTERNAL_IF inet static
+
''VMs in this category are deployed to [[Infrastructure:Meltdown|meltdown]]''
−
    address $EXTERNAL_IPv4
 
−
    gateway 128.173.88.1
 
−
    broadcast 128.173.91.255
 
−
    netmask 255.255.252.0
 
−
    # Nat Settings
 
−
    # TODO this probably doesn't work
 
−
    pre-up tc action nat egress 10.99.0.0/24 $EXTERNAL_IP
 
−
    # Enable ARP Proxying so internal v4 address are accessible
 
−
    pre-up echo 1 > /proc/sys/net/ipv4/conf/all/proxy_arp
 
−
    pre-up echo 1 > /proc/sys/net/ipv4/ip_forward
 
−
    # Route internal v4 addresses
 
−
    ip route add $JOEY_EXTERNAL_IPv4/24 dev $INTERNAL_IF
 
−
    ip route add $CRASHANDBURN_EXTERNAL_IPv4/24 dev $INTERNAL_IF
 
−
    ip route add $SCZI_EXTERNAL_IPv4/24 dev $INTERNAL_IF
 
−
    ip route add $ACIDBURN_EXTERNAL_IPv4/24 dev $INTERNAL_IF
 
−
    ip route add $ZEROCOOL_EXTERNAL_IPv4/24 dev $INTERNAL_IF
 
−
    ip route add $MIRROR_EXTERNAL_IPv4/24 dev $INTERNAL_IF
 
−
</nowiki>
 
  
−
Next, set up NDP proxying
+
Prerequisites:
−
Configure /etc/ndppd.conf: (May not already exist)
+
* Clone <code>https://github.com/vtluug/scripts</code>. This is referred to as 'SCRIPTS' in this guide.
 +
* Clone <code>https://github.com/vtluug/ansible</code> and install ansible. This repo is referred to as 'ANSIBLE' in this guide.
 +
* Have access to the vtluug-admin repo on gitolite.
 +
* Understand the [[Infrastructure:Network|Network]] and [[Infrastructure]].
 +
* Put your SSH key on [[Infrastructure:Meltdown|meltdown]]
  
 +
=== Configure the network ===
 +
* Decide on a MAC address for the host and add it to <code>SCRIPTS/router/lan/local_hosts</code>
 +
* Add an entry to <code>SCRIPTS/router/lan/dnsmasq.conf</code> for static DHCP leases.
 +
* If a new IP in 128.173.88.1/22 is being added, also add it to <code>SCRIPTS/router/proxy/arp_proxy.sh</code>
  
−
<nowiki>
+
'''Note:''' It is '''not''' recommended that you do the following steps if nobody is on campus in case something breaks.
−
# Rather than only listenting on each individual IPv6 address, we
 
−
#  simply forward all soliciations. The main advantage is that we
 
−
#  don't have to add any additional routing rules if a new internal
 
−
#  device is added.
 
−
route-ttl 30000
 
−
address-ttl 30000
 
  
−
# External interface to listen on
+
Pull the latest changes to <code>/root/scripts</code>, update the configuration files, and restart the services:
−
proxy $EXTERNAL_IF {
+
* Dnsmasq configuration is at <code>/etc/dnsmasq.conf</code>
−
    router yes
+
* ARP Proxy configuration is in <code>/usr/local/bin</code>
−
    timeout 500 
 
−
    autowire no
 
−
    keepalive yes
 
−
    retries 3
 
−
    promiscuous no
 
−
    ttl 30000
 
  
−
    # Prefix to listen on
+
=== Add the VM configuration to ansible ===
−
    rule ::0/ { # TODO might change prefix
+
Edit <code>ANSIBLE_PATH/roles/deploy-vms/defaults/main.yml</code> and add a new entry, following the existing format.
−
        # Internal interface to forward everything to
 
−
        iface $INTERNAL_IF
 
−
        autovia no
 
−
    }
 
−
}
 
−
</nowiki>
 
  
−
Now start '''and''' enable ndppd.service.
+
'''Note:''' if there are any entries in this file that are '''not''' present on the VM host, they will also be created. Comment out entries that shouldn't be created. Existing hosts are skipped.
  
−
=== Everything Else not run under oVirt ===
+
Run <code>ansible-playbook -v deploy.yml -i hosts.cfg -u papatux -k -K -e @VTLUUG_ADMIN_REPO/accounts.yml</code>, using the correct vtluug-admin repo path.
−
==== Debian ====
 
−
Configure /etc/network/interfaces:
 
  
−
<nowiki>
+
=== Testing ===
−
# v6
+
The new host should be accessible by papatux on via SSH port 2222 (and 22) over IPv6 and IPv4 from the internal network. Check 10.98.1.0/24 to see if it had any issues getting a static DHCP lease and if the MAC is correct.
−
iface $INTERFACE inet6 auto
 
−
auto $INTERFACE
 
−
iface $INTERFACE inet static
 
−
    address $INTERNAL_IPv4
 
−
    gateway 10.99.0.1
 
−
    netmask 255.255.255.0
 
  
−
# Additional IPs - Only do this if this box has an external IP
+
== Adding a User VM ==
−
iface $INTERFACE inet static
+
''VMs in this category are deployed to [[Infrastructure:Spectre|spectre]]''
−
    address $EXTERNAL_IPv4
 
−
    gateway 128.173.88.1
 
−
    netmask 255.255.252.0
 
−
</nowiki>
 
  
−
==== Centos ====
+
Prerequisites:
−
Configure /etc/sysconfig/network-scripts/ifcfg-$INTERFACE:
+
* Clone <code>https://github.com/vtluug/scripts</code>. This is referred to as 'SCRIPTS' in this guide.
 +
* Clone <code>https://github.com/vtluug/ansible</code> and install ansible. This repo is referred to as 'ANSIBLE' in this guide.
 +
* Understand the [[Infrastructure:Network|Network]] and [[Infrastructure]].
 +
* Have root on [[Infrastructure:Spectre|spectre]]
 +
* Put your SSH key on [[Infrastructure:Spectre|spectre]]
  
−
<nowiki>
+
=== Configure the network ===
−
ONBOOT="yes"
+
* Decide on a MAC address for the host and add it to <code>SCRIPTS/router/lan/local_hosts</code>
−
NM_CONTROLLED="no"
+
* Add an entry to <code>SCRIPTS/router/lan/dnsmasq.conf</code> for static DHCP leases. (If applicable; you might not care for a test/temp VM).
−
BOOTPROTO="static"
 
−
IPADDR0="$INTERNAL_IPv4"
 
−
GATEWAY0="10.99.0.1"
 
−
NETMASK0="255.255.255.0"
 
−
# Addition IPs - Only do this if this box has an external IP
 
−
IPADDR1="$EXTERNAL_IPv4"
 
−
GATEWAY1="128.173.88.1"
 
−
NETMASK1="255.255.252.0"
 
−
</nowiki>
 
  
−
== Other stuff ==
+
'''Note:''' It is '''not''' recommended that you do the following steps if nobody is on campus in case something breaks.
 +
 
 +
Pull the latest changes to <code>/root/scripts</code>, update the configuration files, and restart the services:
 +
* Dnsmasq configuration is at <code>/etc/dnsmasq.conf</code>
 +
 
 +
=== Add the VM configuration to ansible ===
 +
Edit <code>ANSIBLE_PATH/roles/deploy-vms/defaults/main.yml</code> and add a new entry under <code>new_vms_spectre</code>, following the existing format.
 +
 
 +
'''Note:''' if there are any entries in this file that are '''not''' present on the VM host, they will also be created. Comment out entries that shouldn't be created. Existing hosts are skipped.
 +
 
 +
Run <code>ansible-playbook -v deploy.yml -i hosts.cfg -u papatux -k -K</code>, using the correct vtluug-admin repo path.
 +
 
 +
'''Important:''' A random root password is set during VM creation and printed to stdout. Record this!
 +
 
 +
=== Testing ===
 +
The new host should be accessible by root on via SSH port 2222 (and 22) over IPv6 and IPv4 from the internal network. Check 10.98.1.0/24 to see if it had any issues getting a static DHCP lease and if the MAC is correct.
  
 
[[Category:Infrastructure]]
 
[[Category:Infrastructure]]
 
[[Category:Howtos]]
 
[[Category:Howtos]]
 +
[[Category:Needs restoration]]

Latest revision as of 16:02, 2 June 2020

This page describes how to manage the infra. See rtfm.txt for a guide to build it from scratch.

This covers setup of a VM on meltdown or spectre depending on if the service is critical or not.

Networks

Further information: Network

We should have the following networks in place:

  • meltdown and spectre br0 on eno1 <--> enp4s0 on joey. This is the main LUUG network.
    • 10.98.0.0/16 for VTLUUG NAT
    • IPv6 via prefix delegation on 607:b400:6:cc80/64
    • Global IPv4s via ARP proxying (See https://github.com/vtluug/scripts). Gateway is 128.173.88.1/22.
  • Static hosts are on 10.98.0.0/24, and DHCP is enabled on 10.98.1.0/24. This is mainly just useful for organization and quickly finding new hosts or other hosts on the network.
    • Static host IPs are assigned via static DHCP leases for IPv4.
    • Since we can't do this with IPv6, physical host IPs are determined upon first boot and VMs are assigned a specific MAC to pre-determine the SLAAC IP.
  • "Internet" (a CNS portal) <--> enp2s0 on joey. LUUG only has one of these, and port security is probably enabled.

DNS/DHCP:

  • All DNS entries for services run by VTLUUG are hosted on Gandi. Ask an officer if you want to change something.
  • jkh and Roddy own ece.vt.edu. DNS updates don't happen. echarlie can add IPv6-only records if needed to wuvt.vt.edu so we have PTRs.
  • joey runs DHCP via dnsmasq on enp4s0 (that is, 10.98.0.0/16). To change anything, modify it on https://github.com/vtluug/scripts first then pull that into root's homedir on joey. Please don't just update it on a machine without pushing your updates.
  • By default, hosts are accessible via SSH on ports 22 and 2222.

Adding a VTLUUG Service VM

VMs in this category are deployed to meltdown

Prerequisites:

Configure the network

  • Decide on a MAC address for the host and add it to SCRIPTS/router/lan/local_hosts
  • Add an entry to SCRIPTS/router/lan/dnsmasq.conf for static DHCP leases.
  • If a new IP in 128.173.88.1/22 is being added, also add it to SCRIPTS/router/proxy/arp_proxy.sh

Note: It is not recommended that you do the following steps if nobody is on campus in case something breaks.

Pull the latest changes to /root/scripts, update the configuration files, and restart the services:

  • Dnsmasq configuration is at /etc/dnsmasq.conf
  • ARP Proxy configuration is in /usr/local/bin

Add the VM configuration to ansible

Edit ANSIBLE_PATH/roles/deploy-vms/defaults/main.yml and add a new entry, following the existing format.

Note: if there are any entries in this file that are not present on the VM host, they will also be created. Comment out entries that shouldn't be created. Existing hosts are skipped.

Run ansible-playbook -v deploy.yml -i hosts.cfg -u papatux -k -K -e @VTLUUG_ADMIN_REPO/accounts.yml, using the correct vtluug-admin repo path.

Testing

The new host should be accessible by papatux on via SSH port 2222 (and 22) over IPv6 and IPv4 from the internal network. Check 10.98.1.0/24 to see if it had any issues getting a static DHCP lease and if the MAC is correct.

Adding a User VM

VMs in this category are deployed to spectre

Prerequisites:

Configure the network

  • Decide on a MAC address for the host and add it to SCRIPTS/router/lan/local_hosts
  • Add an entry to SCRIPTS/router/lan/dnsmasq.conf for static DHCP leases. (If applicable; you might not care for a test/temp VM).

Note: It is not recommended that you do the following steps if nobody is on campus in case something breaks.

Pull the latest changes to /root/scripts, update the configuration files, and restart the services:

  • Dnsmasq configuration is at /etc/dnsmasq.conf

Add the VM configuration to ansible

Edit ANSIBLE_PATH/roles/deploy-vms/defaults/main.yml and add a new entry under new_vms_spectre, following the existing format.

Note: if there are any entries in this file that are not present on the VM host, they will also be created. Comment out entries that shouldn't be created. Existing hosts are skipped.

Run ansible-playbook -v deploy.yml -i hosts.cfg -u papatux -k -K, using the correct vtluug-admin repo path.

Important: A random root password is set during VM creation and printed to stdout. Record this!

Testing

The new host should be accessible by root on via SSH port 2222 (and 22) over IPv6 and IPv4 from the internal network. Check 10.98.1.0/24 to see if it had any issues getting a static DHCP lease and if the MAC is correct.

Retrieved from "https://vtluug.org/w/index.php?title=Infrastructure:Sysadmin_Handbook&oldid=7738"