I am moving away from Foreman managed PXE functionality to a standalone PXE boot server.
Homelab Update
I’ve been using Katello to manage my homelab for some years now, and it’s done the job well. Upgrades can be painful at times but you learn to read the CHANGELOG.
My main area of interest has shifted in the last twelve months. I’ve been focused on adopting Docker and getting into the world of containers and infrastructure as a code. Due to global pandemic, the Red Hat Summit 2020 was held online as a Virtual Experience, and I was able to attend it for the first time. I was surprised to see so many OpenShift use cases: from Public Health England using OpenShift to support scientific computing to automotive industry building flexible production infrastructures. I will admit that I did not anticipate the scale of it.
Fast forward some months, I changed jobs to work on a Kubernetes project. As a result, my homelab requirements have changed. I fell in love with Kubernetes. I migrated from Puppet to Ansible. I’ve found myself using Katello less and less. Its system requirements have also increased by 50%. When I started with Katello I could get away with 8GB of memory. This has grown to 12GB over time and has become a resource hog.
What I use My PXE Boot Server For
I use KVM to run my virtual machines (it’s enterprise quality and free). When I deploy a new VM, I normally do the following:
$ VM_ID="08" $ virt-install \ --connect qemu+ssh://[email protected]/system \ --name srv${VM_ID} \ --network bridge=br0,model=virtio,mac=C0:FF:EE:D0:5E:${VM_ID} \ --disk path=/var/lib/libvirt/images/srv${VM_ID}.qcow2,size=16 \ --pxe \ --ram 2048 \ --vcpus 1 \ --os-type linux \ --os-variant centos7.0 \ --sound none \ --rng /dev/urandom \ --virt-type kvm
This uses PXE boot to provision a new CentOS 7 server, issues a static DHCP lease based on my pre-defined list of MAC addresses (that does not change), and creates a dynamic DNS entry. Once the server is up and running, I use Ansible to configure it without having to SSH into the VM manually.
Using this approach I can spin up half a dozen of servers to deploy my multi-master Kubernetes homelab, or re-build the workstations that also run on CentOS 7.
Software
A PXE boot server requires DHCP, TFTP and FTP (or NFS) to function properly, but these services don’t have to be deployed on a single machine. My DHCP server configuration is described here and won’t be covered in this article.
Software used in this article:
- CentOS 7
- tftp-server 5.2
- vsftpd 3.0
SELinux set to enforcing mode and firewalld is enabled.
PXE Boot Server Hardware Requirements
- 1 CPU
- 1GB RAM
- 16GB disk
- The IP address of the PXE boot server is 10.11.1.20
Install and Configure FTP Server
We use FTP to share OS installation media to PXE boot clients.
Install vsftpd package and ensure that the service is enabled:
$ sudo yum install vsftpd -y $ sudo systemctl enable vsftpd
Configure /etc/vsftpd/vsftpd.conf
:
anonymous_enable=YES local_enable=NO write_enable=NO local_umask=022 dirmessage_enable=YES xferlog_enable=YES connect_from_port_20=YES xferlog_std_format=YES ftpd_banner=Welcome to homelab FTP service. listen=YES listen_ipv6=NO listen_port=21 pam_service_name=vsftpd userlist_enable=YES tcp_wrappers=YES pasv_enable=YES pasv_address=10.11.1.20 pasv_min_port=60000 pasv_max_port=60029
Configure firewall rules including passive port range:
$ sudo firewall-cmd --permanent --add-service=ftp $ sudo firewall-cmd --permanent --add-port=60000-60029/tcp $ sudo firewall-cmd --reload
Start the service:
$ sudo systemctl start vsftpd
Install TFTP Server
Install tftp-server package and ensure that the service is enabled:
$ sudo yum install tftp-server -y $ sudo systemctl enable tftp && sudo systemctl start tftp
Configure firewall rules:
$ sudo firewall-cmd --permanent --add-service=tftp $ sudo firewall-cmd --reload
Download and Create CentOS 7 FTP Installation Media
Download a CentOS 7 ISO image. Make sure to download the full DVD version, e.g.:
$ sudo yum install curl -y $ curl -# -O http://mirror.ox.ac.uk/sites/mirror.centos.org/7.9.2009/isos/x86_64/CentOS-7-x86_64-DVD-2009.iso
Mount the image and copy its content to the FTP location:
$ sudo mkdir -p /mnt/iso /var/ftp/pub/pxe/CentOS7 $ sudo mount CentOS-7-x86_64-DVD-2009.iso /mnt/iso $ sudo cp -prv /mnt/iso/* /var/ftp/pub/pxe/CentOS7/ $ sudo umount /mnt/iso
Verify:
$ curl ftp://10.11.1.20/pub/pxe/CentOS7/ -rw-r--r-- 1 0 0 14 Oct 29 21:14 CentOS_BuildTag drwxr-xr-x 3 0 0 35 Oct 26 16:25 EFI -rw-rw-r-- 1 0 0 227 Aug 30 2017 EULA -rw-rw-r-- 1 0 0 18009 Dec 09 2015 GPL drwxr-xr-x 2 0 0 43 Oct 26 16:25 LiveOS drwxr-xr-x 2 0 0 225280 Nov 04 11:30 Packages -rw-rw-r-- 1 0 0 1690 Dec 09 2015 RPM-GPG-KEY-CentOS-7 -rw-rw-r-- 1 0 0 1690 Dec 09 2015 RPM-GPG-KEY-CentOS-Testing-7 -r--r--r-- 1 0 0 2883 Nov 04 11:36 TRANS.TBL drwxr-xr-x 3 0 0 57 Oct 26 16:26 images drwxr-xr-x 2 0 0 198 Nov 02 16:17 isolinux drwxr-xr-x 2 0 0 4096 Nov 04 11:35 repodata
Create Kickstart File
Kickstart files contain answers to all questions normally asked by CentOS installation program. By providing a prepared kickstart file when the installation begins we can perform deployments automatically.
This is the kickstart file /var/ftp/pub/pxe/centos7-ks.cfg
that I use for my CentOS 7 servers (a 16GB disk required).
#version=CentOS7 # Install OS instead of upgrade install # System authorisation information auth --enableshadow --passalgo=sha512 # Use network installation url --url="ftp://10.11.1.20/pub/pxe/CentOS7" # Use graphical install graphical # Keyboard layouts keyboard --vckeymap=gb --xlayouts='gb' # System language lang en_GB.UTF-8 # SELinux configuration selinux --enforcing # Firewall configuration firewall --enabled --ssh firstboot --disable # Network information network --bootproto=dhcp --device=eth0 --nameserver=10.11.1.2,10.11.1.3 --noipv6 --activate # Reboot after installation reboot ignoredisk --only-use=vda # Root password rootpw --iscrypted $6$7YZ0gnLkLPrl6rRO$NTjTQx1nesw5JLjtiAVdn3UBSbahUBGDFSiGGfrMNfGBum5aFs.TQcNX1SEuoWX/TmQ/ZMfiMnyHDs9uu9VH9. # System services services --enabled="chronyd" # System timezone timezone Europe/London --isUtc # System bootloader configuration bootloader --location=mbr --timeout=1 --boot-drive=vda # Clear the Master Boot Record zerombr # Partition clearing information clearpart --all --initlabel # Disk partitioning information part /boot --fstype="xfs" --ondisk=vda --size=1024 --label=boot --asprimary part pv.01 --fstype="lvmpv" --ondisk=vda --size=15359 volgroup vg_os pv.01 logvol /tmp --fstype="xfs" --size=1024 --label="lv_tmp" --name=lv_tmp --vgname=vg_os logvol / --fstype="xfs" --size=14331 --label="lv_root" --name=lv_root --vgname=vg_os %packages @^minimal @core chrony %end %addon com_redhat_kdump --disable --reserve-mb='auto' %end %anaconda pwpolicy root --minlen=6 --minquality=1 --notstrict --nochanges --notempty pwpolicy user --minlen=6 --minquality=1 --notstrict --nochanges --emptyok pwpolicy luks --minlen=6 --minquality=1 --notstrict --nochanges --notempty %end
Copy the content and save it as /var/ftp/pub/pxe/centos7-ks.cfg
.
Install Syslinux and Configure PXE Boot Menu
The Syslinux project covers lightweight bootloaders for network booting (PXELINUX) and more.
Install syslinux package:
$ sudo yum install syslinux -y
Copy syslinux boot loaders to the tftp server’s boot directory:
$ sudo cp -prv /usr/share/syslinux/* /var/lib/tftpboot/
Copy initrd.img
and vmlinuz
from the CentOS 7 installation media to /var/lib/tftpboot/networkboot/CentOS7/
:
$ sudo mkdir -p /var/lib/tftpboot/networkboot/CentOS7 $ sudo cp -pv /var/ftp/pub/pxe/CentOS7/images/pxeboot/{initrd.img,vmlinuz} /var/lib/tftpboot/networkboot/CentOS7/
Create the PXE configuration directory:
$ sudo mkdir -p /var/lib/tftpboot/pxelinux.cfg
Create the PXE boot configuration file /var/lib/tftpboot/pxelinux.cfg/default
with the following content:
default menu.c32 prompt 0 timeout 30 menu title Homelab PXE Menu label Install CentOS 7 Server kernel /networkboot/CentOS7/vmlinuz append initrd=/networkboot/CentOS7/initrd.img inst.repo=ftp://10.11.1.20/pub/pxe/CentOS7 ks=ftp://10.11.1.20/pub/pxe/centos7-ks.cfg
A Note About DHCP next-server
DHCP server has to be configured to use 10.11.1.20 as the next-server, e.g.:
allow booting; allow bootp; next-server 10.11.1.20; # TFTP filename "pxelinux.0";
For DHCP server configuration, see here.
Perform a Network Boot
Create a VM that uses Network Boot (PXE). Start the VM and observe the automated process.
You should do this with UEFI as well.
That’s a good point actually. My workstations that I run hypervisors on are 9 years old. I simply did not have a case for UEFI.
how to add Ubuntu image with anaconda cfg file?
is it work?
For Ubuntu 18.04 (and possibly others), you need to use NFS. Ubuntu Desktop uses casper to boot into Live DVD mode. Casper supports network boot via NFS only. As a result, in order to boot Ubuntu 18.04 and any Ubuntu derivatives via PXE, we need to have a fully functional NFS server accessible over the network. See Ansible role here:
https://github.com/lisenet/homelab-ansible/tree/master/roles/hl.pxe-boot-server
If you want to deploy with anaconda, add a seed file. See example below:
https://github.com/lisenet/homelab-ansible/blob/master/roles/hl.pxe-boot-server/templates/ubuntu1804-desktop.seed.j2
can you up guide how to do that with UEFI Support?
I don’t use UEFI, sorry.
are you have guide to installation kickstart with uefi?
I’m afraid I don’t. My homelab hardware is nearly 10 years old (I use decommissioned business hardware in order to save costs), it was shipped with a BIOS only firmware option. I could check whether subsequent firmware updates introduced the option for UEFI, but I wouldn’t count on that.
can you explaine me what i need to add for uefi in configuration?
Have a look here for how to create a kickstart file which will be used for both UEFI and legacy boots: https://access.redhat.com/solutions/3081331
You can sign up for a free-of-charge Red Hat developer subscription if you don’t have one yet.
can you tell me please how to configure the PXE server over a Rocky 8.6 machine ?
Hi Ibrahim, please use instructions provided in this article.
Alternativelly, you can use the following Ansible role from my homelab:
https://github.com/lisenet/homelab-ansible/tree/master/roles/hl.pxe-boot-server
Hi Lisenet,
thanks for your update,
actually i’ve did what you mentioned here and build the PXE server on a rocky VM
and the other VM was able to detect the PXE server
but i’m facing an issue now that
curl ftp://10.11.1.20/pub/pxe/CentOS7/
isn’t working and i’m unable to upload the image to the other VM’s too
do you have any idea about this issue ?
and i keep getting this alert when run the curl command :
curl: (8) Got a 500 ftp-server response when 220 was expected
Can you post your FTP server logs please?
I followed every step, and ensure everything was done the way you have it listed.
When I get to the step to test ftp:
$ curl ftp:///pub/pxe/Rocky8
I get the following:
curl: (8) Got a 500 ftp-server response when 220 was expected