The latest innovation in Red Hat Enterprise Linux (RHEL) was the introduction of picture mode just over 90 days ago, however, working with image-based operations is not a new concept. Most of our daily devices update using images. The container revolution brought image-based updates to the world of applications. In the realm of desktops, image-based deployment and updates are now quite common. Except for certain areas like high-performance computing (HPC), image-based operations for server operating systems (OS) have not been widely adopted. While the tools may not have been widely used, I believe the hesitation has been on our part. No, not Red Hat, but us, the practitioners.

Pioneers of the past

Why was it named “picture mode” to begin with? According to Oxford, “mode” means “a specific way of doing something.” This new introduction is simply that, a specific way of managing a RHEL host but utilizing an image-based workflow instead of the traditional RPM package-based workflows.

Image-based operations have been around for at least 25 years, dating back to when I first encountered a commercially viable data center virtualization platform at LinuxWorld in San Jose. Virtualization essentially entails image-based disks with accompanying metadata to inform the system of the virtual machine (VM) capabilities to create. Besides a few notable features such as snapshots and cloning, we treated VMs as if they were physical servers. Consequently, the focus shifted from image operations towards the standard server-based tools we were accustomed to. This transition made sense due to the inherent challenges surrounding change, such as political factors, budget constraints, and time limitations.

Another opportunity to engage with image-focused operations for servers emerged with the advent of cloud computing. Reflecting on the past, many opponents of cloud computing based their arguments on maintaining traditional data center setups and were reluctant to embrace a new paradigm centered around ephemeral, image-based infrastructure. A decade ago, I delved into a workshop at OSCON to grasp how Netflix had established cloud infrastructure utilizing their AMI Bakery tools like AMInator, Asgard, and the Simian Army. These tools were open-source, highly automated, and heavily focused on images.

However, change is challenging, thereby leading us to advocate for familiar tools and techniques that allowed us to leverage our existing server-centric processes. Sadly, this shift was not as successful as the push for virtualization since we transitioned from controlling the environment to merely consuming it. For most individuals, their cloud operations mirrored their traditional data centers unless they had the chance to embark on a fresh project with a novel approach. As a result, a new set of challenges arose, characterizing the world in which most of us currently operate.

This retrospective brings to light a crucial question: if conventional methods are still functional, why should one consider a change? This is precisely what I aim to examine here. While there are alternative resources available that can guide you in experimenting with image mode for RHEL, I would like to shed light on why adopting a novel perspective could prove advantageous.

Benefits in store

There are numerous advantages to be gained from embracing a picture mode mentality, but here are a few that resonate with me.

Seamless updates

Single transactions offer greater safety compared to those combining multiple components. Recently, I spent half an hour updating my Fedora laptop. It wasn’t the volume of packages nor any catastrophic failures; I simply had to run it twice. Why? A momentary glitch. The software itself was functioning well. DNF (Fedora’s package manager) executed its task correctly. A single package installation error resulted in the entire transaction failing. The hiccup might have been caused by a WiFi interruption, a sudden spike in CPU activity, or a surge in disk usage. Regardless, it was purely environmental and transient. I reinitiated the update, and everything progressed smoothly. After rebooting, I resumed my tasks.

Had I been utilizing something with image-based updates, the update would have been compiled from my system and presented as a unified update to download, prepare, and activate upon the subsequent reboot. Thirty minutes on a particular day may appear inconsequential, but let’s extrapolate this to a cluster of 100 production servers during a maintenance window. Assuming each DNF transaction consumes 10 minutes, troubleshooting requires 5 minutes, and reboots last 5 minutes to restore full functionality. This process isn’t entirely sequential, hence, we are not discussing 16+ hours, but each 15-minute interval spent troubleshooting and re-running an update accumulates rapidly. What if you realize you need to create additional space in /var just to download the packages on one system? How time-consuming would that task be, especially when operating within the confines of a service level agreement (SLA)?

Enhanced recovery and transparency

Suppose an issue arises not from the update process itself but from the updates that were installed. Let’s presume one of the updates involved a new version of OpenSSL which inadvertently disrupted SSL certificates in the production environment without affecting development. Do you have a contingency plan in place? Do you attempt to troubleshoot SSL reconfiguration in a production setting? Is the issue tied to your application? Image-based systems incorporate a safety net: the rollback option. Since these systems consist of complete disk images, reverting to a previously functional installation is feasible if required, permitting further troubleshooting. Moreover, armed with the knowledge of the specific image powering the production system, you can replicate an exact copy in a development environment for diagnostic purposes. This approach allows prompt identification of the root cause, be it the system installation, application configurations, component settings, or other factors.

Image-based operations offer the additional advantage of swiftly determining the exact software versions installed on any host. Drift occurs when minor modifications are made locally. Drift arises when package versions lack stringent control across environments. Drift emerges when we commence with a standard build but subsequently incorporate numerous changes to support diverse applications without establishing downstream standards. Image-based systems track a known entity. With the appropriate tools, you can not only visualize the software components on a host but also inspect the contents of a specific image.

Streamlined update procedures, rollback capabilities, and traceability are valuable features for control, albeit slightly defensive. While these aspects contribute to operational efficiency, are they adequate? Can image-based systems accelerate our pace?

Expedited experimentation

If the process of swapping out entire operating systems for updates is simplified, it can be extended to altering the role of a host for various purposes. One of the largest server clusters I managed was dedicated to the development environment. We had to sustain multiple parallel stacks to support various experiments related to the applications we developed. Some experiments focused on application-level modifications, others delved into component-level changes such as exploring new Java versions. Each of these undertakings necessitated new hardware and complete installations of the application and data. What if, instead, you could deploy a new OS version incorporating the new component and seamlessly integrate it into an existing application stack? This approach would enable you to conduct A/B testing on the latest crucial changes to your PHP application with a mere reboot of the application server.

Diverse possibilities

In line with A/B testing or blue/green deployments, you could introduce features gradually into production through controlled experiments, much like the deployment of applications today. You could construct a sophisticated array of operational models that are notably challenging to execute with most package-based operating systems.

These represent just a fraction of the myriad reasons driving my enthusiasm for this new phase of image-based operations. Over the past 25 years, tools have made significant advancements, and image mode for RHEL mirrors a multitude of contemporary infrastructure solutions at our disposal currently. Imagine the possibilities that open up when you adopt a novel perspective towards your operating systems?

What lies ahead?

If the prospects of image mode for RHEL resonate with you, I urge you to delve deeper into this approach. Additionally, by incorporating containers to oversee image creation and curation, you can elevate this methodology beyond the boundaries of current image-based operations.

Additional resources for image mode

If you wish to explore further into image mode, why not embark on our quick start guide or engage in one of the learning exercises? Alternatively, you can embark on a self-guided exploration from any subscribed RHEL system equipped with podman.