A Guideline for Stream Management for Timber Control
Docker is a remarkable instrument for promptly establishing and launching the necessary applications. Nonetheless, we frequently disregard a critical element of its functionality: the recordings it generates.
Why do I articulate that? Recently, I stumbled upon the fact that one of my servers was depleting its available disk space, nearly reaching a crucial minimum. After examining, I unveiled that a Docker container had amassed a 14 GB log file in just a year.
To prevent encountering a similar scenario, this manual will support you in understanding how to inspect the dimensions of all Docker log files swiftly, condense them if necessary, and configure Docker to avert them from expanding excessively in the future.
Docker containers are transient, indicating their logs are eradicated when the container is regenerated. However, many neglect a crucial detail at this juncture. Consequently, let’s first clarify an essential point before advancing.
Initiate vs. Regenerate Docker Container
Let’s revisit the instance mentioned above. Over the course of the past year, the server has been rebooted multiple times, signifying that Docker containers have been halted and initiated again. Nevertheless, their log files have persisted and continued to escalate.
The reason why this occurred is that we are discussing restarting here and not regenerating the containers. When the server undergoes a reboot, the Docker daemon gracefully shuts down the operational containers and after startup, automatically relaunches them if stipulated to do so.
Similar to the scenario when manually restarting the Docker service employing the “systemctl restart docker” directive. It’s crucial to comprehend that Docker logs are exclusively reset when the Docker container is obliterated and regenerated. Halting and initiating the container solely do not reset the logs.
Having clarified that, let’s delve into the core of this guide and learn how to promptly inspect the magnitude of space Docker container logs exploit on our system.
Verifying Docker Log Dimensions
Docker logs are generally preserved (if you have not explicitly altered Docker’s default data directory) in the subsequent location on the host machine:
/var/lib/docker/containers/<container-id>/<container-id>-json.logLanguage: HTML, XML (xml)Every Docker container possesses its own folder under “/var/lib/docker/containers/.” Within each container’s folder, you’ll encounter a document concluding in “-json.log,” comprising the logs for that specific container.
By default, these files are structured in JSON format and capture the standard output (stdout) and standard error (stderr) streams of the container.
To ascertain the size of all Docker container logs sorted from the largest to the smallest, you can leverage the subsequent find directive:
find /var/lib/docker/containers/ -name "*json.log" | xargs du -h | sort -hrLanguage: Bash (bash)Superb, now we have all the log file magnitudes from our Docker containers. Nevertheless, how do we associate these IDs to the actual designations of the containers to discern which one generated each log file? Here’s the method.
Uncover Docker’s Unit Title by Its ID
Suppose we intend to pinpoint the Docker container accountable for birthing the largest log file mentioned earlier, sized at 5.9M. We’ll operate its ID, which is “d2e9228f92b66ac09fa35dcab36abba2eb4a7f46baa1d03b65d71ed8d42de977.”
The directive we necessitate is:
docker inspect --format='{{.Name}}' <container_id>Language: Bash (bash)Subsequently, insert the string including the container ID and insert it into the specified directive.
docker inspect --format='{{.Name}}' d2e9228f92b66ac09fa35dcab36abba2eb4a7f46baa1d03b65d71ed8d42de977Language: Bash (bash)And just like that, enchantment unfolds.
We’ve verified that the Docker container responsible for the log file is denoted as “php-fpm-valente” in our illustration.
To validate that this is indeed the accurate container that spawned the log file, you can authenticate this by utilizing the docker inspect <container_name> directive to retrieve additional particulars about it and validate that it’s the provenance of the log file. In our scenario, it would appear as follows:
docker inspect php-fpm-valente
We possess all the requisite information. Now, let me demonstrate how to purge all the contents of the designated log file, thereby liberating disk space.
Scrubbing Docker Container Logs
You can apply the truncate directive to eradicate all the contents of an individual log file. Just specify the whole path to the log file you wish to delete as a parameter. For instance:
truncate -s 0 /var/lib/docker/containers/d2e9228f92b66ac09fa35dcab36abba2eb4a7f46baa1d03b65d71ed8d42de977/d2e9228f92b66ac09fa35dcab36abba2eb4a7f46baa1d03b65d71ed8d42de977-json.logLanguage: JavaScript (javascript)
If you opt for a more drastic approach, you can erase all Docker log files with just one directive. However, ensure you comprehend what you’re executing before enacting the forthcoming directive.
truncate -s 0 /var/lib/docker/containers/*/*-json.logLanguage: Bash (bash)Establishing Docker Log Dimensions Restrictions
The methodology delineated previously offers a temporal solution for governing extensive Docker log files and freeing storage space. Nonetheless, it demands habitual utilization of specific directives, which might prove inconvenient.
A notably superior and enduring solution would be to restrict the utmost size of Docker log files. Once this magnitude is reached, the Docker daemon will automatically rotate those log files and archive them in compliance with naming norms such as “<container_id>-json.log.1,” “<container_id>-json.log.2,” etc., contingent on the demonstrated options.
This guarantees that the log files do not expand unbridledly. Here’s the approach.
To apply log rotation settings globally to all containers managed by the Docker daemon, you can modify the Docker daemon configuration file, typically situated at “/etc/docker/daemon.json.” If the file is absent, you can forge it by executing a directive akin to:
sudo nano /etc/docker/daemon.jsonSubsequently, paste the ensuing contents into the file, salvage it, and exit:
{
"log-driver": "json-file",
"log-opts": {
