Essential and Useful docker logs command Tips Revealed

Introduction to the `docker logs` Command

Docker, as a platform, facilitates containerization, allowing developers and system administrators to encapsulate applications within isolated environments known as containers. To understand the behavior, performance, and potential issues of these containers, accessing their logs becomes crucial. The primary tool for this purpose in the Docker ecosystem is the docker logs command.

How the Command Works

At its core, the docker logs command retrieves logs from a specified container. This command works by targeting the container’s ID or name:

docker logs [CONTAINER_NAME_OR_ID]

Additionally, there are numerous options and flags that can be paired with the command to refine the output, such as:

-f or --follow: Stream the logs in real-time.
--since: Show logs since a specific time.
--tail: Display only the last N lines of logs.
--timestamps: Show timestamps in the log output.

Logging Drivers in Docker

Docker supports multiple logging mechanisms through the concept of logging drivers. A logging driver is responsible for handling and forwarding the logs produced by a Docker container. When you start a Docker container, you can specify which logging driver to use.

Here’s a brief overview:

Default json-file Driver:

Description: This is the default logging driver for Docker. When using this driver, logs are written in a JSON format to a file on the host machine.
Benefits: Easy to use and doesn’t require any additional setup. It’s beneficial for simple use cases or development environments.
Limitations: The log files can grow indefinitely, potentially consuming all available disk space on the host. Some rotation or cleanup might be needed.

Supported Logging Drivers:

syslog: Sends logs to the syslog daemon. Syslog is a standard for message logging on Unix systems.
journald: Sends logs to the journald daemon, which is used by systemd.
gelf (Graylog Extended Log Format): Sends logs to a Graylog instance using the GELF protocol.
fluentd: Sends logs to Fluentd, an open-source data collector.
awslogs: Sends logs to Amazon CloudWatch Logs.
Others: Docker also supports other logging drivers like splunk, etwlogs (for Windows Event logging), gcplogs (for Google Cloud Platform logging), and more

Using a Logging Driver:

When running a container, you can specify which logging driver to use with the --log-driver option:

docker run --log-driver=syslog my-image

Choosing a Logging Driver:

The best logging driver often depends on the specific requirements of your infrastructure, the tools you’re using, and your preferred logging strategy. For instance, if you’re fully invested in the AWS ecosystem, awslogs might be your driver of choice. On the other hand, if you’re using the ELK (Elasticsearch, Logstash, Kibana) or EFK (Elasticsearch, Fluentd, Kibana) stack for centralized logging, you might opt for fluentd or gelf.

Accessing Container Logs

When running applications or services in Docker containers, logging becomes essential for monitoring, debugging, and ensuring proper functioning. Docker provides an intuitive command for fetching these logs.

Using the `docker logs` Command

The primary command to access logs of a container is docker logs. It’s used as follows:

docker logs [OPTIONS] CONTAINER

Where CONTAINER can be either the container’s ID or its name.

Supported Options

Tail with --tail:

Shows only the last n lines of logs, which can be useful if you’re only interested in recent log entries. This is especially beneficial if a container has been running for a long time and has amassed a large volume of logs.

docker logs --tail 100 CONTAINER

This will show the last 100 lines of logs from the specified container.

Follow Logs with -f or --follow:

This option allows you to stream the logs in real-time. As new log entries are added, they’re displayed on your terminal. This is similar to the tail -f command in UNIX systems.

docker logs -f CONTAINER

docker logs --follow CONTAINER

You can combine this with the --tail option to start streaming from a specific point in the log. For instance, docker logs -f --tail 10 CONTAINER will display the last 10 lines and then continue to stream new entries.

Timestamps with --timestamps or -t:

If you want to see timestamps alongside each log entry, which can be very valuable for determining the exact timing of events, use the --timestamps option.

docker logs --timestamps CONTAINER

docker logs -t CONTAINER

The output will include the date and time for each log entry.

Combining Options:

You can combine these options to get more refined log outputs. For example, if you wish to view the last 50 lines of logs, stream new logs, and display timestamps, you’d use:

docker logs --tail 50 -f -t CONTAINER

Managing Log Files in Docker

Properly managing log files is critical in a containerized environment, not only for performance but also for maintaining an efficient disk space usage. Let’s delve into how Docker manages log files and why it’s essential to keep logs in check.

1. How Docker Manages Log Files

Log Storage: By default, Docker uses the json-file logging driver, storing logs in a JSON format on the host filesystem. For each container, logs are usually stored in a location like /var/lib/docker/containers/[container-id]/[container-id]-json.log.
Log Drivers: Docker supports various logging drivers (syslog, journald, fluentd, and more) that can redirect logs to different destinations, potentially reducing the size of log files on the host system.

2. Log Rotation and Retention Policies

Native Log Rotation: Docker provides native support for log rotation with the json-file and journald logging drivers. To set up log rotation, you need to configure the --log-opt parameter when starting the Docker daemon or in the Docker configuration file.For example, for json-file:

--log-opt max-size=10m --log-opt max-file=5

This configuration will ensure that each log file doesn’t exceed 10MB, and it retains a maximum of 5 rotated log files. When the limit is reached, the oldest file is removed.

Filtering and Formatting Logs in Docker

When working with Docker, especially in environments with multiple containers or high levels of activity, the ability to filter and format logs is crucial. Properly filtered and formatted logs can provide clearer insights, faster debugging, and improved log readability.

1. Filtering Logs by Container:

The primary method to view logs of a specific container is by using the docker logs command followed by the container name or ID:

docker logs CONTAINER_NAME_OR_ID

However, if you’re using a centralized logging solution or have logs from multiple containers aggregated together, you’d rely on the mechanisms provided by that solution to filter logs by container. For example, if using a logging tool like Fluentd or Logstash, you’d set up filters based on container metadata.

2. Customizing Log Output Formats:

Customizing log output format is especially relevant when you’re using a logging driver that sends logs to an external system. You might want the logs in a specific format to integrate better with your logging tools or to provide clearer information.

With Docker, the --log-opt flag lets you customize the log output when using certain logging drivers:

Example with json-file driver:

The json-file driver lets you control log file size and log file count, but it doesn’t allow customization of the log message format. The log messages are always in JSON format.

Example with syslog driver:

If you’re using the syslog driver, you can customize the log format using the syslog-format option:

docker run --log-driver=syslog --log-opt syslog-format=rfc5424micro my_container_image

The syslog-format option can have values like rfc3164 (default), rfc5424, or rfc5424micro that correspond to different syslog message formats.

Example with fluentd driver:

With the fluentd driver, you can customize tags, which can be helpful in formatting and categorizing log messages:

docker run --log-driver=fluentd --log-opt tag="docker.{{.Name}}" my_container_image

The tag option can use variables like {{.Name}} to include container-specific information.

Conclusion

In this article, we have learned about viewing the logs of the docker containers. I hope this article makes it clear to you and I suggest everyone try it out.

In case you have any doubts, please feel free to add your questions in the comment section below and I shall try to respond at the earliest.

Happy learning!!

References

https://docs.docker.com/config/containers/logging/
https://stackoverflow.com/questions/66360898/docker-tail-n-not-showing-correct-no-of-lines

Introduction to the docker logs Command