observability

Logrotate in Linux: Configuration and Troubleshooting

Sébastien Puyet

Sébastien Puyet

— 10 min read
Logrotate in Linux: Configuration and Troubleshooting

Logrotate manages Linux log files by rotating them, compressing older copies, and removing archives according to your configuration. This guide covers /etc/logrotate.conf, application rules in /etc/logrotate.d/, daily and size-based rotation, and the checks to run when logs are not rotating.

Logrotate runs when a scheduler invokes it, usually through cron or logrotate.timer on systemd systems. It does not continuously watch file sizes: a rule is evaluated only when logrotate runs. Start with the configuration example below, test it in debug mode, then verify the scheduler and the application’s log-reopening behavior.

What is Logrotate and Why You Need It

Logrotate solves the fundamental problem of unbounded log growth. Without proper log management, applications and services will continue writing to log files indefinitely, eventually consuming all available disk space. This leads to system crashes, failed deployments, and potential data loss.

To catch disk space issues before they become critical, set up monitoring disk space on Linux with automated alerts.

With normal rename-and-create rotation, the active log becomes an archive and the application must write to the new file. Archives are usually numbered; dateext uses date suffixes instead. The rotate directive controls the number of retained archives, while maxage adds an age limit checked during rotation.

Key benefits include:

  • Disk space management: Prevents log files from consuming all available storage
  • Performance optimization: Keeps active log files small for faster I/O operations
  • Compliance support: Maintains logs for required retention periods while automatically purging old data
  • Operational reliability: Reduces risk of disk-full incidents that can crash services

However, logrotate isn't always the right solution. In containerized environments where logs are streamed to stdout/stderr, or when using centralized logging systems like ELK stack or Splunk, log rotation may be handled by other components. Modern cloud-native applications often rely on log aggregation services rather than local file rotation.

Quick Start: Get Logrotate Working in 5 Minutes

Most Linux distributions include logrotate by default. Verify your installation and check the version:

logrotate --version

If logrotate isn't installed, install it using your package manager:

# Ubuntu/Debian
sudo apt update && sudo apt install logrotate

# RHEL/CentOS/Fedora
sudo yum install logrotate

Let's create a simple configuration for a custom application log. Create a new file in the logrotate configuration directory:

sudo nano /etc/logrotate.d/myapp

Add this basic configuration:

/var/log/myapp/*.log {
    daily
    rotate 7
    compress
    delaycompress
    missingok
    notifempty
    create 0644 www-data www-data
}

Replace www-data with the user and group that write your logs, and choose permissions appropriate for their contents. This example creates a new file but does not tell a running application to reopen it; use that application’s documented reopen command in a postrotate block, as shown in the Nginx example below. The directives mean:

  • daily - Rotate when the daily interval is due and logrotate is invoked
  • rotate 7 - Keep seven rotated archives; this is not necessarily seven days
  • compress - Compress old log files with gzip
  • delaycompress - Wait one rotation cycle before compressing
  • missingok - Don't error if log files are missing
  • notifempty - Don't rotate empty files
  • create 0644 www-data www-data - Create new log files with specific permissions and ownership

Test your configuration without actually rotating files:

sudo logrotate -dv /etc/logrotate.d/myapp

-d enables debug mode without changing logs or the state file; -v requests verbose output. Running a snippet directly does not load defaults from /etc/logrotate.conf, so also check the complete configuration with sudo logrotate -d /etc/logrotate.conf before relying on the scheduled run.

For an intentional test that changes files and executes rotation hooks, use -f below. Check the application’s reopen command first and use a test environment where possible:

sudo logrotate -vf /etc/logrotate.d/myapp

Understanding Logrotate Configuration Structure

Logrotate uses a hierarchical configuration system with global defaults and service-specific overrides. The main configuration file is /etc/logrotate.conf, which sets system-wide defaults and includes service-specific configurations from /etc/logrotate.d/.

The main configuration file typically looks like this:

# Global options
weekly
rotate 4
create
dateext
compress

# Include service-specific configurations
include /etc/logrotate.d

# System-specific logs
/var/log/wtmp {
    monthly
    create 0664 root utmp
    rotate 1
}

Keep application-specific rules in /etc/logrotate.d/ and reserve /etc/logrotate.conf for global defaults and includes. This separation provides several advantages:

  • Package management: Software packages can install their own logrotate configurations
  • Maintainability: Changes are isolated and easier to track
  • Override capability: Service-specific settings override global defaults

Configuration files use a simple syntax where log file paths are followed by configuration blocks in curly braces. You can specify multiple files or use wildcards:

# Single file
/var/log/myapp.log { ... }

# Multiple specific files
/var/log/app1.log /var/log/app2.log { ... }

# Wildcard pattern
/var/log/myapp/*.log { ... }

Settings in service-specific files override global defaults. For example, if the global configuration sets weekly rotation but your service configuration specifies daily, daily rotation takes precedence for that service.

Essential Logrotate Directives and Options

The upstream logrotate manual documents each directive. Size limits are checked when logrotate runs; they are not hard caps on disk usage between scheduled runs.

Rotation Triggers

  • daily, weekly, monthly - Time-based rotation intervals
  • size 100M - Rotate above the size threshold; size and time-interval directives override one another, with the last specified taking precedence
  • minsize 10M - Rotate based on time, but only if file exceeds minimum size
  • maxsize 500M - Allow rotation above this size before the time interval is due, on the next invocation

File Management

  • rotate 30 - Number of old log files to keep
  • compress - Compress rotated files with gzip
  • delaycompress - Compress files on the next rotation cycle
  • create 0644 user group - Create new log file with specified permissions
  • copytruncate - Copy and then empty the same file when the application cannot reopen its log; some entries can be lost

Error Handling

  • missingok - Continue if log files don't exist
  • notifempty - Don't rotate empty files
  • ifempty - Rotate even if files are empty (opposite of notifempty)

Prefer rename-and-create rotation with the application’s supported reopen signal or command. Use copytruncate only when reopening is unavailable and the risk of lost entries is acceptable: writes between the copy and truncate steps can disappear. With copytruncate, create has no effect because the original file remains in place.

Script Integration

You can execute scripts before and after rotation:

/var/log/nginx/*.log {
    daily
    rotate 30
    compress
    delaycompress
    prerotate
        if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
            run-parts /etc/logrotate.d/httpd-prerotate; \
        fi
    endscript
    postrotate
        systemctl reload nginx
    endscript
}

Step-by-Step Configuration Examples

Web Server Logs (Nginx)

Web servers generate high-volume access and error logs. Here's a complete Nginx configuration:

/var/log/nginx/*.log {
    daily
    rotate 52
    compress
    delaycompress
    missingok
    notifempty
    create 0644 www-data adm
    sharedscripts
    postrotate
        if [ -f /var/run/nginx.pid ]; then
            kill -USR1 `cat /var/run/nginx.pid`
        fi
    endscript
}

The sharedscripts directive ensures the postrotate script runs only once, even when multiple log files are rotated. The postrotate script sends a USR1 signal to Nginx, instructing it to reopen log files.

Application Logs with Service Restart

Some applications require a full restart to recognize new log files:

/var/log/myapp/application.log {
    weekly
    rotate 12
    compress
    delaycompress
    missingok
    notifempty
    create 0640 myapp myapp
    postrotate
        systemctl restart myapp
    endscript
}

High-Volume Database Logs

Database logs can grow rapidly and may need size-based rotation:

/var/log/mysql/mysql.log /var/log/mysql/error.log {
    size 100M
    rotate 10
    compress
    delaycompress
    missingok
    notifempty
    create 0640 mysql mysql
    postrotate
        mysqladmin flush-logs
    endscript
}

Advanced Logrotate Configuration

Container Environments

For Docker’s own container logs, use the logging driver’s rotation settings. Do not apply logrotate or copytruncate to Docker-managed *-json.log files; Docker documents these files as exclusive to the daemon. If you need to keep json-file, merge settings such as these into /etc/docker/daemon.json:

{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "20m",
    "max-file": "5"
  }
}

Preserve other daemon settings when merging this example. Restart Docker during a planned maintenance window; existing containers must be recreated to adopt the new logging options. Docker also recommends the local driver for many deployments because it rotates logs by default. Application-owned plain-text logs in mounted volumes are a separate case and can use an application-specific logrotate rule.

High-Frequency Rotation

For extremely high-volume logs, you might need hourly rotation. Create a custom cron job:

Need help writing cron schedules? Use our cron expression generator to build the right timing for your rotation jobs.

# Add to /etc/cron.hourly/logrotate-hourly
#!/bin/bash
/usr/sbin/logrotate /etc/logrotate.d/high-volume-app

Configure the logrotate file for hourly execution:

/var/log/high-volume-app/*.log {
    size 1G
    rotate 24
    compress
    delaycompress
    missingok
    notifempty
    copytruncate
}

Network Filesystems

For logs on NFS or another network filesystem, verify permissions, rename behavior, and the application’s ability to reopen files. Network storage alone is not a reason to enable shred: it changes how expired files are deleted, not how rotation is coordinated. The following fallback uses copytruncate only for an application that cannot reopen its log, with the data-loss tradeoff described above:

/mnt/nfs/logs/*.log {
    daily
    rotate 30
    compress
    delaycompress
    missingok
    notifempty
    copytruncate
}

Security and Permissions Best Practices

Proper security configuration prevents unauthorized access to sensitive log data and ensures logrotate can perform its operations.

File Permissions

Always specify appropriate permissions in the create directive:

# Restrictive permissions for sensitive logs
/var/log/auth.log {
    create 0600 root root
}

# Web server logs accessible to web group
/var/log/apache2/*.log {
    create 0640 root adm
}

SELinux Considerations

On SELinux-enabled systems, logrotate may encounter permission denials. Check for SELinux alerts:

sudo ausearch -m AVC -ts recent | grep logrotate

Common solutions include setting appropriate SELinux contexts or creating custom policies for non-standard log locations.

User Privilege Management

Use the su directive when logrotate needs to run with different privileges:

/var/log/myapp/*.log {
    su myapp myapp
    daily
    rotate 7
    compress
    create 0644 myapp myapp
}

Testing and Validation Techniques

Thorough testing prevents production issues and ensures your rotation strategy works as expected.

Command-Line Testing

Use these flag combinations for different testing scenarios:

# Test configuration syntax without rotation
sudo logrotate -d /etc/logrotate.conf

# Verbose debug output for specific config
sudo logrotate -dv /etc/logrotate.d/myapp

# Force actual rotation for testing
sudo logrotate -vf /etc/logrotate.d/myapp

Status File Monitoring

Logrotate maintains a state file for rotation timing. Its default path varies by distribution and can be overridden with -s; check logrotate --version and the scheduled command. On systems using /var/lib/logrotate/status, inspect it with:

sudo cat /var/lib/logrotate/status

This file helps troubleshoot rotation timing issues and verify that rotation is occurring as expected.

Validation Workflow

  1. Test configuration syntax with -d flag
  2. Create test log files with sample content
  3. Force rotation with -f flag
  4. Verify rotated files exist with correct permissions
  5. Check that services restart properly (if using postrotate scripts)
  6. Monitor for several rotation cycles to ensure consistency

Common Issues

Rotation Not Happening

First identify the scheduler your distribution actually uses. On a systemd installation, inspect the timer and the service journal. If the distribution uses cron instead, inspect its installed logrotate job and cron logs:

# For a systemd timer
systemctl list-timers --all logrotate.timer
systemctl status logrotate.timer
sudo journalctl -u logrotate.service --since "24 hours ago" --no-pager

# For a cron-based installation
ls -l /etc/cron.daily/logrotate
# Cron service names and log locations vary by distribution.

Permission Problems

Common permission issues include:

# Check file ownership and permissions
ls -la /var/log/myapp/

# Verify logrotate can write to log directory
sudo -u root touch /var/log/myapp/test-file

# Review SELinux denials (if applicable)
sudo sealert -a /var/log/audit/audit.log

Service Restart Failures

Debug postrotate script issues:

# Test postrotate commands manually
sudo systemctl reload nginx

# Add error handling inside the application rule
postrotate
    if ! systemctl reload nginx; then
        logger "Failed to reload nginx after logrotate"
        exit 1
    fi
endscript

Disk Space Issues

When rotation fails due to insufficient disk space:

Use our disk space calculator to plan storage needs based on your log volume and retention settings.

# Check the filesystem containing the logs
df -h /var/log

# Inspect old archives for this application; do not delete them blindly
sudo find /var/log/myapp -type f -name "*.gz" -mtime +30 -print

# Confirm retention requirements before changing the application rule.

Monitoring and Maintaining Logrotate

Proactive monitoring ensures logrotate continues working reliably in production environments.

Key Metrics to Monitor

  • Rotation success rate: Track failed rotations via syslog
  • Disk usage trends: Monitor log directory sizes
  • Rotation timing: Verify rotations occur at expected intervals
  • Service restart success: Monitor postrotate script execution

Checking Scheduled Runs

On systems using logrotate.service, inspect the latest service result and its recent journal. A recent state-file timestamp alone is not proof that every rotation succeeded. These commands expose evidence for diagnosis; an empty journal is not an all-clear:

# Inspect the last service execution
systemctl show logrotate.service \
  -p Result -p ExecMainStatus -p ExecMainStartTimestamp

# Review messages from the last 24 hours
sudo journalctl -u logrotate.service --since "24 hours ago" --no-pager

Rotation rules and disk monitoring solve different parts of the problem. With Fivenines Linux server monitoring, you can track partition usage and set threshold alerts before available space becomes critical. Use the scheduler’s results and your application checks to verify rotation itself.

For a complete overview of what to track beyond log rotation, see our guide on monitoring your Linux servers.

Performance Optimization and Scaling

Large log files and high-volume environments require optimization to prevent logrotate from impacting system performance.

I/O Optimization

For large files, consider these strategies:

  • Use copytruncate judiciously: It requires reading the entire file, which can be slow
  • Schedule rotation during low-activity periods: Move logrotate cron jobs to off-peak hours
  • Implement size-based rotation: Prevent files from becoming too large

Compression Strategies

Balance compression ratio against CPU usage:

# Use faster compression for high-volume logs
/var/log/high-volume/*.log {
    compress
    compresscmd /bin/gzip
    # Fastest gzip compression
    compressoptions -1
    compressext .gz
}

Avoid Overlapping Rotation Jobs

Keep the normal scheduled run responsible for the included application rules. Concurrent logrotate processes sharing a state file contend for its lock; a second process can exit without rotating anything. If separate schedules are necessary, ensure each log has one owner and plan state files deliberately.

# Validate the combined configuration before changing schedules
sudo logrotate --debug /etc/logrotate.conf

Integration with Modern Log Management

Modern infrastructure often combines traditional log rotation with centralized logging and cloud-based solutions.

Log Shipper Coordination

Configure your shipper to follow both active and rotated files. For Filebeat, use a stable filestream input ID and matching paths, as described in the Filebeat rotation guidance. The application writing the log must still reopen it; reloading Filebeat is not a substitute. Confirm compressed-file support for your installed Filebeat version before choosing a compression policy.

filebeat.inputs:
  - type: filestream
    id: application-logs
    paths:
      - /var/log/app/*.log*

Cloud Integration

Archive completed, compressed logs in a separate job with checked upload results and an explicit retention policy. A postrotate hook runs before compression, so it cannot assume the newly rotated .gz file exists. For example, copy one finished archive to a unique object key, then verify the uploaded object before any local cleanup:

# Replace the bucket and date with your archive destination
aws s3 cp /var/log/archive/app.log.2.gz \
  s3://YOUR-BUCKET/logs/app/2026-09-23/app.log.2.gz

Hybrid Architectures

In hybrid environments, maintain local rotation for immediate troubleshooting while shipping logs to centralized systems for long-term analysis. This approach provides both real-time local access and comprehensive historical analysis capabilities.

With proper configuration and monitoring, logrotate becomes an invisible but essential component of your infrastructure, ensuring systems remain stable and performant while maintaining the log data you need for troubleshooting and compliance.

Read more