Dec 21, 2025

How to Fix Terraform Too Many Open Files on Ubuntu 20.04

Troubleshooting Guide: Terraform “Too Many Open Files” on Ubuntu 20.04

As Senior DevOps Engineers, we often encounter resource limitations in dynamic environments. One common pitfall when working with Terraform, particularly on systems managing a large number of resources or using many providers, is the “Too Many Open Files” error. This guide addresses this specific issue on Ubuntu 20.04.

1. The Root Cause

The “Too Many Open Files” error (often manifested as Error: too many open files or a panic within a provider) occurs when a process attempts to open more file descriptors than the operating system allows it.

On Ubuntu 20.04 (and most Linux distributions), there are default limits on the number of file descriptors a single process can open. These are typically set to a relatively low value (e.g., 1024 or 4096).

Why Terraform Hits This Limit:

Provider Execution: Terraform heavily relies on provider plugins. Each provider often runs as a separate child process, and these processes, along with the main Terraform process, open files for various purposes:
- Communicating with remote APIs.
- Reading/writing state files.
- Managing temporary files.
- Inter-process communication (IPC) sockets for plugin communication.
- Reading configuration files.
Resource Count: As your infrastructure grows, the number of resources managed by Terraform increases. A single terraform plan or apply operation can involve hundreds or thousands of individual operations, each potentially requiring file descriptors.
Concurrent Operations: When Terraform performs concurrent operations (e.g., creating multiple resources in parallel), the demand for file descriptors can spike rapidly, quickly exhausting the default limits.

When these limits are breached, Terraform or one of its provider plugins will fail, reporting the “Too Many Open Files” error.

2. Quick Fix (CLI)

For immediate relief or testing, you can temporarily increase the open file limit for the current shell session. This is not persistent across reboots or new sessions.

Check Current Limit: To see the current soft limit for your session:
```
ulimit -n
```
Increase Limit: To increase the limit for the current shell session (e.g., to 65536, a common recommended value for heavy I/O applications):
```
ulimit -n 65536
```
Note: You might need to be root or have sudo privileges to set a very high ulimit if your hard limit is lower. The value you set for the soft limit cannot exceed the hard limit.
Execute Terraform: After setting the ulimit, execute your terraform commands from the same shell session:
```
terraform plan
# or
terraform apply
```

This temporary fix will allow you to complete your current Terraform operation. For a permanent solution, refer to the “Configuration Check” section.

3. Configuration Check

To permanently resolve the “Too Many Open Files” issue, you need to modify system-wide or user-specific configuration files.

A. User/Group Specific Limits (`/etc/security/limits.conf`)

This is the most common approach for setting persistent limits for specific users or groups.

Edit limits.conf: Open the file /etc/security/limits.conf with sudo:
```
sudo nano /etc/security/limits.conf
```
Add Limit Entries: Append the following lines to the end of the file. Replace terraform_user with the actual username running Terraform, or use * for all users (which is generally acceptable for nofile limits if not too extreme).
```
# <domain>      <type>  <item>         <value>
# Terraform user specific limits
terraform_user  soft    nofile         65536
terraform_user  hard    nofile         65536

# Or for all users (less specific, but often effective)
# *               soft    nofile         65536
# *               hard    nofile         65536
```
- domain: The user or group (e.g., myuser, @mygroup, * for all).
- type: soft (the current limit that can be increased by the user up to the hard limit) or hard (the maximum limit that cannot be exceeded by a non-root user).
- item: The resource to limit, nofile for the number of open files.
- value: The new limit.
Verify PAM Configuration: Ensure that the pam_limits.so module is enabled, which is responsible for applying these limits. This is typically already configured on Ubuntu 20.04. Check /etc/pam.d/common-session and /etc/pam.d/common-session-noninteractive for the line:
```
session required pam_limits.so
```
If it’s missing, add it.
Apply Changes: For these changes to take effect, the user must log out and log back in, or the system needs to be rebooted.

B. Systemd Service Limits (If Terraform Runs as a Service)

If you are running Terraform operations via a systemd service (e.g., a CI/CD runner executing terraform apply as part of a service), the limits in limits.conf might not directly apply to the service’s process.

Edit Systemd Service Unit: Locate the systemd service file (e.g., /etc/systemd/system/my-terraform-ci.service).
Add LimitNOFILE: Within the [Service] section, add or modify the LimitNOFILE directive:
```
[Service]
# ... other service configurations ...
LimitNOFILE=65536:65536  # soft:hard limit
```
You can also specify just a single value, e.g., LimitNOFILE=65536, which sets both soft and hard limits to that value.
Reload Systemd and Restart Service: Apply the changes by reloading the systemd daemon and restarting your service:
```
sudo systemctl daemon-reload
sudo systemctl restart my-terraform-ci.service
```

C. Kernel-Wide File Handle Limit (Less Common for this specific error)

While less frequently the direct cause of “Too Many Open Files” for a single process, if many processes on the system are collectively opening a vast number of files, the system-wide limit (fs.file-max) might be a bottleneck.

Edit sysctl.conf:
```
sudo nano /etc/sysctl.conf
```
Add/Modify fs.file-max: Add or modify the following line:
```
fs.file-max = 2097152 # Example: 2 million
```
Apply Changes: Apply the new kernel parameter without rebooting:
```
sudo sysctl -p
```
This change is persistent across reboots.

4. Verification

After implementing any of the above configuration changes, it’s crucial to verify that the new limits have been successfully applied.

Verify User Session Limit: After logging out and logging back in (if modifying limits.conf), open a new terminal and run:
```
ulimit -n
```
The output should reflect your newly configured soft limit (e.g., 65536).
Verify for a Running Process: If Terraform is already running or you want to check a background process:
- Find the Process ID (PID) of the Terraform process (or its parent process if Terraform is a child process):
```
pgrep -a terraform
```
  This will show PIDs and command lines. Note down the PID.
- Check the limits for that specific PID:
```
cat /proc/<PID>/limits
```
  Look for the Max open files line. It should show your configured soft and hard limits.
Test with Terraform: Execute the Terraform command that previously failed. For example:
```
terraform plan -var-file=large_environment.tfvars
```
Monitor the output to ensure the “Too Many Open Files” error no longer appears.

By following these steps, you can effectively diagnose and resolve the “Too Many Open Files” error, ensuring your Terraform operations run smoothly on Ubuntu 20.04.