How to Fix Ansible 403 Forbidden on Ubuntu 20.04

As a Senior DevOps Engineer, encountering “Ansible 403 Forbidden” is a clear signal: an HTTP request made by an Ansible module has been denied by the target server. This error explicitly indicates an authorization issue at the HTTP level, distinct from SSH connection failures. This guide will clarify the root causes specific to Ubuntu 20.04 environments and provide actionable steps for resolution.

The Root Cause: Ansible 403 Forbidden on Ubuntu 20.04

The “Ansible 403 Forbidden” error almost universally means that an Ansible module, specifically those making HTTP requests (e.g., ansible.builtin.uri, ansible.builtin.get_url), received an HTTP 403 status code from a web server or API endpoint it attempted to contact. This signifies that the server understood the request but refuses to authorize access to the requested resource.

On Ubuntu 20.04, where your target web server or API might be running, or from where Ansible is trying to fetch resources, the common underlying reasons for this HTTP 403 response include:

  1. Missing or Invalid Authentication:

    • The request lacks necessary credentials (e.g., API key, basic authentication headers, bearer token, cookies).
    • The provided credentials are incorrect or have expired.

  2. Insufficient Authorization:

    • Even with valid authentication, the user or service account associated with the request does not have the specific permissions to access the requested resource or perform the requested action.
    • IP Address Restrictions: The web server or API is configured to only allow access from a specific whitelist of IP addresses, and your Ansible control node’s IP is not included.
    • Incorrect URL Path: The requested URL path is incorrect, points to a forbidden directory listing, or the resource simply does not exist at that exact path.
    • HTTP Method Not Allowed: The server is configured to disallow the specific HTTP method (e.g., PUT, DELETE) for the requested resource, although this often results in a 405 Method Not Allowed rather than 403.

  3. File System Permissions on the Target Web Server (Indirect):

    • If Ansible is trying to download a file from a web server (e.g., Nginx, Apache2) on an Ubuntu 20.04 host, the web server’s process owner (commonly www-data) might lack read access to the specific file or its parent directories. While an OS-level permission issue, it manifests as an HTTP 403 from the web server.

  4. AppArmor Restrictions (Rarely Direct, but Possible):

    • Ubuntu 20.04 uses AppArmor as its Mandatory Access Control (MAC) system. In rare cases, if the web server process (e.g., Nginx, Apache2) is confined by an AppArmor profile, and that profile prevents it from accessing specific directories or files, it could lead to a 403 when trying to serve content from those paths. This is less common for standard web serving but can happen with custom configurations.

Quick Fix (CLI)

These commands address the most common scenarios.

Scenario 1: Missing Authentication/Authorization for uri or get_url

If your Ansible playbook is using uri or get_url and hitting a protected resource:

  1. Add HTTP Headers (e.g., for API keys or custom tokens):

    - name: Fetch data from authenticated API
  ansible.builtin.uri:
    url: "https://api.example.com/data"
    method: GET
    headers:
      Authorization: "Bearer your_api_token_here" # Or 'X-API-Key: your_key'
      Content-Type: "application/json"
    status_code: 200
    validate_certs: yes # Set to 'no' only for self-signed or untrusted certs in dev
  register: api_response

  2. Add Basic Authentication (user/password):

    - name: Download file with Basic Auth
  ansible.builtin.get_url:
    url: "https://repo.example.com/software.zip"
    dest: "/tmp/software.zip"
    user: "repo_user"
    password: "repo_password" # Consider using Ansible Vault for sensitive data
    validate_certs: yes

  3. Bypass SSL Certificate Validation (Use with Caution for Development): Only use validate_certs: no in development or testing environments where self-signed certificates are in use and security is not paramount. Never use this in production if the target server is expected to have valid certificates.

    - name: Get URL with ignored certs (DEV ONLY)
  ansible.builtin.get_url:
    url: "https://untrusted-dev-server/file.txt"
    dest: "/tmp/file.txt"
    validate_certs: no

Scenario 2: File System Permissions on the Target Web Server

If the 403 is from a web server on an Ubuntu 20.04 host and implies the web server cannot read the file it’s trying to serve:

  1. Correct File/Directory Ownership: Ensure the web server process user (e.g., www-data for Apache/Nginx) has ownership or is in a group that owns the files.

    # On the target Ubuntu 20.04 server (via SSH or Ansible 'shell' module)
sudo chown -R www-data:www-data /var/www/html/your_app

  2. Correct File/Directory Permissions: Grant appropriate read permissions.

    # On the target Ubuntu 20.04 server
sudo find /var/www/html/your_app -type d -exec chmod 755 {} \; # Directories: rwx for owner, rx for group/others
sudo find /var/www/html/your_app -type f -exec chmod 644 {} \; # Files: rw for owner, r for group/others

Scenario 3: AppArmor Interference (Less Common)

If you suspect AppArmor is blocking access for the web server process (e.g., Nginx, Apache), you might see denials in dmesg or /var/log/syslog.

  1. Temporarily Disable AppArmor Profile for a Service (for testing): 
    # On the target Ubuntu 20.04 server
sudo aa-disable /etc/apparmor.d/usr.sbin.nginx # Or /etc/apparmor.d/usr.sbin.apache2
sudo systemctl restart nginx # Or apache2
    WARNING: Disabling AppArmor profiles reduces security. Re-enable or fine-tune profiles after testing.

Configuration Check

Thoroughly review these configurations to pinpoint the issue.

1. Ansible Playbook/Task Definition

  • uri / get_url module parameters:
    • url: Is the URL correct, including scheme (http/https), hostname, and path? Double-check for typos.
    • headers: Are all required headers (e.g., Authorization, X-API-Key, Referer, Content-Type) present and correctly formatted?
    • user / password: If using basic authentication, are these provided and correct?
    • validate_certs: Is it yes and is the target server using a valid, trusted SSL certificate? If no, confirm this is intentional and for development purposes only.
    • method: Is the HTTP method (GET, POST, PUT, DELETE) appropriate for the resource?

2. Target Ubuntu 20.04 Web Server/API

  • Web Server Access Logs: Check the access logs of the web server (e.g., Nginx: /var/log/nginx/access.log, Apache2: /var/log/apache2/access.log). Look for the 403 entry and the IP address it’s coming from. This can often reveal the exact URL that was requested and any other relevant request details.
  • Web Server Error Logs: Check the error logs (e.g., Nginx: /var/log/nginx/error.log, Apache2: /var/log/apache2/error.log). These might provide more detail about why the 403 was returned, e.g., “client denied by server configuration,” “permission denied,” or configuration issues.
  • Web Server Configuration:
    • Nginx: Check .conf files in /etc/nginx/sites-available/ and /etc/nginx/nginx.conf. Look for deny directives, allow directives (IP restrictions), auth_basic or other authentication mechanisms, or incorrect root directives.
    • Apache2: Check .conf files in /etc/apache2/sites-available/ and /etc/apache2/apache2.conf. Look for Require all denied, Allow from, Deny from, AuthType, or Options -Indexes directives.
  • File System Permissions: For any files or directories the web server serves, verify the following: 
    # Example for web root
ls -la /var/www/html
ls -la /var/www/html/your_specific_file.html
# Output should show 'www-data' as owner or group with read permissions
    Ensure the path specified in the web server config (e.g., root in Nginx, DocumentRoot in Apache) points to an accessible directory.
  • IP Whitelisting: Confirm if the web server/API has IP-based restrictions. If so, ensure the Ansible control node’s IP address is explicitly whitelisted.
  • AppArmor Status: 
    sudo aa status
# This lists active AppArmor profiles. If you suspect an issue,
# look for profiles related to your web server (e.g., /usr/sbin/nginx)
# and then check specific denials in dmesg:
sudo dmesg | grep 'apparmor="DENIED"'
sudo grep -i "apparmor" /var/log/syslog

Verification

Once you’ve made changes, verify your fix with these steps:

  1. Re-run the Ansible Playbook with Verbosity:

    ansible-playbook your_playbook.yml -vvv

    The increased verbosity can provide more detailed output about the HTTP request and response, including any intermediate redirects or specific error messages before the 403.

  2. Manual curl Test (from Ansible Control Node): Attempt to access the problematic URL directly from your Ansible control node using curl, mimicking the request Ansible makes. This helps isolate whether the problem is with Ansible’s module configuration or the target server’s configuration.

    # With headers
curl -v -H "Authorization: Bearer your_api_token_here" "https://api.example.com/data"

# With basic auth
curl -v -u "repo_user:repo_password" "https://repo.example.com/software.zip"

# With ignored certs (like validate_certs: no)
curl -v --insecure "https://untrusted-dev-server/file.txt"

    A successful curl request (HTTP 200 OK) indicates the issue is likely within your Ansible playbook’s module parameters. A persistent 403 with curl points to an issue on the target web server/API configuration.

  3. Manual File System Access Test (on Target Ubuntu 20.04 Host): If the issue was related to web server file permissions, log directly into the Ubuntu 20.04 server and manually try to access the files as the web server’s user.

    # Try to read the file as the www-data user
sudo -u www-data cat /var/www/html/your_app/index.html

    If this command fails with a “Permission denied” error, you’ve confirmed the file system permission issue.

By systematically following these steps, you should be able to identify and resolve the “Ansible 403 Forbidden” error efficiently in your Ubuntu 20.04 environments.

⚡ Related Troubleshooting Guides