Bug Report: mbedTLS SAN IP Address Parsing Failure

Executive Summary

ESP8266 and ESP32 devices running ESP-IDF versions that utilize TCP_IP_ADAPTER.h fail to complete SSL/TLS handshakes when connecting to servers presenting X.509 v3 certificates containing IP addresses in the Subject Alternative Name (SAN) extension field. This limitation stems from mbedTLS's x509_get_subject_alt_name function not supporting IP address parsing in SAN fields.

Environment

Affected Hardware

Primary: ESP8266 (ESP8266 RTOS SDK based on ESP-IDF v3.4)
Secondary: ESP32 (ESP-IDF versions using TCP_IP_ADAPTER.h)

Software Stack

TLS Library: mbedTLS (embedded in ESP-IDF)
Network Stack: TCP_IP_ADAPTER.h-based implementations
Certificate Standard: X.509 v3 with SAN extension
Use Case: MQTT over TLS connections to local broker

Test Environment

Server: Mosquitto MQTT broker running on Raspberry Pi
Certificate Type: Self-signed X.509 v3 certificates with SAN extension
Network: Local area network with mDNS (avahi-daemon)

Issue Description

Symptoms

When an ESP8266 or ESP32 device attempts to establish an SSL/TLS connection to a server whose certificate contains an IP address in the SAN field, the handshake fails with the following error:

E (9628) esp-tls-mbedtls: mbedtls_ssl_handshake returned -0x2700
I (9638) esp-tls-mbedtls: Failed to verify peer certificate!
I (9648) esp-tls-mbedtls: verification info:   ! The certificate Common Name (CN) does not match with the expected CN
  ! The certificate is not correctly signed by the trusted CA
E (9658) esp-tls: Failed to open new connection
E (9658) TRANS_SSL: Failed to open a new connection
E (9668) MQTT_CLIENT: Error transport connect

The critical error message is:

The certificate Common Name (CN) does not match with the expected CN

Unexpected Behavior

The error is particularly confusing because:

The same IP address is present in both the CN and SAN fields
The certificate validates successfully with other MQTT clients (MQTT Explorer, Node.js clients, Raspberry Pi devices)
The server-side broker configuration is correct and functional for non-ESP devices

Root Cause Analysis

Technical Deep Dive

The issue originates from mbedTLS's incomplete implementation of SAN field parsing, specifically in the function x509_get_subject_alt_name located in components/mbedtls/mbedtls/library/x509_crt.c.

Code Analysis

According to ESP-IDF maintainer ESP-Marius (Feb 22, 2021):

"From my reading it seems like the SubjectAltName feature in mbedtls do not support parsing IPs: see description of x509_get_subject_alt_name in components/mbedtls/mbedtls/library/x509_crt.c"

Parsing Logic Failure

When mbedTLS encounters a certificate with a SAN extension:

It attempts to parse the SAN field using x509_get_subject_alt_name
The function does not support IP address types in SAN fields
Unable to parse the SAN IP, the library fails to match it against the connection target
Even though the CN field contains the correct IP, the presence of the unparseable SAN field causes the entire validation to fail
The error message incorrectly reports a CN mismatch when the actual issue is SAN parsing failure

Affected ESP-IDF Versions

All ESP-IDF versions using TCP_IP_ADAPTER.h are affected:

ESP8266 RTOS SDK (based on ESP-IDF v3.4)
Early ESP32 ESP-IDF versions prior to IDF v4.x migration to esp_netif

Why This Matters

The SAN extension was introduced to address security limitations of relying solely on the CN field. Modern browsers and TLS implementations prioritize SAN over CN. However, the incomplete implementation in mbedTLS creates a catch-22:

Including an IP in SAN (modern best practice) - ESP devices fail
Omitting SAN entirely (deprecated approach) - Other clients may warn or fail

Reproduction Steps

Prerequisites

ESP8266 or affected ESP32 device with ESP-IDF
MQTT broker (Mosquitto) on Raspberry Pi or similar
OpenSSL for certificate generation

Step 1: Generate X.509 v3 Certificate with SAN IP

Create a certificate configuration file cert.conf:

[req]
distinguished_name = req_distinguished_name
x509_extensions = v3_ca

[req_distinguished_name]

[v3_ca]
subjectAltName = IP:192.168.1.100

Generate the certificate:

openssl req -new -x509 -days 365 -nodes \
  -out server.crt \
  -keyout server.key \
  -subj "/CN=192.168.1.100" \
  -config cert.conf

Step 2: Configure Mosquitto Broker

Update mosquitto.conf:

listener 8883
certfile /path/to/server.crt
keyfile /path/to/server.key
cafile /path/to/ca.crt
require_certificate true

Restart broker:

sudo systemctl restart mosquitto

Step 3: Configure ESP8266/ESP32 Client

const char *mqtt_broker = "192.168.1.100";
const int mqtt_port = 8883;

esp_mqtt_client_config_t mqtt_cfg = {
    .uri = "mqtts://192.168.1.100:8883",
    .cert_pem = (const char *)server_cert_pem_start,
    // Additional config...
};

esp_mqtt_client_handle_t client = esp_mqtt_client_init(&mqtt_cfg);
esp_mqtt_client_start(client);

Step 4: Observe Failure

Monitor serial output for the handshake failure error.

Step 5: Verify Server Functionality

Test with MQTT Explorer or mosquitto_pub from another device to confirm the broker works correctly.

Workaround / Solution

Implemented Solution

Replace the IP address with a fully qualified domain name (FQDN) in both the certificate and connection string.

Step 1: Configure mDNS on Raspberry Pi

Ensure avahi-daemon is running:

sudo systemctl status avahi-daemon

Set hostname (e.g., mqttbroker):

sudo hostnamectl set-hostname mqttbroker

The device will be accessible as mqttbroker.local

Step 2: Regenerate Certificate with FQDN

Update cert.conf:

[req]
distinguished_name = req_distinguished_name
x509_extensions = v3_ca

[req_distinguished_name]

[v3_ca]
subjectAltName = DNS:mqttbroker.local

Generate new certificate:

openssl req -new -x509 -days 365 -nodes \
  -out server.crt \
  -keyout server.key \
  -subj "/CN=mqttbroker.local" \
  -config cert.conf

Step 3: Update ESP Client Configuration

esp_mqtt_client_config_t mqtt_cfg = {
    .uri = "mqtts://mqttbroker.local:8883",
    .cert_pem = (const char *)server_cert_pem_start,
};

Results

After implementing the FQDN-based approach:

ESP8266 devices successfully complete SSL handshakes
ESP32 devices successfully complete SSL handshakes
Node.js clients continue to function correctly
Raspberry Pi devices maintain connectivity
No additional network configuration required (mDNS handles resolution)

Alternative Workarounds

Option 1: Disable Certificate Validation (NOT RECOMMENDED)

esp_mqtt_client_config_t mqtt_cfg = {
    .uri = "mqtts://192.168.1.100:8883",
    .skip_cert_common_name_check = true,
};

Warning: This defeats the purpose of TLS and should only be used for debugging.

Option 2: Upgrade ESP-IDF

For ESP32 devices, upgrading to ESP-IDF v4.x or later may provide improved support, though the fundamental mbedTLS limitation may persist.

Option 3: Patch mbedTLS

Modify x509_get_subject_alt_name to support IP address parsing. This requires:

Understanding ASN.1 encoding for IP addresses in SAN
Modifying mbedTLS source code
Maintaining a forked version of ESP-IDF

This approach is not recommended for production use.

Impact Assessment

Severity: HIGH

Security Impact: Forces developers to either disable certificate validation or use workarounds
Development Impact: Delays project timelines, as experienced during deployment
User Experience: Silent failures with misleading error messages
Portability: Certificates that work with standard clients fail with ESP devices

Affected Use Cases

IoT Devices in Local Networks: Devices connecting to local MQTT brokers via IP
Embedded Systems: Resource-constrained environments where DNS may be unavailable
Development/Testing: Local development environments using IP-based configurations
Closed Networks: Industrial or isolated networks without DNS infrastructure

Recommendations

For ESP-IDF Developers

Document the limitation: Clearly state in official documentation that SAN IP addresses are not supported
Improve error messages: Report "SAN IP parsing not supported" instead of "CN does not match"
Backport improvements: Consider backporting mbedTLS updates to older IDF versions
Provide migration path: Offer guidance for affected users

For Application Developers

Always use FQDNs: Configure all SSL/TLS connections using domain names, not IPs
Deploy mDNS: Use avahi-daemon or similar for local network name resolution
Test early: Validate certificate configurations with ESP devices before production
Plan infrastructure: Ensure DNS/mDNS availability in deployment environments

For System Administrators

Standardize on DNS: Avoid IP-based certificate configurations entirely
Use mDNS for local: Implement .local domains for LAN-based services
Document quirks: Maintain internal documentation of ESP-specific requirements

References

Official Sources

ESP32 Forum Discussion: mbedTLS fails SSL handshake using certificate with alternative name (SAN)
ESP-IDF GitHub: ESP-IDF Repository
mbedTLS Documentation: SubjectAltName Support

Earlier CN Mismatch Issues: ESP32 Forum Thread
TCP_IP_ADAPTER Migration: ESP-IDF v4.0 release notes

Technical Standards

RFC 5280: Internet X.509 Public Key Infrastructure Certificate and CRL Profile
- Section 4.2.1.6: Subject Alternative Name
RFC 6125: Representation and Verification of Domain-Based Application Service Identity
CA/Browser Forum Baseline Requirements: Deprecation of CN field in favor of SAN

Appendix

Error Code Reference

| Error Code | Meaning | |------------|---------| | -0x2700 | MBEDTLS_ERR_X509_CERT_VERIFY_FAILED | | -0x2800 | MBEDTLS_ERR_SSL_BAD_HS_SERVER_HELLO |

Verification Commands

Test certificate SAN fields:

# View certificate details
openssl x509 -in server.crt -text -noout

# Check SAN extension
openssl x509 -in server.crt -text -noout | grep -A1 "Subject Alternative Name"

# Verify certificate chain
openssl verify -CAfile ca.crt server.crt

# Test TLS connection
openssl s_client -connect mqttbroker.local:8883 -CAfile ca.crt

Useful Diagnostics

Enable verbose ESP-IDF logging:

esp_log_level_set("esp-tls", ESP_LOG_VERBOSE);
esp_log_level_set("esp-tls-mbedtls", ESP_LOG_VERBOSE);
esp_log_level_set("MQTT_CLIENT", ESP_LOG_VERBOSE);

Acknowledgments

ESP-Marius (Espressif Staff): For identifying the root cause in the forum discussion
RichPiano: For detailed bug reporting and testing
ESP32/ESP8266 Community: For collaborative troubleshooting

Document Metadata

Report Date: February 24, 2021 (Original Forum Post)
Analysis Date: June 16, 2024
Last Updated: June 16, 2024
Tested ESP-IDF Versions: v3.4 (ESP8266), v4.x (ESP32)
Status: Workaround identified, fundamental limitation persists

Bug Report: mbedTLS SAN IP Address Parsing Failure on ESP8266/ESP32