Troubleshooting Office 365 SMTP Relay and Client Submission Failures

Setting up an application, printer, or legacy device to send email via the Office 365 (Microsoft 365) SMTP gateway (smtp.office365.com) is a cornerstone of modern business workflows. It enables critical notifications, automated reports, and scan-to-email functionality. However, despite entering the correct hostname and port (587), this configuration is notoriously prone to persistent, frustrating failures.

As depicted in the accompanying image, an “SMTP Authentication Error” in the Microsoft 365 Admin Center often leaves administrators staring at error logs and configuration screens, trying to determine why a setup that seems straightforward refuses to connect. When these configurations break, essential business processes stop, leading to lost time and diagnostic headaches.

The difficulty lies in the collision between modern security protocols and the decades-old architecture of SMTP AUTH (Authenticated SMTP). Microsoft has been aggressively moving away from “Basic Authentication”—which relies solely on a username and password—in favor of Modern Authentication (OAuth 2.0). Since many applications and legacy devices only support simple credentials, the default security stance of a new M365 tenant is designed to block the very connection you are trying to establish. This is almost always the starting point for any SMTP troubleshooting journey.

Here is a comprehensive guide to diagnosing and resolving the most common causes of Office 365 SMTP failures.

Part 1: Validating the Fundamentals

Before diving into complex policy changes, verify that the application or device has the absolutely correct foundational settings. M365 is unforgiving of even slight variations.

The standard configuration for SMTP Client Submission (the most common method) is:

• Server/Host Name: smtp.office365.com
• Port: 587
• Encryption Method: STARTTLS (or TLS, sometimes labeled as Explicit SSL/TLS)
• Authentication: Requires a valid Microsoft 365 mailbox username and password.

If you are attempting to use Port 25, be aware that many ISPs and corporate firewalls now block this outbound port to prevent spam. Port 465 is generally reserved for Implicit SSL/TLS and is not the standard for M365 client submission.

Part 2: The Critical Security Checkpoints

If the fundamentals are correct but the error persists, the problem is almost certainly security-related. You must review three critical areas in the Microsoft 365 ecosystem:

1. The Global “Authenticated SMTP” Setting

Microsoft now disables SMTP AUTH by default at the tenant level for all new tenancies. Even if you have created a licensed mailbox and believe the credentials are valid, the exchange online service may be rejecting the connection before checking the password.

Action Required: You must explicitly enable this setting.

• In the Microsoft 365 Admin Center, navigate to Users > Active users.
• Select the specific user account you are using for SMTP submission.
• In the details flyout menu, click the Mail tab.
• Under the Email apps section, click Manage email apps.
• Ensure the checkbox for Authenticated SMTP is checked. If it was unchecked, check it and save the changes.

Tip: It may take up to 24 hours for this setting to fully propagate, though it often works within an hour.

2. Security Defaults and Multi-Factor Authentication (MFA)

This is the largest hurdle for legacy devices. By default, Microsoft enforces Security Defaults, which require all users to register for MFA and block all legacy authentication protocols—including standard SMTP—globally.

Action Required: If the SMTP account has MFA enabled or is subject to Security Defaults, you cannot use the account’s standard password for SMTP. You have two primary paths to resolve this conflict:

• Pathway A: Use an App Password (Requires disabling Security Defaults).
  1. First, you must disable Security Defaults in the Azure Active Directory properties.
  2. Once Security Defaults are off, ensure the SMTP user account is configured to allow legacy authentication and enforce MFA.
  3. Log in to that specific user account at myaccount.microsoft.com.
  4. Under Security Info, add a new method: App Password.
  5. Copy the long, randomly generated 16-character password. Use this password instead of your standard account password in your device’s SMTP configuration.
• Pathway B: Disable MFA entirely for the SMTP User (Use with Caution). This pathway is less secure but often necessary for very old hardware. It requires disabling Security Defaults and ensuring that no Conditional Access policies apply to the SMTP user. You must use a very complex, long password to mitigate the increased risk.

Part 3: Identifying Environmental and Legacy Hardware Blockers

Sometimes, the configuration within M365 is perfect, but the environmental factors prevent the connection from completing.

1. Local Network and Firewall Restrictions

The device or application must be able to initiate an outbound TCP connection on port 587. Corporate firewalls are often locked down by default.

Action Required:

• Consult your network team to verify that a firewall rule allows the source IP address of the device (e.g., the scanner’s internal IP) to make outbound connections to any IP address on Port 587 (or specifically to smtp.office365.com if your firewall supports FQDN-based rules).
• Test the connection from the same network segment using a tool like Telnet or PowerShell:Test-NetConnection smtp.office365.com -Port 587 If this fails, the issue is environmental, not an M365 configuration error.

2. TLS Version Compatibility

Office 365 requires TLS 1.2 or higher for all connections. Many legacy devices (old printers, older versions of ERP software, etc.) were manufactured when TLS 1.0 or 1.1 were the standard. They structurally cannot negotiate a TLS 1.2 connection.

Symptoms: You will see connection termination errors immediately after the initially successful TCP handshake (e.g., “The connection was closed by the remote host” or “SSL/TLS Handshake Failed”).

Action Required:

1. Check the manufacturer’s documentation to see if a firmware update is available to add TLS 1.2 support.
2. If the device is too old, you cannot use SMTP Client Submission. You must use Method 3: Microsoft 365 High Volume Email (previously a specialized connector or a full relay) which can accept traffic from specific static IP addresses, but this approach has its own complexities and requirements (like a static public IP).

Summary of Diagnostic Steps

By strictly methodically eliminating the possibilities, you can resolve the frustration shown in the image. Validate the basic hostname/port (587/STARTTLS), ensure SMTP AUTH is enabled for the specific mailbox, check the MFA/Security Defaults status (and use an App Password if needed), and confirm the network path is clear. Once these four layers are verified, your SMTP setup should begin functioning correctly.

Also Read: BigPond POP3 Mail Server Not Working

Source: SMTP server for Office 365 settings, port, and auth guide