Check for non-ascii local part on emails depending on SMTP configuration

Closes #41994

Signed-off-by: rmartinc <rmartinc@redhat.com>

docs/documentation/server_admin/topics/realms/email.adoc

@@ -72,12 +72,7 @@ Auth Token Client Secret::
 Allow UTF-8::
   Enable to UTF-8-encode email address when sending them to the server. This should only be enabled if the mail server supports UTF-8 via the SMTPUTF8 extension. If disabled, domain names containing non-ASCII characters will be encoded using punycode, and addresses containing non-ASCII characters in the local part of the address will return an error.
 +
-If you do not enable this option, take additional measures to prevent non-ASCII characters in users' email addresses:
-+
---
-. Verifying that no email addresses of existing users have non-ASCII characters in the local part of the email address.
-. Updating the validation of email addresses to prevent non-ASCII characters in the local part of the email address, for example, by adding a regex pattern validation in the user profile for the email address field similar to `\p{ASCII}*@.*` with an error message similar to `Local part of the address must contain only ASCII characters`.
---
+If the realm is configured to send emails (this SMTP configuration is setup) and *Allow UTF-8* option is disabled, the built-in <<user-profile, user profile>> email validator checks the local part of the address contains only ASCII characters. This way, {project_name} prevents user emails that cannot be notified.
 
 ifeval::[{project_community}==true]
 

docs/documentation/server_admin/topics/users/user-profile.adoc

@@ -269,7 +269,7 @@ The list below provides a list of all the built-in validators:
 *error-message*: the key of the error message in i18n bundle. If not set a generic message is used.
 
 |email
-|Check if the value has a valid e-mail format.
+|Check if the value has a valid e-mail format. If <<_email, the realm is configured to send emails>> and the option *Allow UTF-8* is not enabled to support internationalized emails, this validator also checks that the local part of the address contains only ASCII characters.
 |
 *max-local-length*: an integer to define the maximum length for the local part of the email. It defaults to 64 per specification.
 

docs/documentation/upgrading/topics/changes/changes-26_4_0.adoc

@@ -133,13 +133,14 @@ The input fields  in the login theme for OTP and recovery codes and have been op
 
 * The input mode is now `numeric`, which will ease the input on mobile devices.
 * The auto-complete is set to `one-time-code` to avoid interference with password managers.
+
 === UTF-8 management in the email sender
 
 Since this release, {project_name} adds a new option `allowutf8` for the realm SMTP configuration (*Allow UTF-8* field inside the *Email* tab in the *Realm settings* section of the Admin Console).
 For more information about email configuration, see the link:{adminguide_link}#_email[Configuring email for a realm] chapter in the {adminguide_name}.
 
 Enabling the option encodes email addresses in UTF-8 when sending them, but it depends on the SMTP server to also supports UTF-8 via the SMTPUTF8 extension.
-If *Allow UTF-8* is disabled, {project_name} will encode the domain part of the email address (second part after `@`) using punycode if non-ASCII characters are used, and will reject email addresses that use non-ASCII characters in the local part.
+If *Allow UTF-8* is disabled, {project_name} will encode the domain part of the email address (second part after `@`) using punycode if non-ASCII characters are used, and will reject email addresses that use non-ASCII characters in the local part. The built-in User Profile email validator also checks that the local part of the address contains only ASCII characters when this option is disabled, avoiding the registration of emails that cannot be used by the SMTP configuration.
 
 If you have an SMTP server configured for your realm, perform the following migration after the upgrade:
 
@@ -147,8 +148,7 @@ If you have an SMTP server configured for your realm, perform the following migr
 . Enable the *Allow UTF-8* option.
 * If your SMTP server does not support SMTPUTF8:
 . Keep the *Allow UTF-8* option disabled.
-. Verify that no email addresses of users have non-ASCII characters in the local part of the email address.
-. Update the validation of email addresses to prevent allow non-ASCII characters in the local part of the email address, for example, by adding a regex pattern validation in the user profile for the email address field similar to `\p{ASCII}*@.*` with an error message similar to `Local part of the address must contain only ASCII characters`.
+. Verify that no email addresses of users have non-ASCII characters in the local part of the email address. If you detect emails with non-ascii characters in the local part you can use the Verify Profile action to force the user to modify the email after the upgrade.
 
 // ------------------------ Deprecated features ------------------------ //
 == Deprecated features

js/apps/account-ui/maven-resources/theme/keycloak.v3/account/messages/messages_en.properties

@@ -188,6 +188,7 @@ webauthn-passwordless-display-name=Passkey
 webauthn-passwordless-help-text=Use your Passkey for passwordless sign in.
 passwordless=Passwordless
 error-invalid-multivalued-size=Attribute {{0}} must have at least {{1}} and at most {{2}} value(s).
+error-non-ascii-local-part-email=Local part of the address must contain only ASCII characters.
 recovery-authn-code=My recovery authentication codes
 recovery-authn-codes-display-name=Recovery authentication codes
 recovery-authn-codes-help-text=These codes can be used to regain your access in case your other 2FA means are not available.

js/apps/admin-ui/maven-resources/theme/keycloak.v2/admin/messages/messages_en.properties

@@ -3034,6 +3034,7 @@ error-person-name-invalid-character='{{0}}' contains invalid character.
 error-user-attribute-required=Please specify '{{0}}'.
 error-username-invalid-character='{{0}}' contains invalid character.
 error-user-attribute-read-only=The field {{0}} is read only.
+error-non-ascii-local-part-email=Local part of the address must contain only ASCII characters.
 missingUsernameMessage='{{0}}': Please specify username.
 missingFirstNameMessage='{{0}}': Please specify first name.
 invalidEmailMessage='{{0}}': Invalid email address.

server-spi-private/src/main/java/org/keycloak/email/EmailSenderProvider.java

@@ -27,6 +27,8 @@ import java.util.Map;
  */
 public interface EmailSenderProvider extends Provider {
 
+    String CONFIG_ALLOW_UTF8 = "allowutf8";
+
     default void send(Map<String, String> config, UserModel user, String subject, String textBody, String htmlBody) throws EmailException {
         send(config, user.getEmail(), subject, textBody, htmlBody);
     }

server-spi-private/src/main/java/org/keycloak/validate/validators/EmailValidator.java

@@ -20,7 +20,9 @@ import java.util.LinkedHashSet;
 import java.util.List;
 import java.util.Set;
 
+import org.keycloak.email.EmailSenderProvider;
 import org.keycloak.models.KeycloakSession;
+import org.keycloak.models.RealmModel;
 import org.keycloak.provider.ConfiguredProvider;
 import org.keycloak.provider.ProviderConfigProperty;
 import org.keycloak.provider.ProviderConfigurationBuilder;
@@ -43,6 +45,8 @@ public class EmailValidator extends AbstractStringValidator implements Configure
 
     public static final String MESSAGE_INVALID_EMAIL = "error-invalid-email";
 
+    public static final String MESSAGE_NON_ASCII_LOCAL_PART_EMAIL = "error-non-ascii-local-part-email";
+
     public static final String MAX_LOCAL_PART_LENGTH_PROPERTY = "max-local-length";
 
     @Override
@@ -61,6 +65,29 @@ public class EmailValidator extends AbstractStringValidator implements Configure
                 ? EmailValidationUtil.isValidEmail(value, maxEmailLocalPartLength)
                 : EmailValidationUtil.isValidEmail(value))) {
             context.addError(new ValidationError(ID, inputHint, MESSAGE_INVALID_EMAIL, value));
+            return;
+        }
+
+        final KeycloakSession session = context.getSession();
+        if (session == null) {
+            return;
+        }
+
+        final RealmModel realm = session.getContext().getRealm();
+        if (realm == null || realm.getSmtpConfig() == null || realm.getSmtpConfig().isEmpty()
+                || "true".equals(realm.getSmtpConfig().get(EmailSenderProvider.CONFIG_ALLOW_UTF8))) {
+            // UTF-8 non-ascii chars allowed because no smtp configuration or allowutf8 is enabled
+            return;
+        }
+
+        final int idx = value.lastIndexOf('@');
+        if (idx < 0) {
+            return;
+        }
+
+        final String localPart = value.substring(0, idx);
+        if (!localPart.chars().allMatch(c -> c < 128)) {
+            context.addError(new ValidationError(ID, inputHint, MESSAGE_NON_ASCII_LOCAL_PART_EMAIL));
         }
     }
     

services/src/main/java/org/keycloak/email/DefaultEmailSenderProvider.java

@@ -233,7 +233,7 @@ public class DefaultEmailSenderProvider implements EmailSenderProvider {
     }
 
     private static boolean isAllowUTF8(Map<String, String> config) {
-        return "true".equals(config.get("allowutf8"));
+        return "true".equals(config.get(CONFIG_ALLOW_UTF8));
     }
 
     private static boolean isDebugEnabled(Map<String, String> config) {

services/src/main/java/org/keycloak/utils/SMTPUtil.java

@@ -58,7 +58,7 @@ public class SMTPUtil {
      * @return The converted email or null (if IDN.toASCII throws an exception)
      */
     public static String convertIDNEmailAddress(String email) {
-        final int idx = email == null ? -1 : email.indexOf('@');
+        final int idx = email == null ? -1 : email.lastIndexOf('@');
         if (idx < 0) {
             return email;
         }

tests/base/src/test/java/org/keycloak/tests/admin/SMTPConnectionTest.java

@@ -25,6 +25,7 @@ import org.junit.jupiter.api.TestMethodOrder;
 import org.keycloak.admin.client.Keycloak;
 import org.keycloak.admin.client.resource.RealmResource;
 import org.keycloak.admin.client.resource.UserResource;
+import org.keycloak.email.EmailSenderProvider;
 import org.keycloak.models.AdminRoles;
 import org.keycloak.models.Constants;
 import org.keycloak.representations.AccessToken;
@@ -246,6 +247,11 @@ public class SMTPConnectionTest {
         assertMailReceived();
 
         // utf8 on address
+        RealmResource realmRes = adminClient.realms().realm(managedRealm.getName());
+        RealmRepresentation realmRep = realmRes.toRepresentation();
+        realmRep.getSmtpServer().put(EmailSenderProvider.CONFIG_ALLOW_UTF8, Boolean.TRUE.toString());
+        realmRes.update(realmRep);
+
         AccessToken token = oAuthClient.parseToken(adminClient.tokenManager().getAccessToken().getToken(), AccessToken.class);
         UserResource userRes = adminClient.realm("default").users().get(token.getSubject());
         UserRepresentation userRep = userRes.toRepresentation();
@@ -267,6 +273,8 @@ public class SMTPConnectionTest {
         } finally {
             userRep.setEmail(previousEmail);
             userRes.update(userRep);
+            realmRep.getSmtpServer().remove(EmailSenderProvider.CONFIG_ALLOW_UTF8);
+            realmRes.update(realmRep);
         }
     }
 
@@ -315,7 +323,7 @@ public class SMTPConnectionTest {
         config.put("replyTo", replyTo);
         config.put("envelopeFrom", envelopeFrom);
         if (allowutf8 != null) {
-            config.put("allowutf8", allowutf8);
+            config.put(EmailSenderProvider.CONFIG_ALLOW_UTF8, allowutf8);
         }
         return config;
     }

testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/user/profile/UserProfileTest.java

@@ -34,6 +34,7 @@ import static org.keycloak.userprofile.config.UPConfigUtils.ROLE_USER;
 import static org.keycloak.userprofile.config.UPConfigUtils.parseSystemDefaultConfig;
 
 import jakarta.ws.rs.core.Response;
+import java.io.IOException;
 import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.Collections;
@@ -52,6 +53,7 @@ import org.keycloak.admin.client.resource.RealmResource;
 import org.keycloak.common.util.MultivaluedHashMap;
 import org.keycloak.component.ComponentModel;
 import org.keycloak.component.ComponentValidationException;
+import org.keycloak.email.EmailSenderProvider;
 import org.keycloak.models.Constants;
 import org.keycloak.models.KeycloakSession;
 import org.keycloak.models.LDAPConstants;
@@ -74,6 +76,7 @@ import org.keycloak.representations.userprofile.config.UPAttributeRequired;
 import org.keycloak.representations.userprofile.config.UPAttributeSelector;
 import org.keycloak.representations.userprofile.config.UPConfig;
 import org.keycloak.testsuite.admin.ApiUtil;
+import org.keycloak.testsuite.updaters.RealmAttributeUpdater;
 import org.keycloak.testsuite.util.ClientScopeBuilder;
 import org.keycloak.testsuite.util.KeycloakModelUtils;
 import org.keycloak.userprofile.Attributes;
@@ -320,10 +323,15 @@ public class UserProfileTest extends AbstractUserProfileTest {
     }
 
     @Test
-    public void testValidation() {
+    public void testValidation() throws IOException {
         getTestingClient().server(TEST_REALM_NAME).run((RunOnServer) UserProfileTest::failValidationWhenEmptyAttributes);
         getTestingClient().server(TEST_REALM_NAME).run((RunOnServer) UserProfileTest::testAttributeValidation);
         getTestingClient().server(TEST_REALM_NAME).run((RunOnServer) UserProfileTest::testEmailAsUsernameValidation);
+        getTestingClient().server(TEST_REALM_NAME).run((KeycloakSession session) -> testNonAsciiEmailValidator(session, false));
+        try (RealmAttributeUpdater updater = new RealmAttributeUpdater(testRealm())
+                .setSmtpServer(EmailSenderProvider.CONFIG_ALLOW_UTF8, Boolean.TRUE.toString()).update()) {
+            getTestingClient().server(TEST_REALM_NAME).run((KeycloakSession session) -> testNonAsciiEmailValidator(session, true));
+        }
     }
 
     private static void failValidationWhenEmptyAttributes(KeycloakSession session) {
@@ -425,6 +433,21 @@ public class UserProfileTest extends AbstractUserProfileTest {
         }
     }
 
+    private static void testNonAsciiEmailValidator(KeycloakSession session, boolean success) {
+        Map<String, Object> attributes = new HashMap<>();
+        UserProfileProvider provider = session.getProvider(UserProfileProvider.class);
+        List<ValidationError> errors = new ArrayList<>();
+        attributes.put(UserModel.USERNAME, "diego");
+        attributes.put(UserModel.EMAIL, "diegø@foo.com");
+        UserProfile profile = provider.create(UserProfileContext.UPDATE_PROFILE, attributes);
+        if (success) {
+            assertTrue(profile.getAttributes().validate(UserModel.EMAIL, errors::add));
+        } else {
+            assertFalse(profile.getAttributes().validate(UserModel.EMAIL, errors::add));
+            assertTrue(containsErrorMessage(errors, EmailValidator.MESSAGE_NON_ASCII_LOCAL_PART_EMAIL));
+        }
+    }
+
     private static boolean containsErrorMessage(List<ValidationError> errors, String message){
     	for(ValidationError err : errors) {
     		if(err.getMessage().equals(message)) {

themes/src/main/resources/theme/base/account/messages/messages_en.properties

@@ -386,3 +386,4 @@ error-invalid-date=Invalid date.
 error-user-attribute-read-only=The field {0} is read only.
 error-username-invalid-character=Username contains invalid character.
 error-person-name-invalid-character=Name contains invalid character.
+error-non-ascii-local-part-email=Local part of the address must contain only ASCII characters.

themes/src/main/resources/theme/base/admin/messages/messages_en.properties

@@ -67,10 +67,11 @@ error-user-attribute-read-only=Attribute {0} is read only.
 error-username-invalid-character={0} contains invalid character.
 error-person-name-invalid-character={0} contains invalid character.
 error-invalid-multivalued-size=Attribute {0} must have at least {1} and at most {2} {2,choice,0#values|1#value|1<values}.
+error-non-ascii-local-part-email=Local part of the address must contain only ASCII characters.
 
 client_account=Account
 client_account-console=Account Console
 client_security-admin-console=Security Admin Console
 client_admin-cli=Admin CLI
 client_realm-management=Realm Management
-client_broker=Broker
+client_broker=Broker

themes/src/main/resources/theme/base/login/messages/messages_en.properties

@@ -274,6 +274,7 @@ error-user-attribute-read-only=This field is read only.
 error-username-invalid-character=Value contains invalid character.
 error-person-name-invalid-character=Value contains invalid character.
 error-reset-otp-missing-id=Please choose an OTP configuration.
+error-non-ascii-local-part-email=Local part of the address must contain only ASCII characters.
 
 invalidPasswordExistingMessage=Invalid existing password.
 invalidPasswordBlacklistedMessage=Invalid password: password is blacklisted.