Repository navigation
Description
Before reporting an issue
- I have read and understood the above terms for submitting issues, and I understand that my issue may be closed without action if I do not follow them.
Area
storage
Bug Description
Two custom changes in jpa-changelog-26.8.0.xml run queries against the generation database while a manual migration script is generated, and they read columns and tables that an earlier changeset has not created yet when the starting version is older than 26.0.0.
26.8.0-org-idp-and-domain-routing-migration (JpaUpdate26_8_0_DomainIdpRoutingMigration) selects IDENTITY_PROVIDER.ORGANIZATION_ID and joins ORG and ORG_DOMAIN. It has no precondition.
26.8.0-idp-mapper-organization-id (JpaUpdate26_8_0_IdpMapperOrganizationId) selects IDENTITY_PROVIDER.ORGANIZATION_ID. Its onSqlOutput="IGNORE" precondition is not evaluated in script mode, so the Java always runs.
ORG and ORG_DOMAIN are created in 25.0.0 and IDENTITY_PROVIDER.ORGANIZATION_ID is added in 26.0.0.
For a database older than that, the generation database does not have them.
CustomKeycloakTask.isApplicable() only checks that REALM exists and has rows, so it does not skip the task, the query fails, and the script generation fails with a CustomChangeException.
Automatic migration is not affected because each earlier changeset is applied before the next one is evaluated.
This was found by reading the code against the guidance in docs/updating-database-schema.md and has not been reproduced yet.
The comments on 26.8.0-idp-mapper-organization-id, 26.8.0-drop-idp-organization-id and 26.8.0-backfill-group-org-id also say the columns "existed before 26.8" or are "always present on the generation database".
That only holds for starting versions after 26.0.0 (ORGANIZATION_ID) or 26.6.0 (KEYCLOAK_GROUP.ORG_ID).
The IGNORE choice on the last two is correct because they only emit SQL, so only the wording is wrong.
Version
26.8.0
Regression
- The issue is a regression
Expected behavior
Manual migration from any supported earlier version, including one that predates 26.0.0 and has realms, generates a script.
A task that reads a table or column that is absent from the generation database emits nothing, because a database without that column has no data to migrate.
Actual behavior
Script generation is expected to fail when the generation database has realms but lacks IDENTITY_PROVIDER.ORGANIZATION_ID, ORG or ORG_DOMAIN.
How to Reproduce?
- Create a database with Keycloak 24.0.4 or earlier, such as 19.0.3, and create a realm with an identity provider.
- Start the 26.8.0 server with
--migration-strategy=manualagainst that database. - Check whether script generation fails in
JpaUpdate26_8_0_DomainIdpRoutingMigrationorJpaUpdate26_8_0_IdpMapperOrganizationId.
The existing tests do not cover this.
CI only runs MigrationTest with -Dmigration.mode=auto from 24.0.4, and ManualMigrationEmptyDatabaseDistTest starts from an empty database with no REALM table, so the tasks are skipped.
Anything else?
Suggested fix:
- Make the two tasks skip when the generation database does not have the table or column they read. Do not change the changesets themselves, because their checksums cover the changes and
modifySql, not comments or preconditions. - Correct the misleading comments in
jpa-changelog-26.8.0.xml. Comments and preconditions are not part of the Liquibase checksum, which I confirmed with Liquibase 4.33.0. - Add a test that generates a manual script from an older version with existing data.
MigrationTestwith-Dmigration.mode=manualis the closest existing one.
The other released changelogs from 25.0.0 to 26.7.0 use TEST with indexExists and columnExists, and were not reviewed individually.
The open 27.0.0 changelog has no problems against the new guidance: it has no preconditions and only reads tables that exist at every starting version.
Metadata
Metadata
Assignees
Labels
Type
Fields
Projects
- StatusShow more project fieldsOpen
Activity