Skip to content

Keycloak-X: Full Database Export/Import

Published by admin on 6. February 2022

Advantage

  • Secrets and Passwords remains unchanged
  • Good for testing different keycloak settings during the development process

Disadvantage

  • Be careful your master realm gets overwriten and your admin password may change ! 

Docker Command to start Keycloak-X

  • keycloak-X instance : quay.io/keycloak/keycloak-x:16.1.1
  • http-relative-path=auth is used to get same baseUrl as we have used in our older JBOSS keycloak installation
# docker run --name keycloak -e KEYCLOAK_ADMIN=admin -e KEYCLOAK_ADMIN_PASSWORD=admin -p 8280:8080 -p 8543:8443 \
    -v "D:/dev/Quarkus/Keycloak/quarkus-keycloak-authorization-sample/imports:/tmp/imports" \
	quay.io/keycloak/keycloak-x:16.1.1 start-dev --http-relative-path=auth

Run full Database Export

  • First you may need to setup your keycloak-X instance with new realms, users, clients, roles, ..
  • After that start a 2.nd keycloack session inside the keycloak container and run the export
Get Container ID
# docker ps
CONTAINER ID   IMAGE                                COMMAND                  CREATED        STATUS        PORTS                                            NAMES
b431264a1fb8   quay.io/keycloak/keycloak-x:16.1.1   "/opt/keycloak/bin/k…"   46 hours ago   Up 46 hours   0.0.0.0:8280->8080/tcp, 0.0.0.0:8543->8443/tcp   keycloak

Open a new bash
# docker exec -it b431264a1fb8 bash

Start the Export
bash-4.4$ /opt/keycloak/bin/kc.sh export --file /tmp/imports/full-db-export-X.json
2022-03-06 08:59:57,930 INFO  [org.key.qua.run.hos.DefaultHostnameProvider] (main) Hostname settings: FrontEnd: <request>, Strict HTTPS: false, Path: <request>, Strict BackChannel: false, Admin: <request>
2022-03-06 08:59:58,538 WARN  [org.inf.PERSISTENCE] (keycloak-cache-init) ISPN000554: jboss-marshalling is deprecated and planned for removal
2022-03-06 08:59:58,639 WARN  [org.inf.CONFIG] (keycloak-cache-init) ISPN000569: Unable to persist Infinispan internal caches as no global state enabled
2022-03-06 08:59:58,657 INFO  [org.inf.CONTAINER] (keycloak-cache-init) ISPN000556: Starting user marshaller 'org.infinispan.jboss.marshalling.core.JBossUserMarshaller'
2022-03-06 08:59:58,884 INFO  [org.inf.CONTAINER] (keycloak-cache-init) ISPN000128: Infinispan version: Infinispan 'Triskaidekaphobia' 13.0.0.Final
2022-03-06 08:59:59,264 INFO  [org.key.con.inf.DefaultInfinispanConnectionProviderFactory] (main) Node name: node_700982, Site name: null
2022-03-06 08:59:59,884 INFO  [org.key.services] (main) KC-SERVICES0033: Full model export requested
2022-03-06 08:59:59,884 INFO  [org.key.exp.sin.SingleFileExportProvider] (main) Exporting model into file /tmp/imports/full-db-export-X.json
2022-03-06 09:00:01,539 INFO  [org.key.services] (main) KC-SERVICES0035: Export finished successfully
2022-03-06 09:00:01,654 ERROR [org.key.services] (main) KC-SERVICES0010: Failed to add user 'admin' to realm 'master': user with username exists
2022-03-06 09:00:01,943 ERROR [org.key.qua.run.cli.ExecutionExceptionHandler] (main) ERROR: Failed to start server using profile (import_export)
2022-03-06 09:00:01,943 ERROR [org.key.qua.run.cli.ExecutionExceptionHandler] (main) ERROR: Unable to start HTTP server
2022-03-06 09:00:01,944 ERROR [org.key.qua.run.cli.ExecutionExceptionHandler] (main) ERROR: java.net.BindException: Address already in use
2022-03-06 09:00:01,944 ERROR [org.key.qua.run.cli.ExecutionExceptionHandler] (main) ERROR: Address already in use
2022-03-06 09:00:01,944 ERROR [org.key.qua.run.cli.ExecutionExceptionHandler] (main) For more details run the same command passing the '--verbose' option.

Validate Export File
# ls -l /tmp/imports/full-db-export-X.json
-rwxr-xr-x 1 keycloak root 228078 Mar  6 09:00 /tmp/imports/full-db-export-X.json

bash-4.4$ grep -i testadmin /tmp/imports/full-db-export-X.json
    "username" : "testadmin",
bash-4.4$ grep -i testuser /tmp/imports/full-db-export-X.json
    "username" : "testuser",

Ignore the Address already in use Error ( still need to find out how to fix this error message).

Run a full database import

Start a clean keycloak instance by mapping the shared volume

  • Start a clean keycloak instance by mapping the shared volume
    • see above Chapter: Docker Command to start Keycloak-X
  • For import use flag -Dkeycloak.profile.feature.upload_scripts=enabled

Run the full database import

Get docker ID
$ docker ps
CONTAINER ID   IMAGE                                COMMAND                  CREATED          STATUS          PORTS                                            NAMES
a85ae8284b58   quay.io/keycloak/keycloak-x:16.1.1   "/opt/keycloak/bin/k…"   27 seconds ago   Up 26 seconds   0.0.0.0:8280->8080/tcp, 0.0.0.0:8543->8443/tcp   keycloak

Login into container
$ docker exec -it a85ae8284b58 bash


Validate that export file is available
bash-4.4$  ls /tmp/imports/full-db-export-X.json
/tmp/imports/full-db-export-X.json

Run the full db import

bash-4.4$ /opt/keycloak/bin/kc.sh import --file /tmp/imports/full-db-export.json   -Dkeycloak.profile.feature.upload_scripts=enabled
2022-03-06 09:13:44,896 WARN  [org.key.com.Profile] (main) Deprecated feature enabled: upload_scripts
2022-03-06 09:13:44,898 WARN  [org.key.com.Profile] (main) Preview feature enabled: scripts
2022-03-06 09:13:44,975 INFO  [org.key.qua.run.hos.DefaultHostnameProvider] (main) Hostname settings: FrontEnd: <request>, Strict HTTPS: false, Path: <request>, Strict BackChannel: false, Admin: <request>
2022-03-06 09:13:45,549 WARN  [org.inf.PERSISTENCE] (keycloak-cache-init) ISPN000554: jboss-marshalling is deprecated and planned for removal
2022-03-06 09:13:45,631 WARN  [org.inf.CONFIG] (keycloak-cache-init) ISPN000569: Unable to persist Infinispan internal caches as no global state enabled
2022-03-06 09:13:45,669 INFO  [org.inf.CONTAINER] (keycloak-cache-init) ISPN000556: Starting user marshaller 'org.infinispan.jboss.marshalling.core.JBossUserMarshaller'
2022-03-06 09:13:45,909 INFO  [org.inf.CONTAINER] (keycloak-cache-init) ISPN000128: Infinispan version: Infinispan 'Triskaidekaphobia' 13.0.0.Final
2022-03-06 09:13:46,493 INFO  [org.key.con.inf.DefaultInfinispanConnectionProviderFactory] (main) Node name: node_988524, Site name: null
2022-03-06 09:13:47,048 INFO  [org.key.services] (main) KC-SERVICES0030: Full model import requested. Strategy: OVERWRITE_EXISTING
2022-03-06 09:13:47,048 INFO  [org.key.exp.sin.SingleFileImportProvider] (main) Full importing from file /tmp/imports/full-db-export.json
2022-03-06 09:13:47,055 INFO  [org.key.exp.uti.ImportUtils] (main) Realm 'master' already exists. Removing it before import
2022-03-06 09:13:50,349 INFO  [org.key.exp.uti.ImportUtils] (main) Realm 'master' imported
2022-03-06 09:13:52,142 INFO  [org.key.exp.uti.ImportUtils] (main) Realm 'RBAC' imported
2022-03-06 09:13:54,010 INFO  [org.key.exp.uti.ImportUtils] (main) Realm 'quarkus' imported
2022-03-06 09:13:54,174 INFO  [org.key.services] (main) KC-SERVICES0032: Import finished successfully
2022-03-06 09:13:54,361 ERROR [org.key.services] (main) KC-SERVICES0010: Failed to add user 'admin' to realm 'master': user with username exists
2022-03-06 09:13:54,644 ERROR [org.key.qua.run.cli.ExecutionExceptionHandler] (main) ERROR: Failed to start server using profile (import_export)
2022-03-06 09:13:54,645 ERROR [org.key.qua.run.cli.ExecutionExceptionHandler] (main) ERROR: Unable to start HTTP server
2022-03-06 09:13:54,645 ERROR [org.key.qua.run.cli.ExecutionExceptionHandler] (main) ERROR: java.net.BindException: Address already in use
2022-03-06 09:13:54,645 ERROR [org.key.qua.run.cli.ExecutionExceptionHandler] (main) ERROR: Address already in use
2022-03-06 09:13:54,646 ERROR [org.key.qua.run.cli.ExecutionExceptionHandler] (main) For more details run the same command passing the '--verbose' option.

Ignore the Address already in use Error ( still need to find out how to fix this error message).

Validate Import Log Files

2022-03-06 09:13:47,048 INFO  [org.key.services] (main) KC-SERVICES0030: Full model import requested. Strategy: OVERWRITE_EXISTING
2022-03-06 09:13:47,048 INFO  [org.key.exp.sin.SingleFileImportProvider] (main) Full importing from file /tmp/imports/full-db-export.json
2022-03-06 09:13:47,055 INFO  [org.key.exp.uti.ImportUtils] (main) Realm 'master' already exists. Removing it before import
2022-03-06 09:13:50,349 INFO  [org.key.exp.uti.ImportUtils] (main) Realm 'master' imported
2022-03-06 09:13:52,142 INFO  [org.key.exp.uti.ImportUtils] (main) Realm 'RBAC' imported
2022-03-06 09:13:54,010 INFO  [org.key.exp.uti.ImportUtils] (main) Realm 'quarkus' imported
2022-03-06 09:13:54,174 INFO  [org.key.services] (main) KC-SERVICES0032: Import finished successfully

References

Published inKeycloak-X

Be First to Comment

Leave a Reply