This procedure covers backing up and restoring the Institutional Vault database state. Use it for:

- **Pre-upgrade backups** — Always take a backup before performing a [wallet upgrade](https://vault.docs.blockdaemon.com/docs/how-to-perform-wallet-upgrade). This is your rollback safety net if the upgrade introduces issues.
- **Disaster recovery** — Restore from a backup in the event of logical data corruption or human error.
- **Environment migration** — Restore a backup into a different environment (requires updating Encryptor Master Password secrets).

If the database is restored within the **same environment**, no secret changes are required. If the restoration is performed in a **different environment**, other secrets such as database connection strings, hostnames, and service credentials can be updated to match the new environment. However, the **Encryptor Master Password** secrets must be carried over unchanged from the original environment — they cannot be regenerated or substituted.

> ⚠️
>
> ### Warning:  
> The Encryptor Master Password secrets are used to decrypt sensitive data stored in the database (key shares, encrypted configuration). If these secrets are lost or changed, the encrypted data becomes irrecoverable. These secrets are stored in your cloud provider's secret manager (Azure Key Vault or AWS Secrets Manager).

## Step 1: Take a backup  
Before any disruptive operation (upgrade, migration, or maintenance), create a point-in-time database backup using your cloud provider's managed database backup service or a manual dump.

**Managed database services (recommended):**

- **Azure Database for PostgreSQL** — Use the Azure portal or CLI to create an on-demand backup, or verify that automatic backups with point-in-time restore are enabled.
- **Amazon RDS for PostgreSQL** — Create a manual snapshot via the AWS Console or CLI (`aws rds create-db-snapshot`).

**Manual backup (self-managed PostgreSQL):**

```undefined
pg_dump --host <hostname> --port 5432 --username postgres --format=custom --file wallet-backup-$(date +%Y%m%d-%H%M%S).dump <dbname>
```

Regardless of method, store the backup in a secure, durable location and record the timestamp. Verify the backup completed successfully before proceeding.

## Step 2: Shutdown services  
Shut down wallet services to ensure a consistent restore without data races. Follow this shutdown order:

1. Stop the **wallet** service.
2. Stop the **policy-node0**, **policy-node1**, and **policy-node2** services.
3. Stop the **nats** service.

## Step 3: Restore database  
There are two scenarios for restoring the database.

### In-place restore  
When restoring to the same database instance, use your provider's point-in-time restore feature or manually restore from the dump.

**Managed database services:**

- **Azure Database for PostgreSQL** — Use point-in-time restore to create a restored instance at the backup timestamp, then swap connection strings.
- **Amazon RDS for PostgreSQL** — Restore the manual snapshot to the same instance or a new one.

**Manual restore (self-managed PostgreSQL):**

```undefined
pg_restore --dbname=<dbname> --host <hostname> --port 5432 --username postgres --clean --if-exists wallet-backup-YYYYMMDD-HHMMSS.dump
```

### Redirected restore  
When restoring to a different database instance (e.g., a new managed cloud SQL instance), update the database connection configuration for the following services:

- **Orchestrator**
- **Policy-node0, policy-node1, policy-node2**
- **Wallet**

For automated deployments (AWS or Azure), modify the `DB_HOST` value in the database configuration secrets:

| Service | Secret location |
| --- | --- |
| Orchestrator | `<environment>.<namespace>.orchestrator.db-info` |
| Policy-node0 | `<environment>.<namespace>.policy-node0.db-info` |
| Policy-node1 | `<environment>.<namespace>.policy-node1.db-info` |
| Policy-node2 | `<environment>.<namespace>.policy-node2.db-info` |
| Wallet | `<environment>.<namespace>.wallet.db-info` |

## Step 4: Start services  
Bring services back online in dependency order:

1. Start **nats**.
2. Start **policy-node0**, **policy-node1**, and **policy-node2**.
3. Start the **wallet** service.

## Step 5: Verify restoration  
After services are running:

1. Check all pods are healthy: `kubectl get pods -n <namespace>`
2. Verify the wallet health endpoint: `curl -s https://<your-wallet-domain>/health`
3. Confirm account data is present by listing accounts in the UI or via `GET /api/v2/accounts`.

> ⚠️
>
> ### Warning:  
> After restoring to a backup's point in time, any transaction history beyond that point will be lost. Account balances will automatically update to reflect on-chain values as the indexer re-syncs. Pending operations that were in-flight at backup time may need to be re-submitted.

## Related documentation  
- [Wallet Upgrade Processes](https://vault.docs.blockdaemon.com/docs/how-to-perform-wallet-upgrade) — Always take a backup before upgrading.
- [Troubleshooting Guide](https://vault.docs.blockdaemon.com/docs/troubleshooting-guide) — Common error codes and resolutions.
- [Institutional Vault Release Notes](https://vault.docs.blockdaemon.com/changelog) — Review breaking changes before any upgrade.

Please contact us via [email](mailto:support@blockdaemon.com) or support chat if you encounter an issue, bug, or need assistance.