Skip to main content

Installing the archive migration package

The archive node Berkeley migration package is sufficient for satisfying the migration from Devnet/Mainnet to Berkeley. However, it has some limitations. For example, the migration package does not migrate a non-canonical chain and it skips orphaned blocks that are not part of a canonical chain.

To mitigate these limitations, the archive node maintenance package is available for use by archive node operators who want to maintain a copy of their Devnet and Mainnet databases for historical reasons.

Install with Google Cloud SDK​

The Google Cloud SDK installer does not always register a google-cloud-sdk apt package. The best way to install gsutil is using the apt repostory:

curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo gpg --dearmor -o /usr/share/keyrings/cloud.google.gpg
echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list
sudo apt-get update && sudo apt-get install google-cloud-sdk

Download the o1labs Mainnet archive database​

We strongly encourage you to perform the migration on your own data to preserve the benefits of decentralization. However, if you want to use the archive data that o1labs runs (for example, to bootstrap a new archive from SQL without waiting all day for the chain to download and replay), you can use the following steps:

  1. Download the Devnet/Mainnet archive data using cURL or gsutil:

    • cURL:

      For Devnet:

      curl https://storage.googleapis.com/mina-archive-dumps/devnet-archive-dump-{date}_0000.sql.tar.gz

      For Mainnet:

      curl https://storage.googleapis.com/mina-archive-dumps/mainnet-archive-dump-{date}_0000.sql.tar.gz

      To filter the dumps by date, replace {date} using the required yyyy-dd-mm format. For example, for March 15, 2024, use 2024-03-15.

      ⚠️ The majority of backups have the 0000 suffix. If a download with that name suffix is not available, try incrementing it. For example, 0001, 0002, and so on.

    • gsutil:

      gsutil cp gs://mina-archive-dumps/mainnet-archive-dump-2024-01-15* .

  2. Extract the tar package.

    tar -xvzf {network}-archive-dump-{date}_0000.sql.tar.gz {network}-archive-dump-{date}_0000.sql

  3. Import the Devnet/Mainnet archive dump into the Berkeley database.

    Run this command at the database server:

    psql -U {user} -f {network}-archive-dump-{date}_0000.sql

    The database in the dump archive_balances_migrated is created with the Devnet/Mainnet archive schema.

    Note: This database does not have any Berkeley changes.

Ensure the location of Google Cloud bucket with the Devnet/Mainnet precomputed blocks​

The recommended method is to perform migration on your own data to preserve the benefits of decentralization.

gsutil cp gs://mina_network_block_data/{network}-*.json .

⚠️ Precomputed blocks for the Mainnet network take ~800 GB of disk space. Plan for adequate time to download these blocks. The Berkeley migration app downloads them incrementally only when needed.

Validate the Devnet/Mainnet database​

The correct Devnet/Mainnet database state is crucial for a successful migration.

Missing blocks is one the most frequent issues when dealing with the Devnet/Mainnet archive. Although this step is optional, it is strongly recommended that you verify the archive condition before you start the migration process.

To learn how to maintain archive data, see Devnet/Mainnet database maintenance.

Download the migration applications​

Migration applications are distributed as part of the archive migration Docker and Debian packages.

Choose the packages that are appropriate for your environment.

Debian packages​

To get the Debian packages:

CODENAME=bullseye
CHANNEL=stable
VERSION=3.0.1-e848ecb

echo "deb [trusted=yes] http://packages.o1test.net $CODENAME $CHANNEL" | tee /etc/apt/sources.list.d/mina.list
apt-get update
apt-get install --allow-downgrades -y "mina-archive-migration=$VERSION"

Docker image​

To get the Docker image:

docker pull gcr.io/o1labs-192920/mina-archive-migration:3.0.1-e848ecb-{codename}

Where supported codenames are:

  • bullseye
  • focal
  • buster

Devnet/Mainnet genesis ledger​

The Mina Devnet/Mainnet genesis ledger is stored in GitHub in the mina repository under the genesis_ledgers subfolder. However, if you are already running a daemon that is connected to the Mina Mainnet or the Devnet network, you already have the genesis ledger locally.

Berkeley database schema files​

You can get the Berkeley schema files from different locations:

  • GitHub repository from the berkeley branch.

    Note: The berkeley branch can contain new updates regarding schema files, so always get the latest schema files instead of using an already downloaded schema.

  • Archive/Rosetta Docker from berkeley version

Example: Downloading schema sources from GitHub​

wget https://raw.githubusercontent.com/MinaProtocol/mina/berkeley/src/app/archive/zkapp_tables.sql

wget https://raw.githubusercontent.com/MinaProtocol/mina/berkeley/src/app/archive/create_schema.sql

Next steps​

Congratulations on completing the essential preparation and verification steps. You are now ready to perform the migration steps in Migrating Devnet/Mainnet Archive to Berkeley Archive.