Add a new storage structure to tally player information.
Upgrade your blockchain in production.
Deal with data migrations and logic upgrades.
If you have been running v1 of your checkers blockchain for a while, games have been created, played on, won, and lost. In this section, you will introduce v1.1 of your blockchain where wins and losses and tallied in a new storage data structure.
This is not done in vain. Instead, looking forward, this is done to support the addition of a leaderboard module for your v2 in the next section.
For now, a good tally should be such that for any player who has ever played it should be possible to access a tally of games won. While you are at it, you will add games lost and forfeited. Fortunately, this is possible because all past games and their outcomes are kept in the chain's state. Migration is a good method to tackle the initial tally.
For the avoidance of doubt, v1 and v1.1 refer to the overall versions of the application, and not to the consensus versions of individual modules, which may change or not. As it happens, your application has a single module, apart from those coming from the Cosmos SDK.
Several things need to be addressed before you can focus all your attention on the migration:
Save and mark as v1 the current data types about to be modified with the new version. Data types that will remain unmodified need not be identified as such.
Prepare your v1.1 blockchain:
Define your new data types.
Add helper functions to encapsulate clearly defined actions.
Adjust the existing code to make use of and update the new data types.
Prepare for your v1-to-v1.1 migration:
Add helper functions to process large amounts of data from the latest chain state of v1.
Add a function to migrate your state from v1 to v1.1.
Make sure you can handle large amounts of data.
Why do you need to make sure you can handle large amounts of data? The full state at the point of migration may well have millions of games. You do not want your process to grind to a halt because of a lack of memory or I/O capacity.
For your convenience, you decide to keep all the migration steps in a new folder, x/checkers/migrations, and subfolders, which needs to be created:
Copy
$ mkdir x/checkers/migrations
Your data types are defined at a given consensus version of the module, not the application level v1. Find out the checkers module's current consensus version:
Copy
func(AppModule)ConsensusVersion()uint64{return2} x checkers module.go View source
Keep a note of it. At some point, you will create a cv2 subfolder (Where cv is short for consensus version) for anything related to the consensus version at this level.
If your migration happened to require the old data structure at an earlier consensus version, you would save the old types here.
It is time to take a closer look at the new data structures being introduced with the version upgrade.
If you feel unsure about creating new data structures with Ignite CLI, look at the previous sections of the exercise again.
To give the new v1.1 information a data structure, you need the following:
Add a set of stats per player: it makes sense to save one struct for each player and to map it by address. Remember that a game is stored at a notionalStoredGame/value/123/(opens new window), where StoredGame/value/(opens new window) is a constant prefix. Similarly, Ignite CLI creates a new constant to use as the prefix for players:
The new PlayerInfo/value/ prefix for players helps differentiate between the value for players and the value for games prefixed with StoredGame/value/.
Now you can safely have both StoredGame/value/123/ and PlayerInfo/value/123/ side by side in storage.
This creates a Protobuf file:
Copy
messagePlayerInfo{string index =1;uint64 wonCount =2;uint64 lostCount =3;uint64 forfeitedCount =4;} proto checkers player_info.proto View source
It also added the map of new objects to the genesis, effectively your v1.1 genesis:
Copy
import"checkers/player_info.proto";messageGenesisState{...+repeatedPlayerInfo playerInfoList =4[(gogoproto.nullable)=false];} proto checkers genesis.proto View source
You will use the player's address as a key to the map.
With the structure set up, it is time to add the code using these new elements in normal operations, before thinking about any migration.
You can easily call this from these public one-liner functions added to the keeper:
Copy
func(k *Keeper)MustAddWonGameResultToPlayer(ctx sdk.Context, player sdk.AccAddress) types.PlayerInfo {returnmustAddDeltaGameResultToPlayer(k, ctx, player,1,0,0)}func(k *Keeper)MustAddLostGameResultToPlayer(ctx sdk.Context, player sdk.AccAddress) types.PlayerInfo {returnmustAddDeltaGameResultToPlayer(k, ctx, player,0,1,0)}func(k *Keeper)MustAddForfeitedGameResultToPlayer(ctx sdk.Context, player sdk.AccAddress) types.PlayerInfo {returnmustAddDeltaGameResultToPlayer(k, ctx, player,0,0,1)} x checkers keeper player_info_handler.go View source
Which player should get +1, and on what count? You need to identify the loser and the winner of a game to determine this. Create another private helper:
This completes your checkers v1.1 chain. If you were to start it anew as is, it would work. However, you already have the v1 of checkers running, so you need to migrate everything.
Your checkers module's current consensus version is 2. You are about to migrate its store, so you need to increment the module's consensus version by 1 exactly (to avoid any future surprises). You should make these numbers explicit:
Coming back to the store migration, in other words, you need to tackle the creation of player information. You will build the player information by extracting it from all the existing stored games. In the map/reduce(opens new window) parlance, you will reduce this information from the stored games.
If performance and hardware constraints were not an issue, an easy way to do it would be the following:
Call keeper.GetPlayerInfo or, if that is not found, create player info both for the black player and the red player.
Do +1 on .WonCount or .LostCount according to the game.Winner field. In the current saved state, there is no way to differentiate between a normal win and a win by forfeit.
Call keeper.SetPlayerInfo for both black and red players.
Of course, given inevitable resource limitations, you would run into the following problems:
Getting all the games in a single array may not be possible, because your node's RAM may not be able to keep a million of them in memory. Or maybe it fails at 100,000 of them.
Calling .GetPlayerInfo and .SetPlayerInfo twice per game just to do +1 adds up quickly. Remember that both of these calls are database calls. You could be facing a 12-hour job, during which your chain is offline.
Doing it all in a sequential manner would take even more time, as each blocking call blocks the whole process.
Fortunately, there exist ways to mitigate these limitations:
You do not need to get all the games at once. The keeper.StoredGameAll(opens new window) function offers pagination. With this, you can limit the impact on the RAM requirement, at the expense of multiple queries.
Within each subset of games, you can compute in memory the player list and how many wins and losses each player has. With this mapping done, you can add the (in-memory) intermediary WonCount and LostCount sums to each player's stored sums. With this, a +1 is potentially replaced by a +k, at once reducing the number of calls to .GetPlayerInfo and .SetPlayerInfo.
You can separate the different calls and computations into Go routines(opens new window) so that a blocking call does not prevent other computations from taking place in the meantime.
The routines will use channels to communicate between themselves and the main function:
A stored-game channel, that will pass along chunks of games in the []types.StoredGame format.
A player-info channel, that will pass along intermediate computations of player information in the simple types.PlayerInfo format.
A done channel, whose only purpose is to flag to the main function when all has been processed.
Each channel should also be able to pass an optional error, so tuples will be used.
The processing routines will be divided as per the following:
The game loading routine will:
Fetch all games in paginated arrays.
Send the separate arrays on the stored-game channel.
Send an error on the stored-game channel if any is encountered.
Close the stored-game channel after the last array, or on an error.
The game processing routine will:
Receive separate arrays of games from the stored-game channel.
Compute the aggregate player info records from them (i.e. map).
Send the results on the player-info channel.
Pass along an error if it receives any.
Close the player-info channel after the last stored game, or on an error.
The player info processing routine will:
Receive individual player info records from the player-info channel.
Fetch the corresponding player info from the store. If it does not exist yet, it will create an empty new one.
Update the won and lost counts (i.e. reduce). Remember, here it is doing += k, not += 1.
Save it back to the store.
Pass along an error if it receives any.
Close the done channel after the last player info, or on an error.
The processing will take your module's store from consensus version 2 to version 3. Therefore it makes sense to add the function in x/checkers/migrations/cv3/keeper.
The player info processing will handle an in-memory map of player addresses to their information: map[string]*types.PlayerInfo. Create a new file to encapsulate this whole processing. Start by creating a helper that automatically populates it with empty values when information is missing:
The helper function passes along the channel a tuple storedGamesChunk that may contain an error. This is to obtain a result similar to when a function returns an optional error .
It uses the paginated query so as to not overwhelm the memory if there are millions of infos.
It closes the channel upon exit whether there is an error or not via the use of defer.
Next, create the routine function to process the games:
This function can handle the edge case where black and red both refer to the same player.
It prepares a map with a capacity equal to the number of games. At most the capacity would be double that. This is a value that could be worth investigating for best performance.
Like the helper function, it passes along a tuple with an optional error.
It closes the channel it populates upon exit whether there is an error or not via the use of defer.
Create the routine function to process the player info:
The migration proper needs to execute the previous main function. You can encapsulate this knowledge in a function, which also makes more visible what is expected to take place:
Copy
package cv3
import("github.com/b9lab/checkers/x/checkers/keeper"
cv3Keeper "github.com/b9lab/checkers/x/checkers/migrations/cv3/keeper"
sdk "github.com/cosmos/cosmos-sdk/types")funcPerformMigration(ctx sdk.Context, k keeper.Keeper, storedGameChunk uint64)error{
ctx.Logger().Info("Start to compute checkers games to player info calculation...")
err := cv3Keeper.MapStoredGamesReduceToPlayerInfo(ctx, k, storedGameChunk)if err !=nil{
ctx.Logger().Error("Checkers games to player info computation ended with error")}else{
ctx.Logger().Info("Checkers games to player info computation done")}return err
} x checkers ... cv3 migration.go View source
This does not panic in case of an error. To avoid carrying on a faulty state, the caller of this function will have to handle the panic.
You have in place the functions that will handle the store migration. Now you have to set up the chain of command for these functions to be called by the node at the right point in time.
The upgrade module keeps in its store the different module versions(opens new window) that are currently running. To signal an upgrade, your module needs to return a different value when queried by the upgrade module. You have already prepared this change from 2 to 3.
The consensus version number bears no resemblance to v1 or v1.1. The consensus version number is for the module, whereas v1 or v1.1 is for the whole application.
You also have to pick a name for the upgrade you have prepared. This name will identify your specific upgrade when it is mentioned in a Plan (i.e. an upgrade governance proposal). This is a name relevant at the application level. Keep this information in a sub-folder of app:
The function that you are going to write needs a Configurator. This is already created as part of your app preparation, but it is not kept. Instead of recreating one, adjust your code to make it easily available. Add this field to your app:
Copy
type App struct{... sm *module.SimulationManager
+ configurator module.Configurator
} app app.go View source
Now adjust the place where the configurator is created:
Create a function that encapsulates knowledge about all possible upgrades, although there is a single one here. Since it includes empty code for future use, avoid cluttering the already long NewApp function:
Copy
import("github.com/b9lab/checkers/app/upgrades/v1tov1_1"
storetypes "github.com/cosmos/cosmos-sdk/store/types")func(app *App)setupUpgradeHandlers(){// v1 to v1.1 upgrade handler
app.UpgradeKeeper.SetUpgradeHandler(
v1tov1_1.UpgradeName,func(ctx sdk.Context, plan upgradetypes.Plan, vm module.VersionMap)(module.VersionMap,error){return app.mm.RunMigrations(ctx, app.configurator, vm)},)// When a planned update height is reached, the old binary will panic// writing on disk the height and name of the update that triggered it// This will read that value, and execute the preparations for the upgrade.
upgradeInfo, err := app.UpgradeKeeper.ReadUpgradeInfoFromDisk()if err !=nil{panic(fmt.Errorf("failed to read upgrade info from disk: %w", err))}if app.UpgradeKeeper.IsSkipHeight(upgradeInfo.Height){return}var storeUpgrades *storetypes.StoreUpgrades
switch upgradeInfo.Name {case v1tov1_1.UpgradeName:}if storeUpgrades !=nil{// configure store loader that checks if version == upgradeHeight and applies store upgrades
app.SetStoreLoader(upgradetypes.UpgradeStoreLoader(upgradeInfo.Height, storeUpgrades))}} app app.go View source
Now you are ready to inform the app proper. You do this towards the end, after the call to app.SetEndBlocker and before if loadLatest. At the correct location:
Be aware that the monitoring module added by Ignite causes difficulty when experimenting below with the CLI. To keep things simple, remove all references to monitoring(opens new window) from app.go.
When done right, adding the callbacks is a short and easy solution.
With changes made in app.go, unit tests are inadequate – you have to test with integration tests. Take inspiration from the upgrade keeper's own integration tests(opens new window).
In a new folder dedicated to your migration integration tests, copy the test suite and its setup function, which you created earlier for integration tests, minus the unnecessary checkersModuleAddress line:
It is necessary to redeclare, as you cannot import test elements across package boundaries.
The code that runs for these tests is always at consensus version 3. After all, you cannot wish away the player info code during the tests setup. However, you can make the upgrade module believe that it is still at the old state. Add this step into the suite's setup:
Now you can add a test in another file. It verifies that the consensus version increases as saved in the upgrade keeper, when calling an upgrade with the right name.
You can also confirm that it panics if you pass it a wrong upgrade name:
Copy
func(suite *IntegrationTestSuite)TestNotUpgradeConsensusVersion(){
vmBefore := suite.app.UpgradeKeeper.GetModuleVersionMap(suite.ctx)
suite.Require().Equal(cv2types.ConsensusVersion, vmBefore[types.ModuleName])
dummyPlan := upgradetypes.Plan{
Name: v1tov1_1.UpgradeName +"no",
Info:"some text here",
Height:123450000,}deferfunc(){
r :=recover()
suite.Require().NotNil(r,"The code did not panic")
suite.Require().Equal(r,"ApplyUpgrade should never be called without first checking HasHandler")
vmAfter := suite.app.UpgradeKeeper.GetModuleVersionMap(suite.ctx)
suite.Require().Equal(cv2types.ConsensusVersion, vmAfter[types.ModuleName])}()
suite.app.UpgradeKeeper.ApplyUpgrade(suite.ctx, dummyPlan)} tests integration ... cv3 upgrade_test.go View source
After that, you can check that the player infos are tallied as expected by adding in storage three completed games and one still in-play, and then triggering the upgrade:
You can already execute a live upgrade from the command line. The following upgrade process takes inspiration from this one(opens new window) based on Gaia. You will:
Check out the checkers v1 code.
Build the v1 checkers executable.
Initialize a local blockchain and network.
Run v1 checkers.
Add one or more incomplete games.
Add one or more complete games with the help of a CosmJS integration test.
Create a governance proposal to upgrade with the right plan name at an appropriate block height.
Make the proposal pass.
Wait for v1 checkers to halt on its own at the upgrade height.
Check out the checkers v1.1 code.
Build the v1.1 checkers executable.
Run v1.1 checkers.
Confirm that you now have a correct tally of player info.
You should not use docker run --rm here because, when checkersd stops, you do not want to remove the container and thereby destroy the saved keys, and the future genesis too. Instead, reuse them all in the next calls.
Give your players the same token amounts that were added by Ignite, as found in config.yml:
Copy
$ ./release/v1/checkersd add-genesis-account \
alice 200000000stake,20000token --keyring-backend test
$ ./release/v1/checkersd add-genesis-account \
bob 100000000stake,10000token --keyring-backend test
Copy
$ docker exec -t checkers \
./release/v1/checkersd add-genesis-account \
alice 200000000stake,20000token --keyring-backend test
$ docker exec -t checkers \
./release/v1/checkersd add-genesis-account \
bob 100000000stake,10000token --keyring-backend test
To be able to run a quick test, you need to change the voting period of a proposal. This is found in the genesis:
From another shell, create a few un-played games with:
Copy
$ export alice=$(./release/v1/checkersd keys show alice -a --keyring-backend test)
$ export bob=$(./release/v1/checkersd keys show bob -a --keyring-backend test)
$ ./release/v1/checkersd tx checkers create-game \
$alice $bob 10 stake \
--from $alice --keyring-backend test --yes \
--chain-id checkers-1 \
--broadcast-mode block
Copy
$ export alice=$(docker exec checkers ./release/v1/checkersd keys show alice -a --keyring-backend test)
$ export bob=$(docker exec checkers ./release/v1/checkersd keys show bob -a --keyring-backend test)
$ docker exec -t checkers \
./release/v1/checkersd tx checkers create-game \
$alice $bob 10 stake \
--from $alice --keyring-backend test --yes \
--chain-id checkers-1 \
--broadcast-mode block
The --broadcast-mode block flag means that you can fire up many such games by just copying the command without facing any sequence errors.
To get a few complete games, you are going to run the integration tests(opens new window) against it. These tests expect a faucet to be available. Because that is not the case, you need to:
For the software upgrade governance proposal, you want to make sure that it stops the chain not too far in the future but still after the voting period. With a voting period of 10 minutes, take 15 minutes. How many seconds does a block take?
Wait for this period. Afterward, with the same command you should see:
Copy
...
status: PROPOSAL_STATUS_PASSED
...
Now, wait for the chain to reach the desired block height, which should take five more minutes, as per your parameters. When it has reached that height, the shell with the running checkersd should show something like:
Copy
...
6:29PM INF finalizing commit of block hash=E6CB6F1E8CF4699543950F756F3E15AE447701ABAC498CDBA86633AC93A73EE7 height=1180 module=consensus num_txs=0 root=21E51E52AA3F06BE59C78CE11D3171E6F7240D297E4BCEAB07FC5A87957B3BE2
6:29PM ERR UPGRADE "v1tov1_1" NEEDED at height: 1180:
6:29PM ERR CONSENSUS FAILURE!!! err="UPGRADE \"v1tov1_1\" NEEDED at height: 1180: " module=consensus stack="goroutine 62 [running]:\nruntime/debug.Stack
...
6:29PM INF Stopping baseWAL service impl={"Logger":{}} module=consensus wal=/root/.checkers/data/cs.wal/wal
6:29PM INF Stopping Group service impl={"Dir":"/root/.checkers/data/cs.wal","Head":{"ID":"ZsAlN7DEZAbV:/root/.checkers/data/cs.wal/wal","Path":"/root/.checkers/data/cs.wal/wal"},"ID":"group:ZsAlN7DEZAbV:/root/.checkers/data/cs.wal/wal","Logger":{}} module=consensus wal=/root/.checkers/data/cs.wal/wal
...
At this point, run in another shell:
Copy
$ ./release/v1/checkersd status \
| jq -r ".SyncInfo.latest_block_height"
Copy
...
7:06PM INF applying upgrade "v1tov1_1" at height: 1180
7:06PM INF migrating module checkers from version 2 to version 3
7:06PM INF Start to compute checkers games to player info calculation...
7:06PM INF Checkers games to player info computation done
...
After it has started, you can confirm in another shell that you have the expected player info with:
Your checkers blockchain is almost done! It now needs a leaderboard, which is introduced in the next section.
synopsis
To summarize, this section has explored:
How to add a new data structure in storage as a breaking change.
How to upgrade a blockchain in production, by migrating from v1 of the blockchain to v1.1, and the new data structures that will be introduced by the upgrade.
How to handle the data migrations and logic upgrades implicit during migration, such as with the use of private helper functions.
Worthwhile unit tests with regard to player info handling.
Integration tests to further confirm the validity of the upgrade.
A complete procedure for how to conduct the update via the CLI.