Persistence: Saving the Chain

So far your blockchain has lived entirely in memory. You mine some blocks, send a payment, print the chain — and then the program exits and everything is gone. That’s fine for a demo, but a real node has to survive restarts. If Bitcoin forgot its ledger every time the process stopped, there would be no ledger at all.

In this chapter we add two functions, save() and load(), that read and write the chain as a JSON file. Along the way we’ll meet serde (Rust’s serialization framework), the std::fs and std::path modules, std::io::Result, and the ? operator. But the most important idea here isn’t any one API — it’s a design decision about what to save.

Why we only save the blocks

Your Blockchain struct has two fields:

pub struct Blockchain {
    pub blocks: Vec<Block>,
    pub utxos: UtxoSet,
}

It would be tempting to serialize the whole struct — blocks and the UTXO set — and load it all back. Don’t. The UTXO set is derived state: it contains nothing that isn’t already implied by the blocks. Every entry in the UTXO set was created by some transaction output in some block and is removed the moment a later transaction spends it. Given the blocks, you can reconstruct the UTXO set exactly, every time, by replaying the chain from genesis.

The chain is the source of truth

This is exactly the distinction drawn in The UTXO Set vs the Chain: the blocks are the canonical history, and the UTXO set is just a fast-access index a node builds over them. Save the history; rebuild the index. Persisting derived state alongside the source of truth only creates a way for the two to disagree.

There’s also a very practical reason. Look at how the UTXO set is keyed — by a (txid, output index) tuple. Stored in a HashMap, that looks like HashMap<(Hash, u32), TxOutput>. JSON has no concept of a tuple as a key: JSON object keys must be strings. So a HashMap with a non-string key like (Hash, u32) can’t be serialized straight to JSON without extra plumbing. Rather than fight that, we sidestep it entirely: we serialize only chain.blocks — a plain Vec<Block>, which serializes cleanly — and rebuild the UTXO set in memory after loading.

The whole module

Here is src/storage.rs in full. It’s short, which is the point — persistence should be boring.

//! Persistence. We only need to save the **blocks** — the UTXO set is derived
//! state, rebuilt by replaying the chain on load.

use std::fs;
use std::path::Path;

use crate::block::Block;
use crate::blockchain::Blockchain;

The use lines pull in what we need. std::fs is the standard library’s filesystem module (reading and writing files). std::path::Path is the standard type for a path on disk — using it instead of a raw &str lets the type system know “this is a filesystem path,” and it works correctly across operating systems. The two crate:: imports bring in our own Block and Blockchain types from elsewhere in the project.

Saving

/// Save the chain's blocks to a JSON file.
pub fn save(chain: &Blockchain, path: &Path) -> std::io::Result<()> {
    let json = serde_json::to_string_pretty(&chain.blocks)
        .expect("blocks are always serializable");
    fs::write(path, json)
}

The function takes the chain by reference (&Blockchain) — we only need to read it, not own or modify it — and a &Path saying where to write. The return type is std::io::Result<()>. std::io::Result<T> is the standard library’s alias for Result<T, std::io::Error>: every filesystem operation can fail (disk full, permission denied), so it returns a Result, and the () means “on success there is no meaningful value, just the fact that it worked.”

serde_json::to_string_pretty(&chain.blocks) is where serde earns its keep. Because Block derives Serialize (we’ll see that below), serde_json can walk the entire Vec<Block> — headers, transactions, hashes, all of it — and produce nicely indented JSON text. We .expect(...) here rather than propagate an error because serialization of our own well-formed structs genuinely can’t fail; if it somehow did, that’s a bug, not a runtime condition to handle.

Finally fs::write(path, json) writes the string to the file, creating or truncating it. fs::write returns std::io::Result<()> — exactly our function’s return type — so we return it directly.

Loading

/// Load a chain from a JSON file, rebuilding the UTXO set. Returns `None` if the
/// file doesn't exist yet.
pub fn load(path: &Path) -> std::io::Result<Option<Blockchain>> {
    if !path.exists() {
        return Ok(None);
    }
    let json = fs::read_to_string(path)?;
    let blocks: Vec<Block> = serde_json::from_str(&json)
        .map_err(|e| std::io::Error::new(std::io::ErrorKind::InvalidData, e))?;
    Ok(Some(Blockchain::from_blocks(blocks)))
}

The return type is std::io::Result<Option<Blockchain>>. That nesting is doing real work: the outer Result covers errors (the file is corrupt or unreadable), while the inner Option covers the perfectly-normal absence of a save file. A brand-new node has nothing saved yet — that’s not an error, it’s just None.

So the first thing we do is check path.exists(). If there’s no file, we return Ok(None) and the caller knows to start a fresh chain.

fs::read_to_string(path)? reads the file into a String. Notice the ? at the end. The ? operator is Rust’s shorthand for “if this is an Err, return it from the function right now; otherwise unwrap the Ok value and keep going.” It only works because our function returns a Result whose error type matches — and read_to_string returns std::io::Result, so it fits.

The next line parses the JSON back into Vec<Block>. The type annotation let blocks: Vec<Block> tells serde_json::from_str what to deserialize into — serde uses it to know the shape to expect. But here the error types don’t line up: serde_json::from_str fails with a serde_json::Error, not a std::io::Error, so a bare ? wouldn’t compile. That’s what map_err fixes: it transforms the error inside an Err while leaving an Ok untouched. We wrap the serde error in a std::io::Error of kind InvalidData, and now the ? applies cleanly. This is a common, idiomatic move — converting one library’s error into the error type your function promises to return.

If parsing succeeds, the last line does the crucial bit:

Ok(Some(Blockchain::from_blocks(blocks)))

Blockchain::from_blocks is the constructor that takes the loaded blocks and replays them to derive the UTXO set:

/// Rebuild a chain from stored blocks, replaying them to derive the UTXO set.
pub fn from_blocks(blocks: Vec<Block>) -> Self {
    let mut utxos = UtxoSet::new();
    for block in &blocks {
        for tx in &block.transactions {
            utxos.add_transaction(tx.txid(), tx);
        }
    }
    Blockchain { blocks, utxos }
}

It starts with an empty UtxoSet and walks every transaction in every block, in order, applying each one. By the time the loops finish, the UTXO set reflects exactly the state implied by the saved history — no more, no less. We never had to serialize that troublesome tuple-keyed HashMap: we recompute it. The blocks are the source of truth; the UTXO set is rebuilt fresh.

Making Block serializable

None of this works unless our types opt into serde. That’s a one-line annotation on each struct, via the derive macro:

#[derive(Clone, Debug, Serialize, Deserialize)]

Serialize lets a value be turned into JSON; Deserialize lets JSON be turned back into a value. Because Block contains a BlockHeader and a Vec<Transaction>, those types derive them too — serde composes recursively, so deriving on the leaves makes the whole tree serializable.

Pretty vs compact

We used to_string_pretty for the save file because a human might want to open it and read it. For data flying over the network in the next chapter, we’ll use plain to_string — compact, one object per line. Same serde, different formatting.

Try it

Run a node that serves and saves, stop it, and start it again — the chain comes back from disk instead of starting empty:

cargo run -- serve 127.0.0.1:9000
# let it mine a couple of blocks, then press Ctrl-C to stop it
cargo run -- serve 127.0.0.1:9000

On the second start you should see the node load the existing blocks rather than beginning from a lone genesis block — proof that save() wrote the history and load() rebuilt the chain (and its UTXO set) from it.

Check your understanding

1. Why do we persist only chain.blocks and not the whole Blockchain struct?
2. What does it mean that the UTXO set is derived state, and which function reconstructs it on load?
3. Why does load return std::io::Result<Option<Blockchain>> — what does the outer Result represent, and what does the inner Option represent?
4. The ? operator works on fs::read_to_string but a bare ? would not work on serde_json::from_str. Why, and how does map_err resolve it?
5. Why can’t a HashMap<(Hash, u32), TxOutput> be serialized directly to JSON?

Show answers

1. Because the UTXO set is derived state — it contains nothing not already implied by the blocks, and can be rebuilt exactly by replaying the chain. Persisting derived state alongside the source of truth only creates a way for the two to disagree; the blocks are the canonical history.
2. It means every UTXO entry was created by some output in some block and removed when a later transaction spent it, so given the blocks you can reconstruct the set exactly. Blockchain::from_blocks reconstructs it on load, replaying every transaction in every block into a fresh UtxoSet.
3. The outer Result represents an error (the file is corrupt or unreadable); the inner Option represents the perfectly-normal absence of a save file (a brand-new node has nothing saved — that’s None, not an error).
4. fs::read_to_string returns std::io::Result, whose error type matches the function’s return type, so a bare ? works. serde_json::from_str fails with a serde_json::Error, a different type, so ? wouldn’t compile — map_err converts that error into a std::io::Error (kind InvalidData) first, after which ? applies cleanly.
5. Because JSON object keys must be strings, and the map’s key is a tuple (Hash, u32), not a string. A non-string key can’t serialize straight to JSON without extra plumbing — which is exactly why we save only chain.blocks (a plain Vec<Block>) and rebuild the set in memory.