Subscribe to the Ardan Labs Insider

You’ll get our FREE Video Series & special offers on upcoming training events along with notifications on our latest blog posts.

Included in your subscription
  • Access to our free video previews
  • Updates on our latest blog posts
  • Discounts on upcoming events

Valid email required.

Submit failed. Try again or message us directly at hello@ardanlabs.com.

Thank You for Subscribing

Check your email for confirmation.

Blockchain In Go: Part I: Digital Accounts, Signatures and Verification

Author image
William Kennedy

Introduction

This is the first post in a series that will explore the semantics and implementation details of the Ardan blockchain project. The code is a reference implementation of a blockchain and not intended to mirror any specific blockchain in use today. Even though the code has been engineered with production level coding standards, I wouldn’t use this project for anything more than learning.

I am using the Ethereum project as a reference and have taken inspiration from that code. I’m hoping that understanding the code inside the Ardan blockchain will give you enough knowledge to understand the code that runs Ethereum.

There are four aspects of a blockchain that this series will explore with a backing implementation provided by the Ardan blockchain project.

This post will focus on the first aspect, how the Ardan blockchain provides support for digital accounts, signatures, and verification.

Source Code

The source code for the Ardan blockchain project can be found at the link below.

https://github.com/ardanlabs/blockchain

It’s important to understand this code is still a work-in-progress and that things are being refactored as more is learned.

Genesis

Each blockchain has a genesis file that provides the global settings and initial state of the blockchain. For the Ardan blockchain, I’ve created a genesis file as well.

Listing 1: Genesis File

{
    "date": "2021-12-17T00:00:00.000000000Z",
    "chain_id": "the-ardan-blockchain",
    "difficulty": 6,
    "transactions_per_block": 2,
    "mining_reward": 700,
    "gas_price": 15,
    "balance_sheet": {
        "0xF01813E4B85e178A83e29B8E7bF26BD830a25f32": 1000000,
        "0xdd6B972ffcc631a62CAE1BB9d80b7ff429c8ebA4": 1000000
    }
}

Note: I was able to find this document that describes what the Ethereum genesis file looks like with good explanations.

In listing 1, you can see the genesis file for the Ardan blockchain. These settings might not make a lot of sense to you at the moment, but later posts will expand on these details. For now, please focus on the origin accounts (hex addresses) under the balance sheet. Both myself and Pavel are starting out with one million units of $ARD. That makes us $ARD millionaires in the books.

Note: Ethereum defines a metric system of denominations described as units of ether. The smallest unit is called a wei, which represents one unit (like a penny) and ether, representing 10^18 units (like 1,000,000,000,000,000,000 pennies).

Accounts and Addresses

Look at the origin accounts once again from the genesis file.

Listing 2: Origin Accounts

"balance_sheet": {
    "0xF01813E4B85e178A83e29B8E7bF26BD830a25f32": 1000000,
    "0xdd6B972ffcc631a62CAE1BB9d80b7ff429c8ebA4": 1000000
}

How do I know that those addresses in listing 2 represent the accounts for myself and Pavel?

Just by looking at them I don’t. One aspect of a blockchain is that accounts are anonymous, they are just large hexadecimal numbers. Eventually you will want to transact with your account, and it could become clear that you are the owner. This could be because of assets you say you own or just general activity with others.

How are those addresses generated?

Different blockchains use different addressing schemes, but generally they are generated using the Elliptic Curve Digital Signature Algorithm (ECDSA). The algorithm generates a private/public key pair that can only be used for signing, it can’t be used for encryption. ECDSA is the algorithm that Ethereum uses.

In the Ardan project, I have added several files under the zblock/accounts folder that contain a set of private ECDSA keys based on the Secp256k1 curve.

Listing 3: Accounts Folder

$ ls -l zblock/accounts
-rw-------  1 bill  staff    64B Feb  1 08:49 baba.ecdsa
-rw-------  1 bill  staff    64B Feb  1 08:49 cesar.ecdsa
-rw-------  1 bill  staff    64B Feb  1 08:49 kennedy.ecdsa
-rw-------  1 bill  staff    64B Feb  1 08:49 pavel.ecdsa

Listing 3 shows the different files in the zblock/accounts folder . The Ardan project uses these files to maintain the private keys for the different test accounts. You will see the files for myself and Pavel in that folder.

Listing 4: kennedy.ecdsa

9f332e3700d8fc2446eaf6d15034cf96e0c2745e40353deef032a5dbf1dfed93

Listing 4 shows the hex-encoded version of my private ECDSA key, which represents my account on the Ardan blockchain. The key is used by the Ardan digital wallet to perform activities against the Ardan blockchain on my behalf. You will soon realize that a digital wallet is nothing more than an application that can make network calls to a blockchain node using a private key as your account’s identity.

Note: Different blockchains use different network protocols. As an example, Ethereum uses JSON-RPC.

The Ethereum project has a crypto package that provides support for working with ECDSA. The Ardan project uses this package to manage digital signatures.

Listing 5: Generating Private Key

01 import "github.com/ethereum/go-ethereum/crypto"
02
03 privateKey, err := crypto.GenerateKey()
04 if err != nil {
05     return err
06 }
07
08 file := filepath.Join("zblock/accounts", "kennedy.ecdsa")
09 if err := crypto.SaveECDSA(file, privateKey); err != nil {
10     return err
11 }

Listing 5 shows how the Ardan wallet is generating private key files in the accounts folder for a new user. You can see the use of the GenerateKey function from the Ethereum crypto package on line 03.

Normally the private key is produced by generating a 12 or 24 word seed phrase. This seed phrase represents your account’s private key and is useful for storing your private key offline (paper wallet) or to restore your account if your machine breaks or if you want to use your account from a different machine (mobile, new computer, etc.).

If someone finds your seed phrase, then they have your private key and can configure a wallet application to transfer money out of your account. Never ever, no matter what, share that seed phrase with anyone that you don’t trust. If you need to share a seed phrase, then one solution is to distribute your seed phrase among a shared group of peers using a system like Shamir secrets. A system like this will allow your private key to be recovered in case of emergency (your untimely demise), with no one person controlling your account.

From the private key, you can generate the corresponding public key. The public key represents the identity of your account

Listing 6: Private to Address

01 privateKey, err := crypto.LoadECDSA("kennedy.ecdsa")
02 if err != nil {
03     log.Fatal(err)
04 }
05
06 address := crypto.PubkeyToAddress(privateKey.PublicKey)
07 fmt.Println(address)

Output:
0xF01813E4B85e178A83e29B8E7bF26BD830a25f32

Listing 6 shows how to use the Ethereum crypto package to generate an address from a public key. This address represents your account’s identity on the blockchain. If anyone knows this address they can see what you own and all of your transactions (sending and receiving) on the blockchain..

As an example, I have an account whose address is 0x01D398ECb403BE33Cd6ED8c9Fefa1712Be48d8d8 that I use on the Ethereum blockchain. With this address, you can look up all my transactions.

Figure 1: Etherscan

Figure 1 shows images from etherscan for my account on Ethereum. You can see how I bought Ether from my Coinbase account and then the transactions I executed to buy a name on the Ethereum Name Service (ENS). Since addresses are easy to type wrong, ENS was created to act like a DNS lookup for addresses.

Figure 2: wkennedy.eth

Figure 2 shows how my ENS name wkennedy.eth resolves to the address associated with my account.

Transaction Types

To submit a transaction to the Ardan blockchain, there is specific information that is required to be sent.

Listing 5: User Transaction Types

01 type UserTx struct {
02     Nonce uint   `json:"nonce"`
03     To    string `json:"to"`
04     Value uint   `json:"value"`
05     Tip   uint   `json:"tip"`
06     Data  []byte `json:"data"`
07 }

Listing 5 shows the UserTx type. This type provides information about who is getting money, how much they are getting, and how much of a tip (bonus) the account associated with the blockchain node will receive for successfully storing this transaction in a block. The Data field allows for any extra information to be associated with the transaction.

Notice that this type is missing a field to identify what account is submitting the transaction. This is because the account submitting the transaction must identify themselves by signing the transaction.

Listing 6: Signed Transaction Type

01 type SignedTx struct {
02     UserTx
03     V *big.Int `json:"v"`
04     R *big.Int `json:"r"`
05     S *big.Int `json:"s"`
06 }

Listing 6 shows the SignedTx type. This type embeds the UserTx type and adds three fields that represent the ECDSA signature of the account submitting the transaction. A value of this type needs to be provided when submitting a transaction to the Ardan blockchain.

Signing A Transaction

How can the UserTx be signed by a wallet so it can be submitted to the blockchain? It starts by hashing the user transaction into a slice of 32 bytes.

Listing 7: Hashing

01 func (tx UserTx) HashWithArdanStamp() ([]byte, error) {
02     txData, err := json.Marshal(tx)
03     if err != nil {
04         return nil, err
05     }
06
07     txHash := crypto.Keccak256Hash(txData)
08     stamp := []byte("\x19Ardan Signed Message:\n32")
09     tran := crypto.Keccak256Hash(stamp, txHash.Bytes())
10
11     return tran.Bytes(), nil
12 }

Listing 7 shows the HashWithArdanStamp method for the UserTx type. This function returns a hash of 32 bytes that represents a user transaction with the Ardan stamp embedded into the final hash. This final hash is used for creating signatures, public key extraction, and signature validation.

On line 02, the receiver value is marshaled into a slice of bytes which is then run through a hash function on line 07 to produce an array of 32 bytes that represent the marshaled transaction data. On line 08, the Ardan stamp is converted into a slice of bytes so it can be joined with the hash of the transaction data on line 09 to produce a final hash of 32 bytes used to represent the transaction.

The Ardan stamp is used to ensure that signatures produced when signing transactions are always unique to the Ardan blockchain. Ethereum does this as well using the same format.

Listing 8: Blockchain Stamps

Ethereum Stamp Format
\x19Ethereum Signed Message:\n + length(message) + message

Ardan Stamp
"\x19Ardan Signed Message:\n32" + dataHash

Note: Ethereum calls this a signature, not a stamp. I find this confusing since this string is being used to salt the hashed transaction data to be signed. It’s less confusing for me to think of this as a stamp.

By forcing the length of the marshaled transaction data’s hash to 32 bytes, the length can be hard-coded into the stamp (\n32) to simplify things. Ethereum uses this trick.

Using the HashWithArdanStamp method, a transaction can now be signed and prepared for submission to the blockchain.

Listing 9: Signing

01 func (tx UserTx) Sign(privateKey *ecdsa.PrivateKey) (SignedTx, error) {
02
03     // Prepare the transaction for signing.
04     tran, err := tx.HashWithArdanStamp()
05     if err != nil {
06         return SignedTx{}, err
07     }
08
09     // Sign the hash with the private key to produce a signature.
10     sig, err := crypto.Sign(tran.Bytes(), privateKey)
11     if err != nil {
12         return SignedTx{}, err
13     }
14
15     // Convert the 65 byte signature into the [R|S|V] format.
16     v, r, s := toSignatureValues(sig)
17
18     // Construct the signed transaction.
19     signedTx := SignedTx{
20         UserTx: tx,
21         V:      v,
22         R:      r,
23         S:      s,
24     }
25
26     return signedTx, nil
27 }

Listing 9 shows the Sign method for the UserTx type. On line 04, the HashWithArdanStamp method we just discussed is used to generate the hash of data to be signed. Then on line 10, a call to the Sign function from the crypto package is used to sign the transaction with the account’s private key.

The signature that is returned is a slice of 65 bytes using the ECDSA format of [R | S | V]. The first 32 bytes represent the R value, the next 32 bytes represent the S value, and the final byte represents the V value.

Note: If you want to learn more about the R, S, and V values, read this excellent post.

Next, the call to the toSignatureValues function on line 16 converts the 65 bytes of the signature into the individual R, S and V values. Since Ethereum stores the signature as R, S, and V inside their different transaction types , I decided to follow suit.

Listing 10: Signature Bytes To Values

01 const ardanID = 29
02
03 func toSignatureValues(sig []byte) (r, s, v *big.Int) {
04    r = new(big.Int).SetBytes(sig[:32])
05    s = new(big.Int).SetBytes(sig[32:64])
06    v = new(big.Int).SetBytes([]byte{sig[64] + ardanID})
07
08    return r, s, v
09 }

Listing 10 shows the toSignatureValues function. Something that Ethereum and Bitcoin do with signatures is add an arbitrary number to V. This is done to make it clear the signature comes from their blockchains. The arbitrary number that Ethereum and Bitcoin use is 27. For the Ardan blockchain, I decided to use 29. It’s important to note that this arbitrary number needs to be subtracted before the signature can be used in any crypto operations.

Listing 11: Signature Values To Bytes

01 func toSignatureBytes(v, r, s *big.Int) []byte {
02    sig := make([]byte, crypto.SignatureLength)
03
04    copy(sig, r.Bytes())
05    copy(sig[32:], s.Bytes())
06    sig[64] = byte(v.Uint64() - ardanID)
07
08    return sig
09 }
10
11 func toSignatureBytesForDisplay(v, r, s *big.Int) []byte {
12     sig := make([]byte, crypto.SignatureLength)
13
14     copy(sig, r.Bytes())
15     copy(sig[32:], s.Bytes())
16     sig[64] = byte(v.Uint64())
17
18     return sig
19 }

Listing 11 shows two functions that convert R, S, and V back into the slice of 65 bytes. The toSignatureBytes function removes the ardanID from V so the value goes back to 0 or 1. The toSignatureBytesForDisplay function keeps the ardanID in V and is used for display purposes.

Signature To Address

Now that you know how to sign a transaction, next you need to see how a blockchain node uses the signature to extract the public key to identify the address of the account that submitted the transaction.

Listing 12: Address

01 type BlockTx struct {
02     SignedTx
03     Gas uint `json:"gas"`
04 }
05
06 func (tx BlockTx) FromAddress() (string, error) {
07
08     // Prepare the transaction for public key extraction.
09     tran, err := tx.HashWithArdanStamp()
10     if err != nil {
11         return "", err
13     }
14
15     // Convert the [R|S|V] format into the original 65 bytes.
16     sig := toSignatureBytes(tx.V, tx.R, tx.S)
17
18     // Capture the public key associated with this signature.
19     publicKey, err := crypto.SigToPub(tran, sig)
20     if err != nil {
21         return "", err
22     }
23
24     // Extract the account address from the public key.
25     return crypto.PubkeyToAddress(*publicKey).String(), nil
26 }

Listing 12 shows the FromAddress method for the BlockTx type. This transaction type is what the blockchain uses after receiving a SignedTx value from a submitted transaction . This method is executed by the blockchain node to retrieve the address of the account that signed the transaction.

On line 09, the HashWithArdanStamp method is used to recreate the hash that was used to produce the received signature. That data needs to be exactly the same or the blockchain node will determine the wrong public key. On line 16, the R, S and V values are converted into the slice of 65 bytes so the signature can be used with the SigToPub function from the crypto package on line 19.

Now with the public key in hand, the PubkeyToAddress function from the crypto package can be used on line 25 to extract the address of the account that submitted and signed the transaction.

Validating A Signature

The last step is the ability for the blockchain node to validate a signature.

Listing 13: Validate a Signature

01 func (tx SignedTx) VerifySignature() error {
       . . . CHECKS ARE HERE . . .
36 }

Listing 13 shows the VerifySignature method from the SignedTx type. There are 3 checks that are performed to validate the signature provided with the transaction data represents the correct account.

Listing 14: Check Recovery ID

03     // Check the recovery id is either 0 or 1.
04     v := tx.V.Uint64() - ardanID
05     if v != 0 && v != 1 {
06         return errors.New("invalid recovery id")
07     }

Listing 14 shows the first check that is performed, making sure the recovery id is set with the ardan id. If this signature was not produced by the Sign function we reviewed earlier, subtracting the ardan id would not result in a value of 0 or 1.

Listing 15: Check Signature Values

09     // Check the signature values are valid.
10     if !crypto.ValidateSignatureValues(byte(v), tx.R, tx.S, false) {
11         return errors.New("invalid signature values")
12     }

Listing 15 shows the second check, to validate the entire signature is valid. This is done with the ValidateSignatureValues function from the crypto package. The function accepts the signature with the individual R, S, and V values. The last parameter of the function is to know if the signature is part of the Homestead release of Ethereum. Homestead is the second major version of the Ethereum platform and the first production release of Ethereum.

Note: There have been 13 different releases of Ethereum with London being the current release as of the writing of this post. To see the different releases, look at this Wikipedia page.

Listing 16: Capture the Public Key

14     // Prepare the transaction for recovery and validation.
15     tran, err := tx.HashWithArdanStamp()
16     if err != nil {
17         return err
18     }
19
20     // Convert the [R|S|V] format into the original 65 bytes.
21     sig := toSignatureBytes(tx.V, tx.R, tx.S)
22
23     // Capture the uncompressed public key associated with this signature.
24     sigPublicKey, err := crypto.Ecrecover(tran, sig)
25     if err != nil {
26         return fmt.Errorf("ecrecover, %w", err)
27     }

Listing 16 doesn’t perform a check, but is code needed to extract the public key that was used to sign the transaction. On lines 15 and 21, the transaction data is hashed with the Ardan stamp and the signature is converted to the slice of 65 bytes. Then on line 24, the public key can be extracted. I’m using the Ecrecover function from the crypto package because I need the public key as an uncompressed slice of 33 bytes for the next call.

Listing 17: Check Public Key Created Signature Over Data

29     // Check that the given public key created the signature over the data.
30     rs := sig[:crypto.RecoveryIDOffset]
31     if !crypto.VerifySignature(sigPublicKey, tran, rs) {
32         return errors.New("invalid signature")
33     }

Listing 17 shows the next call and last check to be performed. The VerifySignature function from the crypto package is called on line 31. This function needs the uncompressed 33 byte public key, the hashed and stamped transaction data, and just the R and S values from the signature. This validation is critical to making sure the right account is associated with the transaction.

When the blockchain node receives the SignedTx value, it doesn’t really know if the fields associated with the UserTx portion of the value were the same when the signature was created. If they are not, then a different public key would be produced and therefore a different account would be used. What a perfect hack to take money out of someone’s account.

It’s important that the UserTx values are rehashed. Now the public key, the rehashed UserTx value, and the R and S values of the signature can be used to validate everything is in sync. If the VerifySignature function does not fail, there is certainty that the signature provided in the SignedTx value was produced by the UserTx value and therefore the public key does represent the correct account.

Conclusion

This first post focused on how the Ardan blockchain provides support for digital accounts, signatures and verification. This represents the first of the four aspects of a blockchain that this series will explore with a backing implementation provided by the Ardan blockchain.

In the next post, I will share how the Ardan blockchain stores transactions in memory and how transactions are shared between the different computers on the network.

Go Training

We have taught Go to thousands of developers all around the world since 2014. There is no other company that has been doing it longer and our material has proven to help jump start developers 6 to 12 months ahead of their knowledge of Go. We know what knowledge developers need in order to be productive and efficient when writing software in Go.

Our classes are perfect for both experienced and beginning engineers. We start every class from the beginning and get very detailed about the internals, mechanics, specification, guidelines, best practices and design philosophies. We cover a lot about "if performance matters" with a focus on mechanical sympathy, data oriented design, decoupling and writing production software.

Capital One
Cisco
Visa
Teradata
Red Ventures

Interested in Ultimate Go Corporate Training and special pricing?

Let’s Talk Corporate Training!

Join Our Online
Education Program

Our courses have been designed from training over 4,000 engineers since 2013 and they go beyond just being a language course. Our goal is to challenge every student to think about what they are doing and why.