This book aims to document the Monero protocol. Currently, work is being done to document Monero's consensus rules.
This being completed as a part of Cuprate, the Rust Monero node.
Consensus Rules
This chapter contains all of Monero's consensus rules, from genesis to now. Some rules are complex so have been split into their own chapter.
Rules that are not bound to consensus (relay rules) are not included here. Also we have not documented "rules" which are enforced by (de)serialization, for example it's impossible to have a ringCT signature in a version 1 transaction, rules that are unclear if they can be omitted or not should always be included.
Index
Definitions
Canonically Encoded Scalar: an Ed25519 scalar which is fully reduced mod l, where \(l = 2^{252} + 27742317777372353535851937790883648493 \).
Canonically Encoded Point: an Ed25519 point which is not the negative identity and with y coordinate fully reduced mod p, where \(p = 2^{255} - 19 \).
Prime Order Point: a point in the prime subgroup.
PoW Hash: the hash calculated from the block hashing blob by using the active proof of work function.
Block Hash:
the keccak hash of the block hashing blob, this is a slightly different hashing blob than the one used to calculate the PoW Hash
.
Transaction Blob: the raw bytes of a serialized transaction.
Block Blob: the raw bytes of a serialized block.
Chain Height: the amount of blocks in the chain, this is different to the height of the top block as blocks start counting at 0.
Ring (transactions inputs): the group of potential outputs of which one is the true spend.
Decoys (transactions inputs):
the fake outputs used to hide the true spend, the length of this is equal to one minus the Rings
length.
MixIns (transactions inputs):
another term for Decoys
Genesis
Monero has a hardcoded genesis block that gets added to the blockchain on the first run of the daemon1. The contents of this block are different depending on the network.
For all networks the timestamp is set to 0, the major and minor version of the block are set to CURRENT_BLOCK_MAJOR_VERSION
and
CURRENT_BLOCK_MINOR_VERSION
2. These two constants are set to 1 and 0 respectively3. The transaction
field is empty, and the previous block hash is not set so that field is zeroed.
Mainnet
The nonce is set to 10,000 and the miner transaction is set to:
013c01ff0001ffffffffffff03029b2e4c0281c0b02e7c53291a94d1d0cbff8883f8024f5142ee494ffbbd08807121017767aafcde9be00dcfd098715ebcf7f410daebc582fda69d24a28e9d0bc890d1
4
The mainnet genesis block will hash to: 418015bb9ae982a1975da7d79277c2705727a56894ba0fb246adaabb1f4632e3
.
The final block:
{
header: {
major_version: 1,
minor_version: 0,
timestamp: 0,
previous: [0; 32],
nonce: 10000
},
miner_tx: "013c01ff0001ffffffffffff03029b2e4c0281c0b02e7c53291a94d1d0cbff8883f8024f5142ee494ffbbd08807121017767aafcde9be00dcfd098715ebcf7f410daebc582fda69d24a28e9d0bc890d1",
txs: [],
}
Testnet
The nonce is set to 10,001 and the miner transaction is set to the same as mainnet5
The testnet genesis block will hash to 48ca7cd3c8de5b6a4d53d2861fbdaedca141553559f9be9520068053cda8430b
.
The final block:
{
header: {
major_version: 1,
minor_version: 0,
timestamp: 0,
previous: [0; 32],
nonce: 10001
},
miner_tx: "013c01ff0001ffffffffffff03029b2e4c0281c0b02e7c53291a94d1d0cbff8883f8024f5142ee494ffbbd08807121017767aafcde9be00dcfd098715ebcf7f410daebc582fda69d24a28e9d0bc890d1",
txs: [],
}
Stagenet
The nonce is set to 10,002 and the miner transaction is set to:
013c01ff0001ffffffffffff0302df5d56da0c7d643ddd1ce61901c7bdc5fb1738bfe39fbe69c28a3a7032729c0f2101168d0c4ca86fb55a4cf6a36d31431be1c53a3bd7411bb24e8832410289fa6f3b
6.
The stagenet genesis block will hash to 76ee3cc98646292206cd3e86f74d88b4dcc1d937088645e9b0cbca84b7ce74eb
.
The final block:
{
header: {
major_version: 1,
minor_version: 0,
timestamp: 0,
previous: [0; 32],
nonce: 10002
},
miner_tx: "013c01ff0001ffffffffffff0302df5d56da0c7d643ddd1ce61901c7bdc5fb1738bfe39fbe69c28a3a7032729c0f2101168d0c4ca86fb55a4cf6a36d31431be1c53a3bd7411bb24e8832410289fa6f3b",
txs: [],
}
Hard Forks
Monero makes use of hard-forks to update its protocol. Although it has never been used, Monero has a system in it's codebase to
allow voting for activation of a hard-fork1. It works by using the blocks minor version
field as a voting field,
when enough blocks vote for a hard fork the fork is activated.
Because Monero has never used hard fork voting, you don't need to implement it but as it's included in the codebase, an explanation is included here.
Blocks version and vote
Monero uses the block's major version
field as an indicator of hard-fork and the minor version
field as an indicator of the blocks
vote. A minor version of 0 is treated as a vote for 1 as legacy blocks use to just set this field to 02.
The block's vote must be greater than or equal to the version, a vote higher than the maximum known hard-fork is interpreted as a vote for the latest hard-fork3. So if a block is at V2 then the vote must be V2 or higher.
Accepting a fork
When a hard-fork is added to Monero's protocol it must specify a threshold
, a number between 0 and 100, this is the proportion of
blocks in the window that must vote for this fork (or a later one) for it to activate. For all current forks the threshold is 0 meaning
that no votes are needed for the fork to activate.
Monero keeps track of a week (10,080 blocks) worth of votes4, when a new block is added Monero works backwards through the list of hard-forks (latest to oldest) tallying the votes and checking if the number of votes is bigger than the amount needed5, votes for later hardforks are also votes for previous hard-forks. The amount needed is calculated:
\( amountNeeded = \frac{windowSize * threshold + 99}{100} \)
If the amount of votes is greater than or equal to the amount needed and the current blockchain height is greater than or equal to the hard-fork height the HF is activated5.
Mainnet Hard-Forks 6
Version | Height | Threshold | Finalized (timestamp) |
---|---|---|---|
1 | 07 | 0 | Jul 04 2012 (1341378000) |
2 | 1009827 | 0 | Sep 20 2015 (1442763710) |
3 | 1141317 | 0 | Mar 21 2016 (1458558528) |
4 | 1220516 | 0 | Jan 05 2017 (1483574400) |
5 | 1288616 | 0 | Mar 14 2017 (1489520158) |
6 | 1400000 | 0 | Aug 18 2017 (1503046577) |
7 | 1546000 | 0 | Mar 17 2018 (1521303150) |
8 | 1685555 | 0 | Sep 02 2018 (1535889547) |
9 | 1686275 | 0 | Sep 02 2018 (1535889548) |
10 | 1788000 | 0 | Feb 10 2019 (1549792439) |
11 | 1788720 | 0 | Feb 15 2019 (1550225678) |
12 | 1978433 | 0 | Oct 18 2019 (1571419280) |
13 | 2210000 | 0 | Aug 23 2020 (1598180817) |
14 | 2210720 | 0 | Aug 24 2020 (1598180818) |
15 | 2688888 | 0 | Jun 30 2022 (1656629117) |
16 | 2689608 | 0 | Jun 30 2022 (1656629118) |
Testnet Hard-Forks 8
Version | Height | Threshold | Finalized (timestamp) |
---|---|---|---|
1 | 07 | 0 | Jul 04 2012 (1341378000) |
2 | 624634 | 0 | Oct 20 2015 (1445355000) |
3 | 800500 | 0 | Aug 28 2016 (1472415034) |
4 | 801219 | 0 | Aug 28 2016 (1472415035) |
5 | 802660 | 0 | Aug 28 2016 (1472415036 + 86400*180) |
6 | 971400 | 0 | Aug 02 2017 (1501709789) |
7 | 1057027 | 0 | Dec 02 2017 (1512211236) |
8 | 1057058 | 0 | Aug 02 2018 (1533211200) |
9 | 1057778 | 0 | Aug 03 2018 (1533297600) |
10 | 1154318 | 0 | Feb 14 2019 (1550153694) |
11 | 1155038 | 0 | Feb 15 2019 (1550225678) |
12 | 1308737 | 0 | Sep 27 2019 (1569582000) |
13 | 1543939 | 0 | Sep 02 2020 (1599069376) |
14 | 1544659 | 0 | Sep 02 2020 (1599069377) |
15 | 1982800 | 0 | May 16 2022 (1652727000) |
16 | 1983520 | 0 | May 17 2022 (1652813400) |
Stagenet Hard-Forks 9
Version | Height | Threshold | Finalized (timestamp) |
---|---|---|---|
1 | 07 | 0 | Jul 04 2012 (1341378000) |
2 | 32000 | 0 | Mar 14 2018 (1521000000) |
3 | 33000 | 0 | Mar 15 2018 (1521120000) |
4 | 34000 | 0 | Mar 16 2018 (1521240000) |
5 | 35000 | 0 | Mar 18 2018 (1521360000) |
6 | 36000 | 0 | Mar 19 2018 (1521480000) |
7 | 37000 | 0 | Mar 21 2018 (1521600000) |
8 | 176456 | 0 | Sep 24 2018 (1537821770) |
9 | 177176 | 0 | Sep 24 2018 (1537821771) |
10 | 269000 | 0 | Feb 14 2019 (1550153694) |
11 | 269720 | 0 | Feb 15 2019 (1550225678) |
12 | 454721 | 0 | Oct 18 2019 (1571419280) |
13 | 675405 | 0 | Aug 23 2020 (1598180817) |
14 | 676125 | 0 | Aug 23 2020 (1598180818) |
15 | 1151000 | 0 | Jun 30 2022 (1656629117) |
16 | 1151720 | 0 | Jun 30 2022 (1656629118) |
Monero C++ sets this to 1 even though the genesis block has a major version of 1.
Blocks
Introduction
This chapter contains all the rules that apply to a block. Miner transactions are included in this section as the rules that apply to them are different to normal transactions.
Index
Block Rules
Block Weight And Size
The block blob
must not be bigger than (2 * the effective median weight + 100)1.
The block weight must not be more than 2 * the median weight for coinbase checks2.
Amount Of Transactions
The amount of transactions in a block (including the miner transaction) must be less than 0x10000000
3.
No Duplicate Transactions
There must be no duplicate transactions in the block or the blockchain4.
Key Images
There must be no duplicate key images in the block5, or the whole chain.
Previous ID
The blocks prev_id
must equal the block hash
of the last block6.
PoW Function
The proof of work function used depends on the hard-fork7:
hard-fork | PoW function |
---|---|
1 to 6 | CryptoNight v0 |
7 | CryptoNight v1 |
8 to 9 | CryptoNight v2 |
10 to 11 | CryptoNight R |
12 onwards | RandomX |
For block 202612 always return the same PoW hash, no matter the network8.
PoW hash:
84f64766475d51837ac9efbef1926486e58563c95a19fef4aec3254f03000000
Checking PoW Hash
See checking PoW in the difficulty chapter.
RandomX Seed
The RandomX seed, which is used to set up the dataset, is a previous block hash in the blockchain.
The seed height is 0 if the current height is below or equal to \( 2048 + 64 \) otherwise is got by:
\( seedHeight = (height - 64 - 1) \land \lnot(2048 - 1) \)
with \( \land \) being a bit-and and \( \lnot \) being a bit-not.
You then get the block hash at seedHeight
which is then the RandomX seed.9
Version And Vote
The block's major version must equal to the current hard-fork and the vote must be greater than or equal to the current hard-fork10.
Vote is not always the same as the minor version, see here.
Timestamp
The block's timestamp must not be more than the current UNIX time + 2 hours11 and the timestamp must not be less than the median timestamp over the last 60 blocks12, if there are less than 60 blocks in the chain then the timestamp is always valid.
https://github.com/monero-project/monero/blob/eac1b86bb2818ac552457380c9dd421fb8935e5b/src/cryptonote_core/blockchain.cpp#L1418-L1428 && https://github.com/monero-project/monero/blob/eac1b86bb2818ac552457380c9dd421fb8935e5b/src/cryptonote_basic/cryptonote_basic_impl.cpp#L107
https://github.com/monero-project/monero/blob/eac1b86bb2818ac552457380c9dd421fb8935e5b/src/cryptonote_core/blockchain.cpp#L5267 && https://github.com/monero-project/monero/blob/67d190ce7c33602b6a3b804f633ee1ddb7fbb4a1/src/cryptonote_core/blockchain.cpp#L4319
Difficulty
Difficulty is a measure used to keep block production at a constant rate, it is the average amount of hashes before a solution is found.
Checking A Block's Proof Of Work
To check a block's PoW hash
you interpret the hash as a little endian integer and multiply it by the difficulty, if the result
does not overflow the hash is valid1:
\(Hash * difficulty <= MAXu256 \)
Calculating Difficulty
To calculate difficulty, Monero keeps a window of the last 7352 timestamps and cumulative difficulties, if there are not enough blocks, then you just use as many as possible.
The genesis block is skipped for these calculations3 so should not be included in the timestamp / CD list but it is included in the cumulative difficulty of the chain.
If the amount of blocks is less than or equal to 1 then 1 is returned as the difficulty4.
The timestamps and cumulative difficulties are then shortened to 7205 blocks so that calculations lag 15 blocks behind the chain.
The timestamps are then sorted, in ascending order. We now need to get a time span value to do this we first remove the outliers:
If the number of timestamps is less than or equal to the amount of blocks we are accounting for (600: 720 - 2 * 60) then the lower is set to 0 and the upper is set to the length of timestamps. Otherwise, if we have enough timestamps, the lower and upper is calculated by6:
\(lower = \frac{len(timestamps) - 600+1}{2} \)
\(upper = lower + 600 \)
We then get the timestamp at position lower
and take this away from the timestamp at position upper -1
to get timeSpan
.
If timeSpan
is 0 we set it to 17.
We also get the cumulative difficulty at position lower
and take this away from the cumulative difficulty at position upper -1
to get totalWork
.
The next difficulty is then calculated by8:
\(difficulty = \frac{totalWork * targetSeconds + timeSpan -1}{timeSpan} \)
Target Seconds
For hard-fork v1 the target seconds is 60, so one block a minute. For hard-fork 2 onwards block time is 1209.
https://github.com/monero-project/monero/blob/67d190ce7c33602b6a3b804f633ee1ddb7fbb4a1/src/cryptonote_config.h#L81 && https://github.com/monero-project/monero/blob/67d190ce7c33602b6a3b804f633ee1ddb7fbb4a1/src/cryptonote_basic/difficulty.cpp#L205
Block Weights
Monero's blockchain, unlike other blockchains, has dynamic block sizes which means blocks expand to handle demand. However Monero does not allow unrestricted block growth, miners will face a penalty for expanding blocks and miners are restricted by how much they can expand a block.
Index
- Penalty Free Zone
- Blocks Weight
- Long Term Block Weight
- Effective Median Weight
- Median Weight For Coinbase Checks
Penalty Free Zone
Monero sets a minimum max block weight so that miners don't get punished for expanding small blocks.
For hf 1 this is 20000 bytes, for hf 2-4 this is 60000 and from 5 onwards this is 300000 bytes1.
Blocks Weight
A blocks weight is the sum of all the transactions weights in a block, including the miner transaction. The block header and transaction hashes are not included2.
Long Term Block Weight
The block's long term weight is the block's weight adjusted with previous block's weights.
Calculating A Blocks Long Term Weight
Up until hard-fork 10, the blocks long term weight is just the block's weight3.
From hard-fork 10 onwards we first get the median long term weight over the last 100,000 blocks, if this is less than the penalty free zone then set the median long term weight to this instead4.
Now we need to set a shot term constraint and adjusted block weight, the way we do this is different depending on the hard-fork.
From hard-fork 10 to 145:
\(adjustedBlockWeight = blockWeight\)
\(shortTermConstraint = medianLongTermWeight * 1.4\)
From 15 onwards6:
\(adjustedBlockWeight = max(blockWeight, \frac{medianLongTermWeight}{1.7})\)
\(shortTermConstraint = medianLongTermWeight * 1.7\)
Now the long term weight is defined as min(adjustedBlockWeight, shortTermConstraint)
7.
Effective Median Weight
The effective median weight is used to calculate block reward and to limit block size.
Calculating Effective Median Weight
For any hard-fork the minimum this can be is the penalty free zone8.
Up until hard-fork 10, this is done by just getting the median block weight over the last 100 blocks9, if there are less than 100 blocks just get the median over all the blocks.
For hf 10 onwards, we first get the median long term weight over the last 100,000 blocks10, if this median is less than the hf 5 penalty free zone set the median to that, this is the long term median.
Now get the median block weight over the last 100 blocks, this is the short term median.
Now we can calculate the effective median, for hard-forks 10 to 14 this is done by11:
\(effectiveMedian = min(max(hf5PenaltyFreeZone, shortTermMedian), 50 * longTermMedian) \)
From 15 onwards this is done by:
\(effectiveMedian = min(max(longTermMedian, shortTermMedian), 50 * longTermMedian) \)
Median Weight For Coinbase Checks
When checking coinbase transactions and block weight Monero uses yet another median weight :).
Calculating Median Weight For Coinbase Checks
Before hf 12 this is the median block weight over the last 100 blocks12.
From hf 12 this is the effective median weight13
https://github.com/monero-project/monero/blob/eac1b86bb2818ac552457380c9dd421fb8935e5b/src/cryptonote_core/blockchain.cpp#L4289 and https://github.com/monero-project/monero/blob/eac1b86bb2818ac552457380c9dd421fb8935e5b/src/cryptonote_core/blockchain.cpp#L4408
Block Reward
The block reward is the amount paid out to a miner for mining a block.
Calculating Base Block Reward
The base block reward is the reward before factoring in the potential penalty for expanding blocks.
To calculate the base block reward you first need the total amount of coins already generated, then define:
1 \(moneySupply = 2^{64} -1 \)
2 \(emissionSpeedFactor = 20 - (targetMinutes - 1) \)
where targetMinutes
is the target block time in minutes.
The baseReward
is then calculated by:
3 \(baseReward = (moneySupply - alreadyGeneratedCoins) >> emissionSpeedFactor \)
If baseReward
falls below the final subsidy (0.3 XMR / minute) them set the baseReward
to that instead 4.
Calculating Block Reward
First calculate the base block reward.
Now we need to get the median weight for block rewards
If the current block weight is not more than the median weight then the block reward is the base reward.
Otherwise the block reward is:5
\(blockReward = baseReward * (1 - (\frac{blockWeight}{effectiveMedianWeight} -1)^2) \)
https://web.getmonero.org/library/Zero-to-Monero-2-0-0.pdf#subsection.7.3.3 && https://github.com/monero-project/monero/blob/eac1b86bb2818ac552457380c9dd421fb8935e5b/src/cryptonote_basic/cryptonote_basic_impl.cpp#L111-L127
Miner Transaction Rules
Introduction
Miner transactions are handled differently to normal transactions, see here for the rules on normal transactions.
Rules
Version
The transactions version must be either 1 or 21.
The version can be 1 or 2 up to hard-fork 12 then it must be 22.
Input
The transaction must only have one input and it must be of type txin_gen
3.
The height specified in the input must be the actual block height4.
RingCT Type
From hard-fork 12 version 2 miner transactions must have a ringCT type of Null
5.
Unlock Time
The unlock time must be the current height + 606.
Output Amounts
The output, when summed, must not overflow7.
For only hard-fork 3 the output amount must be a valid decomposed amount8, which means the amount must be in this table.
Total Outputs
The reward from the block + the total fees must not be more than the summed output amount9.
For hard-fork 1 and from 12 onwards the summed output amount must equal the reward + fees10 this means from 2 till 11 miners can collect less if they want less dust.
Output Type
The output type allowed depends on the hard-fork11:
hard-fork | output type |
---|---|
1 to 14 | txout_to_key |
15 | txout_to_key and txout_to_tagged_key |
16 onwards | txout_to_tagged_key |
For hard-fork 15 both are allowed but the transactions outputs must all be the same type.
Zero Amount V1 Output
Monero does not explicitly ban zero amount V1 outputs on miner transactions but the database throws an error if a 0 amount output doesn't have a commitment 12 meaning they are banned.
V2 Output Pool
When adding version 2 miner transactions to the blockchain, put the outputs into the 0 amount pool and create dummy commitments of:13
\(commitment = G + amount * H \)
Transaction Rules
Introduction
This chapter does not include miner, coinbase, transactions as they are handled elsewhere, the rules for them are under blocks
Index
Miscellaneous Rules
Version
Version 0 is never allowed1.
The max transaction version is 1 up to hard fork 4 then the max is 22.
The minimum tx version is 1 up until version 6 then if the number of un-mixable inputs is 0 the minimum is 2 otherwise 13 so a version 1 transaction is allowed if the amount it's spending does not have enough outputs with the same amount to mix with.
Transaction Size
The size of the transaction blob
must not be bigger than 1 million bytes4.
From v8, the transaction's weight must not be bigger than half of the block penalty free zone minus 6005.
Calculating Transaction Weight
For all transactions that don't use bulletproofs or bulletproofs+ the weight is just the length of the transaction blob.6
For bulletproofs(+) transactions we add a "clawback" onto the transaction.
To calculate the "clawback" we fist define a bpBase
which is the size of a 2 output proof, normalized to 1 proof by dividing by 27:
for bulletproofs: \(fields = 9\)
for bulletproofs+: \(fields = 6\)
\(bpBase = \frac{(32 * (fields + 7 * 2))}{2}\)
Next we calculate the size of the bulletproofs(+) field by first getting the first power of 2 above or equal to the number of outputs: firstPower2AboveNumbOuts
.
If firstPower2AboveNumbOuts
is <= 2 then the \(clawback = 0\)8.
Next define the number of L and R elements9: \(nlr = firstPower2AboveNumbOuts + 6\)
now the size of the bulletproofs(+) field is10:
\(bpSize = 32 * (fields + 2 * nlr)\)
now the clawback
is11:
\( clawback = \frac{(bpBase * firstPower2AboveNumbOuts - bpSize) * 4}{ 5} \)
To get the transaction weight now you just get the length of the transaction blob and add this clawback
12.
https://github.com/monero-project/monero/blob/eac1b86bb2818ac552457380c9dd421fb8935e5b/src/cryptonote_core/cryptonote_core.cpp#L791 and https://github.com/monero-project/monero/blob/eac1b86bb2818ac552457380c9dd421fb8935e5b/src/cryptonote_basic/cryptonote_basic_impl.cpp#L78
https://github.com/monero-project/monero/blob/eac1b86bb2818ac552457380c9dd421fb8935e5b/src/cryptonote_core/tx_pool.cpp#L117 && https://github.com/monero-project/monero/blob/eac1b86bb2818ac552457380c9dd421fb8935e5b/src/cryptonote_core/tx_pool.cpp#L221
Transaction Inputs
Introduction
These rules apply to transaction inputs, excluding miner transactions.
Index
Necessary Functions/Definitions
Default Minimum Decoys
This is the default number of decoys an input must at least have.
There are exceptions to this being the minimum decoy size for all transactions. See further down in Rules.
Hard-Fork | Minimum Decoys1 |
---|---|
1 | N/A |
2 to 5 | 2 |
6 | 4 |
7 | 6 |
8 to 14 | 10 |
15+ | 15 |
Minimum And Maximum Decoys Used
To check a transaction input's ring
size we must first get the minimum and maximum number of decoys
used in the transactions inputs2.
So if this was our transactions:
Input | 1 | 2 | 3 |
---|---|---|---|
Ring size | 12 | 8 | 16 |
The minimum and maximum amount of decoys would be 7 and 15 respectively.
Mixable And Un-Mixable Inputs
A mixable input is one that has enough outputs on the chain with the same amount to be able to build a ring with the minimum amount of decoys needed.
A ringCT input, aka an output with 0 amount, is always considered mixable3.
For other inputs you first get the amount of outputs on chain with that amount and check if that's less than or equal to the default minimum amount of decoys if it is then the input is un-mixable otherwise it is mixable4.
Rules
No Empty Inputs
The transaction must have at least 1 input5.
No Empty decoys
All inputs must have decoys6.
Input Type
All inputs must be of type txin_to_key
7.
Inputs Must Not Overflow
The inputs when summed must not overflow a u64
and the outputs when summed must not either8.
Unique Ring Members
From hard-fork 6, all ring members in an input must be unique, this is done by checking that
no key_offset
after the first is 09.
Unique Key Image
The key image must be unique in a transaction10 and the whole chain 11.
Torsion Free Key Image
The key image must be a canonical prime order point12.
Minimum Decoys
These rules are in effect from hard fork 2.
First you get the minimum number of decoys used in the transaction.
Then you get the amount of mixable and un-mixable inputs.
Now get the default minimum decoys allowed for the current hard-fork.
If the minimum amount of decoys used in the transaction is less than the default minimum decoys allowed then the transaction is only allowed if there is at least one input which is un-mixable13.
If there is an un-mixable then the transaction is not allowed to have more than 1 mixable input as well.
Special rules14:
- For hard-fork 15, both 10 and 15 decoys are allowed.
- From hard-fork 8 upwards, the minimum amount of decoys used in a transaction must be equal to the minimum allowed.
Equal Number Of Decoys
From hard-fork 12, all inputs must have the same number of decoys15.
Sorted Inputs
From hard-fork 7, the inputs must be sorted by key image, in descending lexicographic order16.
10 Block Lock
From hard-fork 12, all ring members must be at least 10 blocks old17.
The Output Must Exist
The output a transaction references must exist in the chain18.
The Output Must Not Be Locked
The outputs, which are referenced in the inputs, unlock time must have passed, see the chapter on unlock time.
Transaction Outputs
Introduction
These rules apply to transaction outputs, excluding miner transaction outputs.
Rules
Outputs Must Not Overflow
The outputs when summed must not overflow a u641.
Output Amount
For version 1, txs sum of the outputs must be less than the sum of the inputs, the difference between the inputs and the outputs is then the fee.2 The amount of each output must also not be zero.3
From hard-fork 2, version 1 transaction output amounts also must be validly decomposed4. A valid decomposed amount is an amount contained in this table
For version 2, txs all outputs must have a zero amount.5
Output Keys Canonical
All output public keys must be canonical points
6.
This was added as a rule in hard-fork 4 but that check is redundant as it was done before that. So how did invalid keys get on the chain? miner txs.
Output Type
The output type allowed depends on the hard-fork7:
hard-fork | output type |
---|---|
1 to 14 | txout_to_key |
15 | txout_to_key and txout_to_tagged_key |
16 onwards | txout_to_tagged_key |
For hard-fork 15, both are allowed but the transactions outputs must all be the same type8.
2 Outputs
From hard-fork 12, version 2 transactions (RCT) must have 2 outputs9.
https://github.com/monero-project/monero/blob/eac1b86bb2818ac552457380c9dd421fb8935e5b/src/cryptonote_core/cryptonote_core.cpp#L1163 and https://github.com/monero-project/monero/blob/eac1b86bb2818ac552457380c9dd421fb8935e5b/src/cryptonote_core/tx_pool.cpp#L190-L204
Unlock Time
To spend an output the output's unlock time must have passed.
Interpreting An Unlock Time
The unlock time is just a 64 bit unsigned number. It is interpreted as a block height if less than 500,000,000 otherwise it's a Unix timestamp1.
Checking The Output Is Unlocked
Block Height
First you get the top blocks height and add one, we do this because we are checking if the transaction is allowed in the next block not the last.
We now check if this height is greater than or equal to the unlock time if it is then accept the block2.
Timestamp
Getting The Current Time
Before hard-fork 13, this was done by just getting the computer's time, from hf 13 onwards, we use an average over the last blocks3.
Monero uses the last 60 blocks to get an average, if the chain height
is less than
60, just use the current time4.
First you get the median timestamp of the last 60 blocks. We then project this timestamp to match approximately when the block being validated will appear, to do this we do5:
\(adjustedMedian = median + \frac{(TimestampWindow + 1) * DifficultyTarget}{2} \)
where:
\(TimestampWindow = 60\)
\(DifficultyTarget = 120\)
You then get the top block's timestamp and add the target seconds per block6.
The timestamp we use is then the minimum out of the adjusted median and adjusted most recent timestamp7.
Checking Timestamp Has Passed
Now with our timestamp we add the target seconds per block and check if this is more than or equal to the unlock time8.
Transaction Version 1 Rules
Introduction
These rules apply only to version 1, pre-ringCT, transactions.
Rules
Amount Of Ring Signatures
The amount of ring signatures must be the same as the number of inputs1.
Amount Of Signatures In A Ring
For a ring signature at a certain index, the input at that same index must have the same amount of ring members as the ring signature has signatures2.
Signatures Must Be Canonical
Every signatures c and r value must be canonical scalars
3.
Ring Members Must Be Valid Points
All outputs used as ring members must be valid canonical points4.
The Ring Signature Must Be Valid
The ring signature must be correctly formed5.
https://github.com/monero-project/monero/blob/eac1b86bb2818ac552457380c9dd421fb8935e5b/src/cryptonote_core/blockchain.cpp#L3485 and https://github.com/monero-project/monero/blob/eac1b86bb2818ac552457380c9dd421fb8935e5b/src/cryptonote_basic/cryptonote_basic.h#L266
https://github.com/monero-project/monero/blob/eac1b86bb2818ac552457380c9dd421fb8935e5b/src/cryptonote_core/blockchain.cpp#L3999 and https://github.com/monero-project/monero/blob/eac1b86bb2818ac552457380c9dd421fb8935e5b/src/cryptonote_basic/cryptonote_basic.h#L271-L282
Ring Confidential Transactions
Introduction
Ring confidential transactions are version 2 Monero transactions which keep amounts hidden. They were activated at hard-fork 4. There are multiple types of RingCT transactions that were activated and deprecated at different hard-forks.
Definitions
OutPK: A pedersen commitment to the output amount.
Pseudo-outs: A pedersen commitment to the true spends amount with a different mask, such that the sum of the pseudo-outs is the same as the sum of the outPKs + fee * H.
Index
- Rules That Apply To All Types
- Simple Types Rules
- Borromean Rules
- MLSAG Rules
- Bulletproofs Rules
- CLSAG Rules
- Bulletproofs+ Rules
Rules That Apply To All Types
Type
RingCT type define the proofs used in the transaction, the ringCT types allowed depend on the hard-fork:
Type (Name) | Short description | Hard Fork allowed | Hard Fork disallowed |
---|---|---|---|
0 (NULL) | No ringCT signatures, used for coinbase transactions | 4 (only miner transactions) 1 | Still allowed |
1 (Full) | A single aggregate MLSAG signature with borromean range proofs | 4 1 | 9 2 |
2 (Simple) | MLSAG signatures per input with borromean range proofs | 4 1 | 9 2 |
3 (Bulletproof) | MLSAG signatures per input with a single bulletproof for all outputs | 8 2 | 11 3 |
4 (Bulletproof2) | Uses the same signatures as type 3 | 10 3 | 14 (except 2 transactions) 4 |
5 (CLSAG) | CLSAG signatures per input with a single bulletproof for all outputs | 13 4 | 16 5 |
6 (Bulletproof+) | CLSAG signatures per input with a single bulletproof+ for all outputs | 15 5 | Still allowed |
6+ | Future type not currently allowed | Not allowed 6 | Not allowed |
There are 2 type 4 RCT transactions that are allowed after hard-fork 13, this was due to a bug in which transactions added to the txpool before a fork
were not being checked for new fork rules they are:
c5151944f0583097ba0c88cd0f43e7fabb3881278aa2f73b3b0a007c5d34e910
and 6f2f117cde6fbcf8d4a6ef8974fcac744726574ac38cf25d3322c996b21edd4c
7.
OutPKs Valid Points
All outPKs must be canonically encoded points8.
Simple Types Rules
These rules apply to all RCT "simple" types, which are all except type "FULL".
Pseudo-outs Valid Points
This rule applies to the pseudo-outs, from type 3 (Bulletproof) the pseudo-outs field moved to the prunable RCT section from the non-prunable section.
The pseudo-outs must all be canonically encoded points9.
Pseudo-outs OutPKs Balance
The sum of the pseudo-outs must equal the sum of the OutPKs + fee * H:10
\(\sum PseudoOuts == \sum outPK + fee * H \)
There is no direct code allowing these types of RingCT, these are the original types that got activated when version 2 transactions got activated
https://github.com/monero-project/monero/blob/master/src/cryptonote_core/blockchain.cpp#L3083-L3107
https://github.com/monero-project/monero/blob/master/src/cryptonote_core/blockchain.cpp#L3108-L3130
https://github.com/monero-project/monero/blob/master/src/cryptonote_core/blockchain.cpp#L3132-L3166
https://github.com/monero-project/monero/blob/master/src/cryptonote_core/blockchain.cpp#L3168-L3193
For simple types: https://github.com/monero-project/monero/blob/ac02af92867590ca80b2779a7bbeafa99ff94dcb/src/ringct/rctSigs.cpp#L1444, For type FULL: https://github.com/monero-project/monero/blob/ac02af92867590ca80b2779a7bbeafa99ff94dcb/src/ringct/rctSigs.cpp#L829-L829
Borromean Rules
Introduction
These rules apply to all ringCT types that use Borromean ring signatures to prove an output amount is in the correct range.
Rules
Number Of Borromean Range Proofs
The amount of Borromean range proofs must be the same as the number of outputs.1
Ci Valid Points
Each Ci (bit commitment) must be canonically encoded points.2
Sum Ci
For a range proof at a certain index the sum of each Ci must equal the outPK at that index.3
Borromean Scalar Encoding
Monero does not check that the scalars s0
and s1
are reduced this leads to them, if not reduced, being interpreted as a different scalar by the slide
function
which calculates the 5-NAF of the number. The slide
function restricts its output to 256 bytes however if the last bit is set on the input this could lead to the
5-NAF of the scalar being 257 bytes long. There are scalars on the chain which have this behavior.4
The scalar ee
must be a fully reduced scalar as it is compared against the raw bytes of an output from the hash_to_scalar
function.5
The Borromean Ring Must Be Valid
To verify a Borromean ring signature is valid you must first set up the public keys that the ring will be verified with, one member of the ring will be a Ci the other will be (\(Ci - H * 2^X \)), where X is the index of the Ci. By setting up the ring like this the prover will only know the discreet log of a ring member if either the Ci is a commitment to 0 or \(2^X\)6.
After setting up the public keys the actual borromean rings must be valid.7
MLSAG Rules
Introduction
These rules are split into 3 sections: Full, Simple and Both. Full is for RCT type Full and Simple are for the other RCT types that use MLSAG signatures.
Simple is not just for RCT type Simple!
Index
Full Rules
Creating The Ring Matrix (Full)
For RCT type full the ring matrix contains every inputs ring members: 1
(The signer owns a whole column)
The last row contains: \(\sum CommitmentsAtIndex - \sum outPK - fee * H \) 2
Where CommitmentsAtIndex are the ring members commitments in that column.
Which means that for the true spends column the entry in the last row will be commitment to 0.
By structuring the matrix like this the true spend has to be a the same index in each inputs ring, which is not good for privacy.
Number Of Ring Members
There must be the same amount of ring members in each inputs ring.3
One MLSAGs
There must be only one MLSAG signature.4
Simple Rules
Creating The Ring Matrix (Simple)
For simple RCT types the ring matrix only contains the ring members of a single input: 5
The last row contains the ring members commitment minus the pseudo-out for this input.6
Simple Number Of MLSAGs
There must be the same amount of MLSAG signatures as there are inputs.4
Rules That Apply To Both
More Than One Ring Member
There must be more than one ring member.7
SS Size
The ss field must be the same length as the key matrix8 and each ss member lengths must be the same as the matrix's rows. 9
SS, CC Canonical Encoding
Every ss element and cc must be fully reduced scalars.10
Key Images Not Identity
All the key images must not be equal to the identity point.11
The MLSAG Signature Must Be Correct
The signature must be valid.12
Bulletproofs Rules
Introduction
These rules apply to all ringCT types that use bulletproofs.
Rules
L & R Length
The Length of the L & R fields must be the same, they must both be equal to \( 6 + log_2(firstPower2AboveNumbOuts) \).1
Where firstPower2AboveNumbOuts
is the first power of 2 above or equal to the amount of outputs in the transaction, so:
If outputs = 3, firstPower2AboveNumbOuts = 4.
If outputs = 8, firstPower2AboveNumbOuts = 8.
Number Of Bulletproofs
There must only be one bulletproof in a transaction.2
Max Outputs
The amount of outputs in the transaction must not be more 16 3
At Least One Output
There must be at least one element of V, which is constructed from the outPKs which must have the same number of elements as the outputs.4
Canonical Encoding
taux
, mu
, a
, b
, t
must all be fully reduced scalars5.
All the elements of V
, L
, R
and A
, T1
, T2
and S
must all be valid, canonically encoded points.6
The Bulletproof Must Be Valid
The bulletproof must pass verification. 7
https://github.com/monero-project/monero/blob/master/src/ringct/rctTypes.cpp#L300-L304 && https://github.com/monero-project/monero/blob/master/src/ringct/bulletproofs.cc#L862-L863
CLSAG Rules
Introduction
These rules apply to all ringCT types that use CLSAG signatures.
Rules
Number Of CLSAGs
There must be the same number of CLSAG signatures as there are inputs.1
s
Size
The s
field must have has many elements as the amount of ring members.2
Canonical Encoding
All s
scalars must be fully reduced, the c1
scalar must be fully reduced3 and the D
point must be canonically encoded.4
Key Images Not Identity
The key image and 8 * D
, the commitment key image, must both not be the identity point.5
The CLSAG Signature Must Be Correctly Formed
The signature must be valid.6
https://github.com/monero-project/monero/blob/ac02af92867590ca80b2779a7bbeafa99ff94dcb/src/ringct/rctSigs.cpp#L895 and https://github.com/monero-project/monero/blob/ac02af92867590ca80b2779a7bbeafa99ff94dcb/src/ringct/rctSigs.cpp#L884
Bulletproofs+ Rules
Introduction
These rules apply to all ringCT types that use bulletproofs+.
Rules
L & R Length
The Length of the L & R fields must be the same, they must both be equal to \( 6 + log_2(firstPower2AboveNumbOuts) \).1
Where firstPower2AboveNumbOuts
is the first power of 2 above or equal to the amount of outputs in the transaction, so:
If outputs = 3, firstPower2AboveNumbOuts = 4.
If outputs = 8, firstPower2AboveNumbOuts = 8.
Number Of Bulletproofs
There must only be one bulletproof in a transaction.2
Max Outputs
The amount of outputs in the transaction must not be more than 16 3
Canonical Encoding
r1
, s2
, d1
must all be canonically encoded, reduced, scalars.4 All the points of V
, L
and R
must be canonically encoded and A1
, B
and
A
must canonically encoded points.5
At Least One Output
There must be at least one element of V, which is constructed from the outPKs which must have the same number of elements as the outputs.6
The Bulletproof Must Be Valid
The bulletproof must pass verification. 7
https://github.com/monero-project/monero/blob/master/src/ringct/rctTypes.cpp#L300-L304 && https://github.com/monero-project/monero/blob/ac02af92867590ca80b2779a7bbeafa99ff94dcb/src/ringct/bulletproofs_plus.cc#L850
https://github.com/monero-project/monero/blob/ac02af92867590ca80b2779a7bbeafa99ff94dcb/src/ringct/bulletproofs_plus.cc#L931-L939 && https://github.com/monero-project/monero/blob/ac02af92867590ca80b2779a7bbeafa99ff94dcb/src/ringct/rctOps.cpp#L415
P2P Network
This chapter contains descriptions of Monero's peer to peer network, including messages, flows, etc.
Levin Protocol
This chapter describes the levin protocol.
Buckets
A Bucket is a single piece of data that the levin protocol parser can decode, it will contain a p2p message or it will be part of a chain of buckets that will be combined into a single message.
Bucket Format
Field | Type | Size (bytes) |
---|---|---|
Header | BucketHeader | 33 |
Body | bytes | dynamic |
BucketHeader
Format1:
Field | Type | Size (bytes) |
---|---|---|
Signature | LE u64 | 8 |
Size | LE u64 | 8 |
Expect Response | bool | 1 |
Command | LE u32 | 4 |
Return Code | LE i32 | 4 |
Flags | LE u32 | 4 |
Protocol Version | LE u32 | 4 |
Signature
The signature field is fixed for every bucket and is used to tell apart peers running different protocols.
Its value should be 0x0101010101012101
2
Size
This field represents the size of the buckets body.
Expect Response
Messages with the expect response field set must be responded to in order, other messages are still allowed in between responses.
Command
This field is an identifier for what specific message the bucket's body contains.
Return Code
This field represents the status of the response from the peer, requests and notifications should set this to 0
and successful
responses should be 1
.
Flags
This is a bit-flag field that determines what type of bucket this is3:
Type | Bits set |
---|---|
Request | 0000_0001 |
Response | 0000_0010 |
Start Fragment | 0000_0100 |
End Fragment | 0000_1000 |
Dummy | 0000_1100 |
Protocol Version
This is a fixed value of 1.
Bucket Body
All bucket bodies are serialized in the epee binary format which is described here: https://github.com/monero-project/monero/blob/cc73fe71162d564ffda8e549b79a350bca53c454/docs/PORTABLE_STORAGE.md
Exact message types are described in the next chapters.
Admin Messages
This chapter describes admin messages, and documents the current admin messages. Admin messages are a subset of messages that handle connection creation, making sure connections are still alive, and sharing peer lists.
Levin
All admin messages are in the request/response levin format. This means requests will set the expect response bit and
responses will set the return code to 1
.
Messages
Handshake
ID: 1001
1
Request 2
Fields | Type | Description |
---|---|---|
node_data | basic node data | Static information about our node |
payload_data | core sync data | Information on the node's sync state |
Response 3
Fields | Type | Description |
---|---|---|
node_data | basic node data | Static information about our node |
payload_data | core sync data | Information on the node's sync state |
local_peerlist_new | A Vec of peer list entry base | A list of peers in the node's peer list |
Timed Sync
ID: 1002
4
Request 5
Fields | Type | Description |
---|---|---|
payload_data | core sync data | Information on the node's sync state |
Response 6
Fields | Type | Description |
---|---|---|
payload_data | core sync data | Information on the node's sync state |
local_peerlist_new | A Vec of peer list entry base | A list of peers in the node's peer list |
Ping
ID: 1003
7
Request 8
No data is serialized for a ping request.
Response 9
Fields | Type | Description |
---|---|---|
status | string | Will be OK for successful pings |
peer_id | u64 | The self assigned id of the peer |
Request Support Flags
ID: 1007
10
Request 11
No data is serialized for a support flags request.
Response 12
Fields | Type | Description |
---|---|---|
support_flags | u32 | The peer's support flags |
Protocol Messages
This chapter describes protocol messages, and documents the current protocol messages. Protocol messages are used to share protocol data like blocks and transactions.
Levin
All protocol messages are in the notification levin format. Although there are some messages that fall under requests/responses, levin will treat them as notifications.
All admin messages are in the request/response levin format. This means requests will set the expect response bit and
responses will set the return code to 1
.
Messages
Notify New Block
ID: 2001
1
Fields | Type | Description |
---|---|---|
b | Block Complete Entry | The full block |
current_blockchain_height | u64 | The current chain height |
Notify New Transactions
ID: 2002
2
Fields | Type | Description |
---|---|---|
txs | A vector of bytes | The txs |
_ | Bytes | Padding to prevent traffic volume analysis |
dandelionpp_fluff | bool | True if this message contains fluff txs, false if stem |
Notify Request Get Objects
ID: 2003
3
Fields | Type | Description |
---|---|---|
blocks | A vector of [u8; 32] serialized as a single string | The block IDs requested |
prune | bool | True if we want the blocks in pruned form, false otherwise |
Notify Response Get Objects
ID: 2004
4
Fields | Type | Description |
---|---|---|
blocks | A vector of Block Complete Entry | The blocks that were requested |
missed_ids | A vector of [u8; 32] serialized as a single string | IDs of any missed blocks |
current_blockchain_height | u64 | The current blockchain height |
Notify Request Chain
ID: 2006
5
Fields | Type | Description |
---|---|---|
block_ids | A vector of [u8; 32] serialized as a single string | A list of block IDs in reverse chronological order, the top and genesis block will always be included |
prune | bool | True if we want the response to contain pruned blocks, false otherwise |
Notify Response Chain Entry
ID: 2007
6
Fields | Type | Description |
---|---|---|
start_height | u64 | The start height of the entry |
total_height | u64 | The height of the peer's blockchain |
cumulative_difficulty | u64 | The low 64 bits of the cumulative difficulty |
cumulative_difficulty_top64 | u64 | The high 64 bits of the cumulative difficulty |
m_block_ids | A vector of [u8; 32] serialized as a single string | The block IDs in this entry |
m_block_weights | A vector of u64 serialized as a single string | The block weights |
first_block | bytes (epee string) | The header of the first block in m_block_ids |
Notify New Fluffy Block
ID: 2008
7
Fields | Type | Description |
---|---|---|
b | Block Complete Entry | The block, may or may not contain txs |
current_blockchain_height | u64 | The current chain height |
Notify Request Fluffy Missing Tx
ID: 2009
8
Fields | Type | Description |
---|---|---|
block_hash | [u8; 32] serialized as a string | The block hash txs are needed from |
current_blockchain_height | u64 | The current chain height |
missing_tx_indices | A vector of u64 serialized as a single string | The indices of the needed txs in the block |
Notify Get Txpool Compliment
ID: 2010
9
Fields | Type | Description |
---|---|---|
hashes | A vector of [u8; 32] serialized as a string | The current txpool txs |
Common P2P Types
This chapter contains definitions of types used in multiple P2P messages.
Support Flags
Support flags specify any protocol extensions the peer supports, currently only the first bit is used:
FLUFFY_BLOCKS = 1
- for if the peer supports receiving fluffy blocks.
Basic Node Data 1
Fields | Type | Description |
---|---|---|
network_id | A UUID (epee string) | A fixed constant value for a specific network (mainnet,testnet,stagenet) |
my_port | u32 | The peer's inbound port, if the peer does not want inbound connections this should be 0 |
rpc_port | u16 | The peer's RPC port, if the peer does not want inbound connections this should be 0 |
rpc_credits_per_hash | u32 | States how much it costs to use this node in credits per hashes, 0 being free |
peer_id | u64 | A fixed ID for the node, set to 1 for anonymity networks |
support_flags | support flags (u32) | Specifies any protocol extensions the peer supports |
Core Sync Data 2
Fields | Type | Description |
---|---|---|
current_height | u64 | The current chain height |
cumulative_difficulty | u64 | The low 64 bits of the cumulative difficulty |
cumulative_difficulty_top64 | u64 | The high 64 bits of the cumulative difficulty |
top_id | [u8; 32] (epee string) | The hash of the top block |
top_version | u8 | The hardfork version of the top block |
pruning_seed | u32 | THe pruning seed of the node, 0 if the node does no pruning |
Network Address 3
Network addresses are serialized differently than other types, the fields needed depend on the type
field:
Fields | Type | Description |
---|---|---|
type | u8 | The address type |
addr | An object whose fields depend on type | The address |
IPv4
type = 1
Fields | Type | Description |
---|---|---|
m_ip | u32 | The IPv4 address |
m_port | u16 | The port |
IPv6
type = 2
Fields | Type | Description |
---|---|---|
addr | [u8; 16] (epee string) | The IPv6 address |
m_port | u16 | The port |
Tor
TODO:
I2p
TODO:
Peer List Entry Base 4
Fields | Type | Description |
---|---|---|
adr | Network Address | The address of the peer |
id | u64 | The random, self assigned, ID of this node |
last_seen | i64 | A field marking when this peer was last seen, although this is zeroed before sending over the network |
pruning_seed | u32 | This peer's pruning seed, 0 if the peer does no pruning |
rpc_port | u16 | This node's RPC port, 0 if this peer has no public RPC port. |
rpc_credits_per_hash | u32 | States how much it costs to use this node in credits per hashes, 0 being free |
Tx Blob Entry 5
Fields | Type | Description |
---|---|---|
blob | bytes (epee string) | The pruned tx blob |
prunable_hash | [u8; 32] (epee string) | The hash of the prunable part of the tx |
Block Complete Entry 6
Fields | Type | Description |
---|---|---|
pruned | bool | True if the block is pruned, false otherwise |
block | bytes (epee string) | The block blob |
block_weight | u64 | The block's weight |
txs | depends on pruned | The transaction blobs, the exact type depends on pruned |
If pruned
is true:
txs
is a vector of Tx Blob Entry
If pruned
is false:
txs
is a vector of bytes.
Message Flows
Message flows are sets of messages sent between peers, that achieve an identifiable goal, like a handshake. Some message flows are complex, involving many message types, whereas others are simple, requiring only 1.
The message flows here are not every possible request/response.
When documenting checks on the messages, not all checks are documented, only the ones notable. This should help to reduce the maintenance burden.
Different Flows
Handshakes
Handshakes are used to establish connections to peers.
Flow
The default handshake flow is made up of the connecting peer sending a handshake request and the receiving peer responding with a handshake response.
It should be noted that not all other messages are banned during handshakes, for example, support flag requests and even some protocol requests can be sent.
Handshake Request Checks
The receiving peer will check:
- The
network_id
is network ID expected.1 - The connection is an incoming connection.2
- The peer hasn't already completed a handshake.3
- If the network zone is public, then the
peer_id
must not be the same as ours.4 - The core sync data is not malformed.5
Handshake Response Checks
The initiating peer will check:
- The
network_id
is network ID expected.6 - The number of peers in the peer list is less than
250
.7 - All peers in the peer list are in the same zone.8
- The core sync data is not malformed.5
- If the network zone is public, then the
peer_id
must not be the same as ours.9
Timed Syncs
A timed sync request is sent every 60 seconds to make sure the connection is still live.
Flow
First the timed sync initiator will send a timed sync request, the receiver will then respond with a timed sync response
Timed Sync Request Checks
- The core sync data is not malformed.1
Timed Sync Response Checks
- The core sync data is not malformed.1
- The number of peers in the peer list is less than
250
.2 - All peers in the peer list are in the same zone.3
New Block
This is used whenever a new block is to be sent to peers. Only the fluffy block flow is described here, as the other method is deprecated.
Flow
First the peer with the new block will send a new fluffy block notification, if the receiving peer has all the txs in the block then the flow is complete. Otherwise the peer sends a fluffy missing transactions request to the first peer, the first peer will then respond with again a new fluffy block notification but with the transactions requested.
New Transactions
Monero uses the dandelion++ protocol to pass transactions around the network, this flow just describes the actual tx passing between nodes part.
Flow
This flow is pretty simple, the txs are put into a new transactions notification and sent to peers.
Hopefully in the future this is changed.
There must be no duplicate txs in the notification.1
Chain Sync
Chain sync is the first step in syncing a peer's blockchain, it allows a peers to find the split point in their chains and for the peer to learn about the missing block IDs.
Flow
The first step is for the initiating peer is to get its compact chain history. The compact chain history must be in reverse chronological
order, with the first block being the top block and the last the genesis, if the only block is the genesis then that only needs to be included
once. The blocks in the middle are not enforced to be at certain locations, however monerod
will use the top 11 blocks and will then go power
of 2 offsets from then on, i.e. {13, 17, 25, ...}
Then, with the compact history, the initiating peer will send a request chain message, the receiving peer will then find the split point and return a response chain entry message.
The response chain entry
will contain a list of block IDs with the first being a common ancestor and the rest being the next blocks that come after
that block in the peer's chain.
Response Checks
Get Blocks
The get block flow is used to download batches of blocks from a peer.
Flow
The initiating peer needs a list of block IDs that the receiving peer has, this can be done with the chain sync flow.
With a list a block IDs the initiating peer will send a get objects request message, the receiving peer will then respond with get objects response.
Request Checks
- The amount of blocks must be less than
100
.1
Pruning
Monero pruning works by having 8 possible pruning seeds, the seed chosen will decide what part of the blockchain's signing data your node will keep. Each pruned peer generates their pruning seed randomly.
Stripes
This is the amount of different blockchain portions that a pruned peer could keep. For Monero this is currently 8, this means the blockchain's signing data is split into 8 portions.
Stripes Size
Depending on your stripe (and therefore your seed) monerod
will store, in a cyclic manner, a portion of blocks while discarding the ones that are out of your stripe. The stripe's size is amount of blocks before another stripe will have to store their portion of blocks, it is set at 4096. That means that in terms of a block's height, the first pruning stripe will store blocks 0 to 4095, the second stripes will store blocks 4096 to 8191, the third stripe will store blocks 8192 to 12288... etc. While a specific stripe is storing a portion of the blockchain, nodes with another stripe can just discard them. This is shown in the table below:
stripe | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 |
---|---|---|---|---|---|---|---|---|
will have blocks | 0 - 4095 | 4096 - 8191 | 8192 - 12287 | .. | .. | .. | .. | .. |
32768 - 36863 | .. | .. | .. | .. | .. | .. | .. | |
.. | .. | .. | .. | .. | .. | .. | .. |
Tip Blocks
Blocks within 5500 of the tip of the chain will not be pruned.
Generating Pruning Seeds
The function in Monero to generate pruning seeds:
uint32_t make_pruning_seed(uint32_t stripe, uint32_t log_stripes)
{
CHECK_AND_ASSERT_THROW_MES(log_stripes <= PRUNING_SEED_LOG_STRIPES_MASK, "log_stripes out of range");
CHECK_AND_ASSERT_THROW_MES(stripe > 0 && stripe <= (1ul << log_stripes), "stripe out of range");
return (log_stripes << PRUNING_SEED_LOG_STRIPES_SHIFT) | ((stripe - 1) << PRUNING_SEED_STRIPE_SHIFT);
}
This function takes in a stripe which is number 1 to 8 including(1 & 8) and a log_stripes which is log2 of the amount of different stripes (8) which is 3.
The constants used in this function:
static constexpr uint32_t PRUNING_SEED_LOG_STRIPES_SHIFT = 7;
static constexpr uint32_t PRUNING_SEED_LOG_STRIPES_MASK = 0x7;
static constexpr uint32_t PRUNING_SEED_STRIPE_SHIFT = 0;
The possible inputs/outputs of this function (log_stripes
is always 3)
input (stripe) | output (seed) |
---|---|
1 | 384 |
2 | 385 |
3 | 386 |
4 | 387 |
5 | 388 |
6 | 389 |
7 | 390 |
8 | 391 |
Getting A Seed's Log Stripes
Monero currently only accepts a log stripes value of 3 and will reject any peers that use a different value. The function to calculate a seed's log stripes is:
constexpr inline uint32_t get_pruning_log_stripes(uint32_t pruning_seed) {
return (pruning_seed >> PRUNING_SEED_LOG_STRIPES_SHIFT) & PRUNING_SEED_LOG_STRIPES_MASK;
}
This will only return 3 for all currently valid Monero seeds.
Getting A Seed's Pruning Stripe
The seed's pruning stripe corresponds, as explain earlier, to the range of blocks we keep. This is the function that gets the stripe from the pruning seed:
inline uint32_t get_pruning_stripe(uint32_t pruning_seed) {
if (pruning_seed == 0) return 0;
return 1 + ((pruning_seed >> PRUNING_SEED_STRIPE_SHIFT) & PRUNING_SEED_STRIPE_MASK); }
A pruning seed of 0 means no pruning. This function is just the inverse of Generating Pruning Seeds so the inputs/outputs of this will just be the other way round.
Getting A Block's Pruning Stripe
A Block's pruning stripe is the stripe that corresponds to keeping that block so for blocks 0 to 4095 this will be 1, for blocks 4096 to 8191 this will be 2. The function in Monero to get the pruning stripe that corresponds to keeping that block is:
uint32_t get_pruning_stripe(uint64_t block_height, uint64_t blockchain_height, uint32_t log_stripes)
{
if (block_height + CRYPTONOTE_PRUNING_TIP_BLOCKS >= blockchain_height)
return 0;
return ((block_height / CRYPTONOTE_PRUNING_STRIPE_SIZE) & (uint64_t)((1ul << log_stripes) - 1)) + 1;
}
This function takes in a number (block_height
) and outputs a number 0 to 8. Zero is a special case for if the block_height is within Tip Blocks, this means every seed should keep this block. For 1 to 8 the output will rotate every 4096 so if I input 0 the output is 1 and if I input 4096 the output is 2 and so
on...
explaining what the function is doing in depth:
As you can see, this function first checks if the block_height is within Tip Blocks and returns 0, because every seed will have this block.
((1ul << log_stripes) - 1)
This sets the last 3 bits: 0000 0111
so when we bitand we
remove every other bit.
(block_height / CRYPTONOTE_PRUNING_STRIPE_SIZE)
:
- for any block 0 to 4095 dividing by 4096 will output 0 (stripe: 1)
- for any block 4096 to 8191 dividing by 4096 will output 1 (stripe: 2)
- for any blocks 32768 to 36863 dividing by 4096 will output 8 (stripe: 1)
Here's an issue, we need the strips to be cyclic. A result of 8 should give an output of 1, and a result of 455 should give an output of 5. To do so we just use the modulo operation. (8 mod 8 = 1, 455 mod 8 = 5) In binary operation, if the divisor is a power of two, then this is equivalent to bitand the value with the divisor -1:
This is why if we bitand this with 7 (0000 0111), this then becomes:
- 0 to 4095 would be 0
- 4096 to 8191 would be 1
- 32768 to 36863 would be 0
now we are close, all we have to do now to get the stripe is add 1
Getting A Block's Pruning Seed
The Block's pruning seed is the seed that will keep that block. This is the function in Monero:
uint32_t get_pruning_seed(uint64_t block_height, uint64_t blockchain_height, uint32_t log_stripes)
{
const uint32_t stripe = get_pruning_stripe(block_height, blockchain_height, log_stripes);
if (stripe == 0)
return 0;
return make_pruning_seed(stripe, log_stripes);
}
This is simple, a call to get_pruning_stripe
and passing that stripe into make_pruning_seed
Getting The Next Un-pruned Block
For a particular seed and block height we can calculate what the height of the next un-pruned block will be. The function to do this in Monero is:
uint64_t get_next_unpruned_block_height(uint64_t block_height, uint64_t blockchain_height, uint32_t pruning_seed)
{
CHECK_AND_ASSERT_MES(block_height <= CRYPTONOTE_MAX_BLOCK_NUMBER+1, block_height, "block_height too large");
CHECK_AND_ASSERT_MES(blockchain_height <= CRYPTONOTE_MAX_BLOCK_NUMBER+1, block_height, "blockchain_height too large");
const uint32_t stripe = get_pruning_stripe(pruning_seed);
if (stripe == 0)
return block_height;
if (block_height + CRYPTONOTE_PRUNING_TIP_BLOCKS >= blockchain_height)
return block_height;
const uint32_t seed_log_stripes = get_pruning_log_stripes(pruning_seed);
const uint64_t log_stripes = seed_log_stripes ? seed_log_stripes : CRYPTONOTE_PRUNING_LOG_STRIPES;
const uint64_t mask = (1ul << log_stripes) - 1;
const uint32_t block_pruning_stripe = ((block_height / CRYPTONOTE_PRUNING_STRIPE_SIZE) & mask) + 1;
if (block_pruning_stripe == stripe)
return block_height;
const uint64_t cycles = ((block_height / CRYPTONOTE_PRUNING_STRIPE_SIZE) >> log_stripes);
const uint64_t cycle_start = cycles + ((stripe > block_pruning_stripe) ? 0 : 1);
const uint64_t h = cycle_start * (CRYPTONOTE_PRUNING_STRIPE_SIZE << log_stripes) + (stripe - 1) * CRYPTONOTE_PRUNING_STRIPE_SIZE;
if (h + CRYPTONOTE_PRUNING_TIP_BLOCKS > blockchain_height)
return blockchain_height < CRYPTONOTE_PRUNING_TIP_BLOCKS ? 0 : blockchain_height - CRYPTONOTE_PRUNING_TIP_BLOCKS;
CHECK_AND_ASSERT_MES(h >= block_height, block_height, "h < block_height, unexpected");
return h;
}
As you can see this is a monstrous function
explaining what the function is doing in depth:
const uint32_t stripe = get_pruning_stripe(pruning_seed);
if (stripe == 0)
return block_height;
if (block_height + CRYPTONOTE_PRUNING_TIP_BLOCKS >= blockchain_height)
return block_height;
This is calculating the stripe of the inputted pruning seed, remember if the seed/stripe is 0
that means no pruning so we can return the current
height as the next un-pruned height and similarly if the block's height is within Tip Blocks of the blockchain's height that also means the block won't be pruned.
const uint32_t seed_log_stripes = get_pruning_log_stripes(pruning_seed);
const uint64_t log_stripes = seed_log_stripes ? seed_log_stripes : CRYPTONOTE_PRUNING_LOG_STRIPES;
const uint64_t mask = (1ul << log_stripes) - 1;
This is calculating the log stripes of the seed, although Monero currently only allows a log stripes of 3 in the future a higher number could be allowed so this function accounts for that.
If the seed's log stripes are zero this will set it to CRYPTONOTE_PRUNING_LOG_STRIPES
which is currently 3
.
Then this sets the value of mask
to one less than the amount of stripes, for Monero the amount of stripes is 8 so mask
will be 7.
const uint32_t block_pruning_stripe = ((block_height / CRYPTONOTE_PRUNING_STRIPE_SIZE) & mask) + 1;
if (block_pruning_stripe == stripe)
return block_height;
This calculates the block's pruning stripe using the same method that we saw in this function.
This then checks if the block's stripe is the same as the seed stripe, if you remember if a seed and block have the same stripe that means the seed will keep the block, so we can just return the entered block_height
.
const uint64_t cycles = ((block_height / CRYPTONOTE_PRUNING_STRIPE_SIZE) >> log_stripes);
This calculates how many cycles of this table we have done:
stripe | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 |
---|---|---|---|---|---|---|---|---|
cycle 0: | 0 - 4095 | 4096 - 8,191 | 8192 - 12287 | .. | .. | .. | .. | .. |
cycle 1: | 32768 - 36863 | .. | .. | .. | .. | .. | .. | .. |
cycle 2: | .. | .. | .. | .. | .. | .. | .. | .. |
.. |
If we think about what this is doing, this makes sense:
\(cycles = \frac{block height}{CRYPTONOTE PRUNING STRIPE SIZE} * \frac{1}{2^{log stripes}} \)
for normal Monero pruning this is the same as:
\(cycles = \frac{block height}{4096 * 2^{3}} = \frac{block height}{32768}\)
const uint64_t cycle_start = cycles + ((stripe > block_pruning_stripe) ? 0 : 1);
This checks if we are a past our seeds stripe in a cycle and if we are past it we add
one to the number of cycles to get cycles_start
which is the start of the cycle our
stripe will next be storing blocks in.
const uint64_t h = cycle_start * (CRYPTONOTE_PRUNING_STRIPE_SIZE << log_stripes) + (stripe - 1) * CRYPTONOTE_PRUNING_STRIPE_SIZE;
If you remember from the table here each stripe will keep a part of the blockchain in a cyclic manner, which replates every 32,768.
- so stripe 1 will keep
numb_of_cycles * 32768 + 0 * 4096
- so stripe 2 will keep
numb_of_cycles * 32768 + 1 * 4096
- so stripe 3 will keep
numb_of_cycles * 32768 + 2 * 4096
Each stripe will stop keeping blocks at one less than the next stripes start.
This can be formalized into the equation:
numb_of_cycles * blocks_in_a_cycle + (stripe - 1) * stripe_size
which also equals:
numb_of_cycles * (stripe_size * amt_of_stripes) + (stripe - 1) * stripe_size
Knowing this, let's split this into 2 parts:
Part 1:
cycle_start * (CRYPTONOTE_PRUNING_STRIPE_SIZE << log_stripes)
This gets the block height at the start of the cycle_start
cycle, so if cycle_start
was:
0
the height would be0
1
the height would be32768
2
the height would be65536
Which is: numb_of_cycles * blocks_in_a_cycle
.
For normal Monero pruning: numb_of_cycles * (4096 * 8)
Part 2:
(stripe - 1) * CRYPTONOTE_PRUNING_STRIPE_SIZE
This gets how many blocks from the start of a cycle until the seeds stripe starts.
For example if the seed's stripe was:
1
the amount of blocks would be0
2
the amount of blocks would be4096
3
the amount of blocks would be8192
which is: (stripe-1) * stripe_size
As you can see if we add the amount of blocks until the start of a cycle (numb_of_cycles * blocks_in_a_cycle
) to the amount of blocks into a cycle the until the seed's stripe "kicks in" ((stripe-1) * stripe_size
) we will get the next un-pruned height.
if (h + CRYPTONOTE_PRUNING_TIP_BLOCKS > blockchain_height)
return blockchain_height < CRYPTONOTE_PRUNING_TIP_BLOCKS ? 0 : blockchain_height - CRYPTONOTE_PRUNING_TIP_BLOCKS;
CHECK_AND_ASSERT_MES(h >= block_height, block_height, "h < block_height, unexpected");
return h;
We now have to check if the height we calculated is above the tip blocks, if it is we get the starting height of the tip blocks and return that or if it isn't over the tip blocks we can just return the calculated height. Yay, we are done!
Getting The Next Pruned Block
For a particular seed and block height we can calculate what the height of the next pruned block will be. The function to do this in Monero is:
uint64_t get_next_pruned_block_height(uint64_t block_height, uint64_t blockchain_height, uint32_t pruning_seed)
{
const uint32_t stripe = get_pruning_stripe(pruning_seed);
if (stripe == 0)
return blockchain_height;
if (block_height + CRYPTONOTE_PRUNING_TIP_BLOCKS >= blockchain_height)
return blockchain_height;
const uint32_t seed_log_stripes = get_pruning_log_stripes(pruning_seed);
const uint64_t log_stripes = seed_log_stripes ? seed_log_stripes : CRYPTONOTE_PRUNING_LOG_STRIPES;
const uint64_t mask = (1ul << log_stripes) - 1;
const uint32_t block_pruning_seed = ((block_height / CRYPTONOTE_PRUNING_STRIPE_SIZE) & mask) + 1;
if (block_pruning_seed != stripe)
return block_height;
const uint32_t next_stripe = 1 + (block_pruning_seed & mask);
return get_next_unpruned_block_height(block_height, blockchain_height, tools::make_pruning_seed(next_stripe, log_stripes));
}
explaining what the function is doing in depth:
const uint32_t stripe = get_pruning_stripe(pruning_seed);
if (stripe == 0)
return blockchain_height;
if (block_height + CRYPTONOTE_PRUNING_TIP_BLOCKS >= blockchain_height)
return blockchain_height;
This is calculating the stripe of the inputted pruning seed, remember if the seed/stripe is 0
that means no pruning so we can return the blockchain height as the next un-pruned height and similarly if the block's height is within Tip Blocks of the blockchain's height that also means the block won't be pruned.
Returning the blockchain's height means the next pruned block doesn't currently exist, its bigger than or equal to blockchain_height - CRYPTONOTE_PRUNING_TIP_BLOCKS or it means it will never exist in the case of a zero pruning seed.
const uint32_t seed_log_stripes = get_pruning_log_stripes(pruning_seed);
const uint64_t log_stripes = seed_log_stripes ? seed_log_stripes : CRYPTONOTE_PRUNING_LOG_STRIPES;
const uint64_t mask = (1ul << log_stripes) - 1;
This is calculating the log stripes of the seed, although Monero currently only allows a log stripes of 3 in the future a higher number could be allowed so this function accounts for that.
If the seed's log stripes are zero this will set it to CRYPTONOTE_PRUNING_LOG_STRIPES
which is currently 3
.
Then this sets the value of mask
to one less than the amount of stripes, for Monero the amount of stripes is 8 so mask
will be 7.
const uint32_t block_pruning_seed = ((block_height / CRYPTONOTE_PRUNING_STRIPE_SIZE) & mask) + 1;
if (block_pruning_seed != stripe)
return block_height;
There is a typo here it should be block_pruning_stripe, think of this as foreshadowing what we are about to do
This calculates the blocks pruning seed STRIPE using the same method that we saw in this function.
This then checks if the block's stripe is NOT the same as the seed stripe, if you remember if a seed and block don't have the same stripe that means the seed will prune that block, so we can just return the entered block_height
.
const uint32_t next_stripe = 1 + (block_pruning_seed & mask);
return get_next_unpruned_block_height(block_height, blockchain_height, tools::make_pruning_seed(next_stripe, log_stripes));
Because the seed's stripe == the block's stripe we need to work out when our stripe ends (when the next stripe starts to get the next pruned block). To do this we can simply calculate the next stripe, make a new pruning seed and pass in that seed, which has a stripe one more than ours, into get next un-pruned block to get the start of the next stripe's un-pruned set and therefore the start of our next pruned set.