This is a map of the European electrical transmission network. Each dot is a connection point where all the consumption from nearby settlements connects to the network, as well as all electricity generation from nearby power plants.

• 🥒 Green dots mean more generation than consumption, so they export power to the network
• 🍇 Purple dots mean more consumption than generation, so they take power from the network
• ▲ Triangles show which way the power is flowing
• 🕹 Hovering over the triangles shows how much power is flowing through the power lines, and each line's total capacity

The numbers shown above are the results of running an optimisation algorithm. Its aim is to find optimal power flow. It goes like this:

I pick a date, and I ask the algorithm to calculate:

• which power plants should produce how much power
• for each 2-hour slot in that day,
• so that the total cost is minimal,
• and all power consumption is covered

To do that, for each connection point and each time slot, the algorithm gathers all the inputs first:

• 💻💡 Get total consumption from the nearby settlements and industry
• 💶 Get the price of generation per MW for solar, wind, hydro, coal, oil, gas & nuclear
• 🏭 Get the total generation capacity available at each point for the old-school power plants (gas, coal, nuclear, etc.)
• ☀️💨 Calculate how much the sun is shining, the wind is blowing, and the rivers are flowing (from historical data for the date I picked), based on which it can
• ⚡️Calculate how much solar, wind and hydro power plants can generate at most
• 🔌 Get how much power each power line can transfer at most

And then it just starts guessing how much each power plant should produce, while making sure that all of the above adds up right. It keeps guessing and improving its guesses until it finds the minimum cost of generation that satisfies all of these conditions.

If you then add up all the power generated at each point, and subtract all the power consumed at each point, you get the numbers which pop up if you hover over the dots.

Where'd You Get the Data? 🥸

The data comes from pypsa-eur, a research paper which is to my knowledge the best summary of the European power grid data that is currently publicly available:

GitHub - PyPSA/pypsa-eur: PyPSA-Eur: An Open Optimisation Model of the European Transmission System

PyPSA-Eur: An Open Optimisation Model of the European Transmission System - GitHub - PyPSA/pypsa-eur: PyPSA-Eur: An Open Optimisation Model of the European Transmission System

GitHubPyPSA

A huge thanks to Fabian Neumann and everyone else from Professor Tom Brown's research group behind pypsa-eur - for the great software, great documentation, and for gathering and prepaing all the data.

The little bit of code that I had to write on top of pypsa-eur to make that pretty-ass map above can be found here:

GitHub - zoltanmaric/coppersushi: Optimal power flow visualisation based on pypsa-eur

Optimal power flow visualisation based on pypsa-eur - GitHub - zoltanmaric/coppersushi: Optimal power flow visualisation based on pypsa-eur

GitHubzoltanmaric

Is This Real? 🦦

The math behind it is real, and it uses the same calculations that real grid operators use to keep the lights on. However, the amount of power that each power plant actually produces depends on the price they offer. This price varies wildly from hour to hour, and neither the price nor the power amount is publicly available.

Instead, pypsa-eur makes assumptions on prices for solar, wind, coal, nuclear, etc. based on historical averages, and then runs an optimisation algorithm to cover all consumption for the least cost of production. In short:

It's the real way of calculating who produces how much, and how the power flows in that case;
but it's calculated based on data that is not real.

For a full discussion of the limitations of the model, have a look at chapter 4 of the original pypsa-eur paper:

PyPSA-Eur: An Open Optimisation Model of the European Transmission System

PyPSA-Eur, the first open model dataset of the European power system at thetransmission network level to cover the full ENTSO-E area, is presented. Itcontains 6001 lines (alternating current lines at and above 220 kV voltagelevel and all high voltage direct current lines), 3657 substations, a new…

arXiv.orgJonas Hörsch

Why Does This Matter? 🤷‍♀️

TL;DR: we can delete most of the emissions of the developed world today by switching heating & car and truck transport to electrical - if we make sure we generate all electricity with near-zero emissions. Just building more renewable generation won't solve it though, as you still need to make sure it covers all consumption everywhere. This map intends to show why that's a complicated problem, and why we should also be smart about where we build renewable generation so that most of the generated energy can also be used and not thrown away because the grid can't handle it.

I wrote a less condensed explanation of this problem in another post:

The Copper Plate Must Die 💀

EU energy market rules are counterproductive to grid congestion & don’t incentivise optimal use of renewables.

1.21 GigawattsZoltán Marić

👋

The Copper Plate Must Die 💀

DIE THE COPPER PLATE MUST I think there’s an opportunity to make a big step towards reaching zero-emissions in Europe, and it doesn’t require any new technologies, but a policy change. This is what I would like to show you today.

Google Docs

I've been pondering. What if getting Europe to zero emissions as soon as possible — was a problem I had to solve? What if I were a benevolent dictator and had all the power to solve this problem? What would I do?

- and I came up with this elaborate plan:

So let's get to it!

What needs to be done

There’s pretty much already a consensus in the energy industry on what needs to be done in order to reach zero emissions: we know how to generate electricity without emissions, so let's just

Electrifying transport is already well underway, we plan to electrify heating with heat pumps, and then for the rest we plan to use hydrogen, which is also extracted using electricity. If we manage to do that, we will have solved the vast majority of today's emissions, so long as all the electricity comes from zero-emission generation 24/7. There’s our “what needs to be done” as a well-defined problem:

How to do it

Figuring out how to generate and deliver zero-emission energy 24/7 is also nothing new. There's plenty of studies about it coming from academia and various think tanks and agencies. They're called capacity expansion studies.

These studies look at:

• Existing and projected regional demand
• Existing regional renewable generation
• Existing grid infrastructure
• Regional solar and wind potentials, based on weather and land suitability

Based on which they estimate:

• where we should build more renewable generation and storage
• where we should expand the grid

Such that it costs the least, but still provides renewable energy 24/7 everywhere.

To do this, they run the data through an optimisation algorithm that may come up with something like this:

The sizes of the pies show how much additional renewable generation or storage we need to install in each region, and the sections of the pie tell us the proportions of the different technologies. For example we see a lot of yellow which is solar along the Mediterranean, and we see a lot of blue along the North and Baltic sea, which is wind, coupled with some pink for hydrogen storage.

The thickness of the links between the pies show the capacities of the transmission grid. If it's grey like here in Spain and in central France, that's the existing grid infrastructure, and if it's coloured brown or green like in Northern France and Germany, that's the required AC or DC transmission expansion.

This particular result is from a study from the Technical University of Berlin, published in March 2022 (link). The study presents best practices for capacity expansion calculations that co-optimise investments in generation, storage, and transmission. This is something I want to stress.

Capacity expansion is a co-optimisation of generation, storage & transmission.

Generation and storage location and capacity strongly influence how transmission should be expanded, and transmission capacity strongly influences how generation capacity should be expanded and where it should be placed.

Do it

In order to understand how we’re doing so far on our path to an optimal zero-emission grid, let me tell you a story of a problem that frequently occurs in Germany.

Say I have a factory in the South of Germany, and I want to buy 20 MW of capacity for tomorrow between 7 and 8 AM. I go to the energy market, and the most competitive sell bid for that time slot happens to come from a wind farm in the North at 20 EUR/MWh. That's a very attractive price, so I give the corresponding buy order, and the trade is executed, meaning I now have a binding agreement to consume at 20 MW for an hour tomorrow at 7, and that wind farm in the north has a binding agreement to produce at 20 MW during the same time.

The forecast for tomorrow says it's going to be very windy in the North, so there’s many other wind farms in the North that offer a large volume of energy at very low prices. Conversely, there's many other factories like mine in the South of Germany as that's where industry is concentrated. So now there's a lot of demand in the South matched with a lot of generation in the North. That amounts to tons of gigawatts having to be transferred from North to South through these few transmission lines connecting the North to the South.

The buyers and the sellers on the energy market have to make sure they consume or produce exactly the amounts of energy that they traded, but they don't have to worry about how that energy will be delivered to them. That's on the grid operators to worry about.

So the grid operators calculate the power flow according to the dispatch plan, and they see that the highlighted purple line above needs to transfer energy with a throughput of 900 MW, while its capacity is rated at 800 MW, and the other routes are already at capacity. So they perform redispatch, which is to say they curtail 100 MW of wind in the North, and they call up some gas power plant in the South to deliver the energy closer to where it's consumed.

The curtailed wind turbines still get to keep their payment, but the grid operator additionally pays the gas power plant, and at a premium, since it was dispatched on short notice.

UPDATE: To help me and others get a better intuition of how these kinds of grid congestion work, I've developed Copper Sushi 🍣 - a pretty, interactive map of the European grid. Go play around with it! 🕹🤓

Copper Sushi 🍣

A Pretty Map of the European Power Grid

1.21 GigawattsZoltán Marić

Now this is already pretty sad as it is, but the worst part is that it's the market rules that exacerbate this problem. There's no penalty or disincentive that would prevent participants from doing it again and again.

This is by design.

The idea is that the market shouldn't have to worry about delivering energy as long as both parties are within the same country. They should just be able to trade freely.

As if the grid weren't a grid,
but a copper plate.

No congestion, no bottlenecks. As far as market is concerned, energy can be transferred instantly between any two places on the copper plate.

And the fact that there’s nothing stopping the market from repeatedly causing the same problem is not the only issue. If we stick to this rule, new generation will keep disregarding impact on congestion, and make things worse. It will keep getting built far from demand, and demand will keep getting built far from generation. This is my biggest worry.

The current market rules
do not incentivise
co-optimisation
of generation and transmission.

Our current path towards zero emissions

I like to imagine our current path towards zero-emissions as shown in the diagram below. The dashed line on the top is where we are today (in 2022), and the dashed line on the bottom is the zero-emission future we want to get to.

If we take co-optimisation seriously, the road to get there might look something like the left side. Generation and transmission influencing each other, growing together more or less straight towards our goal.

Instead, today's market rules are letting generation grow more or less randomly, as shown on the right side, and then we say we’ll just catch up to new generation with more grid expansion.

This more expensive, less energy-efficient, and most importantly - wayyy slower.

Generation and storage development is mostly performed by the private sector, meaning many independent parties who move fast.

Transmission, on the other hand, is built by heavily regulated regional monopolies. That's not exactly the magic formula to move fast. That means they'll likely always lag significantly behind the development of generation and storage. More reason to co-optimise.

And by the way, that gap that the blue line lags behind the yellow line translates to generation capacity we cannot use in the short run, i.e. wasted renewable energy which we'll have to cover for with old- school power plants.

To be fair

The idea of the copper plate comes from a time before renewables, and it's well-intentioned. It's based on the thought that people who live far from power plants shouldn't have to pay more than those who live near. It probably works well if a grid grows gradually and organically.

But today we're talking about replacing almost all old- school generation with renewable generation and storage. And we want to do it as fast as possible. That's the opposite of gradual and organic. It's a complete overhaul of the grid!

Alright, I think I complained enough. Here’s what I’m proposing:

In the spot market, it has to be more expensive to buy energy from "far away" than from "close by".

Yes, in the short term, this may lead to price volatility. But this is exactly the fix that will give the price signal to co-optimise: More generation closer to excess demand areas; more industry, electrolysis and storage near excess generation areas.

The social injustice in the short term caused by potential high prices in some regions can be subsidised by the saved redispatch costs. It makes more sense to subsidise people than to subsidise the band-aiding of technical problems that we're causing ourselves with an outdated market design.

How do we do that?

The good news is that this already seems to be in consideration. The EU’s Agency for the Cooperation of Energy Regulators (ACER) is considering a reconfiguration of bidding zones. In other words – thinking about how to split up the country-wide copper plates into multiple copper saucers.

Following the results of the Locational Marginal Pricing (LMP) analysis carried out by Transmission System Operators (TSOs), the EU Agency for the Cooperation of Energy Regulators (ACER) has initiated the procedure to decide on the TSOs’ proposal of alternative bidding zone configurations.

National Grid in the UK is conducting a Net Zero Market Reform research, and it’s recently said that they’re considering moving to local pricing.

Moving from a national electricity price to more local pricing will reduce consumer costs and unlock further investment in renewables.

On the other hand, EPEX Spot, the company running the biggest short-term electricity market in Europe, doesn’t really like the idea of splitting up into smaller zones. They’ve outlined their concerns in a memo in 2016.

Bidding zone split would necessitate considerable amount of time and effort across power trading sector.

I understand that figuring out and implementing the actual solution is a nuanced matter and is anything but simple. I’m not saying that I have a ready solution. I am saying I want to help work on it. And I think we need to work on it because the current market design is broken and it’s leading us astray.

Feel free to comment on this post on Twitter or on LinkedIn 🙌

Have I told you about how climate change grinds my gears yet? Well, it does. So let's do something about it! But where do we start? How about looking at the largest human-made contributors to CO₂ emissions?

This tells us that in 2014, the largest global contributors were electricity and heat production at almost 50%, followed by transport just above 20%. By the way, that chart is totally interactive, go ahead and click around on it. 🪀🤓 E.g. you can show the same chart for your particular country if you want.

Anyway, having electricity production among the highest contributors to CO₂ emissions is kind of good news. It's good news because we already have the technologies for generating electricity with low emissions all figured out! 🎉

The other part of that largest contributor ― heat production, also has a very promising emerging technology: heat pumps. Heat pumps are rad, because they produce more energy in heat than they consume in electricity. E.g. it's not unusual for a ground source heat pump to produce 4 kWh of heat from 1 kWh of electricity. (Source)

Almost. The missing 3 kWh actually come from the thermal energy of the heat pump's ground source, which itself is a renewable source. Heat pumps are like renewable energy amplifiers 🔥🎸🔥

Next up, we have transport at 20% of global CO₂ emissions. Let's first have a look at how that breaks down further into modes of transport. I couldn't find recent global data for that, so we'll just look at the EU this time:

To be fair, these are greenhouse gas (GHG) emissions, not just CO₂ emissions, but since we're only trying to figure out who's the biggest polluter, it doesn't make that much of a difference.

Here we have a pretty clear winner: road transport, with over 70% of all transport emissions.

By the way, the transport sector makes up 25% of total greenhouse gas emissions in the EU. That means road transport alone accounts for 17% of all of the EU's greenhouse gas emissions combined!

Luckily, we're well under way of figuring out low-emission alternatives for road transport ― with the automotive industry's announced shift to electric and/or hydrogen-powered vehicles.

I say "luckily", because it would be much more worrying if e.g. aviation had a larger share in total emissions, since we didn't yet get very far with low-emission energy sources for planes.

So What About It?

I'm starting to pick up a theme here. Let's put the pieces together:

• 50% of global CO₂ emissions come from electricity and heat production
• Heat production can be electrified
• 17% of EU GHG emissions come from road transport
• Road transport can be electrified
• We've got low-emission electricity generation figured out

Yeah, I wish it were, Kermit 🥺 We may have low-emission, renewable electricity generation figured out, but generation alone is not enough for ⚡ELECTRICAL ENERGY WORLD DOMINATION⚡. The other big piece of the puzzle is getting that electrical energy to all the places it's needed, also known as infrastructure, also known as the power grid.

You see, the power grid we have today was for the most part built with the following assumptions:

• Power will be generated at a few large power plants
• The transmission lines will fan out from these few power plants
• The amount of power generated can be controlled

However, these assumptions are on increasingly shaky ground nowadays. The reason for that is a combination of 3 things.

1. Most Newly Installed Generation is Renewable

In 2019, 72% of newly installed generation was renewable (that includes hydro, wind, solar, bioenergy, and geothermal - just like Captain Planet):

2. Most Newly Installed Renewable Generation Is Non-Dispatchable

Generation is defined as non-dispatchable when it can't be turned on or off on demand. Think wind turbines and solar panels. You don't decide how much energy you're going to produce, or when. You produce when the wind is blowing or the sun is shining, and you produce no more than the wind is blowing or the sun is shining.

In 2019, newly installed wind and solar capacity accounted for 90% of all newly installed renewable sources.

So if 72% of all newly installed generation is renewable, and 90% of all renewable generation is wind and solar, then 65% of all newly installed generation is non-dispatchable, and that share is trending upwards.

3. Wind & Solar Are Decentralised

Pardon the buzzword. Remember how the grid was built with the assumption that there will be few big power plants, with transmission lines fanning outward from there? Well, wind and solar are small and all over the place 🤷‍♂️

On top of all of these, electrifying heating and road transport means that we're bringing significantly more load onto the power grid.

The Grid Needs Our Help

The combination of the trends above means that the power grid is increasingly faced with the following problems:

• Power is not generated when it's needed
• Power is not generated where it's needed
• The power supply is becoming more volatile

All of this is giving headaches to grid operators, because they're the ones responsible for keeping the grid in balance. Their life used to be easier when power generation came from nothing but dispatchable power plants so they could control everything. If only there were a way of taming those erratic renewable energy sources 🤔

That's Where Virtual Power Plants Come In!

Virtual Power Plants are software systems that provide control over a bunch of distributed energy resources like solar panels, wind turbines, batteries, or flexible loads. They're like the shepherds of the new power grid ― and pretty advanced ones at that.

This is just what grid operators need ― flexibility. As uncertainty in the grid increases with more renewables coming in, so does the need for more sophisticated control of both power generation and consumption. The simplest example of this is a virtual power plant controlling batteries which charge when there's too much production and discharge when there's too much demand (which I covered in this post). However, there's actually a variety of nuanced problems that grid operators face, and for which they increasingly turn to virtual power plants for help. In the next several posts, I will dive deeper into what those problems are, and how we at sonnen solve them.

The best part is that the optimal way of solving these problems, particularly using batteries, often happens to be the most sustainable way ― because batteries essentially attempt to capture excess renewable energy, and feed it back into the grid when there's a lack of it. Today, a huge share of this excess renewable energy is simply thrown away. 🗑️

I guess what I am trying to say is this:

Virtual power plants are crucial for solving the modern problems of the grid, and solving the modern problems of the grid will bring big reductions in greenhouse gas emissions.

In addition to that, they're a lovely engineering problem. 🤖

Anyway, this is how I explain to myself why what I'm working on is important. With that out of the way, now I can tell you about why it's fun! Stay tuned! 🤓

In the mean time, if you have any questions or remarks, let me know in the comments below or on LinkedIn or on Twitter. 👋

In episode 5 of The Building Blocks of Blockchain, we explained in detail how the mechanics of blockchain work. We didn’t really show how all of that makes blockchain immutable, though. Because immutability is one of the crucial features that makes blockchain interesting for application in the energy industry, I thought it worth explaining in more detail.

Immutable is just a fancy way of saying a blockchain’s contents cannot be changed. Let’s see what happens if Bugs Bunny attempts to change the contents of a block.

Let’s take the same block we used in the “Putting It All Together” post:

1. Daffy Duck receives 12.5 Bitcoins
2. Yosemite Sam gives Wile E. Coyote 4 Bitcoins
3. Porky Pig gives Pepé Le Pew 3 Bitcoins
4. Bugs Bunny gives Marvin the Martian 1 Bitcoin
5. Wile E. Coyote gives Elmer Fudd 2 Bitcoins
6. Elmer Fudd gives Porky Pig 7 Bitcoins
---
previous_block: 00bc30f8fae8f8b3ca113390fb4cfa16fd01084189fe285805a43c22bd81c739
nonce: 117
hash: 00639a07198476c9436cbbe1836ca8ca23c6a4792f8ebcba68732b56e9b81fae

And let’s say this is block number 5 in the blockchain.

Bugs’ first idea is to just change transaction number 2 in the block:

2. Yosemite Sam gives Wile E. Coyote 4 Bitcoins

so that he becomes the recipient instead:

2. Yosemite Sam gives Bugs Bunny 4 Bitcoins

He quickly realises this won’t work, because Yosemite Sam’s digital signature will not check out any more, and he can’t create a valid signature without knowing Sam’s private key.

He then figures he could just remove the transaction in which he gave Marvin his Bitcoin, essentially stealing it back. Now block number 5 looks like this:

1. Daffy Duck receives 25 Bitcoins
2. Yosemite Sam gives Wile E. Coyote 4 Bitcoins
3. Porky Pig gives Pepé Le Pew 3 Bitcoins
4. Wile E. Coyote gives Elmer Fudd 2 Bitcoins
5. Elmer Fudd gives Porky Pig 7 Bitcoins
---
previous_block: 00bc30f8fae8f8b3ca113390fb4cfa16fd01084189fe285805a43c22bd81c739
nonce: 127
hash: 0091ba2887d8a267c462c8d6f0a538e87188748308e15c644e25e81e479eeaf6

However, removing the transaction changes the hash of the block:

The above calculation was made using this online hash calculator. The new hash is:

1f2d89d4cb8ed12233ea4a4c9578b64ff5071dfd2a1213eadf99f2d92e587bae

which no longer matches the hash declared above. What’s more, it no longer provides a valid solution to the puzzle, as the new hash does not start with 2 zeroes! That means Bugs Bunny needs to solve the puzzle all over again!

That’s still not enough setback for Bugs Bunny to give up, so he goes about solving the puzzle. It takes him quite a while, but he manages to solve it eventually. He finds that the nonce 398 results in the following hash for his updated block number 5:

00be3133fa9810016c39305eb0dffc206ab98871b39fceec1b7d7c35911cb356

Now he has a valid-looking block which he could start spreading around. However, while he was solving the puzzle, there were several new blocks added to the blockchain! The next block, number 6, contains the old hash of block number 5 in its previous_block field. If Bugs Bunny were to update the previous_block field in block 6, its hash would change, which means he would need to solve a puzzle again, and then again for block 7, and 8, and each subsequent block.

Any change to a block makes the hash of that block and all subsequent blocks invalid.

Just call it a day, Bugs.

Alright! We’ve learned how hashes work, how digital signatures work, and how distributed ledgers and mining works. We’ve learned how each of them works conceptually and in isolation. They are the basic building blocks we need to understand well — in order to understand blockchain well. Now we’re ready to put the blocks together and build the big picture so you can finally experience that “Aha!” moment I owe you.

So let’s get to it!

In the last episode, the relationship between Bugs Bunny and Marvin the Martian was somewhere along these lines:

And guess what? Bugs still owes Marvin that Bitcoin! Unlike last time though, now Bugs Bunny has one to spare, so all he has to do is submit a transaction to Marvin’s address. Let’s look at how he does that.

Submitting a Transaction

The usual way for users to exchange Bitcoins is by using a wallet. A wallet is an app on your phone or computer which holds your private key, and which you use to submit transactions. In the earlier post on digital signatures, we mentioned how a wallet is used to generate a key pair which we need to produce digital signatures.

In much the same way, Bugs Bunny uses the wallet on his phone to submit the following transaction:

Amount: 1 Bitcoin
Recipient: (Marvin) bc1q428wf72dmujnz58l57tm3n76gyhkxad7y5rrqh
Signature: MEUCIQDnFAMFHCJjkxGu+dvFZqxzmvN7uMVf5Y1wSinebiTJ6wIgCFoN2Ak04YgGnRVM12uy7wXsoo45xM8MChNwAov46TY=

That strange string of characters in the recipient field is Marvin’s Bitcoin address. This is an identifier in blockchain which works just like a bank account number. The Signature field contains a digital signature produced with Bugs Bunny’s private key. This proves that the transaction came from Bugs Bunny, as described in the digital signatures building block.

In the rest of this post, we’ll keep using the more friendly-looking transaction notation from our previous post:

Bugs Bunny gives Marvin the Martian 1 Bitcoin

I just thought it’s worthwhile reminding ourselves that each of these transactions contains a digital signature.

Once created, the wallet submits this transaction to one of the miners.

The Anatomy of a Block

As we described in the previous post, miners are the ones that write transactions into a blockchain. They don’t write them into the blockchain one by one, but in batches. Each batch of transactions is organised into a block, so miners write blocks of transactions into the blockchain. Let’s see what these blocks look like and how they get added to the blockchain.

Daffy Duck is still a miner in our blockchain network, and he just received Bugs’ transaction. He now has 5 verified transactions submitted to him, which he wants to put in a block and add that block to the blockchain. Here’s what this block might look like:

1. Daffy Duck receives 12.5 Bitcoins
2. Yosemite Sam gives Wile E. Coyote 4 Bitcoins
3. Porky Pig gives Pepé Le Pew 3 Bitcoins
4. Bugs Bunny gives Marvin the Martian 1 Bitcoin
5. Wile E. Coyote gives Elmer Fudd 2 Bitcoins
6. Elmer Fudd gives Porky Pig 7 Bitcoins
---
previous_block: 00bc30f8fae8f8b3ca113390fb4cfa16fd01084189fe285805a43c22bd81c739

Where Do Bitcoins Come From?

That first transaction in that block above looks weird.

“Daffy Duck receives 12.5 Bitcoins.”

Receives… from whom? From no one. The first transaction in any block is the so-called coinbase transaction, and it’s a transaction that creates Bitcoins out of thin air. In fact, this is the one and only way how Bitcoins are created. Apart from creating Bitcoins, the other purpose of the coinbase transaction is to provide an incentive for miners to mine, i.e. an incentive to verify transactions they receive and write them into the ledger. It’s essentially the miners’ paycheck.

The amount that miners receive in the coinbase transaction is called the block reward, and it is defined in Bitcoin’s protocol. It started with 50 Bitcoins in 2008, and it halves every 4 years. At the time of writing this post, the block reward is 12.5 Bitcoins, and it’s poised to halve to 6.25 Bitcoins this year (in 2020).

There’s another thing we haven’t seen before, but appeared in our sample block above: the previous_block field. This field contains the hash of the last block in the blockchain. Each block contains the hash of the block that came before it, which ensures the blocks are sequential, and prevents changes to the contents of the blocks in the blockchain. In other words, this field is what makes the blockchain a chain.

Something’s still missing, though. We learned in the post on distributed ledgers that Daffy needs to solve a puzzle first in order to add the block to the blockchain. Where does that fit in to the story?

Where’s the Puzzle?

The puzzle that miners have to solve is actually ingeniously simple. Check it out:

The hash of the block has to start with a certain number of zeroes.

Let’s see what that means. First, let’s compute the hash of our block as it is now. For this, I just used this online tool for computing hashes.

It uses Keccak-256, the same hash function that both Bitcoin and Ethereum use. The resulting hash is

cea391ea414e9d02dece0db721c0c35de3e50153f13b30160fd7b9260599b771

This hash starts with a c, which is definitely not a certain number of zeroes. There’s a separate rule that determines what that certain number of zeroes is, i.e. how many zeroes the hash has to start with, but for our example, let’s assume that our current puzzle requires the hash to start with 2 zeroes. How can we make the hash of our block be different, though? The hash is the hash — there’s no changing it.

For exactly this purpose, Bitcoin’s blocks contain another field, the nonce. That means a block actually looks more like this:

1. Daffy Duck receives 12.5 Bitcoins
2. Yosemite Sam gives Wile E. Coyote 4 Bitcoins
3. Porky Pig gives Pepé Le Pew 3 Bitcoins
4. Bugs Bunny gives Marvin the Martian 1 Bitcoin
5. Wile E. Coyote gives Elmer Fudd 2 Bitcoins
6. Elmer Fudd gives Porky Pig 7 Bitcoins
---
previous_block:
00bc30f8fae8f8b3ca113390fb4cfa16fd01084189fe285805a43c22bd81c739
nonce: 0

The nonce is a number which the miner can set freely in order to change the resulting hash, i.e. to make the hash of the block start with the required number of zeroes. It is essentially the solution of the puzzle.

Now that we’ve introduced the nonce, we can express the puzzle more accurately:

Find a nonce, such that the hash of the block starts with a certain number of zeroes.

Again, for our example, we’ll assume that the target is 2 zeroes. So let’s have at it! We add the new field, set the nonce to 0 (the lowest possible value) and use that online tool again to compute the hash:

89b3e49dc09b0028d3d0ce48ab30314438fd5a08a759302c3d12a6fecd781094

Still no cigar. Daffy has to keep trying different numbers for the nonce, until the resulting hash starts with 2 zeroes. Here are some of his attempts:

1:  e464f509fed70efa387ef32dd4172e1205900377094780d2ba6fe9f7c7523afc
2:  938c428a450b550c16e686ebf097e9811978b921beb4b84da7a90f5095d4fb13
3:  2d57d47654c2da7d0f37234c4e88bc7f4f0ec0e01624e7c91df533238a83973c
27: b8ef539375af495738e0bccc6f96f02ddfd34a409b8631ca5abae7e870bf1a02

After trying for quite a while, Daffy found that the nonce 117 results in the following hash:

00639a07198476c9436cbbe1836ca8ca23c6a4792f8ebcba68732b56e9b81fae

Bingo! The hash starts with 2 zeroes, so he’s solved the puzzle!

As you may remember from the post on hashes, we said that cryptographic hash functions have an avalanche effect, i.e. that changing even a single digit in the input data results in a completely different hash. In Daffy’s attempts with different nonces above, you can see the avalanche effect in action.

Time to Gossip

Having solved the puzzle, Daffy Duck now has a valid block that he can tack onto the blockchain. He’s said to have mined a new block. This is what his new block looks like:

1. Daffy Duck receives 12.5 Bitcoins
2. Yosemite Sam gives Wile E. Coyote 4 Bitcoins
3. Porky Pig gives Pepé Le Pew 3 Bitcoins
4. Bugs Bunny gives Marvin the Martian 1 Bitcoin
5. Wile E. Coyote gives Elmer Fudd 2 Bitcoins
6. Elmer Fudd gives Porky Pig 7 Bitcoins
---
previous_block:
00bc30f8fae8f8b3ca113390fb4cfa16fd01084189fe285805a43c22bd81c739
nonce: 117
hash: 00639a07198476c9436cbbe1836ca8ca23c6a4792f8ebcba68732b56e9b81fae

Now he needs to let the world know about it. He sends his new block to all the other miners he knows about. They calculate the hash of the new block, see that it starts with 2 zeroes, and therefore accept it as a valid block. These miners then forward it to all the miners they know about, and soon the block becomes part of the blockchain of all the miners. This process of announcing the new block is called the gossip protocol. I’m sure you can see why 🤭

Of course, Marvin has also been following the blockchain closely, so it’s not long before he finds out Bugs’ transaction came through.

Why Are We Solving Puzzles, Again?

At any time, there could be thousands of miners online, all competing to add the next block to the blockchain to get their reward. We want to make sure that they don’t add new blocks at the same time, because that would lead to different versions of the ledger. Instead, we’re looking for a fair way to always pick one random miner of all the miners who are online, and only allow that one miner to write the next block. Making them all solve the puzzle is the fair way to always pick one miner, because the puzzle is engineered so that statistically, every 10 minutes only one miner can find a solution.

The puzzle is the magic ingredient that replaces a central party that would decide who gets to write into the blockchain. The puzzle is what makes blockchain decentralised.

One Solution Every 10 Minutes, No Matter How Many Miners?

That sounds suspicious, doesn’t it? How can it be that there’s always only 1 solution every 10 minutes, even if the number of miners increases? Shouldn’t it take less time to find a solution if the number of miners rises? You’re absolutely right. It should, and it does. But there’s a simple way Bitcoin solves this.

In our earlier example, we said that Daffy had to find a hash which starts with 2 zeroes. The number of zeroes the hash has to start with is called the difficulty, and it’s updated every 2 weeks. It’s called the difficulty, because the more zeroes the hash is required to start with, the more difficult it is to find the right nonce, i.e. to solve the puzzle.

Every 2 weeks, the average time between blocks is calculated. If it’s less than 10 minutes, the difficulty is increased accordingly. If it’s more than 10 minutes, the difficulty is decreased. As I’m writing this, the hash of the most recent block starts with 18 zeroes, and its nonce is 1,284,630,554.

How Did Bugs Bunny Know Which Miner to Submit the Transaction To?

In our example, Bugs Bunny sent his transaction to Daffy Duck, and Daffy conveniently mined the next block, including Bugs’ transaction. In reality, there could be thousands of miners online, and only one of them will mine a block every 10 minutes. This means it can take quite a while before Daffy actually mines the next block. So how is Bugs supposed to pick the right miner to send his transaction to, i.e. guess who is going to mine the next block?

Luckily, he doesn’t need to. Just as miners use the gossip protocol to announce their newly mined block, they also use it to share newly received transactions. They check if the transaction is valid according to the current state of the ledger, and then they share the transactions with all other miners they know about.

What If There’s Too Many Transactions?

As of February 2020, there are around 2000 transactions in a Bitcoin block on average. If there’s 1 new block every 10 minutes, that means Bitcoin can execute about 3.6 transactions per second. That gives Bugs Bunny a mischievous idea: he’ll create another Bitcoin address for himself, and submit millions of transactions just passing 1 Bitcoin between his 2 addresses. This will make the miners dance to Bugs’ tune,

while all other transactions will be stuck in line waiting for Bugs’ transactions to clear. What a rascal.

As you may have guessed, Bitcoin has a remedy for such mischievous behaviour:

Transactions are not free.

Transactions actually include a fee, so that transaction that Bugs Bunny sent to Marvin the Martian looked more like this:

Withdraw 1.01 Bitcoins from Bugs Bunny, deposit 1 Bitcoin to Marvin

That missing 0.01 Bitcoin is awarded to the miner who eventually mines this transaction. If the block reward is a miner’s paycheck, the transaction fee is the miner’s tip. Without a high enough tip, miners will just ignore the transaction and move on with their lives. The amount of the fee is not part of the Bitcoin protocol — it’s at the discretion of each miner to decide what they will accept as a sufficient fee to include a transaction in their next block. This basically means transaction fees create a market of their own.

Having learned this, Bugs Bunny now has 2 options: either he submits his transactions without a fee, resulting in the miners ignoring them; or he includes a fee, which makes his plan look more like this:

Congratulations!

There we have it! You should now have a fairly deep understanding of how blockchain works, all based on the practical details of how Bitcoin — the first blockchain — works.

One often-repeated feature of blockchain is its immutability, i.e. that its contents cannot be changed. This is a key feature that makes blockchain attractive for the energy industry. In case you’re interested in learning how what we learned here makes blockchain immutable, you can find out in this 3-minute post I wrote.

If you found any of this confusing or unclear, please let me know in the comments below, or on LinkedIn, or on Twitter.

Even though we still haven’t talked about the arguably most exciting part of blockchain — smart contracts, I thought this would be an appropriate moment for me to thank you, and for you to thank yourself, since you’ve just learned the complete story of blockchain as Bitcoin defined it. Smart contracts were only introduced 6 years after Bitcoin’s release — with Ethereum.

In the next and final episode of the Building Blocks series, we will talk about how smart contracts tie into what we learned about blockchain so far, and what the energy industry had in mind with them.

I would like to take this opportunity to praise the outstanding Coursera course “Bitcoin and Cryptocurrency Technologies” given by Princeton University. I completed this course in 2017, and this was the one resource that gave me my first in-depth understanding of blockchain.

All copyrighted material used under Fair Use for educational purposes.

The first blockchain ever made is the blockchain behind Bitcoin. The purpose of creating Bitcoin was creating a completely digital, decentralized currency system. We’ll talk about what “decentralized” means shortly— what I want to emphasize now, is that blockchain was invented to implement a currency system. It’s a simple goal, and in this post I want to explain how blockchain achieves this goal. Once you understand that, you will understand the most important part of how blockchain works.

A Book of Transactions

Bitcoin uses a blockchain to store information about how much money everyone has, and how they’re transferring their “money” between each other. To put it differently, Bitcoin’s blockchain is a database containing information on balances and transactions. It’s basically a book of transactions, also called a ledger. I’m not even exaggerating too much when I tell you that this is what a blockchain looks like:

1. Sylvester creates 25 Bitcoins
2. Sylvester gives Yosemite Sam 10 Bitcoins
3. Yosemite Sam gives Porky Pig 4 Bitcoins
4. Porky Pig gives Wile E. Coyote 1 Bitcoin

It’s what you would expect any ledger to look like, right? Each line is a transaction with an amount, a sender, and a recipient. The first line is a bit of an exception, in that it has no sender. Instead, it declares the creation of 25 Bitcoins and awarding them to Sylvester. I’ll describe how Bitcoins are created in the next post. For now, let’s focus on balances and transactions.

The ledger is the source of truth. You only have Bitcoin if the ledger says you have Bitcoin, and if you want to give someone Bitcoin, you can only do that by adding a transaction that says so into the ledger. From our example ledger above, we can clearly tell that Sylvester has 15 Bitcoins, Porky Pig has 3 Bitcoins, Wile E. Coyote has 1 Bitcoin, and I’m sure you can figure out for yourself how many Bitcoins Yosemite Sam has.

So that’s our currency system. We can tell everyone’s current balance by summing up all the transactions in the ledger, and participants can exchange the currency by submitting new transactions into the ledger. But how does that actually happen? Where is the ledger, and who writes the new transactions into it?

The simple solution to that question would be — an almighty accountant. In other words, a single participant that safeguards the ledger and is the only one allowed to write new transactions into it.

As you can see, in our story, Sylvester is the almighty accountant. If Yosemite Sam wants to give Bugs Bunny a Bitcoin, all Sam needs to do is tell Sylvester that he is giving 1 Bitcoin to Bugs. Sylvester checks the ledger and sees that Sam has 6 Bitcoins available, meaning the new transaction is valid, so Sylvester records the new transaction in the ledger. Now the ledger looks like this:

1. Sylvester creates 25 Bitcoins
2. Sylvester gives Yosemite Sam 10 Bitcoins
3. Yosemite Sam gives Porky Pig 4 Bitcoins
4. Porky Pig gives Wile E. Coyote 1 Bitcoin
5. Yosemite Sam gives Bugs Bunny 1 Bitcoin

This is a completely legit solution, and I’m sure you’ve met such almighty accountants before — Visa, PayPal, your bank… They’re all parties whom you trust and who have the exclusive right to carry out transactions in their respective payment systems. E.g. when you want to send some money to your friend via PayPal, you tell PayPal you’re doing so, and PayPal records this transaction in their ledger. This is what we call a centralised payment system, because it relies on a trusted central party for managing the ledger.

But in 2008, along came this dude called Satoshi Nakamoto, who said “hold my beer” and then published a white paper called “Bitcoin: A Peer-to-Peer Electronic Cash System” that described how you can make digital payments work without a trusted central party. By the way, it turns out that Satoshi is not even a real person, and so to this day we have no idea who came up with Bitcoin 🤷

A Distributed Ledger

Bitcoin enables digital payments without a central party by using a distributed ledger. But what is a distributed ledger? The basic idea is that anyone can download a copy of the ledger, and, instead of an almighty accountant, anyone can write new transactions into the ledger.

Wait, what? How can that work?!

Well let’s have a look-see! Say Daffy Duck decided to download a copy of the ledger. At this point, both Daffy Duck and Sylvester hold identical copies of the ledger. Daffy Duck then announces that he is now also a trusted accountant, and transactions can also be submitted to him to enter into the ledger.

Meanwhile, Bugs Bunny owes both Elmer Fudd and Marvin the Martian 1 Bitcoin. He hears the news that there are now 2 trusted accountants, so he decides to be a wise guy about it. He tells Sylvester that he’s giving 1 Bitcoin to Marvin the Martian, and he tells Daffy Duck that he is giving 1 Bitcoin to Elmer Fudd. Now the ledgers look like this:

Sylvester's Ledger

1. Sylvester creates 25 Bitcoins
2. Sylvester gives Yosemite Sam 10 Bitcoins
3. Yosemite Sam gives Porky Pig 4 Bitcoins
4. Porky Pig gives Wile E. Coyote 1 Bitcoin
5. Yosemite Sam gives Bugs Bunny 1 Bitcoin
6. Bugs Bunny gives Marvin the Martian 1 Bitcoin

Daffy Duck's Ledger

1. Sylvester creates 25 Bitcoins
2. Sylvester gives Yosemite Sam 10 Bitcoins
3. Yosemite Sam gives Porky Pig 4 Bitcoins
4. Porky Pig gives Wile E. Coyote 1 Bitcoin
5. Yosemite Sam gives Bugs Bunny 1 Bitcoin
6. Bugs Bunny gives Elmer Fudd 1 Bitcoin

Sufferin’ succotash, Daffy! It looks like Bugs committed a double spend!

Sylvester discovers that his and Daffy’s ledger no longer match, and what’s worse, Bugs Bunny seems to have given his only Bitcoin to two different people! They quickly realise that their currency system is not going to work this way, so they decide to hire an expert.

Remember we said anyone should be able to write into the ledger? This means we can’t hope that Sylvester and Daffy are going to be the only 2 accountants for good. Soon there could be tens, maybe hundreds of accountants, and they should all be able to write into the ledger. How can Wile E. make them all maintain an identical copy of the ledger?

He suspects the key will be to make sure only one accountant at a time writes a transaction into the ledger. But how can he make sure only one accountant at a time writes a transaction into the ledger, even as new accountants can join any time? Wile E. has to ponder on this one.

Puzzles!

That’s it! Wile E. comes up with an ingenious rule:

Before an accountant can write a new transaction into the ledger, they have to solve a puzzle!

The puzzle will be so complicated, that even as all the accountants try hard to solve it, all of them at the same time, it will take about 10 minutes before one of them manages to solve it. The accountant that solves the puzzle gets to write a transaction into the ledger.

She then shares the transaction she just added, along with the solution to the puzzle, to all the other accountants. They verify that the solution is correct, and therefore accept the new transaction and add it to their copy of the ledger.

The puzzle an accountant has to solve is based on the last transaction in the ledger and the new transaction the accountant intends to add. Since the last transaction now changed, they all have to solve a new puzzle, and it takes them 10 minutes again to solve it, as the process repeats.

The purpose of the puzzle is to randomly pick one accountant as the one who writes the next transaction into the ledger. This prevents creating different versions of the ledger, without giving anyone full control over the entire ledger.

Bugs Bunny the Wise Guy, Part 2

Wile E. Coyote explains his new rules to Sylvester and Daffy, but they don’t really quite get them, so they figure they’ll probably understand them better if they just try them. As a first step though, they decide to remove Bugs Bunny’s phony double-spend transactions. Now their ledgers are identical, and they look like this:

Sylvester's Ledger

1. Sylvester creates 25 Bitcoins
2. Sylvester gives Yosemite Sam 10 Bitcoins
3. Yosemite Sam gives Porky Pig 4 Bitcoins
4. Porky Pig gives Wile E. Coyote 1 Bitcoin
5. Yosemite Sam gives Bugs Bunny 1 Bitcoin

Daffy Duck's Ledger

1. Sylvester creates 25 Bitcoins
2. Sylvester gives Yosemite Sam 10 Bitcoins
3. Yosemite Sam gives Porky Pig 4 Bitcoins
4. Porky Pig gives Wile E. Coyote 1 Bitcoin
5. Yosemite Sam gives Bugs Bunny 1 Bitcoin

Meanwhile, Bugs Bunny is feeling cheeky, so he decides to just pull the same stunt again: he tells Sylvester he’s giving Marvin the Martian 1 Bitcoin, and he tells Daffy Duck he’s giving Elmer Fudd 1 Bitcoin.

This time though, Daffy and Sylvester don’t add the transactions into their ledgers immediately, because they have to solve the puzzle first. So they go about doing that.

As it happens, Daffy is the one who solves the puzzle first, so he adds the transaction he has to his ledger:

Daffy's Ledger

1. Sylvester creates 25 Bitcoins
2. Sylvester gives Yosemite Sam 10 Bitcoins
3. Yosemite Sam gives Porky Pig 4 Bitcoins
4. Porky Pig gives Wile E. Coyote 1 Bitcoin
5. Yosemite Sam gives Bugs Bunny 1 Bitcoin
6. Bugs Bunny gives Elmer Fudd 1 Bitcoin

He then shares this new transaction with Sylvester, also telling him the solution to his puzzle. Sylvester verifies the solution to Daffy’s puzzle and thus accepts the new transaction and adds it to his own ledger. Now Sylvester’s ledger looks like this:

Sylvester's Ledger

1. Sylvester creates 25 Bitcoins
2. Sylvester gives Yosemite Sam 10 Bitcoins
3. Yosemite Sam gives Porky Pig 4 Bitcoins
4. Porky Pig gives Wile E. Coyote 1 Bitcoin
5. Yosemite Sam gives Bugs Bunny 1 Bitcoin
6. Bugs Bunny gives Elmer Fudd 1 Bitcoin

Pending Transactions:
** Bugs Bunny gives Marvin the Martian 1 Bitcoin

Sylvester now sees that, according to the new state of the ledger, Bugs Bunny doesn’t have any Bitcoin left, so he just rejects the pending transaction that was supposed to give Bugs’ Bitcoin to Marvin the Martian.

But hey, at least Wile E. Coyote’s rules now have a clear way of deciding who gets to write into the ledger next, and thus make the ledger secure against double spending!

What’s All This About, Again?

You may get the impression that this analogy has become a bit too looney, but in fact, all of what you’ve just read is very very close to how Bitcoin actually works! The process of solving puzzles that I’m describing is really the process of mining, and the accountants are miners. Bitcoin miners really are competing with each other to solve hash puzzles.

What I hoped to get across from this story, is an intuitive understanding of the problem Bitcoin is trying to solve, and how it tries to solve it:

• Bitcoin intends to implement a ledger that cannot be controlled by a single party, but is controlled by many, in a fair way.
• Control by many is achieved by always giving control to a single participant for a limited time.
• Fairness is achieved by each participant gaining control for a limited time based on chance (being lucky enough to be the one who solves the next puzzle).

In the next post, we’ll cover in detail how all of this actually looks in Bitcoin — including how these hash puzzles work, how transactions are organised into blocks, how blocks are chained together, and how Bitcoins are created. We can finally put all the building blocks together and build the big picture to have our well-deserved “Aha!” moment. What a time to be alive!

Next building block: Putting It All Together

Was that interesting and understandable? Did I share some horrible misinformation? Do you disagree with me? Do you have any questions? Please let me know in the comments or on LinkedIn, or on Twitter :)

If you haven’t yet, you can read the introductory post on what it is I even do here.

Image Credits

Sylvester writing, Daffy surprised, Wile E. business card, Wile E. pondering, Daffy pondering, Sylvester pondering, Marvin angry. All copyrighted material used under Fair Use for educational purposes.

On a heavy oak stool, there lay a small scroll. She carefully unrolled the scroll, as two keys slipped out from inside it. One of them was copper, the other one golden. She noticed a strange scripture on the scroll:

That which gold petrifies, none but copper shall undo.

“What’s this?” she asked. “It’s an enchanted key pair.”, he replied. “Enchanted?” she asked, as her eyes lit up with thrill. “Try it! Take the golden key and lock that chest over there.” Timberdoodle turned around and saw a small wooden chest, full of brass gears, nuts, and bolts. There was a rusty padlock hanging on its staple. She shut the chest and locked the padlock on it with the golden key.

A piercing cracking sound enveloped them, as the chest suddenly turned into stone. She quickly tried to unlock the padlock in panic, but the golden key wouldn’t turn. “I can’t open it!” she panicked. “Relax.” Hokumpoke comforted her — “Remember what the scroll said.”

Timberdoodle pondered for a second, and then picked up the copper key. She put it in the padlock and turned it to the left. The padlock clicked open, and the chest stood agape in a flash, just as wooden as before.

I’d like you to give yourself a moment to really visualise this little scene of Timberdoodle and her two keys.

Good!

Timberdoodle herself was rather confused by the event, and she didn’t quite get why anyone would want to make such a silly pair of keys. She decided to go home and sleep on it.

While she’s asleep, I’d like to tell you a story from the other end of the forest.

Contracts Are Weird

At least I always thought so. You have a big pile of papers saying all kinds of things you’re supposed to agree to. If you scribble your name on the last page of the pile, you’ve signed the contract. Anyone who holds this pile of papers now has proof that you have agreed to everything that’s written on it.

What I always found weird about them, was that it seemed so easy to game the system. I could easily swap out one of the pages in the contract to add a clause saying that you owe me a lifelong supply of peanuts. I wouldn’t even have to touch your signature. But it wouldn’t be that hard to fake your signature either. It’s illegal to do either of those things, but it’s not impossible. If someone did do that to you, you would have to pursue legal action, which is such a drag, and the process takes ages.

Now, when it comes to cryptocurrencies, their purpose is to carry out transactions. But what’s in a transaction? Not much more than this:

• Amount
• Recipient
• Signature of the sender

It’s basically a contract saying that the sender is giving the recipient a certain amount of money. Since thousands of these transactions have to be carried out every second, it should be quite obvious that making it possible but illegal to change the transaction data or to fake the signature just won’t cut it. It has to be impossible.

Fortunately for us, digital signatures don’t quite work like paper signatures. Like paper signatures, they guarantee the contents of what is signed and they guarantee the identity of the party signing it. Unlike paper signatures, they make it impossible to change contents, and impossible to falsify identity.

Timberdoodle’s Enchanted Key Pair

The magical ingredient that makes digital signatures possible are so-called asymmetric key pairs. They work just like Timberdoodle’s enchanted key pair. They’re called asymmetric because each of them only works one way. The golden key can only lock the chest, and the copper key can only unlock the chest.

In reality, these keys are just very very large numbers. The golden key, i.e. the one that’s used for locking, is called the private key; and the copper key, i.e. the one that’s used for unlocking, is called the public key. Locking in this sense means encrypting a message, and unlocking means decrypting it.

• Golden key = private key, a large number used for encrypting messages
• Copper key = public key, a large number used for decrypting messages
• Locking/unlocking = encrypting/decrypting a message

Here’s an example of such a key pair, to give you an idea about what “very very large number” means:

Private key:
93155050515612370983163764633558747339007116928401713662183993588125059212302
Public key:
65044426400941893018565888471404011024052903416091989803275131610548528439975929483145666179707753254797354298707623581314170512855135822986652814617423560

Yes, each of those two is just one large number, and yes, they’re so big they don’t fit in one row. It’s very easy for a computer to generate such a pair of keys, but once they’re generated, it’s practically impossible to figure out the private key based on the public key.

A message encrypted with a private key can only be decrypted with its corresponding public key that was generated with it, and no other. The reverse is also true — if you can decrypt a message with a public key, you are certain that it was encrypted with its corresponding private key. This is the secret sauce of digital signatures, so I’ll repeat it.

If you can decrypt a message with a public key, you are certain that the message came from the person who has the corresponding private key.

Let me help you visualise that for a second. Let’s say Hokumpoke decides to keep the golden key, and Timberdoodle takes the copper key home with her. A couple of days later, Meerkat the mail man delivers a locked chest to Timberdoodle. She unlocks the chest with the copper key and finds a message inside. Because she was able to open the chest with the copper key, Timberdoodle is now sure that the message came from Hokumpoke, since only he has the golden key.

The message reads:

So Timberdoodle decides to hide her grapes.

How Does This Work in a Blockchain?

If you’ve ever traded cryptocurrency, it probably all started with you having to create an account using a wallet app. What the wallet app actually does when creating an account, is — you guessed it —it generates a new private and public key. The private key is never shown, and never sent. Using the wallet app usually requires a password. This password is used to encrypt your private key before it’s stored on your hard drive, to prevent others from getting your private key if they steal your laptop. The wallet then shows you your new address. An address is like a bank account number in cryptocurrencies. In effect, it’s just a truncated variation of your public key. You then possibly shared this address with a friend of yours to send you some of that cryptocurrency.

To send you some cryptocurrency really just means that they submitted a signed transaction to be added to the blockchain. Remember what a transaction looked like:

• Amount
• Address of the recipient
• Signature info of the sender

The signature information of the sender consists of the signature and the sender’s public key. And now for the million dollar question.

What is the Signature of the Sender?

To create the signature, the wallet app performs the following 2 steps:

Generating a Signature

1. Compute the hash of the transaction data (amount and recipient address)
2. Encrypt that hash using the private key of the sender

This encrypted hash of the transaction is in effect the digital signature of the transaction. For a quick refresher on how hashes work, I recommend reading my previous post. The following is an actual example on some sample data. It’s just there to help you visualise the steps, there’s no need to understand any of the values themselves.

Amount: 0.001 ETH
Address: 0x48e233335976fb09c99c04b6232b3e5619deeefd
---
Hash of transaction data above: 0xe1b01aa0e707e0a7dfcf1122dee01ce02a8dc581ad4f5320711a177f76830207

Signature (above hash encrypted using the sender's private key):
MEUCIQDnFAMFHCJjkxGu+dvFZqxzmvN7uMVf5Y1wSinebiTJ6wIgCFoN2Ak04YgGnRVM12uy7wXsoo45xM8MChNwAov46TY=

Sender's public key:
0x0454f9a87062a097d7d171032d394ef1ef1d05232bd31226bc4abda15bfd18427613b20b9038b02994b7c851b4433703ec0e7046ee201f1c2b740af20f28cb1410

So how can someone check if this signature is legit?

Validating a Signature

1. Compute the hash of the transaction data (amount and recipient)
2. Decrypt the signature using the public key of the sender
3. Check if the decrypted signature from step 2 is equal to the computed hash of the transaction from step 1

If the comparison in step 3 checks out, the signature is valid.

Amount: 0.001 ETH
Address: 0x48e233335976fb09c99c04b6232b3e5619deeefd
---
Computed Hash of transaction data above:
0xe1b01aa0e707e0a7dfcf1122dee01ce02a8dc581ad4f5320711a177f76830207

Signature:
MEUCIQDnFAMFHCJjkxGu+dvFZqxzmvN7uMVf5Y1wSinebiTJ6wIgCFoN2Ak04YgGnRVM12uy7wXsoo45xM8MChNwAov46TY=

Decrypted signature (using the sender's public key):
0xe1b01aa0e707e0a7dfcf1122dee01ce02a8dc581ad4f5320711a177f76830207

The decrypted signature above is equal to the computed hash, so this signature is valid.

Meerkat the Mail Man Meddles

This probably doesn’t look like much of a signature to you yet. To understand why it really is a signature, let’s see if it makes it impossible to change the contents, and impossible to falsify identity. Let’s ask Meerkat the mail man to help us with that by trying to meddle with our transactions.

What if Meerkat changes the transaction data?

If Meerkat changes e.g. the address of the recipient in the transaction, the hash computed in step 1 of the validation will be different from what we get in step 2, so the validation will fail. Meerkat is busted.

What if Meerkat changes the signature data?

If Meerkat just randomly messes with the signature data, step 2 of validation will just fail, because it will not be possible to decrypt the signature at all. Meerkat is busted.

What if Meerkat changes the data AND generates a new signature?

Meerkat will only be able to generate a new signature using a private key. He doesn’t have the original sender’s private key. The only private key he may have is his own private key. If he generates a new signature with his own private key, Meerkat is practically making himself the sender of the transaction. This means it would be Meerkat’s money that would get transferred. Meerkat just “falsified” a transaction by sending us his own money. Nice going, Meerkat.

What if Meerkat forges the sender’s signature?

It is impossible to generate a valid signature without knowing the original sender’s private key. Meerkat gives up on this idea.

What if Meerkat steals the sender’s private key?

That’s a pickle. The sender’s funds are as good as gone. Meerkat now has access to all of the sender’s money. This is why people go to great lengths to keep their private key safe.

There we have it. A fundamental building block for security in blockchain. It boils down to the following 3 points:

• A sender proves a message came from them by signing the message using the private key
• The proof lies in the fact that only the sender can possibly know the private key
• Anyone who knows the public key of the sender can validate the sender’s digital signature

I should note 2 more important points regarding digital signatures.

1. Just like hashes, digital signatures are not a mechanism of encryption. A digital signature is sent alongside the data that is signed, but that data is not encrypted. You can choose to encrypt the data, but that’s done separately. In Bitcoin and Ethereum, none of the transaction data is encrypted.
2. Digital signatures are absolutely ubiquitous, even outside of blockchain. See that padlock in the address bar of your browser as you’re reading this? That says that your computer received digitally signed information that the page you’re reading really came from Medium.
   You know when you touch your credit card on one of those payment devices in a store? There’s actually a tiny computer on your card that uses the private key stored in the card to digitally sign the transaction.

Is This Relevant for the Energy Industry?

Digital signatures are absolutely groundbreaking for the future of the energy industry. One big use case that I’m personally particularly thrilled about is making the origin of renewable energy traceable.T he big enabler for this will be smart meters which are able to digitally sign their measurements. The idea is that these smart meters would be connected to sources of renewable energy like e.g. solar panels on a rooftop. By knowing which public keys belong to smart meters connected to renewable sources, we can prove that a piece of energy came from a renewable source.

This is a big topic which deserves a post of its own, and I’ll make sure to write one once I’m done with the building blocks series. The good news though, is that Germany is already in the process of rolling out smart meters which can produce digital signatures. What a time to be alive! 🙌

Next building block: Distributed Ledgers & Mining

Was that interesting and understandable? Did I share some horrible misinformation? Do you disagree with me? Do you have any questions? Please let me know in the comments or on LinkedIn, or on Twitter :)

Image Credits

The illustration for the Tale of the Enchanted Key Pair was made by Roza for this post. You may reuse it under a Creative Commons (CC) license, so please link to this post if you do.

Will Ferrel thinking, Greedy meerkat, Pop-up meerkat

Hashes are certainly a core part of blockchain jargon. People talk about transaction hashes, block hashes, hash rate, hash power, even hash paprikaš! And even though the name — hash — describes pretty accurately how they’re made, it may not be that clear what their purpose is, or how they fulfil that purpose. I’ve decided to cover hashes as the first building block of blockchain because a) they’re what makes a blockchain a block-chain, and not a block-list; b) because they’re fundamental to all of the other building blocks I will cover.

What Do I Need Hashes For?

The simplest use of hashes is being able to check whether somebody changed some piece of data that you didn’t want to be changed.

You probably heard people saying that blockchains are immutable. This is just a fancy way of saying they can’t be changed. Hashes are what blockchains use to ensure their data can’t be changed. I will explain how exactly that works in this post.

On top of that, hashes are also the basis of blockchain mining. Mining is a pretty huge topic, which is why it will get its very own post in this series. I just wanted to mention it though, to get you amped-up for understanding hashes 😬. For now, though, let’s focus on getting a comfortable understanding of what hashes are and how they work.

Hashing Potatoes

The verb “to hash” in itself is not a computer science term at all. It actually means “to chop up” in plain English. That’s also where hash-browns got their name from — they’re made of chopped-up potatoes, pressed back together into a patty!

In much the same way — what we call a hash in the context of blockchain, is simply chopped up data!

I knew you would be looking at me like that. Anyway, I just wanted to tell you what the word means, and why that’s relevant. Now we can break it down step by step, and figure out what the heck I just said up there.

What Is Data?

I mentioned “some input data” a few times now, but it may not be clear what that means. It is really any piece of information. It may be some text, it may be an image, or it may be transaction information. When it comes to computers, they really don’t care. Whatever the data you want to work with, under the hood, it’s always stored as a sequence of numbers.

A simple example of this is text. For text, your computer can represent each character as a single number. E.g. the text “The Frog Prince” can be represented as the following sequence of numbers:

84 104 101 32 70 114 111 103 32 80 114 105 110 99 101
T   h   e     F   r   o   g     P   r   i   n  c   e

Similarly, a picture can be represented with a number for each of its pixels, and a transaction with 2 account numbers and an amount. In the end, it always boils down to a sequence of numbers like the one above.

How Do You “Chop Up” Data?

Now that we know that — in a computer — data is just a sequence of numbers, it will be easier to understand what we mean by chopping it up. The process of chopping up data, i.e. hashing, is done by a hash function. But what is a hash function?

Think of it as a cooking recipe for making hash-browns. The ingredients are the input data, the dish you’re making is the hash, and the recipe is the way how you chop up and stick together the ingredients to make that hash.

In addition, this recipe has to fulfil the following properties:

• It can take data of any length as input
• Its output (the hash) is always the same length, regardless of the length of input
• The same input always produces the same output
• It’s one-way, i.e. you can’t recreate the input from the hash by reversing the recipe

Alright, let’s see an example of such a hash recipe right now:

1. Divide the input data into groups of 4 numbers
2. Add the numbers together group-wise
3. If any of the resulting numbers is 256 or larger, subtract 256 from it until it’s less than 256

We’ll call this the Goldilocks hash recipe. Let’s see how it works on our example text— “The Frog Prince” — from the previous section.

Steps 1 & 2 —divide the data into groups of 4 numbers and add them together group-wise:

   84 104 101  32
+  70 114 111 103
+  32  80 114 105
+ 110  99 101
-----------------
= 296 397 427 240

Step 3 — if any of the resulting numbers is 256 or larger, subtract 256 from it until it’s less than 256:

  296 397 427 240
- 256 256 256
-----------------
=  40 141 171 240

Bam! There we have it — our first hash 🎉: 40 141 171 240. This really is the basic principle of how hash functions commonly used today work. They may get a bit weirder on the operations they perform on the numbers, but in essence, they divide the data into groups, they do arithmetic, shuffle their order, shuffle their digits, and out comes a new sequence of numbers.

Has Someone Been Eating Your Porridge?

Alright, we now know how to compute the hash of some data, but we haven’t yet shown how hashes tell us whether some data changed. This is best shown on an example.

Imagine someone sends you a message, and in addition, they tell you that the Goldilocks hash of the message is 40 141 171 240(the same hash we computed above). When you receive the message, it reads “The Frog Quince”:

84 104 101 32 70 114 111 103 32 81 117 105 110 99 101
T   h   e     F   r   o   g     Q   u   i   n  c   e

That message sounds suspicious to you. In order to figure out whether this was really the message that was sent to you, you decide to compute its Goldilocks hash and compare the computed hash to the hash you received.

   84 104 101  32
+  70 114 111 103
+  32  81 117 105
+ 110  99 101
-----------------
= 296 398 430 240
- 256 256 256
-----------------
=  40 142 174 240

The hash you computed is 40 142 174 240, which does not match the received 40 141 171 240, so you conclude that someone has fiddled around with your message!

How Do You Know Someone Didn’t Change The Hash Too?

Aha! Excellent question. What if somebody changes both the message and the hash you use for verifying the data? To prevent changing the hash, the sender will usually digitally sign the hash. If somebody changes the hash, the signature will not check out. There’s no way of changing the signature because only the original sender can generate a valid signature.

We’ll cover digital signatures in the next post. The only thing you need to know now is that they are a way of mathematically proving that a message came from a particular sender.

Why Do Goldilocks Hashes Look Different Than The Hashes I’ve Seen Before?

That’s a good point. If you’ve seen actual hashes on blockchains before, you may remember they always start with 0x and look something like this:

0xec620a4b0422641eabc024686c608903aa43299f621619d94242f039a556b08f

So why does our Goldilocks hash look like 40 141 171 240? Well, they’re really not that different. It’s just a convention for hashes to write each number in your sequence not as a “normal” decimal number, but as a hexadecimal number. This is really just another way of writing numbers. I won’t get into how it works now — instead, I’ll resort to witchcraft to convert the numbers in our example hash into hexadecimal notation:

40 141 171 240

28 8d ab f0

This means 40 is 28 in hexadecimal notation, 141 is 8d in hexadecimal notation, and so on. We can now just paste all the numbers together, add a 0x at the beginning, and there you have it — our Goldilocks hash 40 141 171 240 is really 0x288dabf0 in hexadecimal notation. By the way, the 0x at the beginning is used as a sign to indicate that this is hexadecimal notation.

Before I wrap up, I should note one more thing: our made-up Goldilocks hash recipe is a legit hash function, but it is not a cryptographic hash function. Cryptographic hash functions additionally have to meet the following conditions, which make them secure for usage in cryptography and blockchain:

• Irreversible: Given a hash, it should be infeasible to make up a message that results in that hash
• Unique: It should be infeasible to find 2 different messages that result in the same hash
• Avalanche Effect: A small change in the input message should result in a completely different hash

Cryptographic hash functions achieve all of this with a way of computing hashes that’s a bit more involved than our Goldilocks recipe. Nevertheless, all the other principles of how hashes work that I’ve described here also apply to cryptographic hash functions.

Note also that hashes are not encrypted messages. This is a common misconception. An encrypted message can be decrypted using a secret key, i.e. encryption is two-way. Cryptographic hashes, on the other hand, are one-way, i.e. there’s no way you can figure out the input if you only have a hash.

🎉 Thanks for Reading This Far 🎉

I hope this story gives you a good idea of how hashes work. In contrast, I expect that the bit about what hashes are for will fall into place better in the upcoming posts as we learn how they’re applied in blockchain. In the meantime, give yourself time to let this settle in, and please let me know if I left something unclear.

Next building block: Digital Signatures

I find the challenges of modern energy generation and power grids fascinating, and I feel galvanized to help solve them as quickly as possible. This is why I will try to clarify these challenges in a series of blog posts, along with our ideas and attempts at tackling them.

Am I doing a good job? Was that interesting and understandable? Did I share some horrible misinformation? Do you disagree with me? Do you have any questions? Please let me know in the comments or on LinkedIn, or on Twitter :)

If you haven’t yet, you can read the introductory post on what it is I even do here.

Because of my job title, I often get asked “Can you explain how blockchain works?”, or “What’s the purpose of blockchain in the energy sector?” When this happens, I feel overwhelmed with thoughts.

“Well, you see, the big thing about it is that it’s a distributed ledger, and every change in the data is cryptographically signed, and it natively supports transaction of value and protects against double spending without a central party, and you can prove how each piece of data is processed, and …”

Then I usually get the following reaction:

But it’s really been bugging me. Why do I suck so much at explaining it? Why is it so hard?

After giving it quite some thought, I’ve come to the conclusion that I should embrace the fact that it is hard, and give blockchain the credit it deserves. Blockchain is not a technology that can be explained in 5 minutes. It’s not even one technology for that matter! It’s actually a collection of well-established cryptographic technologies. However, although all of these technologies existed before the invention of blockchain, they’re not exactly common knowledge for people outside of computer science. In fact, some of them are quite obscure even for people in computer science.

In order to understand blockchain on a level deeper than Ron above, it’s essential to understand the technologies that it comprises, and how they are applied to form the blockchain. It’s how these technologies are combined that makes blockchain such a seismic and pure genius breakthrough.

The Building Blocks Of Blockchain

This post is in fact an announcement for a series of posts I intend to write in the following weeks. Each of these posts will explain a single topic (a building block) in an approachable way, topped off by a final post explaining how the blocks fit together into the big picture.

What are the building blocks, then? I’m glad you asked! I believe you can have a sufficiently deep understanding of blockchain and what it could be used for if you understand the following 5 topics:

• Proving That Some Data Hasn’t Changed: Cryptographic Hashes
• Proving That Some Data Came from Me: Digital Signatures
• Distributed Ledgers & Mining
• Putting it all Together
• What Are Smart Contracts and What Are They Good For?

Each of these topics will be covered in a dedicated post in the series, which is why I won’t get into explaining them further here.

Why Should I Care?

There is one simple reason, and one not-so-simple reason that may not apply to all readers.

The simple reason is — because you read this far. Blockchain is surrounded by so much hype, and it’s such an intriguing topic. Still, it’s hard to really get it, regardless of how many explainer videos you watched about it. Yes, I’ll dare to promise I’ll explain blockchain in a way that makes you finally get it, as long as you promise me that you will openly complain and let me know if I did a poor job and there’s still something unclear.

The other reason is that a lot of people see great hope in blockchain for the energy industry. There are many different ideas and approaches, but the one idea that resonated with me, was the one proposed by Michael Merz in his book “Blockchain for B2B Integration”. In the book, Michael argues that because the rapid rise of renewable energy drove energy prices and transaction sizes down, the cost of transactions is becoming a significant factor. If we want to embrace the fact that the grid is moving toward many small distributed energy resources, we will soon need to handle huge numbers of very small transactions. If these transactions are to be financially viable, we need a platform that can support transaction costs which are lower than a Euro cent. This is one of the big problems the energy industry hopes blockchain will help solve.

These are the ideas I want to drive home so that if you happen to read through to the end of the series, you feel more like this:

See you there!

Dive right in with the first building block: cryptographic hashes.

I find the challenges of modern energy generation and power grids fascinating, and I feel galvanized to help solve them as quickly as possible. This is why I will try to clarify these challenges in a series of blog posts, along with our ideas and attempts at tackling them.

Am I doing a good job? Was that interesting and understandable? Did I share some horrible misinformation? Do you disagree with me? Do you have any questions? Please let me know in the comments or on LinkedIn, or on Twitter :)

If you haven’t yet, you can read the introductory post on what it is I even do here.

In the introductory post of this series, I talked about virtual power plants and how they help stabilise the grid. When there is too much power in the grid, they pull power from it; and when there’s a lack of power in the grid, they feed in power. How do they know whether there’s too much or not enough power in the grid? They know it by measuring the frequency of the grid. But what does that even mean? What is the “grid frequency” a frequency of? Like a lot of things in power grids, this topic is not exactly rocket science — in fact, it’s quite simple. In this post, I hope to explain it in a way that makes you think so too.

Before we dive in, here’s a little heads up, though. In the next two paragraphs, I’m going to have to talk about nasty terms like voltage and alternating current. Ugh. We’ll get through them in no time, though, and then I promise it’ll be all be back to fun and games. Let’s get to it then!

The Flow of Electrons

When you plug a device into a socket and turn it on, electrons from the power outlet start flowing through your device. Your device works by converting the flow of electrons into other types of energy like light, heat, moving something, etc. This is the first important thing to note — your vacuum cleaner doesn’t eat up electrons that come from your socket, it just uses the energy those flowing electrons have to spin a fan. It’s just like a watermill using the flow of water to turn.

Now, unlike a stream under a watermill, those electrons don’t need to flow in one direction the whole time for us to be able to make use of them. They can also “flow” back and forth. Some devices like a nice, constant flow in the same direction — called direct current, DC; while others prefer a back and forth flow — called alternating current, AC. It is called alternating because it alternates the direction of the current of electrons. As an example — your computer and other small electronics usually work on DC, while devices containing motors, like your washing machine, usually work on AC.

It is fairly easy to convert between AC and DC (which is why you can have both kinds of devices in your household), but for our quest to understand what grid frequency is, it is important to note that in the power grid, electrons are flowing back and forth, meaning it is an AC grid. This was in fact no easy decision, but it is the result of a series of decisive events called The War of the Currents (no kidding). I would like to take this opportunity to commend the 19th century reporters for coming up with such a rad name.

Back and Forth

One of the reasons why AC won The War of the Currents was that devices for generating large amounts of AC power (AC generators) were fairly simple in design. Let’s take a look at how they work.

Most conventional power plants use steam, water, or wind to rotate a turbine. The turbine then either rotates a magnet inside some copper coils, causing the electrons in the coils to move around; or it rotates coils inside a magnetic field, again causing the electrons in the coils to move around (pictured below). In both cases, changing the direction of the poles of the magnet with respect to the coil causes the stream of electrons flowing inside the coil to change direction, i.e. produces alternating current.

What’s actually changing in the generator is the voltage — alternating between positive and negative. But voltage is such an alien word —humans don’t really experience the effect of voltage in everyday life (and let’s keep it that way), as we do with other physical quantities like force, heat, energy and so on. Luckily, voltage has two alter egos that are much more descriptive — electrical pressure and electrical tension. What these alter egos are trying to tell us is that we can actually imagine voltage as the difference in pressure between two ends of a wire. This difference in pressure causes electrons to flow, just like the difference in air pressure causes wind to blow.

As for the AC generator, I like to imagine it as if the turbine was controlling a piston, and the piston was pushing and pulling the electrons in a pipe. When the piston is pushing into the pipe, it’s increasing the pressure on that end of the pipe, causing the electrons to flow away from it. When it’s pulling back, it’s creating tension on its end of the pipe, causing the electrons to flow towards it.

I thought I Came Here to Read About Frequency

I think we can finally answer the question “What is the grid frequency a frequency of?” It is the frequency of voltage cycles in the grid, i.e. of the voltage going from the highest to its lowest, and back to its highest point. In other words, it is the frequency of the electrons flowing back and forth. In Europe, the grid frequency is 50 Hz, meaning the electrons in the power lines go back and forth 50 times in one second, or the voltage goes from 325V down to -325V and back to 325V 50 times in a second in your power outlet. Most importantly, this frequency is directly dependent on and synchronised with the rate of turning of the turbines (their revolutions per minute) in all the power plants.

How Does that Relate to the Balance of Production & Consumption?

When you turn on a device, the flow of electrons in it provides the device with energy, but this represents a resistance (or load) on their flow, i.e. a load on the grid. As you keep increasing the load, it’s going to be harder and harder for the turbines in the power plants to keep pushing and pulling the electrons, i.e. harder to keep spinning at the same rate, and they’ll start slowing down (less revolutions per minute), thus making the frequency drop.

It’s like riding a bike. I’m going to have to go full hipster, and ask you to imagine riding a fixie at a constant speed on a flat road. Since you’re riding at a constant speed, you are turning your pedals at a constant frequency.

This is how all the power plants feel when our consumption is exactly as they planned, and they’re purring away happily at 50 Hz.

Next, the road starts to go uphill, making it harder for you to maintain your speed, and you start pedalling slower, i.e. you reduce the frequency of your pedalling.

When consumption (load) on the grid rises above planned levels, the power plants feel like they are now cycling uphill, causing their turbines to spin slower — below 50 Hz.

You now reach the top of the hill, and start going downhill, causing your fixie to accelerate and your pedals to turn faster than you’d like them to.

When there’s less load than planned in the grid, the power plants are producing with too much power, causing their turbines to spin faster (as if they were cycling downhill) and the frequency to rise — above 50 Hz.

Errm… Why are you telling me this, again?

Because virtual power plants! When the grid frequency is too high — meaning there’s too little load on the grid — the sonnen VPP increases the load by charging the sonnen batteries in customers’ homes from the grid. When the frequency drops — meaning there’s too much load or conversely too little production — the VPP discharges power from the batteries into the grid, thus increasing production. I explained how virtual power plants work in more detail in this post.

I find the challenges of modern energy generation and power grids fascinating, and I feel galvanized to help solve them as quickly as possible. This is why I will try to clarify these challenges in a series of blog posts, along with our ideas and attempts at tackling them.

Am I doing a good job? Was that interesting and understandable? Did I share some horrible misinformation? Do you disagree with me? Do you have any questions? Please let me know in the comments or on LinkedIn, or on Twitter :)

When I first saw the job ad for sonnen back in September 2017, I was speechless. Whoa. Home batteries. Blockchain. Virtual power plant. In Berlin! What the heck is a virtual power plant? I love it.

Well, needless to say, I immediately started preparing to apply for the position, and in a couple of months, I got it indeed.

However, it really took me quite a while to understand why a bunch of batteries working together trying to stabilise the power grid was called a “virtual power plant”. And what is it about a power grid that needs to be stabilised?

What is a power plant?

A power plant is one of those factory-looking things that generates electricity, right? One of these:

Lovely. Very Pink Floyd. So how can a bunch of batteries be a power plant? They don’t generate electricity — they store electricity. That’s not what a power plant does!

Technically, that is correct, however, the catch is that generating electricity is not all a power plant does. Its responsibility is also to generate electricity at a strictly defined rate. In other words, a power plant also has to control how fast it is producing energy, i.e. how much energy it is producing over time. This rate, or “speed” of producing energy is called power, and it’s usually measured in megawatts for power plants.

Here are some quick numbers to help you get some perspective:

• A power plant producing at 300 MW will have produced 300 MWh of energy in an hour (duh)
• In 2014, an average household in the EU was consuming 3600 kWh in a year (source: EnerData)

Why does a power grid need to be stabilised?

Here’s something I had been completely clueless about, before I started working in the energy industry: How can it be that I can just decide to start my washing machine or a hair dryer at any time, and no one cares, i.e. the power outlet just covers it. Like where does that extra energy suddenly come from? Surely that far-away power plant didn’t increase its production just to cover that extra energy consumption I just caused.

Well, it kinda did.

In order for a power grid to operate properly, consumption has to equal production at all times, within a very small margin. But how does a power plant know that consumption increased or decreased? Conveniently, there’s actually a very simple measure that tells exactly this — the grid frequency.

In Europe, the grid frequency equals 50 Hz when production exactly matches consumption. If it’s above 50 Hz, production is higher than consumption, and if it’s below 50 Hz … you get the picture.

Certain types of power plants reserve a part of their capacity for the ability to decrease or increase their production as the frequency increases or decreases, thus stabilising the grid. This reserve capacity is called frequency containment reserve (FCR). If you would like to understand in more detail how grid frequency works, perhaps my post “What’s the Deal with Grid Frequency?” can help you with that.

Will you please tell us what a Virtual Power Plant is already?

Why yes! Generally speaking, a virtual power plant is a group of grid assets and a piece of software that can control their production or load. It could consist of generating assets, like small biogas power plants and wind farms; or load assets, like an aluminium furnace. The VPP software controls power generation and load of its assets as required by the grid.

This is exactly it —while power plants reserve a part of their generation capacity in order to be able to increase or decrease production, the sonnen Virtual Power Plant achieves the same effect with a swarm of household batteries. It feeds power from the batteries into the grid when there’s excess demand, or pulls power from the grid and stores them into the batteries when there’s excess supply.

OK, what’s the big deal about that?

More and more energy is coming from solar and wind. Unlike in the case of conventional power plants, generation from wind and solar cannot be increased to meet demand, and it can only be curtailed (i.e. shut off) to decrease excess production, meaning part of the renewable energy that could have been produced is wasted, because there wasn’t enough demand to consume it.

In addition, because solar and wind energy is so volatile, the grid frequency also becomes more volatile, requiring more e.g. gas plants to perform frequency containment.

Contrast that idea with using batteries for frequency containment, where the same excess renewable energy that was stored when there was excess production can be released back into the grid at a later time, when there’s lack of (renewable) production.

sonnen recently released a pretty video on this exact topic, you may like it:

I find the challenges of modern energy generation and power grids fascinating, and I feel galvanized to help solve them as quickly as possible. This is why I will try to clarify these challenges in a series of blog posts, along with our ideas and attempts at tackling them.

Am I doing a good job? Was that interesting and understandable? Did I share some horrible misinformation? Do you disagree with me? Do you have any questions? Please let me know in the comments or on LinkedIn, or on Twitter :)