Bits & Blocks Press

Mastering Ethereum - Building Blockchain Service Contracts and Decentralized Apps (WORK-IN-PROGRESS / DRAFT)

by Andreas M. Antonopoulos, Gavin Wood, et al

Contents

Notes

This is the original source reformatted in a single-page book edition (using the Manuscripts format).

See the source repo for how the book gets auto-built with "plain" Jekyll - of course - and hosted on GitHub Pages.

Onwards.

What is Ethereum?

Contents

Ethereum is “the World Computer”. That’s one of the more common descriptions of the Ethereum platform. But what does that mean? Let’s try to start with a computer science focused description, and then try to decipher that with a more practical analysis of Ethereum’s capabilities and characteristics, while comparing it to bitcoin and other distributed ledger technologies (that - for simplicity’s sake - we will often refer to as “blockchains”).

From a computer science perspective, Ethereum is a deterministic but practically unbounded state-machine with two basic functions; the first being a globally accessible singleton state, and the second being a virtual machine that applies changes to that state.

From a more practical perspective, Ethereum is an open-source, globally decentralized computing infrastructure that executes programs called smart contracts. It uses a blockchain to synchronize and store the system state along with a cryptocurrency called ether to meter and constrain execution resource cost.

The Ethereum platform enables developers to build powerful decentralized applications with built-in economic functions. While providing continuous uptime, it also reduces or eliminates censorship, third party interface, and counterparty risk.

Compared to Bitcoin

Many people will come to Ethereum with some prior experience of cryptocurrencies, specifically bitcoin. Ethereum shares many common elements with other open blockchains: a peer-to-peer network connecting participants, a consensus algorithm for synchronization of state (Proof-of-Work), a digital currency (ether) and a global ledger (the blockchain).

Components of a blockchain

The components of an open, public, blockchain are (usually):

All or most of these components are usually combined in a single software client. For example, in Bitcoin, the reference implementation is developed by the bitcoin Core open source project, implemented as the +bitcoind+ client. In Ethereum, rather than a reference implementation, there is a reference specification, a mathematical description of the system in the [yellowpaper]. There are a number of clients, which are built according to the reference specification.

In the past, we used the term blockchain to represent all of the components above, a short-hand reference to the combination of technologies that encompass all of the characteristics above. Today, however, the term blockchain has become diluted by marketers and profiteers, looking to hype their projects and attain unrealistic valuations for their startups. It’s effectively meaningless on its own. We need qualifiers to help us understand the characteristics of the blockchain in question, such as open, public, global, decentralized, neutral, and censorship-resistant, to identify the important emergent characteristics of a “blockchain” system that these components allow.

Not all blockchains are created equal. When you are told that something is a blockchain, you have not received an answer, rather you need to start asking a lot of questions to clarify what “blockchain” means. Start by asking for a description of the components above, then ask about whether this “blockchain” exhibits the open, public, etc. characteristics.

Development of Ethereum

In many ways, both the purpose and construction of Ethereum are strikingly different from the open blockchains that preceded it, including Bitcoin.

Ethereum’s purpose is not primarily a digital currency payment network. While the digital currency ether is both integral and necessary for the operation of Ethereum, ether is intended as a utility currency to pay for use of the Ethereum platform.

Unlike Bitcoin, which has a very limited scripting language, Ethereum is designed to be a general purpose programmable blockchain that runs a virtual machine capable of executing code of arbitrary and unbounded complexity. Where Bitcoin’s Script language is constrained to simple TRUE/FALSE evaluation of spending conditions, Ethereum’s language is Turing Complete, meaning that it is equivalent to a general purpose computer that can run any computation a theoretical Turing Machine can run.

The birth of Ethereum

All great innovations solve real problems, Ethereum is no exception. Ethereum was conceived at a time when people recognized the power of the Bitcoin model, and were trying to move beyond application of cryptocurrency applications, into other projects. But developers faced a conundrum. They either needed to build on top of Bitcoin or start a new blockchain. Building upon Bitcoin meant living within the intentional constraints of the network and trying to find workarounds. The limited types and sizes of data storage seemed to limit the types of applications that could run on top, as second layer solutions. Programmers needed to build systems that utilized only the limited set of variables, transaction types, and data. For projects that needed more freedom, more flexibility, starting a new blockchain was the only option. But starting a new blockchain meant bootstrapping all the infrastructure elements, testing, etc.

Towards the end of 2013, Vitalik Buterin, a young programmer and Bitcoin enthusiast, started thinking about further extending the capabilities of Bitcoin and Mastercoin (an overlay protocol that extended Bitcoin to offer rudimentary smart contracts). In October of 2013, Vitalik proposed a more generalized approach to the Mastercoin team, one that allowed flexible and scriptable (but not Turing complete) contracts to replace the specialized contract language of Mastercoin. While the Mastercoin team was impressed, this proposal was too radical a change to fit into their development roadmap.

In December 2013, Vitalik started sharing a “white paper”, which outlined the idea behind Ethereum: a Turing complete programmable and general purpose blockchain. A few dozen people saw this early draft and offered feedback to Vitalik, helping him gradually evolve the proposal.

Both of the authors of this book received a copy of the white paper and commented on this early draft. Andreas M. Antonopoulos was intrigued by the idea and asked Vitalik many questions about the use of a separate blockchain to enforce consensus rules on smart contract execution and the implications of a Turing complete language. Andreas continued to follow Ethereum’s progress with great interest but was in the early stages of writing his book “Mastering Bitcoin” and did not participate directly in Ethereum until much later. Dr. Gavin Wood, however, was one of the first people to reach out to Vitalik and offer to help with his C++ programming skills. Gavin became Ethereum’s co-founder, co-designer and CTO.

As Vitalik recounts in his “Ethereum Prehistory” post:

This was the time when the Ethereum protocol was entirely my own creation. From here on, however, new participants started to join the fold. By far the most prominent on the protocol side was Gavin Wood.

…

Gavin can also be largely credited for the subtle change in vision from viewing Ethereum as a platform for building programmable money, with blockchain-based contracts that can hold digital assets and transfer them according to pre-set rules, to a general-purpose computing platform. This started with subtle changes in emphasis and terminology, and later this influence became stronger with the increasing emphasis on the “Web 3” ensemble, which saw Ethereum as being one piece of a suite of decentralized technologies, the other two being Whisper and Swarm.

Starting in December 2013, Vitalik and Gavin refined and evolved the idea, together building the protocol layer that became Ethereum.

Ethereum’s founders were thinking about a blockchain that didn’t aim for a specific purpose, but instead could support a broad variety of applications by being programmed. The idea was that by using a general purpose blockchain like Ethereum, a developer could program their particular application without having to bootstrap the underlying mechanisms of peer-to-peer networks, blockchains, consensus algorithms, etc. The Ethereum platform was designed to abstract these details and provide a deterministic and secure programming environment for decentralized blockchain applications.

Much like Satoshi, Vitalik and Gavin didn’t just invent a new technology, they combined new inventions with existing technologies in a novel way and delivered the prototype code to prove their ideas to the world.

The founders worked for years, building and refining the vision. And on July 30th 2015, the first Ethereum block was mined. The world’s computer started serving the world…

Vitalik Buterin’s article “A Prehistory of Ethereum” was published in September 2017 and provides a fascinating first-person view of Ethereum’s earliest moments.

You can read it at http://vitalik.ca/general/2017/09/14/prehistory.html

Ethereum’s four stages of development

The birth of Ethereum was the launch of the first stage, named “Frontier”. Ethereum’s development is planned over four distinct stages, with major changes occurring in each new stage. Each stage may include sub-releases, known as “hard forks” that change functionality in a way that is not backwards compatible.

The four main development stages are codenamed Frontier, Homestead, Metropolis and Serenity. The intermediate hard forks are codenamed “Ice Age”, “DAO”, “Tangerine Whistle”, “Spurious Dragon”, “Byzantium”, and “Constantinople”. They are listed below, by the block number in which the hard fork occurred:

Past transitions

Block #0:: “Frontier” - The initial “test” stage of Ethereum, lasted from July 30th 2015 to March 2016.

Block #200,000:: “Ice Age” - A hard fork to introduce an exponential difficulty increase, motivating a transition to Proof-of-Stake.

Block #1,150,000:: “Homestead” - The second stage of Ethereum, launched in March 2016.

Block #1,192,000:: “DAO” - The hard fork that reversed the hacked DAO contract and caused Ethereum and Ethereum Classic to split into two competing systems.

Block #2,463,000:: “Tangerine Whistle” - A hard fork to change the gas calculation for certain IO-heavy operations and to clear the accumulated state from a denial of service attack, which exploited the low gas cost of those operations.

Block #2,675,000:: “Spurious Dragon” - A hard fork to address more denial of service attack vectors, and another state clearing. Also, a replay attack protection mechanism.

Current state

We are currently in the Metropolis stage, which was planned as two sub-release hard forks (see «hard_fork») codenamed Byzantium and Constantinople. Byzantium went into effect in October 2017 and Constantinople is anticipated by mid-2018.

Block #4,370,000:: “Metropolis Byzantium” - Metropolis is the third stage of Ethereum, current at the time of writing this book, launched in October 2017. Byzantium is the first of two hard forks for Metropolis.

Future plans

After Metropolis’ Byzantium hard fork, there is one more hard fork planned for Metropolis. Metropolis is followed by the final stage of Ethereum’s deployment, codenamed Serenity.

Constantinople:: - The second part of the Metropolis stage, planned for mid-2018. Expected to include a switch to hybrid Proof-of-Work/Proof-of-Stake consensus algorithm, among other changes.

Serenity:: The fourth and final stage of Ethereum. Serenity does not yet have a planned release date.

Ethereum: A general purpose blockchain

The “original” blockchain, bitcoin’s blockchain, tracks the state of units of bitcoin and their ownership. You can think of bitcoin as a distributed consensus state machine, where transactions cause a global state transition, altering the ownership of coins. The state transitions are constrained by the rules of consensus, allowing all participants to (eventually) converge on a common (consensus) state of the system, after several blocks are mined.

Ethereum is also a distributed state machine. But instead of tracking only the state of currency ownership, Ethereum tracks the state transitions of a general-purpose data store. By general purpose we mean any data that can be expressed as a key-value tuple. A key-value data store simply stores any arbitrary value, referenced by some key. For example storing the value “Mastering Ethereum”, referenced by the key “Book Title”. In some ways, this serves the same purpose as the data storage model of Random Access Memory (RAM) used by a general purpose computer. Ethereum has memory that stores both code and data and it uses the blockchain to track how this memory changes over time. Like a general-purpose stored-program computer, Ethereum can load code into its state machine and run that code, storing the resulting state changes in the blockchain. Two of the critical differences from a general purpose computer are that Ethereum state changes are governed by the rules of consensus and the state is distributed globally on a shared ledger. Ethereum answers the question: “What if we could track any arbitrary state and program the state machine, to create a world-wide computer operating under consensus?”.

Ethereum’s components

In Ethereum, the components of a blockchain system described in «blockchain_components» are, more specifically:

P2P Network:: Ethereum runs on the Ethereum Main Network, which is addressable on TCP port 30303, and runs a protocol called ÐΞVp2p.

Consensus rules:: Ethereum’s consensus rules, are defined in the reference specification, the [yellowpaper].

Transactions:: Ethereum transactions (see «transactions») are network messages, that include (among other things) a sender, recipient, value and data payload.

State Machine:: Ethereum state transitions are processed by the Ethereum Virtual Machine (EVM), a stack-based virtual machine that executes bytecode (machine-language instructions). EVM programs called “smart contracts” are written in high-level languages (e.g. Solidity) and compiled to bytecode for execution on the EVM.

Blockchain:: Ethereum’s blockchain is stored locally on each node as a database (usually Google’s LevelDB), which contains the transactions and system state in a serialized hashed data structure called a Merkle Patricia Tree.

Consensus Algorithm:: Ethereum currently uses a Proof-of-Work algorithm called Ethash, but there are plans to transition to a Proof-of-Stake system codenamed Casper in the near future.

Clients:: Ethereum has several interoperable implementations of the client software, the most prominent of which are Go-Ethereum (Geth) and Parity.

Further references

The Ethereum Yellow Paper: https://ethereum.github.io/yellowpaper/paper.pdf

The “Beige Paper”: a rewrite of the “Yellow Paper” for a broader audience in less formal language: https://github.com/chronaeon/beigepaper

ÐΞVp2p network protocol: https://github.com/ethereum/wiki/wiki/%C3%90%CE%9EVp2p-Wire-Protocol

Ethereum Virtual Machine - a list of “Awesome” resources: https://github.com/ethereum/wiki/wiki/Ethereum-Virtual-Machine-(EVM)-Awesome-List

LevelDB Database (used most often to store the local copy of the blockchain): http://leveldb.org

Merkle Patricia Trees: https://github.com/ethereum/wiki/wiki/Patricia-Tree

Ethash Proof-of-Work Consensus Algorithm: https://github.com/ethereum/wiki/wiki/Ethash

Casper Proof-of-Stake v1 Implementation Guide: https://github.com/ethereum/research/wiki/Casper-Version-1-Implementation-Guide

Go-Ethereum (Geth) Client: https://geth.ethereum.org/

Parity Ethereum Client: https://parity.io/

Ethereum and Turing Completeness

As soon as you start reading about Ethereum, you will immediately hear the term “Turing Complete”. Ethereum, they say, unlike bitcoin, is “Turing Complete”. What exactly does that mean?

The term “Turing Complete” is named after English mathematician Alan Turing who is considered the father of computer science. In 1936 he created a mathematical model of a computer consisting of a state machine that manipulates symbols, by reading and writing them on sequential memory (resembling an infinite-length magnetic tape). With this construct, Alan Turing went on to provide a mathematical foundation to answer (in the negative) questions about universal computability, meaning whether all problems are solvable. He proved that there are classes of problems that are uncomputable. Specifically, he proved that the Halting Problem (trying to evaluate whether a program will eventually stop running) is not solvable.

Alan Turing further defined a system to be Turing Complete, if it can be used to simulate any Turing Machine. Such a system is called a Universal Turing Machine (UTM).

Ethereum’s ability to execute a stored program, in a state machine called the Ethereum Virtual Machine, while reading and writing data to memory makes it a Turing Complete system and therefore a Universal Turing Machine. Ethereum can compute any algorithm that can be computed by any Turing Machine, given the limitations of finite memory.

Ethereum’s groundbreaking innovation is to combine the general-purpose computing architecture of a stored-program computer with a decentralized blockchain, thereby creating a distributed single-state (singleton) world computer. Ethereum programs run “everywhere”, yet produce a common (consensus) state that is secured by the rules of consensus.

Turing Completeness as a “feature”

Hearing that Ethereum is Turing Complete, you might arrive at the conclusion that this is a feature that is somehow lacking in a system that is Turing Incomplete. Rather, it is the opposite. It takes a very focused effort to constrain a system so that it is not Turing Complete. Turing completeness arises in even the simplest state machines. In fact the simplest Turing Complete state machine known (Rogozhin, 1996) has 4 states and uses 6 symbols, with a state definition that is only 22 instructions long.

Not only is Turing completeness achievable in the simplest of systems, but systems that are designed to be constrained so that they are Turing Incomplete, are often found to be “Accidentally Turing Complete”. A constrained system that is Turing Incomplete is harder to design and must be carefully maintained so that it remains Turing Incomplete.

A fun reference of systems that are “Accidentally Turing Complete” can be found here: http://beza1e1.tuxen.de/articles/accidentally_turing_complete.html

The fact that Ethereum is Turing Complete means that any program of any complexity can be computed in Ethereum. But that flexibility brings some thorny security and resource management problems.

Implications of Turing Completeness

Turing proved that you cannot predict whether a program will terminate, by simulating it on a computer. In simple terms, we cannot predict the path of a program without running it. Turing Complete systems can run in “infinite loops”, a term used (in oversimplification) to describe a program that does not terminate. It is trivial to create a program that runs a loop that never ends. But unintended never-ending loops can arise without warning, due to complex interactions between the starting conditions and the code. In Ethereum, this poses a challenge: every participating node (client), must validate every transaction, running any smart contracts it calls. But as Turing proved, Ethereum can’t predict if a smart contract will terminate, or how long it will run, without actually running it (possibly running forever). Whether by accident, or on purpose, a smart contract can be created such that it runs forever when a node attempts to validate it, effectively a denial of service attack. Of course, between a program that takes a millisecond to validate and one that runs forever there is an infinite range of nasty, resource hogging, memory-bloating, CPU overheating programs that simply waste resources. In a world computer, a program that abuses resources gets to abuse the world’s resources. How does Ethereum constrain the resources used by a smart contract if it cannot predict resource use in advance?

To answer this challenge, Ethereum introduces a metering mechanism called gas. As the EVM executes a smart contract, it carefully accounts for every instruction (computation, data access, etc.). Each instruction has a pre-determined cost in units of gas. When a transaction triggers the execution of a smart contract, it must include an amount of gas that sets the upper limit of computation that can be consumed running the smart contract. The EVM will terminate execution if the amount of gas consumed by computation exceeds the gas available in the transaction. Gas is the mechanism Ethereum uses to allow Turing Complete computation while limiting the resources that any program can consume.

In 2015 an attacker exploited an EVM instruction that cost far less gas than it should have. this allowed the attacker to create transactions that use a lot of memory and take several minutes to validate. To fix this attack, Ethereum had to change its gas accounting formula for certain instructions in a backwards incompatible (hard fork) change. Even with this change, however, Ethereum clients have to skip validating these transactions or waste weeks trying to validate them.

From general purpose blockchains to Decentralized Applications (DApps)

Ethereum started as a way to make a general purpose blockchain that could be programmed for a variety of uses. But very quickly, Ethereum’s vision expanded to become a platform for programming Decentralized Applications (DApps). DApps represent a broader perspective than “smart contracts”. A DApp is, at the very least, a smart contract and a web user-interface. More broadly, a DApp is a web application that is built on top of open, decentralized, peer-to-peer infrastructure services.

A DApp is composed of at least:

In addition, many DApps include other decentralized components, such as:

::: [TIP] You may see DApps spelled as ÐApps. The Ð character is the Latin character called “ETH”, alluding to Ethereum. To display this character, use decimal entity +#208+ in HTML, and Unicode characters +0xCE+ (UTF-8), or +0x00D0+ (UTF-16). :::

Evolving the World Wide Web

In 2004, the term “Web 2.0” came to prominence, describing an evolution of the web towards user-generated content, responsive interfaces and interactivity. Web 2.0 is not a technical specification, but rather a term describing the new focus of web applications.

The concept of DApps is meant to take the World Wide Web to its next natural evolution, introducing decentralization with peer-to-peer protocols into every aspect of a web application. The term used to describe this evolution is Web3, meaning the third “version” of the web. First proposed by Gavin Wood, web3 represents a new vision and focus for web applications: from centrally owned and managed applications, to applications built on decentralized protocols.

In later chapters we’ll explore the Ethereum +web3js+ JavaScript library which bridges JavaScript applications that run in your browser with the Ethereum blockchain. The +web3.js+ library also includes an interface to a P2P storage network called Swarm and a P2P messaging service called Whisper. With these three components included in a JavaScript library running in your web browser, developers have a full application development suite that allows them to build web3 DApps:

web3: A suite of decentralized application components for the next evolution of the web:

![i/web3suite.png]

Ethereum’s development culture

So far we’ve talked about how Ethereum’s goals and technology differ from other blockchains that preceded it, like bitcoin. Ethereum also has a very different development culture.

In bitcoin, development is guided by conservative principles: all changes are carefully studied to ensure that none of the existing systems are disrupted. For the most part, changes are only implemented if they are backwards compatible. Existing clients are allowed to “opt-in”, but will continue to operate if they decide not to upgrade.

In Ethereum, by comparison, the development culture is focused on speed and innovation. The mantra is “move fast and break things”. If a change is needed, it is implemented, even if that means invalidating prior assumptions, breaking compatibility, or forcing clients to update. Ethereum’s development culture is characterized by rapid innovation, rapid evolution and a willingness to engage in experimentation.

What this means to you as a developer, is that you must remain flexible and be prepared to rebuild your infrastructure as some of the underlying assumptions change. Do not assume anything will be static or permanent. One of the big challenges facing developers in Ethereum is the inherent contradiction between deploying code to an immutable ledger and a development platform that is still rapidly evolving. You can’t simply “upgrade” your smart contracts. You must be prepared to deploy new ones, migrate users, apps and funds, and start over.

Ironically, this also means that the goal of building systems with more autonomy and less centralized control cannot be realized. Autonomy and decentralization requires a bit more stability in the platform than you’re likely to get in Ethereum, in the next few years. In order to “evolve” the platform, you have to be ready to scrap and restart your smart contracts, which means you have to retain a certain degree of control over them.

But, on the positive side, Ethereum is moving forward very fast. There’s very little opportunity for “bike-shedding” - an expression that means arguing over minor details such as how to build the bicycle shed in the back of the building. If you start bike-shedding, you might suddenly discover the rest of the development team changed the plan, and ditched bicycles in favor of autonomous hovercrafts. There are very few sacred principles, final standards, or fixed interfaces in Ethereum.

Eventually, Ethereum core protocol development will slow down and its interfaces will become fixed. But in the meantime, innovation is the driving principle. You’d better keep up, because no one will slow down for you.

Why learn Ethereum?

Blockchains have a very steep learning curve, as they combine multiple disciplines into one domain: programming, information security, cryptography, economics, distributed systems, peer-to-peer networks etc. Ethereum makes this learning curve a lot less steep, so you can get started very quickly. But just below the surface of a deceptively simple environment, lies a lot more. As you learn and start looking deeper, there’s always another layer of complexity and wonder.

Ethereum is a great platform for learning about blockchains and it’s building a massive community of developers, faster than any other blockchain platform. More than any other blockchain, Ethereum is a developer’s blockchain, built by developers, for developers. A developer familiar with JavaScript applications can drop into Ethereum and start producing working code very quickly. For the first years of Ethereum, it was common to see t-shirts announcing that you can create a token in just five lines of code. Of course, this is a double-edged sword. It’s easy to write code, but it’s very hard to write good code and secure code.

What this book will teach you?

This book dives into Ethereum and examines every component. You will start with a simple transaction, dissect how it works, build a simple contract, make it better and follow its journey through the Ethereum system.

You will learn how Ethereum works, but also why it is designed the way it is. You will be able to understand how each of the pieces work, but also how they fit together and why.

Tokens

Contents

What are tokens?

The word token derives from the Old English “tacen” meaning a sign or symbol. Commonly used to mean privately-issued coin-like items that have insignificant value, as used in transportation tokens, laundry tokens, arcade tokens.

Nowadays, tokens based on blockchains are redefining the word to mean blockchain-based abstractions that can be owned and that represent assets, currency, or access rights.

The association between the word “token” and insignificant value has a lot to do with the limited use of physical tokens. Often restricted to specific businesses, organizations or locations, physical tokens are not easily exchangeable and cannot be used for more than one function. With blockchain tokens, these restrictions are erased. Many of these tokens serve multiple purposes globally and can be traded for each other or for other currencies in global liquid markets. With those restrictions gone, the “insignificant value” expectation is also a thing of the past.

In this section, we look at various uses for tokens and how they are created. We also discuss attributes of tokens such as fungibility and intrinsicality. Finally, we examine the standards and technologies that they are based on and experiment by building our own tokens.

How are tokens used?

The most obvious use of tokens is as digital private currencies. However, this is only one possible use. Tokens can be programmed to serve many different functions, often overlapping. For example, a token can simultaneously convey a voting right, an access right, and ownership of a resource. Currency is just the first “app”.

Currency:: A token can serve as a form of currency, with a value determined through private trade. For example, ether or bitcoin.

Resource:: A token can represent a resource earned or produced in a sharing-economy or resource-sharing environment. For example, a storage or CPU token representing resources that can be shared over a network.

Asset:: A token can represent ownership of an intrinsic or extrinsic, tangible or intangible asset. For example, gold, real-estate, a car, oil, energy etc.

Access:: A token can represent access rights and even convey access to a digital or physical property, such as a discussion forum, an exclusive website, a hotel room, a rental car.

Equity:: A token can represent shareholder equity in a digital organization (e.g. a DAO) or legal fiction (e.g. a corporation)

Voting:: A token can represent voting rights in a digital or legal system.

Collectible:: A token can represent a digital (e.g. CryptoPunks) or physical collectible (e.g. a painting)

Identity:: A token can represent a digital (e.g. avatar) or legal identity (e.g. national ID).

Attestation:: A token can represent a certification or attestation of fact by some authority or by a decentralized reputation system (e.g. marriage record, birth certificate, college degree).

Utility:: A token can be used to access or pay for a service.

Often, a single token encompasses several of these functions. Sometimes it is hard to discern between them, as the physical equivalents have always been inextricably linked. For example, in the physical world, a driver’s license (attestation) is also an identity document (identity) and the two cannot be separated. In the digital realm, previously commingled functions can be separated and developed independently (e.g. an anonymous attestation).

Tokens and fungibility

From Wikipedia:

In economics, fungibility is the property of a good or a commodity whose individual units are essentially interchangeable.

Tokens are fungible when we can substitute any single unit of the token for another without any difference in its value or function. For example, ether is a fungible token, as any unit of ether has the same value and use as any other unit of ether.

Strictly speaking, if a token’s historical provenance can be tracked, then it is not entirely fungible. The ability to track provenance can lead to blacklisting and whitelisting, reducing or eliminating fungibility. We will examine this further in «privacy».

Non-fungible tokens are tokens that each represent a unique tangible or intangible item and therefore are not interchangeable. For example, a token that represents ownership of a specific Van Gogh painting is not equivalent to another token that represents a Picasso. Similarly, a token representing a specific digital collectible such as a specific CryptoKitty (see «cryptoKitties») is not interchangeable with any other CryptoKitty. Each non-fungible token is associated with an unique identifier, such as a serial number.

We will see examples of both fungible and non-fungible tokens later in this section.

Counterparty risk

Counterparty risk is the risk that the other party in a transaction will fail to meet their obligations. Some types of transactions create additional counterparty risks because of the addition of more than two parties in the transaction. For example, if you hold a certificate of deposit for a precious metal and you sell that to someone, there are at least 3 parties in that transaction: the seller, the buyer and the custodian of the precious metal. Someone holds the physical asset and by necessity, they become a party and add counterparty risk to any transaction involving that asset. When an asset is traded indirectly through the exchange of a token of ownership, there is additional counterparty risk from the custodian of the asset. Do they have the asset? Will they recognize (or allow) the transfer of ownership based on the transfer of a token (such as a certificate, deed, title or digital token)? In the world of digital tokens, it is important to understand who holds the asset that is represented by the token and what rules apply to that underlying asset.

Tokens and intrinsicality

The word “intrinsic” derives from the Latin “intra”, meaning “from within”.

Some tokens represent digital items that are intrinsic to the blockchain. Those digital assets are governed by consensus rules, just like the tokens themselves. This has an important implication: tokens that represent intrinsic assets do not carry additional counterparty risk. If you hold the keys to 1 ether, there is no other party holding that ether for you. The blockchain consensus rules apply and your ownership (control) of the private keys is equivalent to ownership of the asset, without any intermediary.

Conversely, many tokens are used to represent extrinsic things, like real-estate, corporate voting shares, trademarks, gold bars. The ownership of these items, which are not “within” the blockchain, is governed by law, custom and policy that are separate from the consensus rules that govern the token. In other words, token issuers and owners may still depend on real world non-smart contracts. As a result, these extrinsic assets carry additional counterparty risk because they are held by custodians, recorded in external registries, or controlled by laws and policies outside the blockchain environment.

One of the most important ramifications of blockchain-based tokens is the ability to convert extrinsic assets into intrinsic assets and thereby remove counterparty risk. A good example is moving from equity in a corporation (extrinsic) to an equity or voting token in a decentralized autonomous organization or similar (intrinsic) organization.

Using tokens: utility or equity

Almost all projects in Ethereum today are launching with some kind of token. But do all these projects really need a token? Are there any disadvantages to using a token, or will we see the slogan “tokenize all the things” come to fruition?

First, let’s start by clarifying the role of a token in a new project. The majority of projects are using tokens in one of two ways: either as “utility tokens” or as “equity tokens”. Very often, those two roles are conflated and difficult to distinguish.

Utility tokens are those where the use of the token is required to pay for a service, application or resource. Examples of utility tokens include tokens that represent resources such as shared storage, access to services such as social media networks, or ether itself as gas for the Ethereum platform. By comparison, equity tokens are those that represent shares in a startup.

Equity tokens can be as limited as non-voting shares for distribution of dividends and profits, or as expansive as voting shares in a decentralized autonomous organization, where management of the platform is through majority votes by the token holders.

It’s not a duck

Just because a token is used to fundraise for a startup, doesn’t mean it has to be used as payment for the service, and vice-versa. Many startups, however, face a difficult problem: tokens are a great fundraising mechanism, but offering securities (equity) to the public is a regulated activity in most jurisdictions. By disguising equity tokens as utility tokens, many startups hope to get around these regulatory restrictions and raise money from a public offering while presenting it as a pre-sale of a utility token. Whether these thinly disguised equity offerings will be able to skirt the regulators remains to be seen.

As the popular saying goes: “If it walks like a duck and quacks like a duck - it’s a duck”. Regulators are not likely to be distracted by these semantic contortions, quite the opposite, they are more likely to see such legal sophistry as an attempt to deceive the public.

Utility tokens: who needs them?

The real problem is that utility tokens introduce significant risks and adoption barriers for startups. Perhaps in a distant future “tokenize all the things” becomes a reality. But, at present, the number of people who have access to, an understanding of, and desire to use a token is a subset of the already small cryptocurrency market.

For a startup, each innovation represents a risk and a market filter. Innovation is taking the road least traveled, walking away from the path of tradition. It is already a lonely walk. If a startup is trying to innovate in a new area of technology, such as storage sharing over P2P networks, that is a lonely enough path. Adding a utility token to that innovation and requiring users to adopt tokens in order to use the service compounds the risk and increases the barriers to adoption. It’s walking off the already lonely trail of P2P storage innovation and into the wilderness.

Think of each innovation as a filter. It limits adoption to the subset of the market that can become early adopters of this innovation. Adding a second filter compounds that effect, further limiting the addressable market. You are asking your early adopters to adopt not one but two completely new technologies: the novel application/platform/service you built, and the token economy.

For a startup, each innovation introduces risks that increase the chance of failure of the startup. If you take your already risky startup idea and add a utility token, you are adding all the risks of the underlying platform (Ethereum), broader economy (exchanges, liquidity), regulatory environment (equity/commodity regulators) and technology (smart contracts, token standards). That’s a lot of risk for a startup.

Advocates of “tokenize all the things” will likely counter that by adopting tokens, they are also inheriting the market enthusiasm, early adopters, technology, innovation and liquidity of the entire token economy. That is true too. The question is whether the benefits and enthusiasm outweigh the risks and uncertainties.

Nevertheless, some of the most innovative business ideas are indeed taking place in the crypto realm. If regulators are not quick enough to adapt laws and support new business models, talents and entrepreneurs will seek to operate in other jurisdictions which are more crypto-friendly. This is actually happening right now.

Finally, at the beginning of this chapter, when introducing tokens we discuss the colloquial meaning of token as “something of insignificant value”. The underlying reason for the insignificant value of most tokens is because they can only be used in a very narrow context: one bus company, one laundromat, one arcade, one hotel, one company store. Limited liquidity, limited applicability, and high conversion costs reduce the value of tokens all the way down until it is only a “token” value. So when you add a utility token to your platform, but the token can only be used on your own one platform with a small market, you are re-creating the conditions that made physical tokens worthless. If in order to use your platform, a user has to convert something into your utility token, use it and then convert the remainder back into something more generally useful, you’ve created company scrip. The switching costs of a digital token are orders of magnitude lower than a physical token without a market. But the switching costs are not zero. Utility tokens that work across an entire industry sector will be very interesting and probably quite valuable. But if you set up your startup to have to bootstrap an entire industry standard in order to succeed, you may have already failed.

One of the benefits of deploying services on general-purpose platforms like Ethereum is exactly being able to connect smart contracts, increasing the potential for liquidity and utility of tokens.

Make this decision for the right reasons. Adopt a token because your application cannot work without a token (e.g. Ethereum). Adopt it because the token solves a fundamental market barrier or access problem. Don’t introduce a utility token because it is the only way you can raise money fast and you need to pretend it’s not a public securities offering.

Token Standards

Blockchain tokens have existed before Ethereum. In some ways, the first blockchain currency, bitcoin, is a token itself. Many token platforms were also developed on bitcoin and other cryptocurrencies before Ethereum. However, the introduction of the first token standard on Ethereum led to an explosion of tokens.

Vitalik Buterin suggested tokens as one of the most obvious and useful applications of a generalized programmable blockchain such as Ethereum. In fact, in the first year of Ethereum, it was common to see Vitalik and others wearing t-shirts emblazoned with the Ethereum logo and a smart contract sample on the back. There were several variations of this t-shirt, but the most common showed an implementation of a token.

ERC20 Token Standard

The first standard was introduced in November 2015 by Fabian Vogelsteller, as an Ethereum Request for Comments (ERC). It was automatically assigned GitHub issue number 20, giving rise to the name “ERC20 token”. The vast majority of tokens are currently based on ERC20. The ERC20 request for comments, eventually became Ethereum Improvement Proposal EIP20, but is mostly still referred to by the original name ERC20. You can read the standard here:

https://github.com/ethereum/EIPs/blob/master/EIPS/eip-20.md

ERC20 is a standard for fungible tokens meaning that different units of an ERC20 token are interchangeable and have no unique properties.

The ERC20 standard defines a common interface for contracts implementing a token, such that any compatible token can be accessed and used in the same way. The interface consists of a number of functions that must be present in every implementation of the standard, as well as some optional functions and attributes that may be added by developers.

ERC20 required functions & events

totalSupply:: Returns the total units of this token that currently exist. ERC20 tokens can have fixed or variable supply.

balanceOf:: Given an address, returns the token balance of that address.

transfer:: Given an address and amount, transfers that amount of tokens to that address, from the balance of the address that executed the transfer.

transferFrom:: Given a sender, recipient and amount, transfers tokens from one account to another. Used in combination with +approve+ below.

approve:: Given a recipient address and amount, authorizes that address to execute several transfers up to that amount, from the account that issued the approval.

allowance:: Given an owner address and a spender address, returns the remaining amount that the spender is approved to withdraw from the owner.

Transfer event:: Event triggered upon successful transfer (call to +transfer+ or +transferFrom+) (even for zero value transfers).

Approval event:: Event logged upon successful call to +approve+.

ERC20 optional functions

name:: Returns a human readable name (e.g. “US Dollars”) of the token.

symbol:: Returns a human readable symbol (e.g. “USD”) for the token.

decimals:: Returns the number of decimals used to divide token amounts. For example, if decimals is 2, then the token amount is divided by 100 to get its user representation.

The ERC20 interface defined in Solidity

Here’s what an ERC20 interface specification looks like in Solidity:

contract ERC20 {
   function totalSupply() constant returns (uint theTotalSupply);
   function balanceOf(address _owner) constant returns (uint balance);
   function transfer(address _to, uint _value) returns (bool success);
   function transferFrom(address _from, address _to, uint _value) returns (bool success);
   function approve(address _spender, uint _value) returns (bool success);
   function allowance(address _owner, address _spender) constant returns (uint remaining);
   event Transfer(address indexed _from, address indexed _to, uint _value);
   event Approval(address indexed _owner, address indexed _spender, uint _value);
}

ERC20 data structures

If you examine any ERC20 implementation, it will contain two data structures, one to track balances and one to track allowances. In Solidity, they are implemented with a data mapping.

The first data mapping implements an internal table of token balances, by owner. This allows the token contract to keep track of who owns the tokens. Each transfer is a deduction from one balance and an addition to another balance.

Balances: a mapping from address (owner) to amount (balance)

mapping(address => uint256) balances;

The second data structure is a data mapping of allowances. As we will see in «transfer_workflows», with ERC20 tokens an owner of a token can delegate authority to a spender, allowing them to spend a specific amount (allowance) from the owner’s balance. The ERC20 contract keeps track of the allowances with a two-dimensional mapping, with the primary key being the address of the token owner, mapping to a spender address and an allowance amount:

Allowances: a mapping from address (owner) to address (spender) to amount (allowance)

mapping (address => mapping (address => uint256)) public allowed;

ERC20 workflows: “transfer” and “approve & transferFrom”

The ERC20 token standard has two transfer functions. You might be wondering why?

ERC20 allows two different workflows. The first is a single-transaction, straightforward workflow using the +transfer+ function. This workflow is the one used by wallets to send tokens to other wallets. The vast majority of token transactions happen with the +transfer+ workflow.

Executing the transfer contract is very simple. If Alice wants to send 10 tokens to Bob, her wallet sends a transaction to the token contract’s address, calling the +transfer+ function with Bob’s address and “10” as the arguments. The token contract adjusts Alice’s balance (-10) and Bob’s balance (+10) and issues a +Transfer+ event.

The second workflow is a two-transaction workflow that uses +approve+, followed by +transferFrom+. This workflow allows a token owner to delegate their control to another address. It is most often used to delegate control to a contract for distribution of tokens, but it can also be used by exchanges. For example, if a company is selling tokens for an ICO, they can +approve+ a crowdsale contract address to distribute a certain amount of tokens. The crowdsale contract can then +transferFrom+ the token contract owner balance to each buyer of the token.

The two-step approve & transferFrom workflow of ERC20 tokens

i/approve_transferFrom_workflow.png

For the +approve & transferFrom+ workflow, two transactions are needed. Let’s say that Alice wants to allow the AliceICO contract to sell 50% of all the AliceCoin tokens to buyers like Bob and Charlie. First, Alice launches the AliceCoin ERC20 contract, issuing all the AliceCoin to her own address. Then, Alice launches the AliceICO contract that can sell tokens for ether. Next, Alice initiates the +approve & transferFrom+ workflow. She sends a transaction to AliceCoin, calling +approve+, with the address of AliceICO and 50% of the +totalSupply+. This will trigger the +Approval+ event. Now, the AliceICO contract can sell AliceCoin.

When AliceICO receives ether from Bob, it needs to send some AliceCoin to Bob in return. Within the AliceICO contract is an exchange rate between AliceCoin and ether. The exchange rate that Alice set when she created the AliceICO determines how many tokens Bob will receive for the amount of ether sent to AliceICO. When AliceICO calls the AliceCoin +transferFrom+ function, it sets Alice’s address as the sender, Bob’s address as the recipient, and uses the exchange rate to determine how many AliceCoin tokens will be transferred to Bob in the “value” field. The AliceCoin contract transfers the balance from Alice’s address to Bob’s address and triggers a +Transfer+ event. The AliceICO contract can call +transferFrom+ an unlimited number of times, as long as it doesn’t exceed the approval limit Alice set. The AliceICO contract can keep track of how many AliceCoin tokens it can sell by calling the +allowance+ function.

ERC20 Implementations

While it is possible to implement an ERC20-compatible token in about thirty lines of Solidity code, most implementations are more complex, to account for potential security vulnerabilities. There are two implementations mentioned in the EIP20 standard:

Consensys EIP20:: A simple and easy to read implementation of an ERC20-compatible token.

You can read the Solidity code for Consensys’ implementation here: https://github.com/ConsenSys/Tokens/blob/master/contracts/eip20/EIP20.sol

OpenZeppelin StandardToken:: This implementation is ERC20-compatible, with additional security precautions. It forms the basis of OpenZeppelin libraries implementing more complex ERC20-compatible tokens with fundraising caps, auctions, vesting schedules and other features.

You can see the Solidity code for OpenZeppelin StandardToken here: https://github.com/OpenZeppelin/zeppelin-solidity/blob/master/contracts/token/ERC20/StandardToken.sol

Launching our own ERC20 token

Let’s create and launch our own token. For this example, we will use the +truffle+ framework (see «truffle»). The example assumes you have already installed +truffle+, configured it, and are familiar with its basic operation.

We will call our token “Mastering Ethereum Token”, with symbol “MET”.

You can find this example in the book’s GitHub repository: https://github.com/ethereumbook/ethereumbook/blob/develop/code/truffle/METoken

First, let’s create and initialize a truffle project directory, the same way we did in «truffle_project_directory». Run these four commands and accept the default answers to any questions:

$ mkdir METoken
$ cd METoken
METoken $ truffle init
METoken $ npm init

You should now have the following directory structure:

METoken/
├── contracts
│   └── Migrations.sol
├── migrations
│   └── 1_initial_migration.js
├── package.json
├── test
├── truffle-config.js
└── truffle.js

Edit the +truffle.js+ or +truffle-config.js+ configuration file to setup your +truffle+ environment, or copy the one we used from:

https://github.com/ethereumbook/ethereumbook/blob/develop/code/truffle/METoken/truffle-config.js

If you use the example +truffle-config.js+, remember to create a file +.env+ in the +METoken+ folder containing your test private keys for testing and deployment on public Ethereum test networks, such as ganache or Kovan. You can export your test network private key from MetaMask.

After that your directory look like:

METoken/
├── contracts
│   └── Migrations.sol
├── migrations
│   └── 1_initial_migration.js
├── package.json
├── test
├── truffle-config.js
├── truffle.js
└── .env *new file*

:: [WARNING] Only use test keys or test mnemonics that are not used to hold funds on the main Ethereum network. Never use keys that hold real money for testing. ::

For our example, we will import the OpenZeppelin StandardContract, which implements some important security checks and is easy to extend. Let’s import that library:

$ npm install zeppelin-solidity

+ zeppelin-solidity@1.6.0
added 8 packages in 2.504s

The +zeppelin-solidity+ package will add about 250 files under the +node_modules+ directory. The OpenZeppelin library includes a lot more than the ERC20 token, but we will only use a small part of it.

Next, let’s write our token contract. Create a new file +METoken.sol+ and copy the example code from GitHub:

https://github.com/ethereumbook/ethereumbook/blob/develop/code/truffle/METoken/contracts/METoken.sol

Our contract is very simple, as it inherits all the functionality from the OpenZeppelin StandardToken library:

METoken.sol : A Solidity contract implementing an ERC20 token

include::code/METoken/contracts/METoken.sol[]

Here, we are defining the optional variables +name+, +symbol+, and +decimals+. We also define an _initial_supply variable, set to 21 million tokens, and two decimals of subdivision (2.1 billion total). In the contract’s initialization (constructor) function we set the +totalSupply+ to be equal to _initial_supply and allocate all of the _initial_supply to the balance of the account (+msg.sender+) that creates the +METoken+ contract.

We now use +truffle+ to compile the +METoken+ code:

$ truffle compile
Compiling ./contracts/METoken.sol...
Compiling ./contracts/Migrations.sol...
Compiling zeppelin-solidity/contracts/math/SafeMath.sol...
Compiling zeppelin-solidity/contracts/token/ERC20/BasicToken.sol...
Compiling zeppelin-solidity/contracts/token/ERC20/ERC20.sol...
Compiling zeppelin-solidity/contracts/token/ERC20/ERC20Basic.sol...
Compiling zeppelin-solidity/contracts/token/ERC20/StandardToken.sol...

As you can see, +truffle+ incorporated necessary dependencies from the OpenZeppelin libraries and compiled those contracts too.

Let’s set up a migration script, to deploy the +METoken+ contract. Create a new file +2_deploy_contracts.js+ in the +METoken/migrations+ folder. Copy the contents from the example on Github repository:

https://github.com/ethereumbook/ethereumbook/blob/develop/code/truffle/METoken/migrations/2_deploy_contracts.js

Here’s what it contains:

2_deploy_contracts.js : Migration to deploy METoken

include::code/METoken/migrations/2_deploy_contracts.js[]

Before we deploy on one of the Ethereum test networks, let’s start a local blockchain to test everything. Start the +ganache+ blockchain, either from the command-line with +ganache-cli+ or from the graphical user interface, as we did in «using_ganache».

Once +ganache+ is started, we can deploy our METoken contract and see if everything works as expected:

$ truffle migrate --network ganache
Using network 'ganache'.

Running migration: 1_initial_migration.js
  Deploying Migrations...
  ... 0xb2e90a056dc6ad8e654683921fc613c796a03b89df6760ec1db1084ea4a084eb
  Migrations: 0x8cdaf0cd259887258bc13a92c0a6da92698644c0
Saving successful migration to network...
  ... 0xd7bc86d31bee32fa3988f1c1eabce403a1b5d570340a3a9cdba53a472ee8c956
Saving artifacts...
Running migration: 2_deploy_contracts.js
  Deploying METoken...
  ... 0xbe9290d59678b412e60ed6aefedb17364f4ad2977cfb2076b9b8ad415c5dc9f0
  METoken: 0x345ca3e014aaf5dca488057592ee47305d9b3e10
Saving successful migration to network...
  ... 0xf36163615f41ef7ed8f4a8f192149a0bf633fe1a2398ce001bf44c43dc7bdda0
Saving artifacts...

On the +ganache+ console, we should see that our deployment has created 4 new transactions:

METoken deployment on Ganache

i/ganache_metoken.png

Interacting with METoken using the truffle console

We can interact with our contract on the +ganache+ blockchain, using the +truffle console+. This is an interactive JavaScript environment that provides access to the truffle environment and, via Web3, to the blockchain. In this case, we will connect the +truffle console+ to the +ganache+ blockchain:

$ truffle console --network ganache
truffle(ganache)>

The +truffle(ganache)>+ prompt shows that we are connected to the +ganache+ blockchain and are ready to type our commands. The +truffle console+ supports all the truffle commands, so we could +compile+ and +migrate+ from the console. We’ve already ran those commands, so let’s go directly to the contract itself. The METoken contract exists as a JavaScript object within the truffle environment. Type +METoken+ at the prompt and it will dump the entire contract definition:

truffle(ganache)> METoken
{ [Function: TruffleContract]
  _static_methods:

[...]

currentProvider:
 HttpProvider {
   host: 'http://localhost:7545',
   timeout: 0,
   user: undefined,
   password: undefined,
   headers: undefined,
   send: [Function],
   sendAsync: [Function],
   _alreadyWrapped: true },
network_id: '5777' }

The +METoken+ object also exposes several attributes, such as the address of the contract (as deployed by the +migrate+ command):

truffle(ganache)> METoken.address
'0x345ca3e014aaf5dca488057592ee47305d9b3e10'

If we want to interact with the deployed contract, we have to use an asynchronous call, in the form of a JavaScript “promise”. We use the +deployed+ function to get the contract instance and then call the +totalSupply+ function:

truffle(ganache)> METoken.deployed().then(instance => instance.totalSupply())
BigNumber { s: 1, e: 9, c: [ 2100000000 ] }
----

Next, let's use the accounts created by +ganache+ to check our METoken balance and send some METoken to another address. First, let's get the account addresses:

[[get_account_addresses]]
----
truffle(ganache)> let accounts
undefined
truffle(ganache)> web3.eth.getAccounts((err,res) => { accounts = res })
undefined
truffle(ganache)> accounts[0]
'0x627306090abab3a6e1400e9345bc60c78a8bef57'

The +accounts+ list now contains all the accounts created by +ganache+, and +account[0]+ is the account that deployed the METoken contract. It should have a balance of METoken, because our METoken constructor gives the entire token supply to the address that created it. Let’s check:

truffle(ganache)> METoken.deployed().then(instance => { instance.balanceOf(accounts[0]).then(console.log) })
undefined
BigNumber { s: 1, e: 9, c: [ 2100000000 ] }

Finally, let’s transfer 1000.00 METoken from +account[0]+ to +account[1]+, by calling the contract’s +transfer+ function:

truffle(ganache)> METoken.deployed().then(instance => { instance.transfer(accounts[1], 100000) })
undefined
truffle(ganache)> METoken.deployed().then(instance => { instance.balanceOf(accounts[0]).then(console.log) })
undefined
truffle(ganache)> BigNumber { s: 1, e: 9, c: [ 2099900000 ] }

undefined
truffle(ganache)> METoken.deployed().then(instance => { instance.balanceOf(accounts[1]).then(console.log) })
undefined
truffle(ganache)> BigNumber { s: 1, e: 5, c: [ 100000 ] }

:: [TIP] METoken has 2 decimals of precision, meaning that 1 METoken is 100 units in the contract. When we transfer 1000 METoken, we specify the value as 100,000 in the transfer function. ::

As you can see, in the console, +account[0]+ now has 20,999,000 MET, and +account[1]+ has 1000 MET.

If you switch to the +ganache+ graphical user interface, you will see the transaction that called the +transfer+ function:

METoken transfer on Ganache

i/ganache_metoken_transfer.png

Sending ERC20 tokens to contract addresses

So far we’ve setup an ERC20 token and transferred from one account to another. All the accounts we used for these demonstrations are externally-owned accounts (EOAs), meaning they are controlled by a private key, not a contract. What happens if we send MET to a contract address? Let’s find out!

First, let’s deploy another contract into our test environment. For this example we will use our first contract +Faucet.sol+. Let’s add it to the METoken project by copying it to the +contracts+ directory. Our directory should look like this:

METoken/
├── contracts
│   ├── Faucet.sol
│   ├── METoken.sol
│   └── Migrations.sol

We’ll also add a migration, to deploy +Faucet+ separately from +METoken+:

var Faucet = artifacts.require("Faucet");

module.exports = function(deployer) {
  // Deploy the Faucet contract as our only task
  deployer.deploy(Faucet);
};

Let’s compile and migrate the contracts, from the truffle console:

$ truffle console --network ganache
truffle(ganache)> compile
Compiling ./contracts/Faucet.sol...
Writing artifacts to ./build/contracts

truffle(ganache)> migrate
Using network 'ganache'.

Running migration: 1_initial_migration.js
  Deploying Migrations...
  ... 0x89f6a7bd2a596829c60a483ec99665c7af71e68c77a417fab503c394fcd7a0c9
  Migrations: 0xa1ccce36fb823810e729dce293b75f40fb6ea9c9
Saving artifacts...
Running migration: 2_deploy_contracts.js
  Replacing METoken...
  ... 0x28d0da26f48765f67e133e99dd275fac6a25fdfec6594060fd1a0e09a99b44ba
  METoken: 0x7d6bf9d5914d37bcba9d46df7107e71c59f3791f
Saving artifacts...
Running migration: 3_deploy_faucet.js
  Deploying Faucet...
  ... 0x6fbf283bcc97d7c52d92fd91f6ac02d565f5fded483a6a0f824f66edc6fa90c3
  Faucet: 0xb18a42e9468f7f1342fa3c329ec339f254bc7524
Saving artifacts...

Great. Now let’s send some MET to the +Faucet+ contract:

truffle(ganache)> METoken.deployed().then(instance => { instance.transfer(Faucet.address, 100000) })
truffle(ganache)> METoken.deployed().then(instance => { instance.balanceOf(Faucet.address).then(console.log)})
truffle(ganache)> BigNumber { s: 1, e: 5, c: [ 100000 ] }

Alright, we have transferred 1000 MET to the +Faucet+ contract. Now, how do we withdraw from the +Faucet+?

Remember, +Faucet.sol+ is a pretty simple contract. It only has one function, +withdraw+, which is for withdrawing ether. It doesn’t have a function for withdrawing MET, or any other ERC20 token. If we use +withdraw+ it will try to send ether, but since the faucet doesn’t have a balance of ether yet, it will fail. The +METoken+ contract knows that +Faucet+ has a balance, but the only way that it can transfer that balance is if it receives a +transfer+ call from the address of the contract. Somehow we need to make the +Faucet+ contract call the +transfer+ function in +METoken+.

If you’re wondering what to do next, don’t. There is no solution to this problem. The MET sent to +Faucet+ is stuck, forever. Only the +Faucet+ contract can transfer it, and the +Faucet+ contract doesn’t have code to call the +transfer+ function of an ERC20 token contract.

Perhaps you anticipated this problem. Most likely, you didn’t. In fact, neither did hundreds of Ethereum users who accidentally transferred various tokens to contracts that didn’t have any ERC20 capability. According to some estimates, tokens worth more than $2.5 million USD have been “stuck” like this and are lost forever.

One of the ways that users of ERC20 tokens can inadvertently lose their tokens in a transfer, is when they attempt to transfer to an exchange or other service. They copy an Ethereum address from the website of an exchange, thinking they can simply send tokens to it. However, many exchanges publish receiving addresses that are actually contracts! These contracts serve a number of different functions, most often sweeping all funds sent to them to a “cold storage” or other centralized wallet. Despite the many warnings saying “do not send tokens to this address”, lots of tokens are lost this way.

Demonstrating the approve & transferFrom workflow

Our +Faucet+ contract couldn’t handle ERC20 tokens. Sending tokens to it, using the +transfer+ function results in the loss of those tokens. Let’s rewrite the contract and make it handle ERC20 tokens. Specifically we will turn it into a faucet that gives out MET to anyone who asks.

For this example, we make a copy of the truffle project directory, call it +METoken_METFaucet+, initialize truffle, npm, install OpenZeppelin dependencies and copy the +METoken.sol+ contract. See our first example «METoken_example» for the detailed instructions.

Now, let’s create a new faucet contract, call it +METFaucet.sol+. It will look like this:

METFaucet.sol: a faucet for METoken

include::code/METoken_METFaucet/contracts/METFaucet.sol

We’ve made quite a few changes to the basic faucet example. Since METFaucet will use the +transferFrom+ function in +METoken+, it will need two additional variables. One will hold the address of the deployed +METoken+ contract. The other will hold the address of the owner of MET who will provide approval the faucet withdrawals. The +METFaucet+ will call +METoken.transferFrom+ and instruct it to move MET from the owner to the address where the faucet withdrawal request came from.

We declare these two variables here:

StandardToken public METoken;
address public METOwner;

Since our faucet needs to be initialized with the correct addresses for +METoken+ and +METOwner+ we need to declare a custom constructor:

// METFaucet constructor, provide the address of METoken contract and
// the owner address we will be approved to transferFrom
function METFaucet(address _METoken, address _METOwner) public {

	// Initialize the METoken from the address provided
	METoken = StandardToken(_METoken);
	METOwner = _METOwner;
}

The next change is to the +withdraw+ function. Instead of calling +transfer+, +METFaucet+ uses the +transferFrom+ function in +METoken+ and asks +METoken+ to transfer MET to the faucet recipient:

// Use the transferFrom function of METoken
METoken.transferFrom(METOwner, msg.sender, withdraw_amount);

Finally, since our faucet no longer sends ether, we should probably prevent anyone from sending ether to +METFaucet+, as we wouldn’t want it to get stuck. We change the fallback payable function to reject incoming ether, using the +revert+ function to revert any incoming payments:

// REJECT any incoming ether
function () public payable { revert(); }

Now that our +METFaucet.sol+ code is ready, we need to modify the migration script to deploy it. This migration script will be a bit more complex, as +METFaucet+ depends on the address of +METoken+. We will use a JavaScript promise to deploy the two contracts in sequence. Create +2_deply_contracts.js+ as follows:

var METoken = artifacts.require("METoken");
var METFaucet = artifacts.require("METFaucet");
var owner = web3.eth.accounts[0];

module.exports = function(deployer) {

	// Deploy the METoken contract first
	deployer.deploy(METoken, {from: owner}).then(function() {
		// then deploy METFaucet and pass the address of METoken
		// and the address of the owner of all the MET who will approve METFaucet
		return deployer.deploy(METFaucet, METoken.address, owner);
  	});
}

Now, we can test everything in the truffle console. First, we use +migrate+ to deploy the contracts. When +METoken+ is deployed it will allocate all the MET to the account that created it, +web3.eth.accounts[0]+. Then, we call the +approve+ function in METoken to approve +METFaucet+ to send up to 1000 MET on behalf of +web3.eth.accounts[0]+. Finally, to test our faucet, we call +METFaucet.withdraw+ from +web3.eth.accounts[1]+ and try to withdraw 10 MET. Here are the console commands:

$ truffle console --network ganache
truffle(ganache)> migrate
Using network 'ganache'.

Running migration: 1_initial_migration.js
  Deploying Migrations...
  ... 0x79352b43e18cc46b023a779e9a0d16b30f127bfa40266c02f9871d63c26542c7
  Migrations: 0xaa588d3737b611bafd7bd713445b314bd453a5c8
Saving artifacts...
Running migration: 2_deploy_contracts.js
  Replacing METoken...
  ... 0xc42a57f22cddf95f6f8c19d794c8af3b2491f568b38b96fef15b13b6e8bfff21
  METoken: 0xf204a4ef082f5c04bb89f7d5e6568b796096735a
  Replacing METFaucet...
  ... 0xd9615cae2fa4f1e8a377de87f86162832cf4d31098779e6e00df1ae7f1b7f864
  METFaucet: 0x75c35c980c0d37ef46df04d31a140b65503c0eed
Saving artifacts...
truffle(ganache)> METoken.deployed().then(instance => { instance.approve(METFaucet.address, 100000) })
truffle(ganache)> METoken.deployed().then(instance => { instance.balanceOf(web3.eth.accounts[1]).then(console.log) })
truffle(ganache)> BigNumber { s: 1, e: 0, c: [ 0 ] }
truffle(ganache)> METFaucet.deployed().then(instance => { instance.withdraw(1000, {from:web3.eth.accounts[1]}) } )
truffle(ganache)> METoken.deployed().then(instance => { instance.balanceOf(web3.eth.accounts[1]).then(console.log) })
truffle(ganache)> BigNumber { s: 1, e: 3, c: [ 1000 ] }

As you can see from the results, we can use the +approve+ and +transferFrom+ workflow to authorize one contract to transfer tokens defined in another token. If properly used, ERC20 tokens can be used by externally-owned addresses and other contracts.

However, the burden of managing ERC20 tokens correctly is pushed to the user interface. If a user incorrectly attempts to transfer ERC20 tokens to a contract address and that contract is not equipped to receive ERC20 tokens, the tokens will be lost.

Issues with ERC20 tokens

The adoption of the ERC20 token standard has been truly explosive. Thousands of tokens have been launched, both to experiment with new capabilities and to raise funds in various “crowdfund” auctions and Initial Coin Offerings (ICOs). However there are some potential pitfalls, as we saw with the issue of transferring tokens to contract addresses.

One of the less obvious issues with ERC20 tokens is that they expose subtle differences between tokens and ether itself. Where ether is transferred by a transaction which has a recipient address as its destination, token transfers occur within the specific token contract state and have the token contract as their destination, not the recipient’s address. The token contract tracks balances and issues events. In a token transfer, no transaction is actually sent to the recipient of the token. Instead, the recipient’s address is added to a map within the token contract itself. A transaction sending ether to an address changes the state of an address. A transaction transferring a token to an address only changes the state of the token contract, not the state of the recipient address. Even a wallet that has support for ERC20 tokens does not become aware of a token balance unless the user explicitly adds a specific token contract to “watch”. Some wallets watch the most popular token contracts to detect balances held by addresses they control, but that’s limited to a small fraction of the available ERC20 contracts.

In fact, it’s unlikely that a user would want to track all balances in all possible ERC20 token contracts. Many ERC20 tokens are more like email spam than usable tokens. They automatically create balances for accounts that have ether activity, to attract users. If you have an Ethereum address with a long history of activity, especially if it was created in the presale, you will find it full of “junk” tokens that appeared out of nowhere. Of course, the address isn’t really full of tokens, it’s the token contracts that have your address in them. You only see these balances if these tokens contracts are being watched by the block explorer or wallet you use to view your address.

Tokens don’t behave the same way as ether. Ether is sent with the +send+ function and accepted by any payable function in a contract or any externally owned address. Tokens are sent using +transfer+ or +approve & transferFrom+ functions that exist only in the ERC20 contract, and do not (at least in ERC20) trigger any payable functions in a recipient contract. Tokens are meant to function just like a cryptocurrency such as ether, but they come with certain subtle distinctions that break that illusion.

Consider another issue. To send ether, or use any Ethereum contract you need ether to pay gas. To send tokens, you also need ether. You cannot pay for a transaction’s gas with a token and the token contract can’t pay the gas for you. This can cause some rather strange user experiences. For example, let’s say you use an exchange or Shapeshift to convert some bitcoin to a token. You “receive” the token in a wallet that tracks that token’s contract and shows your balance. It looks the same as any of the other cryptocurrencies you have in your wallet. Now try sending the token and your wallet will inform you that you need ether to do that. You might be confused - after all you didn’t need ether to receive the token. Perhaps you have no ether. Perhaps you didn’t even know the token was an ERC20 token on Ethereum, maybe you thought it was a cryptocurrency with its own blockchain. The illusion just broke.

Some of these issues are specific to ERC20 tokens. Others are more general issues that relate to abstraction and interface boundaries within Ethereum. Some can be solved by changing the token interface, others may need changes to fundamental structures within Ethereum (such as the distinction between EOAs and contracts, and between transactions and messages). Some may not be “solvable” exactly and may require user interface design to hide the nuances and make the user experience consistent regardless of the underlying distinctions.

In the next sections we will look at various proposals that attempt to address some of these issues.

ERC223 - a proposed token contract interface standard

The ERC223 proposal attempts to solve the problem of inadvertent transfer of tokens to a contract (that may or may not support tokens) by detecting whether the destination address is a contract or not. ERC223 requires that contracts designed to accept tokens implement a function named +tokenFallback+. If the destination of a transfer is a contract and the contract does not have support for tokens (i.e. does not implement +tokenFallback+), the transfer fails.

To detect whether the destination address is a contract, the ERC223 reference implementation uses a small segment of inline bytecode, in a rather creative way:

function isContract(address _addr) private view returns (bool is_contract) {
	uint length;
	assembly {
		  //retrieve the size of the code on target address, this needs assembly
		  length := extcodesize(_addr)
	}
	return (length>0);
}

You can see the discussion around the ERC223 proposal here:

https://github.com/ethereum/EIPs/issues/223

The ERC223 contract interface specification is:

interface ERC223Token {
  uint public totalSupply;
  function balanceOf(address who) public view returns (uint);

  function name() public view returns (string _name);
  function symbol() public view returns (string _symbol);
  function decimals() public view returns (uint8 _decimals);
  function totalSupply() public view returns (uint256 _supply);

  function transfer(address to, uint value) public returns (bool ok);
  function transfer(address to, uint value, bytes data) public returns (bool ok);
  function transfer(address to, uint value, bytes data, string custom_fallback) public returns (bool ok);

  event Transfer(address indexed from, address indexed to, uint value, bytes indexed data);
}

ERC223 is not widely implemented and there is some debate in the ERC discussion thread about backwards compatibility and trade-offs between implementing changes at the contract interface level versus the user interface. The debate continues.

ERC777 - a proposed token contract interface standard

Another proposal for an improved token contract standard is ERC777. This proposal has several goals, including:

The details and ongoing discussion on ERC777 can be found here: https://github.com/ethereum/EIPs/issues/777

The ERC777 contract interface specification is:

interface ERC777Token {
    function name() public constant returns (string);
    function symbol() public constant returns (string);
    function totalSupply() public constant returns (uint256);
    function granularity() public constant returns (uint256);
    function balanceOf(address owner) public constant returns (uint256);

    function send(address to, uint256 amount) public;
    function send(address to, uint256 amount, bytes userData) public;

    function authorizeOperator(address operator) public;
    function revokeOperator(address operator) public;
    function isOperatorFor(address operator, address tokenHolder) public constant returns (bool);
    function operatorSend(address from, address to, uint256 amount, bytes userData, bytes operatorData) public;

    event Sent(address indexed operator, address indexed from, address indexed to, uint256 amount, bytes userData, bytes operatorData);
    event Minted(address indexed operator, address indexed to, uint256 amount, bytes operatorData);
    event Burned(address indexed operator, address indexed from, uint256 amount, bytes userData, bytes operatorData);
    event AuthorizedOperator(address indexed operator, address indexed tokenHolder);
    event RevokedOperator(address indexed operator, address indexed tokenHolder);
}

A reference implementation of ERC777 is linked in the proposal. ERC777 depends on a parallel proposal for a registry contract, specified in ERC820. Some of the debate on ERC777 is about the complexity of adopting two big changes at once: a new token standard and a registry standard. The discussion continues.

ERC721 - non-fungible token (deed) standard

All the token standards we have looked at so far are fungible tokens, meaning that each unit of a token is entirely interchangeable. The ERC20 token standard only tracks the final balance of each account and does not (explicitly) track the provenance of any token.

The ERC721 proposal is for a standard for non-fungible tokens, also known as deeds.

From the Oxford Dictionary:

deed: A legal document that is signed and delivered, especially one regarding the ownership of property or legal rights.

The use of the word deed is intended to reflect the “ownership of property” part, even though these are not recognized as “legal documents” in any jurisdiction, at least not currently.

Non-fungible tokens track ownership of a unique thing. The thing owned can be a digital item, such as a game item, or digital collectible. Or, the thing can be a physical item whose ownership is tracked by a token, such as a house, a car, artwork. A deed could also represent things with negative value, such as loans (debt), liens, easements, etc. The ERC721 standard places no limitation or expectation on the nature of the thing whose ownership is tracked by a deed, only that it can be uniquely identified, which in the case of this standard is achieved by a 256-bit identifier.

The details of the standard and discussion are tracked in two different GitHub locations:

Initial proposal: https://github.com/ethereum/EIPs/issues/721

Continued discussion: https://github.com/ethereum/EIPs/pull/841

To grasp the basic difference between ERC20 and ERC721, it is sufficient to look at the internal data structure used in ERC721:

// Mapping from deed ID to owner
mapping (uint256 => address) private deedOwner;

Whereas ERC20 tracks the balances that belong to each owner, with the owner being the primary key of the mapping, ERC721 tracks each deed ID and who owns it, with the deed ID being the primary key of the mapping. From this basic difference flow all the properties of a non-fungible token.

The ERC721 contract interface specification is:

interface ERC721 /* is ERC165 */ {
    event Transfer(address indexed _from, address indexed _to, uint256 _deedId);
    event Approval(address indexed _owner, address indexed _approved, uint256 _deedId);
    event ApprovalForAll(address indexed _owner, address indexed _operator, bool _approved);

    function balanceOf(address _owner) external view returns (uint256 _balance);
    function ownerOf(uint256 _deedId) external view returns (address _owner);
    function transfer(address _to, uint256 _deedId) external payable;
    function transferFrom(address _from, address _to, uint256 _deedId) external payable;
    function approve(address _approved, uint256 _deedId) external payable;
    function setApprovalForAll(address _operateor, boolean _approved) payable;
    function supportsInterface(bytes4 interfaceID) external view returns (bool);
}

ERC721 also supports two *optional* interfaces, one for metadata and one for enumeration of deeds and owners.

The ERC721 optional interface for metadata is:

interface ERC721Metadata /* is ERC721 */ {
    function name() external pure returns (string _name);
    function symbol() external pure returns (string _symbol);
    function deedUri(uint256 _deedId) external view returns (string _deedUri);
}

The ERC721 optional interface for enumeration is:

interface ERC721Enumerable /* is ERC721 */ {
    function totalSupply() external view returns (uint256 _count);
    function deedByIndex(uint256 _index) external view returns (uint256 _deedId);
    function countOfOwners() external view returns (uint256 _count);
    function ownerByIndex(uint256 _index) external view returns (address _owner);
    function deedOfOwnerByIndex(address _owner, uint256 _index) external view returns (uint256 _deedId);
}

Token standards

In this section we’ve reviewed several proposed standards, and a couple of widely-deployed standards for token contracts. What exactly do these standards do? Should you use these standards? How should you use them? Should you add functionality beyond these standards? Which standards should you use? We will examine all those questions next.

What are token standards? What is their purpose?

Token standards are a minimum specification for an implementation. What that means is that in order to be compliant with, say ERC20, you need to, at-minimum, implement the functions and behavior specified by ERC20. You are also free to add to the functionality by implementing functions that are not part of the standard.

The primary purpose of these standards is to encourage interoperability between contracts. Thus, all wallets, exchanges, user interfaces and other infrastructure components can interface in a predictable manner with any contract that follows the specification.

The standards are meant to be descriptive, rather than prescriptive. How you choose to implement those functions is up to you - the internal function of the contract is not relevant to the standard. They have some functional requirements, which govern the behavior under specific circumstances, but they do not prescribe an implementation. An example of this is the behavior of a transfer function if the value is set to zero.

Should you use these standards?

Given all these standards, each developer faces a dilemma: use the existing standards or innovate beyond the restrictions they impose?

This dilemma is not easy to resolve. Standards necessarily restrict your ability to innovate, by creating a narrow “rut” that you have to follow. On the other hand, the basic standards have emerged from experience with hundreds of applications and often fit well with 99% of the use-cases.

As part of this consideration is an even bigger issue: the value of interoperability and broad adoption. If you choose to use an existing standard, you gain the value of all the systems designed to work with that standard. If you choose to depart from the standard, you have to consider the cost of building all of the support infrastructure on your own, or persuading others to support your implementation as a new standard. The tendency to forge your own path and ignore existing standards is known as “Not Invented Here” and is antithetical to the open source culture. On the other hand, progress and innovation depends on departing from tradition sometimes. It’s a tricky choice, so consider it carefully!

Wikipedia “Not Invented Here” (https://en.wikipedia.org/wiki/Not_invented_here)

Not invented here is a stance adopted by social, corporate, or institutional cultures that avoid using or buying already existing products, research, standards, or knowledge because of their external origins and costs, such as royalties.

Security by maturity

Beyond the choice of standard, there is the parallel choice of implementation. When you decide to use a standard, such as ERC20, you have to then decide how to implement a compatible token. There are a number of existing “reference” implementations that are broadly used in the Ethereum ecosystem. Or you could write your own from scratch. Again, this choice represents a dilemma that can have serious security implications.

Existing implementations are “battle tested”. While it is impossible to prove that they are secure, many of them underpin millions of dollars of tokens. They have been attacked, repeatedly and vigorously. So far, no significant vulnerabilities have been discovered. Writing your own is not easy - there are many subtle ways that a contract can be compromised. It is much safer to use a well-tested broadly-used implementation. In our examples above, we used the OpenZeppelin implementation of the ERC20 standard, as this implementation is security focused from the ground up.

If you use an existing implementation you can also extend it. Again, be careful with this impulse. Complexity is the enemy of security. Every single line of code you add expands the attack surface of your contract and could represent a vulnerability lying in wait. You may not notice a problem until you put a lot of value on top of the contract and someone breaks it.

Extensions to token interface standards

The token standards discussed in this section start with a very minimal interface, with limited functionality. Many projects have created extended implementations, to support features that they need for their application. Some of these include:

Owner Control:: Specific addresses, or set of addresses (multi-signature) are given special capabilities, such as blacklisting, whitelisting, minting, recovery etc.

Burning:: A token burn is when tokens are deliberately destroyed by transfer to an unspendable address or by erasing a balance and reducing the supply.

Minting:: The ability to add to the total supply of tokens, at a predictable rate, or by “fiat” of the creator of the token.

Crowdfunding:: The ability to offer tokens for sale, for example through an auction, market sale, reverse-auction, etc.

Caps:: Pre-defined and immutable limits on the total supply, the opposite of the “minting” feature.

Recovery “Back Doors”:: Functions to recover funds, reverse transfers, or dismantle the token that can be activated by a designated address or set of addresses (multi-signature).

Whitelisting:: The ability to restrict token transfers only to listed addresses. Most commonly used to offer tokens to “accredited investors” after vetting by the rules of different jurisdictions. There is usually a mechanism for updating the whitelist.

Blacklisting:: The ability to restrict token transfers by disallowing specific addresses. There is usually a function for updating the blacklist.

There are some reference implementations for many of these functions, for example in the OpenZeppelin library. Some of these are use-case specific and only implemented in a few tokens. There are, as of now, no widely accepted standards for the interfaces to these functions.

As previously discussed, the decision to extend a token standard with additional functionality represents a tradeoff between innovation/risk and interoperability/security.

Tokens and ICOs

Tokens have become an explosive development in the Ethereum ecosystem. It is likely that they will be a very important, foundational, component of all smart-contract platforms like Ethereum.

Nevertheless, the importance and future impact of these standards should not be confused with an endorsement of the current token offerings. As in any early stage technology, the first wave of products and companies will almost all fail, and some will fail spectacularly. Many of the tokens on offer in Ethereum today are barely disguised scams, pyramid schemes and money grabs.

The trick is to separate the long-term vision and impact of this technology, which is likely to be huge, from the short term bubble of token ICOs, which is rife with fraud. Both can be true at the same time. The token standards and platform will survive the current token mania, and then they will likely change the world.

Fork me on GitHub