One of the big factors that made web 2.0 so successful and modular is access of data through APIs. You can fetch weather information, your twitter feed, your YouTube playlist all through APIs. This remains a roadblock when creating Dapps and the graph aims to solve this problem by making decentralised data easily available. The Graph decentralized query protocol for blockchains. The Graph lets you build dApps more easily on Ethereum and IPFS using GraphQL.According to its official website, The Graph has been adopted by DeFi leaders including Uniswap, Synthetix, Decentraland, and Aragon. In this post we will explore:

• How The Graph works
• Understanding Subgraphs
• The GRT token
• Feature Roadmap

Okay so lets get started!

How The Graph Works

The Graph retrieves data on Ethereum based on the description file of the subgraph, which is subgraph manifest. This description file defines the smart contracts and events that the subgraph is interested in as well as how to map event data to the nodes of The Graph and store them.

Once the description file of the subgraph is written, you can use the Graph CLI (command line) to store it in IPFS and inform the online service to start checking the data for this subgraph.

The following process clarifies how a smart contract is processed on Ethereum after the description file is deployed by a subgraph.

1. DApp sends a transaction with data to the smart contract of Ethereum;
2. When processing a transaction, the smart contract triggers one or several events;
3. The Graph node continuously scans new blocks on Ethereum to obtain the data contained in the subgraph;
4. The Graph node discovers Ethereum events and runs the defined mapping processing function. The mapping file is a WASM module, which creates or updates the entity data of the Graph node that corresponds to the event in Ethereum;
5. The DApp queries the data obtained by retrieving the Ethereum chain in the Graph node. The node used here is the GraphQL endpoint. It converts GraphQL queries into queries of data stored in its database;
6. The DApp uses a rich text UI to display the query results to the end user, and so on.

The Graph network is powered by the following entities:

Indexers

Those who want to offer data must run their own node. For this, the GRT token is needed, which node operators can stak. If a provider provides faulty or wrong data, he receives a penalty and loses a part of his deposit. Thus, the provider strives to sell only correct data.

Curators

Curators decide whether a source is trustworthy and provides valuable data. They signal to indexers which the network should index subgraphs. They do this by depositing GRT tokens in the respective subgraph. They then receive a share of the fees.

Delegators

Delegators are participants who deposit shares in indexers and thus contribute to securing the network. However, they do not have to operate their own node but participate in an already existing one. In return, they can also earn a share of the node’s fees.

Consumers

Consumers are end-users who want to purchase qualitative data. These can be trading venues and decentralized applications. They pay indexers and can then integrate the provided data into their application using API.

Understanding subgraphs

A subgraph defines which data The Graph will index from Ethereum, and how it will store it. Once deployed, it will form a part of a global graph of blockchain data.

The subgraph definition consists of a few files:

• subgraph.yaml: a YAML file containing the subgraph manifest
• schema.graphql: a GraphQL schema that defines what data is stored for your subgraph, and how to query it via GraphQL
• AssemblyScript Mappings: AssemblyScript code that translates from the event data to the entities defined in your schema

How subgraph query works

The Graph being a decentralized protocol works slightly different from a traditional data point. Because the consumer needs to perform a trustless transaction the journey starts with a discovery phase:

• Service Discovery. The consumer asks The Graph which Indexers have the data they are interested in.

• Indexer Selection. The consumer selects an Indexer to transact with based on which they deem most likely to provide the highest quality service at the best price. This involves weighing the utility of performance, freshness of data, economic security, as well as subjective measures of Indexer reputation.

Indexers express their prices using a cost model which a Consumer can use to produce a GRT-denominated cost based on the features of a query and variables provided by the Indexer such as database statistics, the USD/GRT price, or a flag to indicate whether the Indexer is throttling queries. Thus, pricing is set between an Indexer and a Consumer, rather than at the network level as is the case in a smart contract platform like Ethereum.

• Query + Conditional Micropayment. The consumer sends the Indexer a query along with a conditional micropayment that specifies how much they are willing to pay for compute and bandwidth.

The query specifies which subgraph to query, as of what block, and what data is expected to be available.

• Response + Attestation. If the Indexer accepts the price offered by the consumer, then they process the query and respond with the resulting data, as well as an attestation that this response is correct. Providing this attestation unlocks the conditional micropayment.

The attestation is produced deterministically and is uniquely attributable to the Indexer for the purposes of verification and dispute resolution elsewhere in the protocol.

A single decentralized application querying The Graph may use multiple subgraphs indexed by different Indexers and in that case would go through the above flow for each subgraph being queried.

Query Verification

In order for the WAVE Locks construction and indexer staking to be meaningful, there must be an effective verification mechanism that is capable of reproducing the work performed by an Indexer, identifying faults and slashing offending Indexers.

In the first phase of The Graph Network, this is handled through an on-chain dispute resolution process, which is decided through arbitration.

For queries, we support two types of disputes:

1. Single Attestation Disputes
2. Conflicting Attestation Disputes

In single attestation disputes, Fishermen submit disputes along with a bond, as well as an attestation signed by an Indexer. If the Indexer is found to have attested to an incorrect query response, the fisherman receives a portion of the slashed amount as a reward. Conversely, the fisherman’s bond is forfeit if the dispute is unsuccessful.

Importantly, the fisherman’s reward must be less than the slashed amount. Otherwise, malicious Indexers could simply slash themselves to get around thawing periods or avoid slashing by someone else.

For single attestation disputes, the Fisherman is assumed to be an actor who got their hands through some extra-protocol means, for example a third party logging service.

We expect the far more common use case to be conflicting attestation disputes. In this case, a Fisherman can submit two attestations for the same query, signed by two different Indexers. If the attestations don’t agree with one another, then it’s guaranteed that one or both Indexers committed a fault.

Consumers, when querying The Graph, may choose to intermittently query multiple Indexers for the same query results–for extra security and the chance of winning a portion of their slashed stake as a reward. This strategy works better as the number of unique Indexers on a subgraph increases.

In the first version of the network, there will be an Arbitrator set via protocol governance, which will decide the outcome of disputes. Similar to indexing disputes, the Arbitrator may exercise judgment when incorrect queries may arise as a result of bugs in the software, Indexers missing events from the blockchain, or other accidental factors that could lead to a slashable offense. Arbitration should settle disputes according to the protocol specification as well as the aforementioned arbitration charter.

Eventually, as the software matures, Indexers will be expected to develop the operational know-how to avoid these sorts of errors.

The GRT Token

To ensure economic security of The Graph Network and the integrity of data being queried, participants stake and use Graph Tokens (GRT). GRT is a work token that is an ERC-20 on the Ethereum blockchain, used to allocate resources in the network. Active Indexers, Curators and Delegators can provide services and earn income from the network, proportional to the amount of work they perform and their GRT stake.

Graph Tokens have two primary uses that are indispensable to the functioning of the protocol:

• Indexer Staking. Indexers deposit Graph Tokens to be discoverable in the query market and to provide economic security for the work they are performing.
• Curator Signaling. Curators deposit Graph Tokens in a curation market, where they are rewarded for correctly predicting which subgraphs will be valuable to the network.

Consumers are encouraged to hold ETH or the stable coin of their choice in their wallets. Payments will be settled, however, in GRT to ensure a common unit of account across the protocol.

In addition to the uses outlined above, having a native utility token enables us to incentivize certain behaviors that benefit the network as a whole, such as the indexing of new subgraphs, through new token issuance.

Feature Roadmap

To date, over 3,000 subgraphs have been deployed by thousands of developers, for DApps like Uniswap, Synthetix, Aragon, AAVE, Gnosis, Balancer, Livepeer, DAOstack, Decentraland and many others. The Graph usage has been growing at over 50% MoM and hit over 7 billion queries during the month of September 2020.

The Graph has a global community, including over 200 Indexer Nodes in the testnet and more than 2,000 Curators in the Curator Program as of October 2020. To fund network development, The Graph raised funds from community members, strategic VCs and influential individuals in the blockchain community including Coinbase Ventures, DCG, Framework, ParaFi Capital, CoinFund, DTC, Multicoin, Reciprocal Ventures, SPC, Tally Capital and others. The Graph Foundation also successfully completed a public GRT Sale with participation from 99 countries (not including the U.S.). To date as of November 2020, The Graph has raised ~$25M. The Graph team has following features set out in their development roadmap:

Subgraph Composition

It should be possible to query entity relationships across subgraphs, and also build subgraphs that process data from other subgraphs. This will do for the open data layer of Web3 what smart contracts and “money legos” did for DeFi.

Optimistic Rollup/ Truebit-style Verification for Proofs of Indexing

Most of the work of indexing a subgraph already takes place in WASM or in a language that can compile to WASM. This lends itself to verifiably computing these at layer 2, using some form of bisection protocol and refereed game as is done in Optimistic Rollup and Truebit. This would allow us to eliminate the Arbitrator’s role in indexing disputes.

Verifiable Queries

Rather than rely on Arbitration for settling query disputes, the validity of queries could be guaranteed using cryptographic proofs. A good first step here would be to implement verifiable Proofs of Indexing as discussed in the previous paragraph. This lays the foundation for the design of PoIs to evolve from being a simple signature over a digest, to being a useful input for verifying queries, such as a polynomial commitment, Merkle tree or some other authenticated data structure.

Optimistic Rollup/ Layer 2 for Protocol Economics

The protocol’s current use of state channels makes the network fairly robust to increased Ethereum gas costs and means that gas costs won’t scale dramatically as usage of the network increases. That being said, there are benefits to being able to settle state channels more frequently and have reward calculations be executed more regularly, which would be feasible on a layer 2 chain.

GraphQL Mutations

GraphQL Mutations provide a more semantically meaningful way to interact with a protocol than a smart contract ABI which is lower-level and whose methods don’t always map to a single conceptual action. Bundling GraphQL mutations with subgraphs will make it dramatically easier for a new developer to start interacting with on-chain protocols, in the same way The Graph has already lowered the bar for consuming data from said protocols.

GraphQL API V2

There are a handful of improvements we’ve identified based on the needs of subgraph developers. These include more efficient pagination, nested sorting and filtering and analytics functionality such as time-series bucketing.

Protocol Modeling & Simulation

We will continue to build on work started earlier this year in collaboration with the BlockScience team to better model the protocol and simulate different protocol parameter choice and possible upgrades. Data currently being collected as part of the currently active Mission Control testnet will be instrumental in this process.

Automated Monetary Policy

In the first version of the network, the issuance rate of Graph Tokens that is used to pay indexer rewards is set via governance. Long term, our goal is to reduce the governance surface area as much as possible. The aforementioned modeling and simulation work will be helpful towards that end. Monetary policy, with its potential to advantage certain stakeholders over others, is a ripe target for an automated rule that removes control from governance, keeping in line with The Graph’s governing principle of “progressive minimization” introduced in The Graph Council introductory blog post.

Conclusion and next steps

The Graph aims has an ambitious aim to organize worlds data in an open decentralized ecosystem whic can build a new generation of the web free from monopolies. Check out exciting subgraphs that you can integrate in your next Dapp on the graph explorer.