DIA Nexus Documentation
  • Nexus Overview
  • Intro to Nexus
    • How it works
    • Nexus vs. Lumina
    • Integrated Chains
  • Data products
    • Token Price Feeds
    • RWA Price Feeds
    • Randomness
    • Fair-value Feeds
  • How-to Guides
    • Fetch Price Data
      • Solidity
      • Vyper
      • Demo Oracles
      • Chain-specific Guide
        • Aleph Zero
        • Alephium
        • Edu Chain
        • Hydration
        • Kadena
        • LUKSO
        • Somnia
        • Stacks
        • Superseed
        • XRP Ledger (XRPL)
    • Generate Randomness
      • Solidity
      • WASM
      • Demo Oracles
      • Chain-specific Guide
        • Alephium
    • Migrate to DIA
    • Fund the Oracle
    • Build a Scraper
      • Development Cluster Stack
      • DIA Test‐Space with Docker Compose
      • DIA Test‐Space with Minikube
      • Add a new exchange scraper
      • Add a new foreign scraper
      • Add a new liquidity scraper
      • Additional notes
  • Request a Custom Oracle
  • Reference
    • Architecture
      • Data sourcing
      • Data computation
      • Data delivery
    • APIs
      • Token prices
        • RestAPI
          • Request Samples
        • GraphQL
          • Request Samples
      • RWA prices
    • Pricing Methodologies
      • IR: Interquartile Range Filter
      • MAIR: Moving Average with Interquartile Range Filter
      • VWAP: Volume Weighted Average Price
      • VWAPIR: Volume Weighted Average Price with Interquartile Range Filter
      • LST fair price
    • Data Sources
      • CEXes Data
      • DEXes Data
    • Smart Contracts
      • DIAOracleV2.sol
      • DIARandomOracle.sol
    • Randomness Protocol
  • Resources
    • Audits
    • Community & Support
    • Security Bounty Program
    • Research
      • Return Rates in Crypto Farming
      • Crypto Volatility Index
      • Compounded Rates
      • Polkadot Medianizer
    • T&C
      • Licence Agreement
      • Contributor Covenant Code of Conduct
      • Disclaimer
Powered by GitBook
On this page
  • Granular trade data sourcing
  • Type of data sources and their aggregation
  • On oracle feed manipulation and risk
Export as PDF
  1. Reference
  2. Architecture

Data sourcing

PreviousArchitectureNextData computation

Last updated 1 month ago

Granular trade data sourcing

By leveraging a network of WebSockets and decentralized node providers, DIA Nexus retrieves token trade ticks and metadata from exchanges. This, in turn, represents an enormous data availability (≈15bn trades/day) to, later on, enable DIA to capture the highest data precision and build flexible and customizable price feeds.

Trades for every asset pair get recorded and collected into blocks of 120 seconds into the DIA Nexus platform. For each block, a range of filters are applied. These are functions to get a single price point from the collection of trades in a block. For example, the Moving Average filter returns the volume-weighted moving average of the trades block.

Filters are calculated for each asset pair on each exchange individually, as well as for each asset on all exchanges combined. This combined filter result represents the result closest to the true "whole market" that can be determined by this system. Learn more about .

Type of data sources and their aggregation

DIA Nexus retrieves trading data from as close to the data source as possible in order to ensure the highest precision possible. The data retrieval is processed through a series of asset collectors:

  • For centralised exchanges such as Coinbase, Kraken or Binance, DIA scrapers collect trades directly from the exchange’s databases. The data collection happens in retrieval periods from 1 to 7 seconds and varies from exchange to exchange. All trading data is retrieved through Rest APIs or WebSocket APIs, developed and maintained by the respective trading platform. More on .

  • For decentralised exchanges, DIA collects data from decentralized exchanges on various blockchains by subscribing to swap events in the corresponding liquidity pools – LPs. In contrast to centralized exchanges, in decentralized exchanges, it is possible to retrieve trading data directly from the respective blockchain by subscribing to the corresponding pool events. Examples of sources are Uniswap, curve.finance or PancakeSwap. .

  • For cross-chain bridges, we collect the swaps, enabling us to port prices and map assets across blockchains. Cross-Chain bridges allow for swapping a crypto asset on a given blockchain to the corresponding asset on a different blockchain. For instance, swap USDC on Ethereum to USDC on Arbitrum. Cross-Chain bridges can be seen as ranging somewhere between centralized and decentralized: Parts of the complete bridging transactions can be seen publicly on the respective blockchain. However, by its very nature, the bridging itself is done by a centralized entity.

On oracle feed manipulation and risk

One of the biggest risks for DeFi and Web3 applications is not having enough sources for its oracle and using a feed that relies on one single market. These single-source oracles, as explained in the beginning, are relatively easily exposed to price manipulations and man-in-the-middle attacks.

DIA is designed to avoid these scenarios by leveraging multiple markets as sources, including CEXs and DEXs and enabling the customisation of methodologies and sources to cater best for the needs of specific use cases. So far in DeFi’s (short) history, a failure in the oracle node network has never been the reason for a loss of funds, but attacks on the markets that provide the price data to the sources. Take for instance the example of the Cream Finance attack in October 2021.

DIA prioritises the transparency aspect of its data and oracle services and enables any dApp developer and user to know precisely how the data feeding the application is built, what data sources are been used, and what methodologies are been applied.

token filters here
CEX trade data collection
More on DEX trade data collection