From Algorithm to Action: Building a DIY Distributed Trading Platform with Open Source

Eugen Geist

Thursday 16:55 in Europium2

Who is this talk for

This talk is ideal for all software engineers interested in financial technology, quantitative developers looking to understand modern trading infrastructure, and technical architects exploring distributed systems in high-stakes environments.
This talk will NOT discuss specific trading strategies or give any financial advice.

Outline

  • Motivation
  • Fundamental trading concepts and market mechanics
  • Market data ingestion and processing
  • Order management and execution
  • Implementation of trading strategies
  • Post-trade analysis
  • Outlook

Motivation

The landscape of financial trading has undergone a dramatic transformation over the past decades. What was once the exclusive domain of institutional players on physical trading floors has evolved into a digitized, accessible marketplace where individual traders can participate from anywhere in the world. The emergence of commission-free trading apps and cryptocurrency exchanges has brought market participation to millions of new retail traders.
This enables everyone to participate with their own trading system in global markets.

In this talk, we'll explore how you can implement your own distributed system for exchange trading leveraging the power of open source without being dependent on trading bot providers. While we won't be able to cover every aspect in depth, we'll address the most essential elements.

Cryptocurrency markets are used as a proving ground for the PoC due to easy availability for everyone.

Fundamental Trading Concepts and Market Mechanics

We'll begin by exploring essential trading concepts:

  • Order book dynamics
  • Orders, Trades and Positions
  • Different types of orders and their implications for system implementation
  • Regulatory requirements
  • Performance of strategies

These lead to different considerations in system design and architecture:

  • De-coupling of exchange interfaces and trading strategies to use same strategy for different markets by using adapter pattern
  • Horizontal scaling to handle data load
  • Need of low latency components and their communication to properly react to market
  • Need of streaming data for real-time risk management
  • Need of persistent storage for regulatory data and post-trading-analysis
  • Need of order action recording and post-trading analysis for performance evaluation

Market Data Ingestion and Processing

The foundation of any trading system is its ability to efficiently process market data. This includes a Python component responsible for real-time normalization and standardization of multi-venue data:

  • Efficient market data representation and storage structures
  • Techniques for handling high-throughput data without compromising latency
  • Market data recording for post-trading analysis using Kafka

Order Management and Execution

Critical components for managing the trading lifecycle. This includes a Python component responsible for normalization and standardization of multi-venue order interfaces:

  • Order action handling (placing orders, modifying orders) and keeping track of orders
  • Global real-time position tracking and risk calculation using Kafka
  • State recovery and system restart procedures
  • Audit trail implementation and transaction logging using Postgres

Implementation of Trading Strategies

We'll explore the practical aspects of implementing trading strategies in Python using the previously discussed system components:

  • Usage of provided market data
  • Placing orders and keeping track of positions
  • Fast communication with market data and order components using gRPC
  • Recording of strategy internals for post-trade analysis

Post-trade Analysis and Performance Monitoring

We will take a closer look to evaluate trading strategies performance with:

  • High-performance post-trade analytics using DuckDB
  • Live performance monitoring using Kafka

Outlook

At the end we will have a brief outlook what other challenges might occur e.g.:

  • Other market types (Finance/Equity/ETF and Energy)
  • Latency considerations
  • Taxes

Eugen Geist

Seasoned Software & Data Engineering Professional with extensive experience in high-frequency trading systems, data warehousing, and cloud solutions. Expert in optimizing mission-critical systems and implementing engineering best practices. Specialized in Python, SQL and cloud technologies.

Currently working as a Freelance Developer focusing on software and data engineering.

Skilled in developing distributed systems, data pipelines, and performance optimization, consistently delivering solutions that maximize business value.