Serverless Orchestration: Exploring the Future of Workflow Automation

Tim Bossenmaier

Thursday 10:55 in Titanium3

Orchestration is a typical challenge in the data engineering world. Scheduling your data transformation jobs via CRON-jobs is cumbersome and error-prone. Furthermore, with an increasing number of jobs to manage it gets in-oversee able. Tools like Apache Airflow, Dagster, Luigi, and Prefect are known for addressing these challenges but often require additional resources or investment. With the advent of serverless orchestration tools, many of these disadvantages are mitigated, offering a more streamlined and cost-effective solution. Beyond data engineering, serverless orchestration holds substantial potential for classical software engineering, especially as organizations explore serverless approaches for optimizing efficiency and reducing overhead.

In this talk you will explore:

  • Basic Introduction to Serverless Orchestration:
    - What is orchestration about?
    - What is serverless about?
    - Why combining the two of them?
    
  • Offerings from Major Cloud Vendors:
    - Analyzing solutions from leading cloud providers in the realm of serverless orchestration
    
  • Patterns and Solutions for Serverless Orchestration in Software Engineering:
    - Exploring how serverless orchestration can be applied within classical software engineering contexts
    

Participants will leave this session equipped with a comprehensive understanding of the serverless orchestration landscape and its applications across different engineering disciplines.

Tim Bossenmaier

Tim is a Data Engineer at Cloudflight, based in Innsbruck. There he architects and builds modern data infrastructures in customer projects, ranging from streaming ETL pipelines to data catalogs and entire data platforms. His focus areas include software engineering, cloud technologies, data platform engineering, and DataOps. Tim is also a passionate Open Source contributor, actively working on projects like Apache StreamPipes and DataHub, among others.