How-Tos

How to start with Baker as a library.

To configure and run a Baker topology, 4 steps are required:

  1. use a TOML configuration file
  2. define a baker.Components object
  3. obtain a Baker configuration object calling baker.NewConfigFromToml
  4. run baker.Main

The example folder in the Baker repositories contains many examples of implementing a Baker pipeline.

Start with the basic example.

Pipeline configuration

How to configure Baker using TOML files

Record and LogLine

Baker operates on objects called “records”, an abstraction over integer-indexable structured data.

Create a custom Record

Record is the interface provided by Baker to represent an “object” of data

baker.Components

How to use and set baker.Components

Create a custom input component

Create a custom filter component

Create a custom output component

Create a custom upload component

Export metrics

How to export metrics from Baker

Read statistics

How to read Baker stats

Sharding setup

How to configure a topology with output sharding?

Tuning concurrency

Last modified November 16, 2020