Getting started
Looking to use Baker and start building pipelines now? Great, let’s see what you need.
Baker is written in Go. To use it you need to import the Baker module into your program.
In this page we describe the simplest way to use Baker. At the end of it, we recommend reading the
Baker Core Concepts and then have a deep dive in the
How-to pages.
Add Baker…
…to a brand-new project
To create a new Go project (using Go modules) and add Baker, these are the suggested steps:
mkdir myProject
cd myProject
go mod init github.com/myUser/myProject
go get github.com/AdRoll/baker
…to an existing project
If you are adding Baker to a project already configured to use Go modules, just type:
cd myProject
go get github.com/AdRoll/baker
Build and run Baker
Once Baker has been added to the project, let’s see how to use it, with a minimalistic example.
This code comes from the cli
example. You can find more examples in the
examples/ folder of the project.
package main
import (
"log"
"github.com/AdRoll/baker"
"github.com/AdRoll/baker/filter"
"github.com/AdRoll/baker/input"
"github.com/AdRoll/baker/output"
"github.com/AdRoll/baker/upload"
)
func main() {
// Add all available components
comp := baker.Components{
Inputs: input.All,
Filters: filter.All,
Outputs: output.All,
Uploads: upload.All,
}
// run Baker
if err := baker.MainCLI(comp); err != nil {
log.Fatal(err)
}
}
To create the binary, just build it:
cd myProject
go build -o baker-example .
Configuration
In the example above we use baker.MainCLI,
an utility function that hides a lot of commonly used setup and requires a
TOML file as first command line parameter.
For details about how to configure Baker,
read the dedicated page.
A simple example, in this case coming from the basic example, is the following:
[fields]
names=["timestamp", "source", "target"]
[input]
name = "List"
[input.config]
files=["testdata/input.csv.gz"]
[[filter]]
name="ReplaceFields"
[filter.config]
ReplaceFields=["replaced", "timestamp"]
[output]
name = "FileWriter"
procs=1
[output.config]
PathString="./_out/output.csv.gz"
Run the program
Running the program is as simple as it sounds, at this point:
$ ./baker-example /path/to/configuration.toml
INFO[0000] Initializing fn=NewFileWriter idx=0
INFO[0000] FileWriter ready to log idx=0
INFO[0000] begin reading f=compressedInput.parseFile fn=testdata/input.csv.gz
INFO[0000] end f=compressedInput.parseFile fn=testdata/input.csv.gz
INFO[0000] terminating f=List.Run
INFO[0000] Rotating current= idx=0
INFO[0000] Rotated current= idx=0
INFO[0000] FileWriter Terminating idx=0
INFO[0000] fileWorker closing idx=0
Next steps
Do you want to know more about Baker?
You can read Baker Core concepts or, if you prefer to jump straight into
the code, you can browse the API reference.
More detailed examples can be found in the How-tos section and the available components are documented in the components pages.