Architecture — Sverlo v2.1

Parsing is a pipeline of five stages. Each stage takes the output of the previous one and knows nothing about the next, so stages can be tested and replaced on their own.

text

source  ->  splitter  ->  joiner  ->  matcher  ->  filter  ->  sink
(Readable)  (lines)       (multiline) (format)     (query)     (your code)

Stages#

Line splitter. Cuts the stream at \n and \r\n, keeps the incomplete tail between chunks and drops lines longer than maxLineLength before they reach memory.
Joiner. Applies the multiline rule. With no rule the stage does nothing and copies no data.
Matcher. Applies the format or the pattern. A pattern compiles to a state machine that reads the line left to right once.
Typer. Turns the matched substrings into Date, numbers and levels. A type error becomes a ParseError, never NaN.
Filter. The function from parser.query passes or drops the finished record.

Decisions and reasons#

Decision	Reason
State machine instead of regular expressions	Run time is linear in the line length; a line of a million spaces cannot hang the process
A record is created only after typing succeeds	No half-filled objects in `parser.errors`
`fields` without a prototype	Keys from a log cannot overwrite object methods
No global state	Two parsers in one process do not affect each other
No `eval` or `new Function`	A query or pattern from outside cannot become code execution

Boundaries#

Sverlo does not open files itself, does not follow log rotation and does not send records over the network. The first is a Readable, the second is tail in the CLI, the third is your code. A narrow scope keeps the library small and predictable.