JSON (1 blogmarks)

JSON Lines is a convenient format for storing structured data that may be processed one record at a time. It works well with unix-style text processing tools and shell pipelines. It's a great format for log files. It's also a flexible format for passing messages between cooperating processes.

JSONL makes it easier to stream and parallelize data which is super useful for LLMs.

Large Language Models (LLMs) generate text in chunks. This means that as LLMs produce output, they can stream text to users in real time, rather than waiting for the entire response to be generated.

source