OpenAF processor

Usage: oafp [file] [options]

Takes an input, usually a data structure such as json, and transforms it to an equivalent data structure in another format or visualization. The output data can be filtered through JMESPath, SQL or OpenAF’s nLinq and provided transformers can also be applied to it.

If a file or file=somefile or file=zipfile::somefile is not provided the input will be expected to be provided through stdin/pipe. Options are expected to be provided as option=value. Check the lists below for all the available options. Use -v or in="?" and out="?" to discover supported inputs, transforms and outputs.

Main options:

Option Description
-h Show this document
help Alternative way to show this document or others (e.g. filters, template)
file The file to parse (if not provided stdin is used)
ingzip Decompress file input with OpenAF gzip streams. Auto-detected from .gz (case-insensitive); true forces decompression and false disables it. The preceding extension selects the input format.
cmd Alternative to file and stdin to execute a command (e.g. kubectl, docker) to get the file contents
data Alternative to file, stdin and cmd to provide data input
out The output format (default: ctree)
idescsource loaded (default) or stream. stream enables file-backed JSON browsing with out=idesc.
idescrecipe Replay a saved idesc selection/query recipe with the supplied source; use out=idesc to reopen interactively.
in The input type (if not provided it will try to be auto-detected)
ifrom An OpenAF nLinq expression to filter input data
isql A SQL expression to filter input data
from An OpenAF nLinq path expression to filter output
sql A SQL expression to filter output
opath A JMESPath expression to filter output data
sqlfilter Enables the forcing of the sql filter parser (values: auto, simple, advanced)
sqlfiltertables A JSON/SLON array composed of ‘table’ name and ‘path’ to each table’s data to be used with the sqlfilter
path A JMESPath expression to filter output
csv If type=csv, the CSV options to use
outkey If defined the map/list output will be prefixed with the provided key
outfile If defined all output will be written to the provided file
outfileappend If ‘true’ and outfile=true the output will be appended on the provided file
parallel auto (default), true, or false. Auto measures eligible pure work before using bounded workers. Record output stays ordered; explicit true also enables supported specialized parallel operations. false overrides OAFP_PARALLEL.
progress auto (default) or off. Shows a delayed wait animation on interactive stderr only.
stream With in=json, processes a top-level array one element at a time using record-level filters and output. Default false.
pause If ‘true’ (or -p) will try to pause contents in alternative to less -r
color If ‘true’ will force colored output if available
url Retrieves data from the provided URL
urlmethod If ‘url’ is provided defines the http method to use if different from GET
urlparams If ‘url’ is provided extra parameters (equivalent to OpenAF’s $rest) can be provided in JSON/SLON
urldata If ‘url’ is provided a JSON/SLON/text data can be provided
insecure If true will ignore SSL/TLS certificate validation
chs A JSON/SLON map or array composed of an OpenAF channel ‘name’, ‘type’ and optional ‘options’
loop If defined will loop the processing by the number of seconds provided
loopcls If ‘true’ and loop is defined it will clear the screen (or file) on each loop cycle
libs Comma delimited list of installed OpenAF’s oPacks to consider to extend oafp’s inputs, transformations and outputs
pipe A JSON/SLON/YAML map for recursive call of oafp similar to using unix pipes (useful with -f)
-f Enables to provide a JSON/SLON/YAML file with all the oafp parameters as a map
-examples Will access an internet based list of oafp examples and list them
examples Will search the provided keyword or ‘category::subcategory’ in the internet based list of oafp examples
version Alternative way to change the input to a map with the tool’s version
-v Changes the input to a map with the tool’s version info
debug If true prints debugging messages

Filter options apply in the following order: path, from and sql.

For path syntax check https://jmespath.org/tutorial.html

You can list inputs by using in=”?”; outputs by out=”?”; transforms by transforms=true

sqlfilterstable assumes and forces sqlfilter=advanced

Use ‘OAFP_RESET=true’ to force resetting the terminal before waiting for input or displaying an output (use this if you experience terminal related issues)

Use ‘OAFP_CODESET=UTF-16’ to force reading files in a different codeset (e.g. UTF-16) different from the default UTF-8.

Gzip file input works with both positional filenames and file=:

oafp file=records.json.gz out=json
oafp records.ndjson.gz out=json
oafp file=records.json.gz stream=true out=json
oafp file=compressed.dat ingzip=true in=csv out=json

Decompression streams directly into text and record readers; whole-document formats still buffer the decompressed text. Path-only readers (jsonprefix, jsondesc, JFR and hsperf) use a temporary decompressed file that is removed after reading. ingzip applies to file input, not stdin, commands or URLs. Use ingzip=false for an uncompressed file whose name ends in .gz.

⬇️ Input types

List of data input types that can be auto-detected (through the file extension or through its contents). You can always override it by using the in option:

Input type Description
ask Interactively asks questions to an user (using JSON/SLON for OpenAF’s askStruct)
base64 A base64 text format
ch An OpenAF channel format
csv A CSV format (auto-detected)
db A JDBC query to a database
gb64json Equivalent to in=base64 and base64gzip=true
hsperf A Java hsperfdata* file (requires file=hsperfdata_user/123)
ini INI/Properties format
javas Tries to list java processes running locally (javasinception=true to include itself). Adds the hsperf file path when the process id matches ow.java.getLocalJavaPIDs()
javagc The Java GC log lines text format
javathread The Java Thread stack dump lines text format
jfr The Java Flight Recorder format
jmx Uses Java JMX to retrieve data from another Java process
json A JSON format (auto-detected)
jsonschema Given a JSON schema format tries to generate sample data for it
jwt Decodes and/or verifies a JSON Web Token (JWT)
lines A given string/text to be processed line by line
llm A large language model input (uses ‘llmenv’ or ‘llmoptions’)
llmdecide Stateless decisions over a whole state and named questions using OpenAF $llm().decide()
llmmodels Lists the large language models available (using ‘llmenv’ or ‘llmoptions’)
ls Returns a list of files and folders for a given directory path or zip or tar or tgz file
md A Markdown format
mdtable A Markdown table format
mdcode A Markdown code blocks format
mcp A Model Context Protocol (MCP) input
minia Executes a mini-a interaction definition (requires the mini-a oPack)
ndjson A NDJSON (new-line delimited JSON) format
ndslon A NDSLON (new-line delimited SLON) format
oaf Takes an OpenAF scripting code or OpenAF scripting file to execute and use the result as input
oafp Takes a JSON/SLON/YAML map input as parameters for calling a sub oafp process (arrays will call multiple oafp processes; inoafpseq=true will process sequentially)
ojob Takes a JSON/SLON/YAML map input with a ‘ojob’ string and a ‘args’ map parameter
openmetrics An OpenMetrics/Prometheus compatible format
raw Passes the input directly to transforms and output
rawhex Tries to read the input char by char converting into lines with the hexadecimal representation
sh Executes a shell command returning stdout, stderr and exitcode as a map
slon A SLON format (auto-detected)
snmp A SNMP device source
sql One or more SQLs statements to AST (Abstract Syntax Tree) or beautified SQL
toml TOML format
toon TOON format
xls A XLSx compatible file (requires file=abc.xlsx)
xml An XML format (auto-detected)
yaml A YAML format (auto-detected)

🚜 Optional transforms:

These options will change the parsed input data included any filters provided.

Option Type Description
arraytomap Boolean If true will try to convert the input array to a map (see arraytomapkey, arraytomapkeepkey)
arraytomapkeepkey Boolean If true and arraytomap=true the defined arraytomapkey won’t be removed from each map
arraytomapkey String For arraytomap=true defines the name of the map property that will be each element key (see arraytomapkeepkey)
allstrings Boolean If true will try to convert all values to strings
cmlt Boolean If true will accumulate the input values into an output array (useful with loop)
correcttypes Boolean If true will try to convert alpha-numeric field values with just numbers to number fields, string date fields to dates and boolean fields
denormalize String Reverses ‘normalize’ given a JSON/SLON map with a normalize schema (see OpenAF’s ow.ai.normalize.withSchema)
diff String A JSON/SLON map with a ‘a’ path and a ‘b’ path to compare and provide diff data
field2byte String A comma delimited list of fields whose value should be converted to a byte abbreviation
field2date String A comma delimited list of fields whose value should be converted to date values
field2si String A comma delimited list of fields whose value should be converted to a SI abbreviation
field2str String A comma delimited list of fields whose value should be converted to a string representation
field4map Boolean A comma delimited list of fields whose value should be converted from JSON/SLON string representation to a map
flatmap Boolean If true a map structure will be flat to just one level (optionally flatmapsep=[char] to use a different separator that ‘.’)
getlist Boolean/Number If true, selects the first nested array; a positive number selects the nth nested array in traversal order. If disabled or no matching array is found, preserves the input, including scalar values.
forcearray Boolean If true and if the input is a map it will force it to be an array with that map as the only element
jsonschema String The JSON schema file to use for validation returning a map with a boolean valid and errors if exist
jsonschemacmd String Alternative option to ‘jsonschema’ to retrieve the JSON schema data to use for validation returning a map with a boolean valid and errors if exist
jsonschemaoptions String/Map JSON/SLON Ajv options for jsonschema or jsonschemacmd, merged with allErrors=true; see JSON Schema
jsonschemagen Boolean If true will taken the provided input map as an example to generate an output json schema
kmeans Number Given an array of ‘normalized’ data will cluster data into the number of centroids provided
llmcontext String If ‘llmprompt’ is defined provides extra context to the model regarding the input data
llmdecide String/Map Per-entry stateless decisions; inline JSON/SLON configuration or JSON/YAML/SLON file. See Decision enrichment.
llmprompt String A large language model prompt to transform the input data to json (uses the same input options ‘llmenv’ and ‘llmoptions’)
maptoarray Boolean If true will try to convert the input map to an array (see maptoarraykey)
maptoarraykey String If maptoarray=true defines the name of the map property that will hold the key for each map in the new array
merge Boolean If input is a list/array of maps will merge each element into one map
normalize String A JSON/SLON map with a normalize schema (see OpenAF’s ow.ai.normalize.withSchema)
numformat String For all number values applies a java.util.Formatter format (e.g. %,d)
oaf String An OpenAF scripting code or OpenAF scripting file to execute taking input as ‘data’ and returning the transformed data
regression String Performs a regression (linear, log, exp, poly or power) over a provided list/array of numeric values
removedups Boolean Removes duplicate array entries, retaining the first occurrence. Map key order is ignored recursively; array order and the distinction between arrays and maps are preserved.
removeempty Boolean If true will remove array/list entries that are either null or undefined
removenulls Boolean If true will try to remove nulls and undefined values from a map or array
searchkeys String Will return a map with only keys that match the provided string
searchvalues String Will return am map with only values that match the provided string
set String Performs set operations (intersection by default) over an ‘a’ and ‘b’ path to an array defined in a JSON/SLON map. Comparisons ignore map key order recursively while preserving array order and distinguishing arrays from maps.
sortmapkeys Boolean Recursively sorts map keys, including maps inside arrays. Preserves arrays, their element order, and scalar values.
spacekeys String Replaces spaces in keys with the provided string (for example, helpful for XML output)
trim Boolean If true all the strings of the result map/list will be trimmed
val2icon String If defined will transform undefined, null and boolean values to emoticons (values can be ‘default’ or ‘simple’)
xjs String A .js file with function code manipulating an input ‘args’. Returns the transformed ‘args’ variable.
xpy String A .py file with Python function code manipulating an input ‘args’. Returns the transformed ‘args’ variable.
xfn String A javascript code, receiving input as ‘args’ and return it’s code evaluation.
xrjs String A .js file with function code to manipulate each input array record as ‘args’. Returns the transformed ‘args’ record.
xrpy String A .py file with function code to manipulate each input array record as ‘args’. Returns the transformed ‘args’ record.
xrfn String A javascript code, receiving each input array record as ‘args’ and return it’s code evaluation.

⬆️ Output formats

List of available formats to use with the output option:

Output format Description
base64 A base64 text format
bstable A borderless Unicode table with separators between records and wrapping to the terminal width (for lists or maps)
btable A borderless Unicode table without separators between records and with wrapping to the terminal width (for lists or maps)
btree A tree-like format without ANSI colors, using plain-text branch characters
ch An OpenAF channel format
chart A line-chart style chart (useful together with ‘loop’)
cjson A JSON forcely colored format
cmd Executes a command for each input data entry
cslon A SLON format forcely colored
csv A CSV format (only for list outputs)
ctable A table-like forcely colored format (only for list outputs)
ctree A tree-like forcely colored format
clkyaml A Kubernetes YAML without new-lines colored format
ckyaml A Kubernetes YAML colored format
cyaml An YAML colored format
db Output to a JDBC database
dsv A delimiter-separated values format with configurable separator, quoting and headers (for lists or maps; see DSV output options)
envs Tries to output the input data as OS environment variables setting commands
gb64json Equivalent to out=base64 and base64gzip=true
grid A multiple output ascii grid (useful together with ‘loop’)
html An HTML format
idesc Interactive full-screen tree browser for maps, arrays and scalar values; expand/collapse, zoom, filter, queries, previews, bookmarks, comparison and export
ini A INI/Properties format (arrays are not supported)
json A JSON format without spacing
jsmap OpenAF’s HTML representation of structured data, printed directly without opening a browser
jwt Signs map data into a JSON Web Token (JWT)
key Stores data into an OpenAF global (used inside OpenAF)
kyaml A Kubernetes YAML format
lkyaml A Kubernetes YAML format without new-lines
lines Given an array of strings prints each line
log If input has Logstash compatible fields outputs a human-readable log
map A rectangle map format
md A Markdown format
mdtable A Markdown table format (only for list outputs)
mdyaml A multi document YAML format (only for list outputs)
mtree A tree-like format without ANSI colors, using Unicode branch characters (falls back to ctree on older OpenAF versions without TREE.mono support)
ndjson A NDJSON (new-line delimited JSON) format
ndslon A NDSLON (new-line delimited SLON) format
ndcslon A NDSLON (new-line delimited SLON) format forcely colored
openmetrics Converts a map or list to OpenMetrics/Prometheus compatible format
oaf An OpenAF scripting code or OpenAF scripting file (together with ‘outoaf’) to execute taking input transformed as ‘data’
pjson A JSON format with spacing (equivalent to prettyjson)
prettyjson A JSON format with spacing
pxml Tries to output the input data into pretty xml
raw Tries to output the internal representation (string or json) of the input transformed data
rawascii Outputs text data line by line with visual representation of non-visual characters
res Outputs data to an OpenAF global ‘res’ (used in oJobs)
schart A static line-chart like chart (for a fixed list/array of values)
slon A SLON format
sql Outputs a series of SQL statements for an input list/array data
stable A table-like format with separation (only for list outputs)
table A table-like format without size constraints (only for list outputs)
template A Handlebars template format
text A string text format
toml A TOML format (arrays will have outkey=list)
toon A TOON format
tree A tree-like format
xls A XLSx output format
xml An XML format
yaml A YAML format

bstable, btable and jsmap are passed through to OpenAF and are not listed by out="?". They require an OpenAF version that supports the corresponding format. The OpenAF btable options __width and __rowsep are not forwarded by oafp; use bstable to enable separators between records.

For ‘template’ check https://docs.openaf.io/docs/guides/oafp/oafp-template.html

For ‘log’ you can use ‘logtheme’ or the environment variable ‘OAFP_LOGTHEME’ with a JSON/SLON map with the colors to use ‘(errorLevel: red, warnLevel: yellow, timestamp: bold)’

⬇️ Input options

🧾 CH input options

List of options to use when in=ch:

Option Type Description
inch String A JSON/SLON configuration string with type and options/url
inchall Boolean A boolean flag to determine if the input map will be used for a getAll query

Example of options provided in JSON: inch=”{type:’mvs’,options:{file:’data.db’}}” Example of options provided in SLON: inch=”(type: remote, url: ‘http://some.host:1234/chname’)”

The input data can be JSON/SLON/YAML and will be used for the ‘get’ or ‘getAll’ query.

You can use sBuckets variables (e.g. secKey, secRepo, secBucket, secPass, secMainPass, secFile) on the ‘options’ map to fill it.

🧾 DB input options

List of options to use when in=db (SQL query):

Option Type Description
indbjdbc String The JDBC URL to access the input database
indbuser String The JDBC access user
indbpass String The JDBC access password
indbtimeout String The JDBC access timeout
indblib String Use a JDBC driver oPack generated by ojob.io/db/getDriver
indbstream Boolean If true the output will be processed record by record
indbexec Boolean If true the input SQL is not a query but a DML statement
indbautocommit Boolean If true the input SQL will be executed with autocommit enabled
indbdesc Boolean If true the output will be a list of column names and types (use ‘LIMIT 1’ for faster results)

JDBC oracle: jdbc:oracle:thin:@[host]:[port]:[database] JDBC postgreSQL: jdbc:postgresql://[host]:[port]/[database] JDBC H2: jdbc:h2:[file]

🧾 DSV input options

List of options to use when in=dsv:

Option Type Description
indsvsep String The separator to use (default is ‘,’)
indsvsepre String The regular expression to use as separator
indsvquote String The quote character to use (default is ‘”’)
indsvescape String The escape character to use for double-quotes
indsvcomment String The comment character to use (default is ‘#’)
indsvheader Boolean Uses the first non-comment line as column names (default is true). When false, provide indsvfields.
indsvtrim Boolean Trims header names and values (default is true). Set false to preserve leading and trailing whitespace.
indsvjoin Boolean Returns one array of records when true; otherwise emits each record once (default is false). Applies to stdin, file, cmd and data input.
indsvfields String Comma separated column names. When provided, every non-comment line is data, including the first; no input header is consumed.

DSV input splits each physical line on indsvsep (or indsvsepre) before removing surrounding quotes. Embedded separators and multiline quoted fields are not supported; use in=csv for CSV records with quoted separators.

For a headerless semicolon-separated file:

oafp in=dsv file=data.txt indsvsep=';' indsvfields=name,score indsvheader=false indsvjoin=true out=json

DSV parsing and header inference stay sequential. Supplying indsvfields avoids inferred field names. Filters, transforms and output run on the calling thread.

🧾 JavaGC input options

List of options to use when in=javagc:

Option Type Description
javagcjoin Boolean If true it will return an array with each processed line.

🧾 JavaThread input options

List of options to use when in=javathread:

Option Type Description
javathreadpid Number Optional you can provider the local java process pid to try to get the thread stack trace (*)

(*) This requires running openaf/oafp with a Java JDK. Keep in mind that it will interrupt the target application to dump the necessary data.

You can extract the input text data by executing kill -3 pid

🧾 HSPERF input options

List of options to use when in=hsperf:

Option Type Description
hsperfmetadata Boolean With in=hsperf, returns {values, header, entries}, including header and per-counter metadata. Defaults to false; requires an updated OpenAF runtime.

Use file= or cmd= with hsperfmetadata=true. Select path=header or path=entries for metadata and path=values.java for Java counters; without metadata, use path=java. Counter longs are exact decimal strings. Missing counter groups are omitted, timers require the reported frequency, and nonzero header overflow indicates counters the JVM could not store.

🧾 JFR input options

List of options to use when in=jfr:

Option Type Description
jfrjoin Boolean If true will join the JFR records to build an output array
jfrdesc Boolean If true it will include a _desc entry with the JFR event description

🧾 JMX input options

List of options to use when in=jmx:

Option Type Description
jmxpid Number The local java process pid to connect to if ‘jmxurl’ is not provided.
jmxurl String The JMX URL to connect to if ‘jmxpid’ is not provided.
jmxuser String The JMX user to use if JMX URL was provided.
jmxpass String The JMX password to use if JMX URL was provided.
jmxprovider String The JMX provider Java class if JMX URL was provided.
jmxop String The operation to perform (see below for options)

Options available to use with ‘jmxop’:

Op Description
all Tries to retrieve all JMX data available.
domains Retrieves just a list of JMX domains available.
query Performs a JMX query from the input data provided (e.g. java.lang:*)
get Retrieves a specific JMX object (e.g. java.lang:type=Memory)

🧾 JSON input options

List of options to use when in=json:

Option Type Description
jsondesc Boolean If true the output will be a list of JSON paths of the original json. Uses metadata-only scanning when OpenAF provides io.scanJSON, avoiding scalar-value loading; older runtimes and lenient JSON use the compatible reader.
jsonprefix String Given the ‘jsondesc=true’ output list you can use each to filter big json files by prefix.

🧾 JWT input options

List of options to use when in=jwt:

Option Type Description
injwtverify Boolean If true the boolean entry ‘__verified’ will be added to the result.
injwtsecret String A string secret for using HS256, HS384 or HS512 depending on secret size used to verify.
injwtpubkey String A public key file used to verify (might require specifying the injwtalg).
injwtalg String Specifies the algorithm used to verify the JWT (HS* or RSA by default). Depends on available algorithms on the current JVM.
injwtraw Boolean If true it won’t try to convert Unix epoch timestamps to dates.

🧾 Lines input options

List of options to use when in=lines:

Option Type Description
linesjoin Boolean If true it will return an array with each processed line
linesvisual Boolean If true it will try to determine header and column position from spaces and tabs
linesvisualsepre String Regular expression representing the separator between columns when linesvisual=true (defaults to ‘ \s+’)
linesvisualheadsep Boolean If true will try to process the second line as header separator aiding on column position determination (if linesvisualsepre is not defined it will default to ‘\s+’)

Lines and visual-header processing stay sequential.

🧾 LLM input options

List of options to use when in=llm:

Option Type Description
llmenv String Environment variable containing the model configuration (defaults to OAFP_MODEL, falling back to OAF_MODEL when absent).
llmoptions String/Map Model configuration as a JSON/SLON string or map; overrides llmenv. See the shared LLM input/transform options below.
llmconversation String JSON file to load an existing conversation from and save the updated conversation to.
llmimage String Image file path or HTTP(S) URL to use with a visual model.

The input text is the prompt. Use out=md, out=mdtable or out=raw for a text response; other formats request JSON. Normal filters and output formats apply.

echo "Summarize this text" | oafp in=llm llmoptions="(type:ollama,model:llama3)" out=raw

🧾 LLM Decide input options

List of options to use when in=llmdecide:

Option Type Description
llmenv String Environment variable containing the model configuration (defaults to OAFP_MODEL, falling back to OAF_DECIDE_MODEL and then OAF_MODEL when absent).
llmoptions String/Map Model configuration as a JSON/SLON string or map; overrides llmenv. Supports the shared sBucket options below.
llmdecidesample String Emit a gemini or ollama request sample without input, credentials or inference. Use out=yaml or out=json.
llmimage String Local PNG/JPEG/WebP image file to encode as options.images for Ollama native decisions. Cannot be combined with options.images.
llmdecidestats Boolean Return {response, stats} from one decideWithStats() execution (defaults to false).

Requires an updated OpenAF runtime with $llm().decide() support (decideWithStats() when llmdecidestats=true). Supply JSON, SLON or YAML via file=, stdin or data= containing {state, questions, options?}:

Field Type Description
state String/Map/Array Entire state to evaluate once, including arrays.
questions Map Nonempty map of named question definitions, each with type and instructions: choice with named criteria, boolean, or score with ordered criteria.
options Map Optional OpenAF decision settings: strategy (auto by default, native or structured), model override, requireProbabilities, providerOptions and images (ordered raw base64 image array for Ollama native decisions).

OpenAF validates questions and options; unknown request fields are rejected. llmconversation, llmcontext and llmprompt are unsupported. No conversation is saved; failures propagate without retry or fallback.

Images require an updated OpenAF runtime, Ollama v0.35.1+ and CLEF/CLEF Flash vision weights. All questions share the images in order; state remains required. Supply a nonempty options.images array of raw base64 strings (no URLs, data URLs or paths), or use llmimage=screenshot.png for one local file. OpenAF validates image options; Ollama validates image contents and model vision support. The serialized UTF-8 request limit is 32 MiB with images (including base64 and JSON), or 64 KiB without images.

Output is {contractVersion, provider, model, strategy, answers}, or {response, stats} with llmdecidestats=true. Normal filters and output formats apply: use path=answers.route.value out=raw (or path=response.answers.route.value with stats). Score levels are zero-based; selectedProbability is the selected alternative’s probability, providerConfidence is separate provider confidence, and expectedScore is the ordinal expectation. Structured probability fields are null.

Generate a request, edit its state and questions, then run it with file= and your model configuration:

oafp in=llmdecide llmdecidesample=gemini out=yaml > gemini-request.yaml
oafp in=llmdecide llmdecidesample=ollama out=json > ollama-request.json

See help=examples for complete request files and provider requirements. Samples use common OpenAF question definitions with provider-specific strategy options.

🧾 LS input options

List of options to use when in=ls:

Option Type Description
lsext String Forces the file format parsing of the provided path or file (between zip, tar, tgz)
lsrecursive Boolean Will list all files and folders recursively (for folders)
lsposix Boolean Tries to add extra posix data if available (for ZIP files)

🧾 MDTable input options

List of options to use when in=mdtable:

Option Type Description
inmdtablejoin Boolean Scans an entire markdown input for tables and returns an array with the data of each markdown table

🧾 MCP input options

List of options to use when in=mcp:

Option Type Description
inmcptoolslist Boolean If true will list all available MCP tools
inmcplistprompts Boolean If true will list all available MCP prompts

When not using the list options, the input data should be a JSON/SLON/YAML map with the following properties:

Required (if not listing):

  • ‘tool’ (string) - The name of the MCP tool to call
  • ‘params’ (map) - Parameters/arguments to pass to the tool

MCP connection:

  • ‘type’ (string) - Connection type: “stdio” for local process or “remote” for HTTP server (default: “stdio”)
  • ‘url’ (string) - Required for remote servers - the MCP server endpoint URL
  • ‘timeout’ (number) - Timeout in milliseconds for operations (default: 60000)
  • ‘cmd’ (string) - Required for stdio type - the command to launch the MCP server
  • ‘options’ (map) - Additional options passed to underlying JSON-RPC client
  • ‘debug’ (boolean) - Enable debug output showing JSON-RPC messages (default: false)
  • ‘strict’ (boolean) - Enable strict MCP protocol compliance (default: true)
  • ‘clientInfo’ (map) - Client information sent during initialization (default: {name: “OpenAF MCP Client”, version: “1.0.0”})

For stdio type connections, the ‘cmd’ field is required. For remote type connections, the ‘url’ field is required.

🧾 mini-a input options

List of options to use when in=minia:

Option Type Description
minianolog Boolean If true hides mini-a interaction logs and shows a temporary progress message instead
minialogfile String Path to append interaction log lines with timestamps
miniametrics String Path to write the mini-a run metrics as JSON (placeholders can include the run id)

Requires the ‘mini-a’ oPack (opack install mini-a). The input data must be a map or array understood by mini-a (for example the prompt, goal and options map).

🧾 ndJSON input options

List of options to use when in=ndjson:

Option Type Description
ndjsonjoin Boolean If true will join the ndjson records to build an output array
ndjsonfilter Boolean If true each line is interpreted as an array before filters execute (this allows to filter json records on a ndjson)

With ndjsonjoin=false, eligible JSON parsing supports bounded workers through parallel=auto or parallel=true. Framing, filters, other transforms and output stay on the calling thread.

🧾 ndSLON input options

List of options to use when in=ndslon:

Option Type Description
ndslonjoin Boolean If true will join the ndslon records to build an output array
ndslonfilter Boolean If true each line is interpreted as an array before filters execute (this allows to filter slon records on a ndslon)

NDSLON framing and parsing stay sequential.

🧾 RAWHEX input options

List of options to use when in=rawhex:

Option Type Description
inrawhexline Number Number of hexadecimal characters per returned array line

🧾 SH input options

List of options to use when in=sh:

Option Type Description
inshformat String The format to parse stdout and stderr between raw, yaml or json (default)

The input data JSON/SLON/YAML map can be composed of:

  • cmd (mandatory string/array) - the command to execute
  • envs (map) - a series of environment variables to use
  • envsall (boolean) - if true all existing environment variables will also be included
  • prefix (string) - if defined will output to the console stdout/stderr with the provided prefix
  • pwd (string) - the command path working directory

🧾 SNMP input options

List of options to use when in=snmp:

Option Type Description
insnmp String A SNMP address in the form ‘udp://1.2.3.4/161’
insnmpcommunity String The SNMP community to use (default ‘public’)
insnmptimeout Number The timeout to wait for a reply
insnmpretries Number Number of retries in case of failure
insnmpversion Number Version of the SNMP server (e.g. 2, 3)
insnmpsec String A JSON/SLON representation of security attributes (see below)

The input data can be either:

  • A single string with an OID
  • Multiple lines each with just an OID
  • A JSON/SLON/YAML array of OID strings
  • A JSON/SLON/YAML map with OID string values

The ‘insnmpsec’ (in case of version 3 or newer) entry should be a JSON/SLON map with:

Entry Description
securityName The security name to use
authProtocol One of: HMAC128SHA224, HMAC192SHA256, HMAC256SHA384, HMAC384SHA512, MD5, SHA
privProtocol One of: 3DES, AES128, AES192, AES256, DES
authPassphrase The authorization passphrase to use
privPassphrase The private passphrase to use
engineId The engine id in hexadecimal format

🧾 SQL input options

List of options to use when in=sql:

Option Type Description
sqlparse Boolean If true instead of returning a SQL AST representation it will beautify the SQL statement(s)
sqloptions String A JSON/SLON map with options for sqlparse=true

SQL options available:

  • indent: the indentation string (defaults to “ “)
  • uppercase: if true will uppercase the SQL (defaults to false)
  • linesBetweenQueries: number of lines between queries (defaults to 1)
  • maxColumnLength: maximum column length (defaults to 50)
  • skipWhitespaceNearBlockParentheses: if true will whitespace near block parentheses (defaults to false)
  • language: the SQL language dialect (Db2, MariaDb, MySql, N1ql, PlSql, PostgreSql, Redshift, SparkSql, StandardSql and TSql)

🧾 XLS input options

List of options to use when in=xls:

Option Type Description
inxlssheet String The name of sheet to consider (default to the first sheet)
inxlsevalformulas Boolean If false the existing formulas won’t be evaluated (defaults to true)
inxlsdesc Boolean If true, instead of retrieving data, either a list of sheet names will be returned, or, if inxlssheet is provided, a table with ‘___’ of empty cells and ‘###’ for non-empty cells will be returned
inxlscol String The column on the sheet where a table should be detected (e.g. “A”)
inxlsrow Number The row on the sheet where a table should be detected (e.g. 1)

🧾 XML input options

List of options to use when in=xml:

Option Type Description
xmlignored String A comma-separated list of XML tags to ignore
xmlprefix String A prefix to add to all XML tags
xmlfiltertag Boolean If true will filter the XML tags

⬇️⬆️ Input/Output options

🧾 Base64 input/output options

List of options to use when in=base64 or out=base64:

Option Type Description
base64gzip Boolean If true the contents will thet gzip/gunzip respectively to reduce the size of the base64 output

🧾 CSV input/output options

List of options to use with the inputcsv input option (when input type=csv) and/or the csv output option (when output=csv). Both expect the corresponding options to be provided in single JSON or SLON value (see below for example):

Option Type Description
format String You can choose between DEFAULT, EXCEL, INFORMIX_UNLOAD, INFORMIX_UNLOAD_CSV, MYSQL, RFC4180, ORACLE, POSTGRESQL_CSV, POSTGRESQL_TEXT and TDF
withHeader Boolean If true tries to automatically use the available header
withHeaders Array A list of headers to use with the corresponding order
quoteMode String You can choose between ALL, ALL_NON_NULL, MINIMAL, NON_NUMERIC and NONE.
withDelimiter String A single character as a custom delimiter
withEscape String A single character as a custom escape
withNullString String String to use as representation of null values

Example of options provided in JSON: csv=”{withHeader:false,withDelimiter:’|’}” Example of options provided in SLON: inputcsv=”(withHeader: false, quoteMode: ALL)”

You can also use incsv as a shortcut for inputcsv

🚜 Transform options

🧾 CMLT transform options

List of options to use when cmlt=true:

Option Type Description
cmltch String A JSON/SLON OpenAF channel configuration string with type and options/url (defaults to simple)
cmltsize Number The number of input data values to keep (default 100). If -1 it will keep without a limit

🧾 Diff transform options

List of options to use when diff=…:

Option Type Description
difftheme String A JSON/SLON map with the colors to use if color = true
diffnlines Boolean If true will append each line with a line number of the final result of the differences between ‘a’ and ‘b’ (just for rough reference)
diffwords Boolean If true and the input is text based will perform the diff at the word level
diffwordswithspace Boolean If true and the input is text based will perform the diff at the word + spaces level
difflines Boolean If true and the input is text based will perform the diff at the lines level
diffsentences Boolean If true and the input is text based will perform the diff at the sentence level
diffchars Boolean If true and the input is text based will perform the diff at the char level

‘difftheme’ example: “(added: GREEN, removed: RED, common: FAINT, linenum: ITALIC, linediv: FAINT, linesep: ‘:’)”

If color=true a visual colored diff will be output instead of an array of differences

If both inputs are array based and color=false (or not provided) the comparison will be performed at the array elements level

The contents of ‘difftheme’ can also be provided through the ‘OAFP_DIFFTHEME’ environment variable

🧾 LLM input/transform options

Decision enrichment

Use _llmdecide=_ as a transform to classify each array entry and add fields before normal output filtering and rendering. A single map or string produces one decision; arrays produce one decision per entry, including array-valued entries. Without `assign`, entries are replaced by full decision responses.

Configuration accepts a map, an inline JSON/SLON expression, or a regular JSON/YAML/SLON file. For a path-like string, an existing file is read first; a missing file falls back to parsing the original string as JSON/SLON. Existing unreadable or malformed files fail without fallback.

Configuration field Type Description
questions Map Required named OpenAF decision questions, as for in=llmdecide.
options Map Optional OpenAF decision options.
statePath String Optional $path expression evaluated against each entry. Defaults to the entire entry; the result must be text, a map or an array.
assign Map Optional map of literal destination field names to $path expressions evaluated against the decision response. Requires map entries. Multiple fields use the same decision call.
overwrite Boolean Allow assign to replace existing own fields; defaults to false.

Example classify.yaml:

questions:
  category:
    type: choice
    instructions: Classify the ticket.
    criteria:
      billing: Payments, charges, and refunds
      technical: Software errors and outages
options:
  strategy: structured
statePath: description
assign:
  classification: answers.category.value

oafp file=tickets.json llmdecide=classify.yaml out=table
oafp file=tickets.json llmdecide=classify.yaml parallel=true out=csv

Input [{"id":1,"description":"I was charged twice"}] becomes [{"id":1,"description":"I was charged twice","classification":"billing"}]. Destination names are literal: class.name adds one field named class.name. Missing result paths that return null are assigned as null; expressions returning undefined or throwing fail with the entry index.

The transform reuses llmoptions, llmenv, secrets, llmimage and llmdecidestats from in=llmdecide, including decision model fallback. With stats enabled, use response.answers.category.value in assign. Conversational parameters are unsupported. State types and field collisions are checked before inference. Empty arrays produce no inference. Provider failures propagate with the entry index and original error code; no retries or conversation persistence occur.

LLM decision execution is sequential by default, including parallel=auto. parallel=true or OAFP_PARALLEL=true enables OpenAF’s adaptive parallel processing with an isolated client for each entry and preserved result order. Older runtimes without pForEach process sequentially. Parallel failures are raised after workers finish; already dispatched requests may complete.

Input filters (path, ifrom, isql) select data before transforms; output filters (opath, from, sql) can select or filter the enriched results. Transform order follows the existing transform registry, rather than CLI argument order. Streaming inputs process each dispatched record/batch separately; join records when one collected array result is needed.

List of options to use when in=llm, in=llmdecide, llmdecide=… or llmprompt=…:

Option Type Description
llmenv String The environment variable containing the value of ‘llmoptions’ (defaults to OAFP_MODEL)
llmoptions String/Map A JSON/SLON string or map with OpenAF’s LLM ‘type’ (e.g. openai/ollama), ‘model’ name, ‘timeout’ in ms for answers, ‘url’ for the ollama type or ‘key’ for openai type; maps are supported by in=llm, in=llmdecide and the llmdecide transform, while llmprompt requires a string
llmconversation String File to keep the LLM conversation
llmimage String With in=llm, image file path or HTTP(S) URL to use with a visual model.

OpenAF sBuckets are supported in llmoptions. You can set any of the environment variables OAFP_SECREPO, OAFP_SECBUCKET, OAFP_SECPASS, OAFP_SECMAINPASS and OAFP_SECFILE OR set the corresponding map values secRepo, secBucket, secPass, secMainPass and secFile.

Tip: Use the ‘getlist=’ optional transform to automatically filter list of data from LLMs prompt responses if relevant. If OAFP_MODEL is not defined but OAF_MODEL is, the latter will be used automatically. Example: OAFP_MODEL="(type:ollama,model:llama3)" oafp llmprompt="hello world"

🧾 Regression transform options

List of options to use when regression=…:

Option Type Description
regressionpath String The path to the array of y values for the regression formulas
regressionx String Optional path to the array of x values for the regression formulas (defaults to 1, 2, 3, …)
regressionoptions String A JSON/SLON configuration with order (defaults to 2) and/or precision (defaults to 5)
regressionforecast String Optional path to an array of x values for which to forecast the corresponding y

Example: oafp data="[1,2,3]" regression=linear regressionforecast="from_slon('[4|5]')" out=ctable

🧾 Set transform options

List of options to use when set=…:

Option Type Description
setop String Allows to choose a different set operation between ‘union’, ‘diffa’, ‘diffb’, ‘diffab’ (symmetric difference), ‘diff’ and ‘intersect’ (default)

Example: set="(a: old, b: new)" setop=diffb ‘setop=diff’ will add an extra column ‘*’ to identify if a line only exists in ‘a’ or in ‘b’

⬆️ Output options

🧭 IDesc output options

out=idesc opens a full-screen tree browser (in the style of out=ctree) over maps, arrays and scalar values. Containers start collapsed and show their length ({3} for a map, [40] for an array); scalars are shown inline. The current location is always visible and quoted path segments preserve literal keys such as a.b, a[0], numeric strings and empty names.

Option Type Description
idescsource String loaded (default) browses the normal processed result. stream scans UTF-8 JSON on demand without loading the whole document.
idescrecipe String File containing a saved version-1 selection/query recipe and named bookmarks. The source is supplied separately.
idescpage Number Entries loaded per expand/”more” step in the tree browser (and per page in the numbered fallback and previews). Default 20.
idescrecords Number Batch size for line-oriented input (NDJSON, NDSLON, lines, DSV) or in=json stream=true. Each batch of processed records opens one explorer session; press n (or :next) to drop it and read the next batch, q to stop reading. Only one batch is held in memory. A selection/query is carried over to the next batch when it still applies. Positive integer, default 100. CSV and explicit join options still browse the whole result.
idescdepth Number Maximum preview depth. Default 3.
idescstring Number Maximum displayed characters per preview string. Default 200. Oversized file-backed scalars may be omitted with a byte-size label.
idescbytes Number Maximum selected JSON bytes admitted for file-backed queries/slices. Default 16777216 (16 MiB).
idescnodes Number Maximum value nodes admitted for file-backed queries/slices and preview traversal. Default 100000.

All numeric options must be positive integers. These loading limits do not bound intermediate allocations inside existing query engines.

# Explore the normal input after standard filters/transforms.
oafp file=response.json path=response out=idesc
oafp file=events.ndjson in=ndjson out=idesc idescrecords=50

# Explore JSON too large to load in memory.
oafp file=large.json out=idesc idescsource=stream
cat large.json | oafp in=json out=idesc idescsource=stream
oafp cmd='some-command --json' in=json out=idesc idescsource=stream
oafp url=https://example.org/data.json in=json out=idesc idescsource=stream

# Replay a saved selection/query without an interactive terminal.
oafp file=large.json idescrecipe=selection.json out=json outfile=selected.json

 oafp ⌂ › inventory                                      │ 💾 file · 1 ops
──────────────────────────────────────────────────────────────────────────
 ▾ inventory {3}                                                       map
 ╭ ▾ items [3]                                                       table
 │ id│name │ok
 │ ──┼─────┼────
▌│ 1 │alpha│true
 │ 22│beta │null
 │ 3 │gamma│
 ├ ▸ tags [12]                                                       array
 ╰ ▸ owner {2} 1.2 KB                                                  map
──────────────────────────────────────────────────────────────────────────
items[0]  5/9    🔍 name
←↑↓→ hjkl move · ⏎ zoom · ⌫ up · ~ top · / filter · t table/tree · : cmd · q quit

The current root is the left-aligned top line; its children hang from it with the same ╭ ├ ╰ ─ lines as out=ctree. Like out=ctree, an expanded array whose entries are all flat maps is shown as a table (one line per entry, with Enter zooming into the entry); t switches the focused array back to a tree (and back again) and T turns tables off or on everywhere. Arrays of scalars, mixed arrays and arrays with nested values always stay trees. The screen has a header (breadcrumbs, source badge and operation count), the tree, a status line (the focused path in a reusable path= form, the row position and the active filter, or messages and prompts) and a key hint line. Colors follow OpenAF’s __colorFormat theme, so the browser matches out=ctree/out=ctable and your customizations: key (property names), number/string/boolean/date/default (values by type), tree.lines (tree lines, dividers and secondary text), table.title/table.lines/table.bandRow (tables: header, lines and every second row, unless __flags.TABLE.bandRows is false; cells take their value-type color), askChoose (the cursor row), askPre/askQuestion/askChooseFilter/askPos (prompts, header, filter and confirmations) and md.heads.h3 (help headings); entries that are missing or empty fall back to built-in colors. Quoted names and [index] rows keep data distinct from commands. Containers whose size is not yet known in file-backed mode show ? plus their size in bytes until expanded or counted. The browser uses the alternate screen: your scrollback is untouched and restored on exit, and nothing is written to stdout unless you export.

Key Action
↑/k, ↓/j Move the cursor. PgUp/PgDn (or Ctrl-U/Ctrl-D) page, g/G or Home/End jump to the first/last row
→/l Expand the focused container (already expanded: move to its first child; on a “more” row: load more)
←/h Collapse; if already collapsed or a scalar, jump to the parent row; on the root row, zoom out
Space Toggle expand. * expands all children of the focused container one level
t / T Switch the focused array between table and tree / turn tables off or on for every array
Enter Zoom: make the focused container the root (recorded). On a scalar opens it in the preview; on a “more” row loads the next idescpage entries
Backspace/u Zoom out one level. ~ or 0 goes back to the original root. U undoes the last zoom or query
/ Filter the child names of the current root as you type (Enter keeps, Esc clears)
p Scrollable bounded preview of the focused node (f cycles ctree/ctable/cyaml)
y Show the reusable path= expression for the focused node
: Command line (Tab completes, ↑/↓ recall history)
? Help overlay. q or Ctrl-C quits

Expanding, collapsing, moving and filtering only change the view: they are never part of a recipe. Zooming, queries and slices are recorded as ordered operations, so a session can be saved and replayed. Queries and slices act on the focused node (zooming to it first); preview, count, field summary, comparison, bookmarks and exports act on the focused node without moving the view.

Command Description
path <expr>, from <expr>, sql <expr> Query the focused node with the existing engines (also query path\|from\|sql <expr>). Examples: path [?status == 'active'], from equals(status, active), sql select id where status = 'active'
slice a:b Keep array entries a to b (end exclusive)
jump <path> Zoom to an exact property/index path from the current root (e.g. items[2].nested)
count / fields Entry count / field summary (samples up to the first 100 records) in an overlay
bm [name], bm go <name>, bm rm <name> List, save (the focused node) and replay/remove bookmarks
compare <bookmark> [values] Diff the focused node with a bookmark (structure by default, bounded values with values)
recipe [print], recipe save <file>, recipe load <file> Show, save or load the recipe
export json\|yaml <file> Write the focused node to a file (asks before replacing). export json - leaves the screen, writes only the selected data to stdout and exits
format ctree\|ctable\|cyaml\|auto Preview format (remembered)
quit Leave without emitting data

Errors (bad queries, missing paths, unknown commands) appear in the status line and leave the view unchanged. A changed source file clears cached locations and bookmarks and restarts at the root. Scans report progress in the status line and Ctrl-C cancels them. The terminal is resized live. Narrow terminals clip rows instead of wrapping.

Terminals without ANSI support (for example TERM=dumb) automatically get a numbered, line-oriented menu with the same capabilities: Preview, Query, Export, Navigate (Parent, Original root, Undo, Next/Previous page, Filter names, Jump to path) and Tools (Slice array, Count entries, Field summary, Bookmarks, Compare with bookmark, Recipe).

Preview uses the existing ctable, ctree and cyaml renderers on bounded data. Automatic format starts with ctable for arrays of maps and ctree otherwise; an explicit choice is remembered. Preview truncation markers are display-only and are never exported as selected data. A simple selection prints a reusable path= expression; compound selections use idescrecipe= replay. Replay uses the same source mode and operation order, including slices. Supply equivalent startup processing again when replaying a loaded-result recipe. Imported query expressions execute through the normal query engines; review recipes from others before replaying them. Saved recipes contain operations/bookmarks rather than source data, credentials or temporary-file paths.

Export writes JSON or YAML to a file, or emits only selected data to stdout and exits. Interactive screens use a separate controlling terminal, including when stdin/stdout are redirected. JSON export of an untransformed file-backed selection streams the original byte range; YAML and derived queries use bounded loading. outfile= with noninteractive recipe replay explicitly replaces the destination after successful export. Quit without exporting emits no data.

File-backed mode accepts local JSON, .json.gz/ingzip, stdin, command output and URL input. Non-file sources are streamed to a temporary file before browsing; source commands/requests run once. Temporary files are removed on session exit. Startup filters, transforms, record-streaming options and loops are rejected in this mode: use its Query action after narrowing or the normal loaded mode. Loaded NDJSON exploration should use ndjsonjoin=true to browse one combined result.

Navigation rescans on demand and caches up to 64 page summaries. Exact counts may require reading the selected container; use Count explicitly. Scans/large JSON copies report progress and Ctrl-C cancels them. Early-stopped scans do not validate the unread suffix. File changes clear cached navigation/bookmarks and restart at root. A persistent index and full disk-backed sorting/aggregation are outside this mode.

Requires OpenAF with askConsole (including read, size, raw and altScreen), askKey, ow.format.string.ansiClip/ansiPad, injectable askChoose/ask, io.scanJSON and io.copyJSONRange; missing capabilities report an update/rebuild requirement. The controlling-terminal implementation uses JLine’s /dev/tty provider on macOS/Linux (requires stty) and independent CONIN$/CONOUT$ handles on Windows. No interactive terminal is required for recipe replay to an ordinary output format.

🧾 CH output options

List of options to use when out=ch:

Option Type Description
ch String A JSON/SLON configuration string with type and options/url
chkey String A comma delimited list of map keys to build a key from each array value
chunset Boolean If true the input data will be used to unset data on the output channel instead of set

Example of options provided in JSON: ch=”{type:’mvs’,options:{file:’data.db’}}” Example of options provided in SLON: ch=”(type: remote, url: ‘http://some.host:1234/chname’)”

You can use sBuckets variables (e.g. secKey, secRepo, secBucket, secPass, secMainPass, secFile) on the ‘options’ map to fill it.

🧾 Chart output options

List of options to use when out=chart:

Option Type Description
chart String Chart definition in the format "<unit> <path:color:legend>... [-min:<n>] [-max:<n>] [-hsize:<n>] [-vsize:<n>]". The ‘unit’ must be one of: int, dec1, dec2, dec3, dec4, dec, bytes or si. The ‘path’ is a JMESPath expression equivalent to the path= filter (use quotes for non-basic expressions; use @ for the current root value). The ‘color’ should be one of the basic ANSI color names. The ‘legend’ is the series label (quote it if it contains spaces). The -min and -max flags set the Y-axis range; -hsize and -vsize override the default chart width and height in terminal columns/rows.
chartcls Boolean If true the screen will be cleared before rendering each update (alternative to loopcls=true)

Examples: oafp cmd="curl -s http://api.open-notify.org/iss-now.json" out=chart loopcls=true chart="dec3 iss_position.latitude:blue:lat iss_position.longitude:red:long" loop=5

oafp cmd="uptime" in=raw path="replace(trim(@), '.+ ([\d\.]+),? ([\d\.]+),? ([\d\.]+)$', '', '$1|$2|$3').split(@,'|')" out=chart chart="dec2 [0]:green:load1m [1]:yellow:load5m [2]:red:load15m -min:0" loop=1 loopcls=true

oafp cmd="ps -p 12345 -o %cpu,%mem" in=lines linesvisual=true linesvisualsepre="\\s+" out=chart chart="int '\"%CPU\"':red:cpu '\"%MEM\"':blue:mem -min:0 -max:100" loop=1 loopcls=true

HOST=1.1.1.1 && PORT=53 && oafp in=oaf data="data=ow.loadNet().testPortLatency('$HOST',$PORT)" out=chart chart="int @:red:latencyMS -min:0" loop=1 loopcls=true

🧾 Cmd output options

List of options to use when out=cmd:

Option Type Description
outcmd String The command to execute receiving, in pipeline, each input entry in json
outcmdjoin Boolean If true and if input is an array the entire array will be the input entry
outcmdseq Boolean If true and if input is an array the commands will be executed in sequence
outcmdnl Boolean If true each command execution output will be appended with a new-line
outcmdparam Boolean If true the input entry will be replaced on the ‘outcmd’ where ‘{}’ is found
outcmdtmpl Boolean If true the input entry will be considered as an HandleBars’ template

If input is an array, without outcmdjoin=true, each entry will result in a command execution in parallel

🧾 DB output options

List of options to use when out=db:

Option Type Description
dbjdbc String The JDBC URL to access the input database
dbuser String The JDBC access user
dbpass String The JDBC access password
dbtimeout String The JDBC access timeout
dblib String Use a JDBC driver oPack generated by ojob.io/db/getDriver
dbtable String The db table in which should be inserted (‘data’ by default)
dbnocreate Boolean If true no table creation command will be executed (if the table already exists set this to true)
dbicase Boolean If true table and field names will try to ignore case
dbbatchsize Number If defined it will changed the default batch data insert process

You can use out=sql to get a preview of the SQL statements the db output type will use

JDBC oracle: jdbc:oracle:thin:@[host]:[port]:[database] JDBC postgreSQL: jdbc:postgresql://[host]:[port]/[database] JDBC H2: jdbc:h2:[file]

🧾 DSV output options

List of options to use when out=dsv:

Option Type Description
dsvsep String The separator to use (default is ‘,’)
dsvquote String Replacement for double quotes inside serialized object/array cells (default is a backslash followed by a double quote). String cells and headers always use double quotes, with embedded quotes doubled.
dsvfields String Comma separated output columns in the requested order. Defaults to the first row’s keys for the whole table. Missing values are empty; additional keys in later rows are omitted. Does not override dsvheader=false.
dsvuseslon Boolean If true the output of value objects will be in SLON format (default is false)
dsvheader Boolean If true will try to output the first line as header (default is true)
dsvnl String Newline sequence to use (default is ‘\n’)

dsvsep applies to both the header and data rows. Column order remains fixed even if later objects list their keys in a different order. Specify dsvfields to include columns that are absent from the first row.

oafp in=json file=records.json out=dsv dsvsep=';' dsvfields=name,score

🧾 Envs output options

List of options to use when out=envs:

Option Type Description
envscmd String If defined will output the provided command to set each environment variable (defaults to ‘export’ or ‘set’ in Windows)
envsprefix String If defined uses the provided prefix for each environment variable key (defaults to ‘OAFP’)
envsnoprefix Boolean Boolean flag to indicate that no envsprefix should be used (defaults to false)

Example of a shell script using ‘out=envs’:

#!/bin/sh
eval $(oafp -v out=envs)
echo Using OpenAF version: $_OAFP_openaf_version - $_OAFP_openaf_distribution
echo On the operating system: $_OAFP_os_name

Setting envsprefix=”” won’t result in an empty prefix. Use envsnoprefix=true instead.

🧾 JWT output options

List of options to use when out=jwt:

Option Type Description
jwtsecret String A string secret for using HS256, HS384 or HS512 depending on secret size used to sign the JWT.
jwtprivkey String A private key file used to sign (might require specifying the jwtalg).
jwtalg String Specifies the algorithm used to sign the JWT (HS* or RSA by default). Depends on available algorithms on the current JVM.

The data map to sign can have the following entries:

  • audience (String)
  • claims (Map)
  • expiration (Date)
  • headers (Map)
  • issuer (String)
  • id (String)
  • issuedAt (Daterrrr)
  • notBefore (Date)
  • subject (String)

🧾 Grid output options

List of options to use when out=grid:

Option Type Description
grid String A JSON/SLON configuration structured as an array of row arrays. Each row array contains one map per column cell (see column cell options below).

Each column cell map supports:

Field Type Description
title String Label displayed as a bold header above the cell content. Auto-set from path if not provided.
type String Rendering type: tree (default for maps/arrays), map, table, chart, bar, area, text, md, func, or empty
path String JMESPath expression to select data from the pipeline input (use @ for root). Auto-sets title if not provided.
obj String Data or format string to render. For chart: same format as chart=; for bar: same format as bar=; for func: JS function body with mx (height) and my (width) args returning a string.
cmd String Shell command whose JSON stdout is used as cell data instead of the pipeline input.
xspan Number Number of columns this cell spans (default: 1).
yspan Number Number of rows this cell spans (default: 1).

Extra fields for bar type cells:

Field Type Description
max Number Maximum value for the bar scale
min Number Minimum value for the bar scale
indicator String Bar fill character (default: ━)
space String Background character (default: ` `)

Examples: oafp cmd="cat /sys/fs/cgroup/cpu.stat | sed 's/ /: /g'" in=yaml out=grid grid="[[(title:cpu.stat,type:tree)|(title:chart,type:chart,obj:'int nr_throttled:red:throttled nr_bursts:blue:bursts -min:0 -vsize:10')]]" loop=1 loopcls=true

HSPERF=/tmp/hsperfdata_user/12345 && oafp $HSPERF in=hsperf path=java out=grid grid="[[(title:Threads,type:chart,obj:'int threads.live:green:live threads.livePeak:red:peak threads.daemon:blue:daemon -min:0')|(title:ClassLoaders,type:chart,obj:'int cls.loadedClasses:blue:loaded cls.unloadedClasses:red:unloaded')]|[(title:Heap,type:chart,obj:'bytes __mem.total:red:total __mem.used:blue:used -min:0')|(title:Metaspace,type:chart,obj:'bytes __mem.metaTotal:blue:total __mem.metaUsed:green:used -min:0')]]" loop=1 loopcls=true

NS=my-namespace && POD=my-pod-123 && oafp cmd="kubectl top -n $NS pod $POD" in=lines linesvisual=true path="[].{name:NAME,cpu:from_siAbbr(\"CPU(cores)\"),mem:from_bytesAbbr(\"MEMORY(bytes)\")}" out=grid grid="[[(title:cpu,type:chart,obj:'dec3 cpu:blue:cpu -min:0')|(title:mem,type:chart,obj:'bytes mem:green:mem -min:0')]]" loop=2 loopcls=true

🧾 HTML output options

List of options to use when out=html:

Option Type Description
htmlcompact Boolean Boolean flag that if true and the input data is a string or markdown the generated html will have a visual compact width format
htmlpart Boolean Boolean flag that if true and the input data is a string or markdown the generated html will be partial and not the complete file
htmlopen Boolean Boolean that if false won’t try to open the output contents in a browser (defaults to true). To use outfile= you need to set htmlopen=false.
htmlwait Number Amount of ms, when htmlopen=true, to wait for the system browser to open an render the html output
htmldark Boolean If supported and true the output html will use a dark theme.

🧾 Log output options

List of options to use when out=log:

Option Type Description
logprintall Boolean If true all original non data (string) lines will be output

🧾 MD output options

List of options to use when out=md:

Option Type Description
mdtemplate Boolean If true will apply a template output without any input data
mdformat String For map or array values: structured (default) renders headings, scalar tables, and lists; json or yaml emits the value in a fenced code block.

🧾 PXML output options

List of options to use when out=pxml:

Option Type Description
pxmlprefix String A prefix added to all XML tags

🧾 RAWASCII output options

List of options to use when out=rawascii:

Option Type Description
rawasciistart Number Starting line number to display
rawasciiend Number Ending line number to display
rawasciitab Number Tab size for tab character visualization (defaults to 8)
rawasciinovisual Boolean If true, non-visual characters won’t be replaced by their visual representation
rawasciinolinenum Boolean If true, line numbers won’t be displayed

🧾 OpenMetrics output options

List of options to use when out=openmetrics:

Option Type Description
metricsprefix String The prefix to use for each metric (defaults to ‘metrics’)
metricstimestamp Number Unix Epoch in seconds for each metric

🧾 SChart output options

List of options to use when out=schart:

Option Type Description
schart String Chart definition in the format "<unit> <path:color:legend>... [-min:<n>] [-max:<n>] [-hsize:<n>] [-vsize:<n>]". Unit must be one of: int, dec1, dec2, dec3, dec4, dec, bytes or si. Path is a JMESPath expression equivalent to the path= filter (use quotes for non-basic expressions). The -min and -max flags set the Y-axis range; -hsize and -vsize override the default chart width and height in terminal columns/rows. Unlike out=chart (which accumulates values over time), schart renders a fixed snapshot from the current input array.

Examples: oafp data="[(x:1,y:2)|(x:2,y:5)|(x:1,y:4)|(x:2,y:5)|(x:1,y:5)]" in=slon out=schart schart="int '[].x':red:x '[].y':blue:y -min:0 -vsize:8"

URL="http://localhost:9090" && METRIC="go_memstats_alloc_bytes" && echo "{query:'max($METRIC)',start:'2024-06-18T20:00:00Z',end:'2024-06-18T20:15:00Z',step:5}" | oafp in=ch inch="(type:prometheus,options:(urlQuery:'$URL'))" inchall=true out=json | oafp path="[].{timestamp:to_date(mul([0],\`1000\`)),value:to_number([1])} | []" out=schart schart="bytes '[].value':green:$METRIC -min:0"

🧾 SQL output options

List of options to use when out=sql:

Option Type Description
sqltable String The table name to use for the SQL statements (defaults to ‘data’)
sqlicase Boolean If true the table and fields names won’t be double-quoted
sqlnocreate Boolean If true the create table statement won’t be generated

🧾 Template output options

List of options to use when out=template:

Option Type Description
template String A file path to a HandleBars’ template or a string template definition if ‘templatetmpl’ is true
templatepath String If ‘template’ is not provided a path to the template definition (pre-transformation)
templatedata String If defined the template data will be retrieved from the provided path
templatetmpl String If true the ‘template’ will be interpreted as the template definition instead of a file path

🧾 XLS output options

List of options to use when out=xls:

Option Type Description
xlsfile String The output filename (if not defined a temporary file will be used to open with the OS’s Excel-compatible application)
xlssheet String The name of sheet to use (default to ‘data’)
xlsformat String A SLON or JSON string with the formatting of the output file (e.g. (bold: true, borderBottom: “medium”, borderBottomColor: “red”))
xlsopen Boolean If false it won’t try to open the OS’s Excel-compatible application (defaults to true)
xlsopenwait Number The amount of time, in ms, to keep the temporary file for the OS’s Excel-compatible application to start and open the file

🧾 XML output options

List of options to use when out=xml:

Option Type Description
outxmlprefix String A prefix added to all XML tags

🔐 sBuckets

To use OpenAF’s sBuckets to retrieve secrets you can use the following parameters or environment variables:

Parameter Env variable Description
secRepo OAFP_SECREPO sBucket repository
secBucket OAFP_SECBUCKET sBucket bucket name
secPass OAFP_SECPASS sBucket bucket password
secMainPass OAFP_SECMAINPASS sBucket repository password
secFile OAFP_SECFILE Optional sBucket file source
secEnv n/a A boolean flag to use environment variables as sBuckets
secKey n/a The mandatory sBucket bucket key to use

Check more in https://docs.openaf.io/docs/concepts/sBuckets

📝 Examples 

# simple processing through pipe
cat someJsonFile.json | oafp

# simple processing through pipe with scrolling
cat someJsonFile.json | oafp output=ctree | less -r

# specifying the input type and output format
cat data.ndjson | oafp input=ndjson output=cslon

# markdown parsing of a file
oafp file=someFile.md input=md

# table with the latest news from Google
curl -s -L https://blog.google/rss | oafp path="rss.channel.item" sql="select title, pubDate" output=ctable

# ask an LLM model to summarise some text
echo "A very long text to be summarized" | oafp llmprompt="summarize this" llmoptions="(type: ollama, model: llama3)"

# list of OpenAF oPacks and their corresponding version
oafp -v path="openaf.opacks" output=ctable

# list of OpenAF processor inputs, transforms and outputs
oafp -v path="oafp.inputs" output=cslon
oafp -v path="oafp.transforms" output=cslon
oafp -v path="oafp.outputs" output=cslon

# list examples with kubectl
oafp examples=kubectl
# list examples for category 'openaf' and sub-category 'oafp'
oafp examples=openaf::oafp
# list examples for category 'kubernetes'
oafp examples=kubernetes::

# yaml options input to link to oafp executions
echo '
data:
  set:
  - 1
  - 2
  - 3
path: "set[].{ x: @ }"
out : json
pipe:
  isql : |
    SELECT sum("x") "SUM"
  opath: "[0].SUM"
' | oafp -f -

📚 Other help documents

Help Description
help=filters Provides more details regarding the use of “path=”, “from=” and “sql=”
help=template Provides more details regarding the use of “output=template”
help=examples Provide several examples
help=jsonschema JSON Schema validation, options, drafts and sample generation
help=in:<format> Input format options, for example oafp help=in:csv
help=out:<format> Output format options, for example oafp help=out:sql
help=readme Returns this document

Format help includes shared input/output options. Format names ignore case, spacing and punctuation (for example help=in:llmdecide and help=in:mini-a). Use out=raw for Markdown or pause=true to page the selected help.

You can use OpenAI’s ChatGPT oAFp GPT to generate commands

Processing large inputs

NDJSON, NDSLON, DSV and lines are processed incrementally when their join option is false (the default), for files, stdin and commands. CSV still produces a complete array. Joined input, sorting across records, SQL over a complete dataset, and whole-document output require memory proportional to their data.

Use in=json stream=true to process a top-level JSON array incrementally:

oafp file=large.json in=json stream=true out=json parallel=auto
cat large.json | oafp in=json stream=true out=json path='[].name'

Each element is processed independently, like an NDJSON record; out=json emits one JSON value per record rather than a single array. Nulls, scalars, nested arrays and empty containers are supported. Whole-document queries require the default stream=false. Streaming does not support jsonprefix or jsondesc. Malformed input can fail after earlier records have already been emitted.

Automatic workers only parse framed JSON records or sort independent map keys. Other transforms, record framing, header inference and rendering stay sequential. Work queues are bounded by record count and input size; one oversized record still requires memory proportional to that record. parallel=false disables these workers, and an explicit CLI value overrides OAFP_PARALLEL. Stateful readers may remain sequential even with parallel=true.

The wait animation starts after 250 ms, refreshes every 150 ms, and clears before output. Use progress=off to disable it. Redirected stderr, nested processing and full-screen outputs do not animate.

October source additions

See decisions and interactive JSON exploration for worked examples. This option reference was synchronized with the bundled 20261014 nightly processor.

This site uses Just the Docs, a documentation theme for Jekyll.