Storage: Difference between revisions

From MemCP
Jump to navigation Jump to search
No edit summary
(Refresh MemCP documentation: accuracy, operational guidance, performance profile and maintained API reference)
Line 1: Line 1:
<!-- Copyright (C) 2026 Carl-Philip Haensch -->
<!-- SPDX-License-Identifier: GPL-3.0-or-later -->
<span id="storage"></span>
= Storage =
= Storage =


The '''Storage''' module provides functions to manage databases, tables, columns, keys, partitions, and large data operations in SCM.
<!-- Generated from MemCP c42e19eba on 2026-08-27; do not edit manually. -->
<div class="mw-message-box mw-message-box-notice">Generated from MemCP commit <code>c42e19eba</code> on 27 August 2026. See [[Full SCM API documentation]].</div>


← Back to [[Full SCM API documentation]]
The '''Storage''' module is the low-level data interface used by generated SQL plans. It manages databases, tables, columns, constraints, scans, indexes, RecSets, computed data, persistence, and schema lifecycle operations.


Most application code should use SQL instead of calling these functions directly. Planner-generated calls rely on exact transaction, locking, callback, ownership, and quoting contracts. Functions that accept table handles or transaction contexts must not be replaced with direct access to shard internals.
For the physical model see [[Columnar Storage]], [[Shards, RecordIDs, Main Storage, Delta Storage]], and [[Persistency and Performance Guarantees]].


== table ==
== table ==
Line 15: Line 23:
=== Parameters ===
=== Parameters ===


* '''schema''' (<code>string</code>):
* '''schema''' (<code>string</code>)
* '''table''' (<code>string</code>):
* '''table''' (<code>string</code>)


<span id="returns"></span>
<span id="returns"></span>
=== Returns ===
=== Returns ===


<code>table</code>
* '''value''' (<code>table</code>)


== scan_estimate ==
== scan_estimate ==
Line 32: Line 40:
=== Parameters ===
=== Parameters ===


* '''table''' (<code>table</code>):
* '''table''' (<code>table</code>)


<span id="returns-1"></span>
<span id="returns-1"></span>
=== Returns ===
=== Returns ===


<code>int</code>
* '''value''' (<code>int</code>)


== table_planner_statistics ==
== table_planner_statistics ==
Line 48: Line 56:
=== Parameters ===
=== Parameters ===


* '''table''' (<code>table</code>):
* '''table''' (<code>table</code>)


<span id="returns-2"></span>
<span id="returns-2"></span>
=== Returns ===
=== Returns ===


<code>any</code>
* '''value''' (<code>any</code>)


== scan_selectivity_estimate ==
== scan_selectivity_estimate ==
Line 65: Line 73:


* '''tx''' (<code>any</code>): transaction context to use for visibility; usually ((context &quot;session&quot;) &quot;__memcp_tx&quot;)
* '''tx''' (<code>any</code>): transaction context to use for visibility; usually ((context &quot;session&quot;) &quot;__memcp_tx&quot;)
* '''table''' (<code>table</code>):
* '''table''' (<code>table</code>)
* '''condition_cols''' (<code>list</code>):
* '''condition_cols''' (<code>list&lt;string&gt;</code>): columns passed to the selectivity predicate
* '''condition''' (<code>any</code>):
** '''column''' (<code>string</code>): column name passed to the corresponding callback parameter
* '''max_rows''' (<code>int</code>):
* '''condition''' (<code>func</code>): predicate sampled to estimate matching rows
** '''Parameters'''
*** '''columns''' (<code>any</code>): one value for each entry in the matching column list, in the same order ''(variadic)''
** '''Returns'''
*** '''result''' (<code>bool</code>): true when the sampled row matches
* '''max_rows''' (<code>int</code>)


<span id="returns-3"></span>
<span id="returns-3"></span>
=== Returns ===
=== Returns ===


<code>list</code>
* '''value''' (<code>list</code>)


<span id="table_empty"></span>
<span id="table_empty"></span>
Line 85: Line 98:
=== Parameters ===
=== Parameters ===


* '''table''' (<code>table</code>):
* '''table''' (<code>table</code>)


<span id="returns-4"></span>
<span id="returns-4"></span>
=== Returns ===
=== Returns ===


<code>bool</code>
* '''value''' (<code>bool</code>)


== scan_recset ==
== scan_recset ==
Line 103: Line 116:
* '''tx''' (<code>any</code>): transaction context to use for visibility; usually ((context &quot;session&quot;) &quot;__memcp_tx&quot;)
* '''tx''' (<code>any</code>): transaction context to use for visibility; usually ((context &quot;session&quot;) &quot;__memcp_tx&quot;)
* '''table''' (<code>any</code>): a table, or an existing recset to narrow further
* '''table''' (<code>any</code>): a table, or an existing recset to narrow further
* '''filterColumns''' (<code>list</code>):
* '''filterColumns''' (<code>list&lt;string&gt;</code>): physical columns passed to filter before map/reduce; $recset_contains supplies a row-bound RecSet membership closure
* '''filter''' (<code>func(columns:any...) -&gt; bool</code>): lambda function that decides whether a row enters the recset
** '''column''' (<code>string</code>): column name passed to the corresponding callback parameter
* '''filter''' (<code>func</code>): lambda function that decides whether a row enters the recset
** '''Parameters'''
*** '''columns''' (<code>any</code>): one value for each entry in the matching column list, in the same order ''(variadic)''
** '''Returns'''
*** '''result''' (<code>bool</code>): true when the row belongs in the recset


<span id="returns-5"></span>
<span id="returns-5"></span>
=== Returns ===
=== Returns ===


<code>recset</code>
* '''value''' (<code>recset</code>)


== recset_count ==
== recset_count ==
Line 120: Line 138:
=== Parameters ===
=== Parameters ===


* '''recset''' (<code>recset</code>):
* '''recset''' (<code>recset</code>)


<span id="returns-6"></span>
<span id="returns-6"></span>
=== Returns ===
=== Returns ===


<code>int</code>
* '''value''' (<code>int</code>)


== recset_project_join ==
== recset_project_join ==
Line 137: Line 155:


* '''tx''' (<code>any</code>): transaction context to use for visibility; usually ((context &quot;session&quot;) &quot;__memcp_tx&quot;)
* '''tx''' (<code>any</code>): transaction context to use for visibility; usually ((context &quot;session&quot;) &quot;__memcp_tx&quot;)
* '''source_recset''' (<code>recset</code>):
* '''source_recset''' (<code>recset</code>)
* '''source_key_columns''' (<code>list</code>):
* '''source_key_columns''' (<code>list</code>)
* '''target_table''' (<code>table</code>):
* '''target_table''' (<code>table</code>)
* '''target_key_columns''' (<code>list</code>):
* '''target_key_columns''' (<code>list</code>)


<span id="returns-7"></span>
<span id="returns-7"></span>
=== Returns ===
=== Returns ===


<code>recset</code>
* '''value''' (<code>recset</code>)


== recset_key_index ==
== recset_key_index ==
Line 157: Line 175:


* '''tx''' (<code>any</code>): transaction context used while reading source keys
* '''tx''' (<code>any</code>): transaction context used while reading source keys
* '''source_recset''' (<code>recset</code>):
* '''source_recset''' (<code>recset</code>)
* '''source_key_columns''' (<code>list</code>):
* '''source_key_columns''' (<code>list</code>)


<span id="returns-8"></span>
<span id="returns-8"></span>
=== Returns ===
=== Returns ===


<code>func(key:any...) -&gt; bool</code>
* '''lookup''' (<code>func</code>): tests whether the recset contains a row with the supplied composite key
** '''Parameters'''
*** '''key''' (<code>any</code>): one value for each source key column, in the same order ''(variadic)''
** '''Returns'''
*** '''present''' (<code>bool</code>): whether the composite key occurs in the recset


== recset_union ==
== recset_union ==
Line 174: Line 196:
=== Parameters ===
=== Parameters ===


* '''recsets''' (<code>list</code>):
* '''recsets''' (<code>list</code>)


<span id="returns-9"></span>
<span id="returns-9"></span>
=== Returns ===
=== Returns ===


<code>recset</code>
* '''value''' (<code>recset</code>)


== recset_intersect ==
== recset_intersect ==
Line 190: Line 212:
=== Parameters ===
=== Parameters ===


* '''recsets''' (<code>list</code>):
* '''recsets''' (<code>list</code>)


<span id="returns-10"></span>
<span id="returns-10"></span>
=== Returns ===
=== Returns ===


<code>recset</code>
* '''value''' (<code>recset</code>)


== recset_difference ==
== recset_difference ==
Line 206: Line 228:
=== Parameters ===
=== Parameters ===


* '''recsets''' (<code>list</code>):
* '''recsets''' (<code>list</code>)


<span id="returns-11"></span>
<span id="returns-11"></span>
=== Returns ===
=== Returns ===


<code>recset</code>
* '''value''' (<code>recset</code>)


== recset_not ==
== recset_not ==
Line 222: Line 244:
=== Parameters ===
=== Parameters ===


* '''recset''' (<code>recset</code>):
* '''recset''' (<code>recset</code>)


<span id="returns-12"></span>
<span id="returns-12"></span>
=== Returns ===
=== Returns ===


<code>recset</code>
* '''value''' (<code>recset</code>)


== scan_exists ==
== scan_exists ==
Line 239: Line 261:


* '''tx''' (<code>any</code>): transaction context to use for visibility; usually ((context &quot;session&quot;) &quot;__memcp_tx&quot;)
* '''tx''' (<code>any</code>): transaction context to use for visibility; usually ((context &quot;session&quot;) &quot;__memcp_tx&quot;)
* '''table''' (<code>table|list|recset</code>):
* '''table''' (<code>table|list|recset</code>)
* '''filterColumns''' (<code>list</code>):
* '''filterColumns''' (<code>list&lt;string&gt;</code>): physical columns passed to filter before map/reduce; $recset_contains supplies a row-bound RecSet membership closure
* '''filter''' (<code>func(columns:any...) -&gt; bool</code>): lambda function that decides whether a row exists
** '''column''' (<code>string</code>): column name passed to the corresponding callback parameter
* '''filter''' (<code>func</code>): lambda function that decides whether a row exists
** '''Parameters'''
*** '''columns''' (<code>any</code>): one value for each entry in the matching column list, in the same order ''(variadic)''
** '''Returns'''
*** '''result''' (<code>bool</code>): true when the row satisfies the existence test


<span id="returns-13"></span>
<span id="returns-13"></span>
=== Returns ===
=== Returns ===


<code>bool</code>
* '''value''' (<code>bool</code>)


== scan ==
== scan ==
Line 259: Line 286:
* '''tx''' (<code>any</code>): transaction context to use for visibility and mutations; usually ((context &quot;session&quot;) &quot;__memcp_tx&quot;)
* '''tx''' (<code>any</code>): transaction context to use for visibility and mutations; usually ((context &quot;session&quot;) &quot;__memcp_tx&quot;)
* '''table''' (<code>table|list|recset</code>): table handle, query-local recset, or a list for temporary data
* '''table''' (<code>table|list|recset</code>): table handle, query-local recset, or a list for temporary data
* '''filterColumns''' (<code>list</code>): physical columns passed to filter before map/reduce; $recset_contains supplies a row-bound RecSet membership closure
* '''filterColumns''' (<code>list&lt;string&gt;</code>): physical columns passed to filter before map/reduce; $recset_contains supplies a row-bound RecSet membership closure
* '''filter''' (<code>func(columns:any...) -&gt; bool</code>): lambda function that decides whether a dataset is passed to the map phase. You can use any column of that table as lambda parameter. You should structure your lambda with an (and) at the root element. Every equal? &lt; &gt; &lt;= &gt;= will possibly translated to an indexed scan
** '''column''' (<code>string</code>): column name passed to the corresponding callback parameter
* '''mapColumns''' (<code>list</code>): physical columns passed to map after filtering; pseudo columns are $update (update/delete current row), $recset_contains (row-bound RecSet membership), $set:<column>, $increment:<column>, and $invalidate:<column> (computed-column maintenance), plus NEW.<column> in trigger plans
* '''filter''' (<code>func</code>): lambda function that decides whether a dataset is passed to the map phase. Equality and range comparisons may be translated into indexed scans
* '''map''' (<code>func(columns:any...) -&gt; any</code>): lambda function to extract data from the dataset. You can use any column of that table as lambda parameter. You can return a value you want to extract and pass to reduce, but you can also directly call insert, print or resultrow functions. If you declare a parameter named '<math display="inline">update', this variable will hold a function that you can use to delete or update a row. Call (</math>update) to delete the dataset, call ($update '(&quot;field1&quot; value1 &quot;field2&quot; value2)) to update certain columns.
** '''Parameters'''
* '''reduce''' (<code>func(any, any)</code>): (optional) lambda function to aggregate the map results. It takes two parameters (a b) where a is the accumulator and b the new value. The accumulator for the first reduce call is the neutral element. The return value will be the accumulator input for the next reduce call. There are two reduce phases: shard-local and shard-collect. In the shard-local phase, a starts with neutral and b is fed with the return values of each map call. In the shard-collect phase, a starts with neutral and b is fed with the result of each shard-local pass. ''(optional)''
*** '''columns''' (<code>any</code>): one value for each entry in the matching column list, in the same order ''(variadic)''
** '''Returns'''
*** '''result''' (<code>bool</code>): true when the row proceeds to map
* '''mapColumns''' (<code>list&lt;string&gt;</code>): physical columns passed to map after filtering; pseudo columns are $update (update/delete current row), $recset_contains (row-bound RecSet membership), $set:<column>, $increment:<column>, and $invalidate:<column> (computed-column maintenance), plus NEW.<column> in trigger plans
** '''column''' (<code>string</code>): column name passed to the corresponding callback parameter
* '''map''' (<code>func</code>): lambda function that extracts or produces one value from the row; it may also use documented pseudo columns for mutations or result output
** '''Parameters'''
*** '''columns''' (<code>any</code>): one value for each entry in the matching column list, in the same order ''(variadic)''
** '''Returns'''
*** '''result''' (<code>any</code>): value passed to reduce, or returned directly when no reducer is supplied
* '''reduce''' (<code>func</code>): optional aggregation function used first within shards and then to combine shard results ''(optional)''
** '''Parameters'''
*** '''accumulator''' (<code>any</code>): current aggregate, initially the neutral value
*** '''value''' (<code>any</code>): next mapped or partially reduced value
** '''Returns'''
*** '''accumulator''' (<code>any</code>): aggregate passed to the next reducer call or returned by the scan
* '''neutral''' (<code>any</code>): (optional) neutral element for the reduce phase, otherwise nil is assumed ''(optional)''
* '''neutral''' (<code>any</code>): (optional) neutral element for the reduce phase, otherwise nil is assumed ''(optional)''
* '''reduce2''' (<code>func(any, any)</code>): (optional) second stage reduce function that will apply a result of reduce to the neutral element/accumulator ''(optional)''
* '''reduce2''' (<code>func</code>): optional final reducer that combines the neutral value with the result produced by reduce ''(optional)''
** '''Parameters'''
*** '''accumulator''' (<code>any</code>): current aggregate, initially the neutral value
*** '''value''' (<code>any</code>): next mapped or partially reduced value
** '''Returns'''
*** '''accumulator''' (<code>any</code>): aggregate passed to the next reducer call or returned by the scan
* '''isOuter''' (<code>bool</code>): (optional) if true, in case of no hits, call map once anyway with NULL values ''(optional)''
* '''isOuter''' (<code>bool</code>): (optional) if true, in case of no hits, call map once anyway with NULL values ''(optional)''


Line 271: Line 318:
=== Returns ===
=== Returns ===


<code>any</code>
* '''value''' (<code>any</code>)


== scan_batch ==
== scan_batch ==
Line 284: Line 331:
* '''tx''' (<code>any</code>): transaction context to use for visibility and mutations; usually ((context &quot;session&quot;) &quot;__memcp_tx&quot;)
* '''tx''' (<code>any</code>): transaction context to use for visibility and mutations; usually ((context &quot;session&quot;) &quot;__memcp_tx&quot;)
* '''table''' (<code>table|list|recset</code>): table handle, query-local recset, or a list for temporary data
* '''table''' (<code>table|list|recset</code>): table handle, query-local recset, or a list for temporary data
* '''filterColumns''' (<code>list</code>): list of columns that are fed into filter; #0, #1, ... address batchdata slots
* '''filterColumns''' (<code>list&lt;string&gt;</code>): columns passed to filter; #0, #1, ... address batchdata slots
* '''filter''' (<code>func(columns:any...) -&gt; bool</code>): lambda function that decides whether a dataset is passed to the map phase
** '''column''' (<code>string</code>): column name passed to the corresponding callback parameter
* '''mapColumns''' (<code>list</code>): list of columns that are fed into map; #0, #1, ... address batchdata slots
* '''filter''' (<code>func</code>): lambda function that decides whether a dataset is passed to the map phase
* '''map''' (<code>func(columns:any...) -&gt; any</code>): lambda function to extract data from the dataset
** '''Parameters'''
*** '''columns''' (<code>any</code>): one value for each entry in the matching column list, in the same order ''(variadic)''
** '''Returns'''
*** '''result''' (<code>bool</code>): true when this table row and batch row proceed to map
* '''mapColumns''' (<code>list&lt;string&gt;</code>): columns passed to map; #0, #1, ... address batchdata slots
** '''column''' (<code>string</code>): column name passed to the corresponding callback parameter
* '''map''' (<code>func</code>): lambda function that extracts data from the table row and batch row
** '''Parameters'''
*** '''columns''' (<code>any</code>): one value for each entry in the matching column list, in the same order ''(variadic)''
** '''Returns'''
*** '''result''' (<code>any</code>): value passed to reduce or returned directly
* '''stride''' (<code>int</code>): number of batchdata entries per batch row
* '''stride''' (<code>int</code>): number of batchdata entries per batch row
* '''batchdata''' (<code>list</code>): flat batch buffer accessed via #N pseudo columns
* '''batchdata''' (<code>list&lt;any&gt;</code>): flat batch buffer accessed via #N pseudo columns
* '''reduce''' (<code>func(any, any)</code>): (optional) lambda function to aggregate the map results ''(optional)''
** '''slot''' (<code>any</code>): one batch value; every stride consecutive slots form a batch row
* '''reduce''' (<code>func</code>): optional lambda function that aggregates mapped values ''(optional)''
** '''Parameters'''
*** '''accumulator''' (<code>any</code>): current aggregate, initially the neutral value
*** '''value''' (<code>any</code>): next mapped or partially reduced value
** '''Returns'''
*** '''accumulator''' (<code>any</code>): aggregate passed to the next reducer call or returned by the scan
* '''neutral''' (<code>any</code>): (optional) neutral element for the reduce phase, otherwise nil is assumed ''(optional)''
* '''neutral''' (<code>any</code>): (optional) neutral element for the reduce phase, otherwise nil is assumed ''(optional)''
* '''reduce2''' (<code>func(any, any)</code>): (optional) second stage reduce function that will apply a result of reduce to the neutral element/accumulator ''(optional)''
* '''reduce2''' (<code>func</code>): optional final reducer that combines the neutral value with the result produced by reduce ''(optional)''
** '''Parameters'''
*** '''accumulator''' (<code>any</code>): current aggregate, initially the neutral value
*** '''value''' (<code>any</code>): next mapped or partially reduced value
** '''Returns'''
*** '''accumulator''' (<code>any</code>): aggregate passed to the next reducer call or returned by the scan
* '''isOuter''' (<code>bool</code>): (optional) if true, in case of no hits, call map once anyway with NULL values ''(optional)''
* '''isOuter''' (<code>bool</code>): (optional) if true, in case of no hits, call map once anyway with NULL values ''(optional)''


Line 298: Line 366:
=== Returns ===
=== Returns ===


<code>any</code>
* '''value''' (<code>any</code>)


== scan_order_batch_accept ==
== scan_order_batch_accept ==
Line 311: Line 379:
* '''tx''' (<code>any</code>): transaction context used consistently by the candidate scan and every batch filter operation; usually ((context &quot;session&quot;) &quot;__memcp_tx&quot;)
* '''tx''' (<code>any</code>): transaction context used consistently by the candidate scan and every batch filter operation; usually ((context &quot;session&quot;) &quot;__memcp_tx&quot;)
* '''table_or_recset''' (<code>table|recset</code>): base table or complete existing query-local RecSet from which ordered candidate batches are drawn
* '''table_or_recset''' (<code>table|recset</code>): base table or complete existing query-local RecSet from which ordered candidate batches are drawn
* '''batchFilter''' (<code>func(input_recset:recset) -&gt; recset</code>): function (lambda (input_recset) accepted_recset). It may naively narrow input_recset with scan_recset, or run arbitrary RecSet projections/search/ACL operations and project back. It must return a same-table, same-transaction subset of input_recset
* '''batchFilter''' (<code>func</code>): function (lambda (input_recset) accepted_recset). It may naively narrow input_recset with scan_recset, or run arbitrary RecSet projections/search/ACL operations and project back. It must return a same-table, same-transaction subset of input_recset
* '''sortcols''' (<code>list</code>): same as scan_order: columns or computed sort functions. Include a unique tie-breaker for a total repeatable order; use an empty list for greedy unsorted collection
** '''Parameters'''
* '''sortdirs''' (<code>list</code>): same as scan_order: one relation per sort column (&lt;, &gt; or collate relation); must also be empty when sortcols is empty
*** '''input_recset''' (<code>recset</code>)
** '''Returns'''
*** '''value''' (<code>recset</code>)
* '''sortcols''' (<code>list&lt;string|func(columns:any...) -&gt; any&gt;</code>): same as scan_order: columns or computed sort functions. Include a unique tie-breaker for a total repeatable order; use an empty list for greedy unsorted collection
** '''sort column''' (<code>string|func</code>): a column name, or a function of row-column values that returns the sortable value
*** '''Parameters'''
**** '''columns''' (<code>any</code>): column values used to compute the sort key ''(variadic)''
*** '''Returns'''
**** '''sort key''' (<code>any</code>): value compared at this sort position
* '''sortdirs''' (<code>list&lt;func(left:any, right:any) -&gt; bool&gt;</code>): same as scan_order: one relation per sort column; must also be empty when sortcols is empty
** '''direction''' (<code>func</code>): strict ordering relation such as &lt;, &gt;, or a collate relation
*** '''Parameters'''
**** '''left''' (<code>any</code>): left sort value
**** '''right''' (<code>any</code>): right sort value
*** '''Returns'''
**** '''ordered''' (<code>bool</code>): true when left belongs before right
* '''limitPartitionCols''' (<code>number</code>): reserved for scan_order signature compatibility; currently must be 0
* '''limitPartitionCols''' (<code>number</code>): reserved for scan_order signature compatibility; currently must be 0
* '''offset''' (<code>number</code>): number of batch-filter-accepted rows to skip; it is not the number of driver candidates already examined
* '''offset''' (<code>number</code>): number of batch-filter-accepted rows to skip; it is not the number of driver candidates already examined
* '''limit''' (<code>number</code>): finite maximum number of accepted rows passed to map; the initial candidate batch size is offset+limit and doubles for every subsequent batch
* '''limit''' (<code>number</code>): finite maximum number of accepted rows passed to map; the initial candidate batch size is offset+limit and doubles for every subsequent batch
* '''mapColumns''' (<code>list</code>): physical columns passed to map after filtering; pseudo columns are $update (update/delete current row), $recset_contains (row-bound RecSet membership), $set:<column>, $increment:<column>, and $invalidate:<column> (computed-column maintenance), plus NEW.<column> in trigger plans; $break is reserved for internal ORC convergence and must not implement SQL OFFSET/LIMIT, which belong in the native offset and limit arguments
* '''mapColumns''' (<code>list&lt;string&gt;</code>): physical columns passed to map after filtering; pseudo columns are $update (update/delete current row), $recset_contains (row-bound RecSet membership), $set:<column>, $increment:<column>, and $invalidate:<column> (computed-column maintenance), plus NEW.<column> in trigger plans; $break is reserved for internal ORC convergence and must not implement SQL OFFSET/LIMIT, which belong in the native offset and limit arguments
* '''map''' (<code>func(columns:any...) -&gt; any</code>): same map callback contract as scan_order; accepted record IDs are passed to its shard mapper in batches
** '''column''' (<code>string</code>): column name passed to the corresponding callback parameter
* '''reduce''' (<code>func(acc:any, val:any) -&gt; any</code>): optional serial reducer over mapped accepted rows, with the same accumulator contract as scan_order ''(optional)''
* '''map''' (<code>func</code>): same map callback contract as scan_order; accepted record IDs are passed to its shard mapper in batches
** '''Parameters'''
*** '''columns''' (<code>any</code>): one value for each entry in the matching column list, in the same order ''(variadic)''
** '''Returns'''
*** '''result''' (<code>any</code>): value passed to reduce or returned directly
* '''reduce''' (<code>func</code>): optional serial reducer over mapped accepted rows, with the same accumulator contract as scan_order ''(optional)''
** '''Parameters'''
*** '''accumulator''' (<code>any</code>): current aggregate, initially the neutral value
*** '''value''' (<code>any</code>): next mapped or partially reduced value
** '''Returns'''
*** '''accumulator''' (<code>any</code>): aggregate passed to the next reducer call or returned by the scan
* '''neutral''' (<code>any</code>): optional neutral element for reduce; defaults to nil ''(optional)''
* '''neutral''' (<code>any</code>): optional neutral element for reduce; defaults to nil ''(optional)''
* '''isOuter''' (<code>bool</code>): optional scan_order-compatible outer behavior: map one NULL row when no accepted row reaches map ''(optional)''
* '''isOuter''' (<code>bool</code>): optional scan_order-compatible outer behavior: map one NULL row when no accepted row reaches map ''(optional)''
Line 327: Line 420:
=== Returns ===
=== Returns ===


<code>any</code>
* '''value''' (<code>any</code>)


== scan_order ==
== scan_order ==
Line 340: Line 433:
* '''tx''' (<code>any</code>): transaction context to use for visibility and mutations; usually ((context &quot;session&quot;) &quot;__memcp_tx&quot;)
* '''tx''' (<code>any</code>): transaction context to use for visibility and mutations; usually ((context &quot;session&quot;) &quot;__memcp_tx&quot;)
* '''table''' (<code>table|list|recset</code>): table handle, query-local RecSet, or a list for temporary data
* '''table''' (<code>table|list|recset</code>): table handle, query-local RecSet, or a list for temporary data
* '''filterColumns''' (<code>list</code>): physical columns passed to filter before map/reduce; $recset_contains supplies a row-bound RecSet membership closure
* '''filterColumns''' (<code>list&lt;string&gt;</code>): physical columns passed to filter before map/reduce; $recset_contains supplies a row-bound RecSet membership closure
* '''filter''' (<code>func(columns:any...) -&gt; bool</code>): lambda function that decides whether a dataset is passed to the map phase. You can use any column of that table as lambda parameter. You should structure your lambda with an (and) at the root element. Every equal? &lt; &gt; &lt;= &gt;= will possibly translated to an indexed scan
** '''column''' (<code>string</code>): column name passed to the corresponding callback parameter
* '''sortcols''' (<code>list</code>): list of columns to sort. Each column is either a string to point to an existing column or a func(cols...)-&gt;any to compute a sortable value
* '''filter''' (<code>func</code>): lambda function that decides whether a dataset is passed to the map phase. Equality and range comparisons may be translated into indexed scans
* '''sortdirs''' (<code>list</code>): list of column directions to sort. Must be same length as sortcols. &lt; means ascending, &gt; means descending, (collate ...) will add collations
** '''Parameters'''
*** '''columns''' (<code>any</code>): one value for each entry in the matching column list, in the same order ''(variadic)''
** '''Returns'''
*** '''result''' (<code>bool</code>): true when the row proceeds to ordering and map
* '''sortcols''' (<code>list&lt;string|func(columns:any...) -&gt; any&gt;</code>): columns used for ordering; each entry corresponds to one relation in sortdirs
** '''sort column''' (<code>string|func</code>): a column name, or a function of row-column values that returns the sortable value
*** '''Parameters'''
**** '''columns''' (<code>any</code>): column values used to compute the sort key ''(variadic)''
*** '''Returns'''
**** '''sort key''' (<code>any</code>): value compared at this sort position
* '''sortdirs''' (<code>list&lt;func(left:any, right:any) -&gt; bool&gt;</code>): one ordering relation per entry in sortcols; &lt; is ascending and &gt; is descending
** '''direction''' (<code>func</code>): strict ordering relation such as &lt;, &gt;, or a collate relation
*** '''Parameters'''
**** '''left''' (<code>any</code>): left sort value
**** '''right''' (<code>any</code>): right sort value
*** '''Returns'''
**** '''ordered''' (<code>bool</code>): true when left belongs before right
* '''limitPartitionCols''' (<code>number</code>): number of leading sort columns that form the partition key for per-partition offset/limit. 0 (default) means global offset/limit.
* '''limitPartitionCols''' (<code>number</code>): number of leading sort columns that form the partition key for per-partition offset/limit. 0 (default) means global offset/limit.
* '''offset''' (<code>number</code>): number of globally ordered, filter-accepted items to skip before map; apply SQL OFFSET here rather than in map
* '''offset''' (<code>number</code>): number of globally ordered, filter-accepted items to skip before map; apply SQL OFFSET here rather than in map
* '''limit''' (<code>number</code>): maximum globally ordered, filter-accepted items passed to map; -1 means unlimited; apply SQL LIMIT here so shard-local Top-K and the global merge can brake early
* '''limit''' (<code>number</code>): maximum globally ordered, filter-accepted items passed to map; -1 means unlimited; apply SQL LIMIT here so shard-local Top-K and the global merge can brake early
* '''mapColumns''' (<code>list</code>): physical columns passed to map after filtering; pseudo columns are $update (update/delete current row), $recset_contains (row-bound RecSet membership), $set:<column>, $increment:<column>, and $invalidate:<column> (computed-column maintenance), plus NEW.<column> in trigger plans; $break is reserved for internal ORC convergence and must not implement SQL OFFSET/LIMIT, which belong in the native offset and limit arguments
* '''mapColumns''' (<code>list&lt;string&gt;</code>): physical columns passed to map after filtering; pseudo columns are $update (update/delete current row), $recset_contains (row-bound RecSet membership), $set:<column>, $increment:<column>, and $invalidate:<column> (computed-column maintenance), plus NEW.<column> in trigger plans; $break is reserved for internal ORC convergence and must not implement SQL OFFSET/LIMIT, which belong in the native offset and limit arguments
* '''map''' (<code>func(columns:any...) -&gt; any</code>): lambda function to extract data from the dataset. You can use any column of that table as lambda parameter. You can return a value you want to extract and pass to reduce, but you can also directly call insert, print or resultrow functions. If you declare a parameter named '<math display="inline">update', this variable will hold a function that you can use to delete or update a row. Call (</math>update) to delete the dataset, call ($update '(&quot;field1&quot; value1 &quot;field2&quot; value2)) to update certain columns.
** '''column''' (<code>string</code>): column name passed to the corresponding callback parameter
* '''reduce''' (<code>func(acc:any, val:any) -&gt; any</code>): (optional) lambda function to aggregate the map results. It takes two parameters (a b) where a is the accumulator and b the new value. The accumulator for the first reduce call is the neutral element. The return value will be the accumulator input for the next reduce call. There are two reduce phases: shard-local and shard-collect. In the shard-local phase, a starts with neutral and b is fed with the return values of each map call. In the shard-collect phase, a starts with neutral and b is fed with the result of each shard-local pass. ''(optional)''
* '''map''' (<code>func</code>): lambda function that extracts or produces one value from each accepted row
** '''Parameters'''
*** '''columns''' (<code>any</code>): one value for each entry in the matching column list, in the same order ''(variadic)''
** '''Returns'''
*** '''result''' (<code>any</code>): value passed to reduce or returned directly
* '''reduce''' (<code>func</code>): optional serial aggregation function over mapped values ''(optional)''
** '''Parameters'''
*** '''accumulator''' (<code>any</code>): current aggregate, initially the neutral value
*** '''value''' (<code>any</code>): next mapped or partially reduced value
** '''Returns'''
*** '''accumulator''' (<code>any</code>): aggregate passed to the next reducer call or returned by the scan
* '''neutral''' (<code>any</code>): (optional) neutral element for the reduce phase, otherwise nil is assumed ''(optional)''
* '''neutral''' (<code>any</code>): (optional) neutral element for the reduce phase, otherwise nil is assumed ''(optional)''
* '''isOuter''' (<code>bool</code>): (optional) if true, in case of no hits, call map once anyway with NULL values ''(optional)''
* '''isOuter''' (<code>bool</code>): (optional) if true, in case of no hits, call map once anyway with NULL values ''(optional)''
* '''notFoundValue''' (<code>any</code>): (optional) result for no hits when isOuter is false; defaults to neutral ''(optional)''
* '''notFoundValue''' (<code>any</code>): (optional) result for no hits when isOuter is false; defaults to neutral ''(optional)''
* '''postOrderFilterColumns''' (<code>list</code>): (optional) columns for a predicate evaluated in global order before OFFSET/LIMIT are counted; use for expensive acceptance checks that cannot participate in index boundaries ''(optional)''
* '''postOrderFilterColumns''' (<code>list&lt;string&gt;</code>): optional columns for a predicate evaluated in global order before OFFSET/LIMIT are counted; use for expensive acceptance checks that cannot participate in index boundaries ''(optional)''
* '''postOrderFilter''' (<code>func(columns:any...) -&gt; bool</code>): (optional) late acceptance predicate. Rejected rows do not count toward OFFSET/LIMIT and never reach map. SQL plans use this instead of callback-driven $break control flow ''(optional)''
** '''column''' (<code>string</code>): column name passed to the corresponding callback parameter
* '''postOrderFilter''' (<code>func</code>): optional late acceptance predicate. Rejected rows do not count toward OFFSET/LIMIT and never reach map ''(optional)''
** '''Parameters'''
*** '''columns''' (<code>any</code>): one value for each entry in the matching column list, in the same order ''(variadic)''
** '''Returns'''
*** '''result''' (<code>bool</code>): true when the ordered row counts toward OFFSET/LIMIT and reaches map


<span id="returns-17"></span>
<span id="returns-17"></span>
=== Returns ===
=== Returns ===


<code>any</code>
* '''value''' (<code>any</code>)


== scan_order_multi ==
== scan_order_multi ==
Line 371: Line 495:


* '''tx''' (<code>any</code>): transaction context
* '''tx''' (<code>any</code>): transaction context
* '''tables''' (<code>list</code>): list of table handles
* '''tables''' (<code>list&lt;table|recset&gt;</code>): scan sources; all per-table lists must have this length
* '''filterColumns''' (<code>list</code>): list of filter column lists, one per table
** '''source''' (<code>table|recset</code>): base table or query-local record set for one input stream
* '''filterFns''' (<code>list</code>): list of filter lambdas, one per table
* '''filterColumns''' (<code>list&lt;list&lt;string&gt;&gt;</code>): filter column lists, one per table
* '''sortcols''' (<code>list</code>): list of sort column lists, one per table
** '''table filter columns''' (<code>list&lt;string&gt;</code>): columns supplied to the matching filterFns entry
* '''sortdirs''' (<code>list</code>): list of sort direction comparators (shared)
*** '''column''' (<code>string</code>): column name in the corresponding table
* '''perTableOffset''' (<code>list</code>): per-table offset (list of int; -1 disables)
* '''filterFns''' (<code>list&lt;func(columns:any...) -&gt; bool&gt;</code>): filter lambdas, one per table
* '''perTableLimit''' (<code>list</code>): per-table limit (list of int; -1 disables)
** '''table filter''' (<code>func</code>): predicate for the corresponding table and filterColumns entry
*** '''Parameters'''
**** '''columns''' (<code>any</code>): one value for each entry in the matching column list, in the same order ''(variadic)''
*** '''Returns'''
**** '''result''' (<code>bool</code>): true when the row enters that table's ordered stream
* '''sortcols''' (<code>list&lt;list&lt;string|func(columns:any...) -&gt; any&gt;&gt;</code>): sort column lists, one per table; every inner list must match sortdirs in length and result domains
** '''table sort columns''' (<code>list&lt;string|func(columns:any...) -&gt; any&gt;</code>): sort expressions for the corresponding table
*** '''sort column''' (<code>string|func</code>): a column name, or a function of row-column values that returns the sortable value
**** '''Parameters'''
***** '''columns''' (<code>any</code>): column values used to compute the sort key ''(variadic)''
**** '''Returns'''
***** '''sort key''' (<code>any</code>): value compared at this sort position
* '''sortdirs''' (<code>list&lt;func(left:any, right:any) -&gt; bool&gt;</code>): shared ordering relations used for every table stream and for the outer merge
** '''direction''' (<code>func</code>): strict ordering relation such as &lt;, &gt;, or a collate relation
*** '''Parameters'''
**** '''left''' (<code>any</code>): left sort value
**** '''right''' (<code>any</code>): right sort value
*** '''Returns'''
**** '''ordered''' (<code>bool</code>): true when left belongs before right
* '''perTableOffset''' (<code>list&lt;int&gt;|nil</code>): optional per-table offsets; nil disables all per-table offsets
** '''offset''' (<code>int</code>): rows skipped in the corresponding table before the outer merge; -1 disables the offset
* '''perTableLimit''' (<code>list&lt;int&gt;|nil</code>): optional per-table limits; nil disables all per-table limits
** '''limit''' (<code>int</code>): maximum rows retained from the corresponding table before the outer merge; -1 disables the limit
* '''limitPartitionCols''' (<code>number</code>): number of leading sort columns forming partition key
* '''limitPartitionCols''' (<code>number</code>): number of leading sort columns forming partition key
* '''offset''' (<code>number</code>): number of items to skip (global)
* '''offset''' (<code>number</code>): number of items to skip (global)
* '''limit''' (<code>number</code>): max number of items to read (global; -1 = unlimited)
* '''limit''' (<code>number</code>): max number of items to read (global; -1 = unlimited)
* '''mapColumns''' (<code>list</code>): list of map column lists, one per table
* '''mapColumns''' (<code>list&lt;list&lt;string&gt;&gt;</code>): map column lists, one per table
* '''mapFns''' (<code>list</code>): list of map lambdas, one per table
** '''table map columns''' (<code>list&lt;string&gt;</code>): columns supplied to the matching mapFns entry
* '''reduce''' (<code>func</code>): (optional) aggregation function ''(optional)''
*** '''column''' (<code>string</code>): column name in the corresponding table
* '''mapFns''' (<code>list&lt;func(columns:any...) -&gt; any&gt;</code>): map lambdas, one per table
** '''table map''' (<code>func</code>): mapper for the corresponding table and mapColumns entry
*** '''Parameters'''
**** '''columns''' (<code>any</code>): one value for each entry in the matching column list, in the same order ''(variadic)''
*** '''Returns'''
**** '''result''' (<code>any</code>): value inserted into the merged stream and passed to reduce
* '''reduce''' (<code>func</code>): optional aggregation function over mapped values from the merged stream ''(optional)''
** '''Parameters'''
*** '''accumulator''' (<code>any</code>): current aggregate, initially the neutral value
*** '''value''' (<code>any</code>): next mapped or partially reduced value
** '''Returns'''
*** '''accumulator''' (<code>any</code>): aggregate passed to the next reducer call or returned by the scan
* '''neutral''' (<code>any</code>): (optional) neutral element for reduce ''(optional)''
* '''neutral''' (<code>any</code>): (optional) neutral element for reduce ''(optional)''
* '''isOuter''' (<code>bool</code>): (optional) if true, emit null row when no hits ''(optional)''
* '''isOuter''' (<code>bool</code>): (optional) if true, emit null row when no hits ''(optional)''
Line 391: Line 549:
=== Returns ===
=== Returns ===


<code>any</code>
* '''value''' (<code>any</code>)


== createdatabase ==
== createdatabase ==
Line 408: Line 566:
=== Returns ===
=== Returns ===


<code>bool</code>
* '''value''' (<code>bool</code>)


== dropdatabase ==
== dropdatabase ==
Line 425: Line 583:
=== Returns ===
=== Returns ===


<code>bool</code>
* '''value''' (<code>bool</code>)


== checktablemaintenance ==
== checktablemaintenance ==
Line 436: Line 594:
=== Parameters ===
=== Parameters ===


* '''schema''' (<code>string</code>):
* '''schema''' (<code>string</code>)
* '''table''' (<code>string</code>):
* '''table''' (<code>string</code>)
* '''operation''' (<code>string</code>):
* '''operation''' (<code>string</code>)


<span id="returns-21"></span>
<span id="returns-21"></span>
=== Returns ===
=== Returns ===


<code>bool</code>
* '''value''' (<code>bool</code>)


== maintenance_capabilities ==
== maintenance_capabilities ==
Line 454: Line 612:
=== Parameters ===
=== Parameters ===


* '''schema''' (<code>string</code>):
* '''schema''' (<code>string</code>)
* '''table''' (<code>string</code>): ''(optional)''
* '''table''' (<code>string</code>) ''(optional)''


<span id="returns-22"></span>
<span id="returns-22"></span>
=== Returns ===
=== Returns ===


<code>list</code>
* '''value''' (<code>list</code>)


== createtable ==
== createtable ==
Line 473: Line 631:
* '''schema''' (<code>string</code>): name of the existing database that will contain the table
* '''schema''' (<code>string</code>): name of the existing database that will contain the table
* '''table''' (<code>string</code>): name of the table to create
* '''table''' (<code>string</code>): name of the table to create
* '''cols''' (<code>list</code>): column and constraint definitions: (&quot;column&quot; name type dimensions typeparams), (&quot;unique&quot; name columns), or (&quot;foreign&quot; name local_columns referenced_table referenced_columns update_mode delete_mode). dimensions is a list of integer type dimensions. typeparams is an alternating key/value list supporting primary (bool), unique (bool), auto_increment (bool), null (bool), default (any), default_expression (string), update (expression), comment (string), collate (string), temp (bool), filtercols (string list), filter (function), sortcols (string list), sortdirs (bool list), partitioncount (integer), mapcols (string list), mapfn (function), reducefn (function), and reduceinit (any). Column lists are string lists; foreign-key modes are restrict, cascade, or set null
* '''cols''' (<code>list&lt;list&gt;</code>): column and constraint definitions
* '''options''' (<code>list</code>): alternating key/value list; supported keys are engine (safe, logged, sloppy, memory, or cache), collation (string), charset (string), comment (string), auto_increment (non-negative integer), and oninit (closed zero-argument function run synchronously once per data generation; concurrent if-not-exists callers wait for it, and memory/cache tables persist the callback so the first idempotent createtable after restart repopulates their empty data)
** '''definition''' (<code>list</code>): one of (&quot;column&quot; name type dimensions typeparams), (&quot;unique&quot; name columns), or (&quot;foreign&quot; name local_columns referenced_table referenced_columns update_mode delete_mode). Column lists contain strings; foreign-key modes are restrict, cascade, or set null. A column definition's dimensions contains integers and its typeparams uses the same fields documented by createcolumn options
* '''options''' (<code>list|assoc</code>): table options as an alternating key/value list
** '''auto_increment''' (<code>int</code>): first automatically assigned value; must be non-negative
** '''charset''' (<code>string</code>): default character set name
** '''collation''' (<code>string</code>): default collation name
** '''comment''' (<code>string</code>): user-visible table comment
** '''engine''' (<code>string</code>): storage engine: safe, logged, sloppy, memory, or cache
** '''oninit''' (<code>func</code>): closed zero-argument initializer run synchronously once per data generation; concurrent if-not-exists callers wait for completion
*** '''Returns'''
**** '''result''' (<code>any</code>): ignored initializer result
* '''ifnotexists''' (<code>bool</code>): when true, return false instead of failing if the table exists; if another caller is still creating it, wait for that caller's after-create-table initialization before returning false ''(optional)''
* '''ifnotexists''' (<code>bool</code>): when true, return false instead of failing if the table exists; if another caller is still creating it, wait for that caller's after-create-table initialization before returning false ''(optional)''


Line 480: Line 647:
=== Returns ===
=== Returns ===


<code>bool</code>
* '''value''' (<code>bool</code>): true when this call created and initialized the table, false when ifnotexists reused an initialized table


== createcolumn ==
== createcolumn ==
Line 491: Line 658:
=== Parameters ===
=== Parameters ===


* '''table''' (<code>table</code>):
* '''table''' (<code>table</code>)
* '''colname''' (<code>string</code>): name of the new column
* '''colname''' (<code>string</code>): name of the new column
* '''type''' (<code>string</code>): name of the basetype
* '''type''' (<code>string</code>): name of the basetype
* '''dimensions''' (<code>list</code>): dimensions of the type (e.g. for decimal)
* '''dimensions''' (<code>list&lt;int&gt;</code>): dimensions of the type, for example precision and scale for decimal
* '''options''' (<code>list</code>): assoc list: primary, unique, auto_increment, null, comment, default, default_expression, collate; ORC: sortcols, sortdirs, partitioncount, mapcols, mapfn, reducefn, reduceinit
** '''dimension''' (<code>int</code>): one type-specific dimension
* '''computorCols''' (<code>list</code>): list of columns that is passed into params of computor ''(optional)''
* '''options''' (<code>list|assoc</code>): column properties and computed-column configuration as an alternating key/value list
* '''computor''' (<code>func(columns:any...) -&gt; any</code>): lambda expression that can take other column values and computes the value of that column ''(optional)''
** '''auto_increment''' (<code>bool</code>): assign increasing values automatically
** '''collate''' (<code>string</code>): collation used for this column
** '''comment''' (<code>string</code>): user-visible column comment
** '''default''' (<code>any</code>): literal value used when an insert omits the column
** '''default_expression''' (<code>string</code>): expression evaluated when an insert omits the column
** '''filter''' (<code>func</code>): predicate limiting which rows are computed
*** '''Parameters'''
**** '''columns''' (<code>any</code>): one value for each entry in the matching column list, in the same order ''(variadic)''
*** '''Returns'''
**** '''result''' (<code>bool</code>): true when the row should be computed
** '''filtercols''' (<code>list&lt;string&gt;</code>): columns supplied to filter before computing a value
*** '''column''' (<code>string</code>): column name passed to the corresponding callback parameter
** '''mapcols''' (<code>list&lt;string&gt;</code>): columns supplied to mapfn for ordered-reduce computation
*** '''column''' (<code>string</code>): column name passed to the corresponding callback parameter
** '''mapfn''' (<code>func</code>): maps one source row into a value for reducefn
*** '''Parameters'''
**** '''columns''' (<code>any</code>): one value for each entry in the matching column list, in the same order ''(variadic)''
*** '''Returns'''
**** '''result''' (<code>any</code>): value passed to reducefn
** '''null''' (<code>bool</code>): whether the column accepts nil values
** '''partitioncount''' (<code>int</code>): number of leading sort columns that define independent reducer partitions
** '''primary''' (<code>bool</code>): whether this column belongs to the primary key
** '''reducefn''' (<code>func</code>): combines ordered mapped values into the computed-column aggregate ''(optional)''
*** '''Parameters'''
**** '''accumulator''' (<code>any</code>): current aggregate, initially the neutral value
**** '''value''' (<code>any</code>): next mapped or partially reduced value
*** '''Returns'''
**** '''accumulator''' (<code>any</code>): aggregate passed to the next reducer call or returned by the scan
** '''reduceinit''' (<code>any</code>): initial accumulator supplied to reducefn
** '''sortcols''' (<code>list&lt;string|func(columns:any...) -&gt; any&gt;</code>): columns or expressions defining ordered-reduce input order
*** '''sort column''' (<code>string|func</code>): a column name, or a function of row-column values that returns the sortable value
**** '''Parameters'''
***** '''columns''' (<code>any</code>): column values used to compute the sort key ''(variadic)''
**** '''Returns'''
***** '''sort key''' (<code>any</code>): value compared at this sort position
** '''sortdirs''' (<code>list&lt;func(left:any, right:any) -&gt; bool&gt;</code>): one ordering relation for every sortcols entry
*** '''direction''' (<code>func</code>): strict ordering relation such as &lt;, &gt;, or a collate relation
**** '''Parameters'''
***** '''left''' (<code>any</code>): left sort value
***** '''right''' (<code>any</code>): right sort value
**** '''Returns'''
***** '''ordered''' (<code>bool</code>): true when left belongs before right
** '''temp''' (<code>bool</code>): whether this is a query-local temporary computed column
** '''unique''' (<code>bool</code>): whether values must be unique
** '''update''' (<code>any</code>): expression evaluated when a row is updated
* '''computorCols''' (<code>list&lt;string&gt;</code>): columns passed to computor in this order ''(optional)''
** '''column''' (<code>string</code>): column name passed to the corresponding callback parameter
* '''computor''' (<code>func</code>): lambda expression that computes this column from the values selected by computorCols ''(optional)''
** '''Parameters'''
*** '''columns''' (<code>any</code>): one value for each entry in the matching column list, in the same order ''(variadic)''
** '''Returns'''
*** '''result''' (<code>any</code>): computed column value


<span id="returns-24"></span>
<span id="returns-24"></span>
=== Returns ===
=== Returns ===


<code>bool</code>
* '''value''' (<code>bool</code>)


== createkey ==
== createkey ==
Line 513: Line 731:
=== Parameters ===
=== Parameters ===


* '''table''' (<code>table</code>):
* '''table''' (<code>table</code>)
* '''keyname''' (<code>string</code>): name of the new key
* '''keyname''' (<code>string</code>): name of the new key
* '''unique''' (<code>bool</code>): whether the key is unique
* '''unique''' (<code>bool</code>): whether the key is unique
Line 521: Line 739:
=== Returns ===
=== Returns ===


<code>bool</code>
* '''value''' (<code>bool</code>)
 
== dropkey ==
 
drops a named unique key from a table
 
'''Allowed number of parameters:''' 2–2
 
<span id="parameters-26"></span>
=== Parameters ===
 
* '''table''' (<code>table</code>)
* '''keyname''' (<code>string</code>): name of the unique key
 
<span id="returns-26"></span>
=== Returns ===
 
* '''value''' (<code>bool</code>)


== createforeignkey ==
== createforeignkey ==
Line 529: Line 764:
'''Allowed number of parameters:''' 7–7
'''Allowed number of parameters:''' 7–7


<span id="parameters-26"></span>
<span id="parameters-27"></span>
=== Parameters ===
=== Parameters ===


* '''table1''' (<code>table</code>):
* '''table1''' (<code>table</code>)
* '''keyname''' (<code>string</code>): name of the new key
* '''keyname''' (<code>string</code>): name of the new key
* '''columns1''' (<code>list</code>): list of columns to include
* '''columns1''' (<code>list</code>): list of columns to include
* '''table2''' (<code>table</code>):
* '''table2''' (<code>table</code>)
* '''columns2''' (<code>list</code>): list of columns to include
* '''columns2''' (<code>list</code>): list of columns to include
* '''updatemode''' (<code>string</code>): restrict|cascade|set null
* '''updatemode''' (<code>string</code>): restrict|cascade|set null
* '''deletemode''' (<code>string</code>): restrict|cascade|set null
* '''deletemode''' (<code>string</code>): restrict|cascade|set null


<span id="returns-26"></span>
<span id="returns-27"></span>
=== Returns ===
=== Returns ===


<code>bool</code>
* '''value''' (<code>bool</code>)


== shardcolumn ==
== shardcolumn ==
Line 551: Line 786:
'''Allowed number of parameters:''' 2–3
'''Allowed number of parameters:''' 2–3


<span id="parameters-27"></span>
<span id="parameters-28"></span>
=== Parameters ===
=== Parameters ===


* '''table''' (<code>table</code>):
* '''table''' (<code>table</code>)
* '''colname''' (<code>string</code>): name of the column
* '''colname''' (<code>string</code>): name of the column
* '''numpartitions''' (<code>number</code>): number of partitions; optional. leave 0 if you want to detect the partiton number automatically or copy the partition schema of the table ''(optional)''
* '''numpartitions''' (<code>number</code>): number of partitions; optional. leave 0 if you want to detect the partiton number automatically or copy the partition schema of the table ''(optional)''


<span id="returns-27"></span>
<span id="returns-28"></span>
=== Returns ===
=== Returns ===


<code>list</code>
* '''value''' (<code>list</code>)


== partitiontable ==
== partitiontable ==
Line 569: Line 804:
'''Allowed number of parameters:''' 2–2
'''Allowed number of parameters:''' 2–2


<span id="parameters-28"></span>
<span id="parameters-29"></span>
=== Parameters ===
=== Parameters ===


* '''table''' (<code>table</code>):
* '''table''' (<code>table</code>)
* '''columns''' (<code>list</code>): associative list of string -&gt; list representing column name -&gt; pivots. You can compute pivots by (shardcolumn ...)
* '''columns''' (<code>list</code>): associative list of string -&gt; list representing column name -&gt; pivots. You can compute pivots by (shardcolumn ...)


<span id="returns-28"></span>
<span id="returns-29"></span>
=== Returns ===
=== Returns ===


<code>bool</code>
* '''value''' (<code>bool</code>)


== altertable ==
== altertable ==
Line 586: Line 821:
'''Allowed number of parameters:''' 3–3
'''Allowed number of parameters:''' 3–3


<span id="parameters-29"></span>
<span id="parameters-30"></span>
=== Parameters ===
=== Parameters ===


* '''table''' (<code>table</code>):
* '''table''' (<code>table</code>)
* '''operation''' (<code>string</code>): one of owner|drop|engine|collation|auto_increment
* '''operation''' (<code>string</code>): one of owner|drop|engine|collation|auto_increment
* '''parameter''' (<code>any</code>): name of the column to drop or value of the parameter
* '''parameter''' (<code>any</code>): name of the column to drop or value of the parameter


<span id="returns-29"></span>
<span id="returns-30"></span>
=== Returns ===
=== Returns ===


<code>bool</code>
* '''value''' (<code>bool</code>)


== altercolumn ==
== altercolumn ==
Line 604: Line 839:
'''Allowed number of parameters:''' 4–4
'''Allowed number of parameters:''' 4–4


<span id="parameters-30"></span>
<span id="parameters-31"></span>
=== Parameters ===
=== Parameters ===


* '''table''' (<code>table</code>):
* '''table''' (<code>table</code>)
* '''column''' (<code>string</code>): name of the column
* '''column''' (<code>string</code>): name of the column
* '''operation''' (<code>string</code>): one of drop|type|collation|auto_increment|comment
* '''operation''' (<code>string</code>): one of drop|type|collation|auto_increment|comment
* '''parameter''' (<code>any</code>): name of the column to drop or value of the parameter
* '''parameter''' (<code>any</code>): name of the column to drop or value of the parameter


<span id="returns-30"></span>
<span id="returns-31"></span>
=== Returns ===
=== Returns ===


<code>bool</code>
* '''value''' (<code>bool</code>)


== droptable ==
== droptable ==
Line 623: Line 858:
'''Allowed number of parameters:''' 2–3
'''Allowed number of parameters:''' 2–3


<span id="parameters-31"></span>
<span id="parameters-32"></span>
=== Parameters ===
=== Parameters ===


* '''schema''' (<code>string</code>):
* '''schema''' (<code>string</code>)
* '''table''' (<code>string</code>):
* '''table''' (<code>string</code>)
* '''ifexists''' (<code>bool</code>): if true, don't throw an error if it already exists ''(optional)''
* '''ifexists''' (<code>bool</code>): if true, don't throw an error if it already exists ''(optional)''


<span id="returns-31"></span>
<span id="returns-32"></span>
=== Returns ===
=== Returns ===


<code>bool</code>
* '''value''' (<code>bool</code>)


== dropcolumn ==
== dropcolumn ==
Line 641: Line 876:
'''Allowed number of parameters:''' 2–2
'''Allowed number of parameters:''' 2–2


<span id="parameters-32"></span>
<span id="parameters-33"></span>
=== Parameters ===
=== Parameters ===


* '''table''' (<code>table</code>):
* '''table''' (<code>table</code>)
* '''column''' (<code>string</code>): name of the column to drop
* '''column''' (<code>string</code>): name of the column to drop


<span id="returns-32"></span>
<span id="returns-33"></span>
=== Returns ===
=== Returns ===


<code>bool</code>
* '''value''' (<code>bool</code>)


== migratedropcolumn ==
== migratedropcolumn ==
Line 658: Line 893:
'''Allowed number of parameters:''' 2–2
'''Allowed number of parameters:''' 2–2


<span id="parameters-33"></span>
<span id="parameters-34"></span>
=== Parameters ===
=== Parameters ===


* '''table''' (<code>table</code>):
* '''table''' (<code>table</code>)
* '''column''' (<code>string</code>): legacy column name
* '''column''' (<code>string</code>): legacy column name


<span id="returns-33"></span>
<span id="returns-34"></span>
=== Returns ===
=== Returns ===


<code>bool</code>
* '''value''' (<code>bool</code>)


== invalidatecolumn ==
== invalidatecolumn ==
Line 675: Line 910:
'''Allowed number of parameters:''' 2–2
'''Allowed number of parameters:''' 2–2


<span id="parameters-34"></span>
<span id="parameters-35"></span>
=== Parameters ===
=== Parameters ===


* '''table''' (<code>table</code>):
* '''table''' (<code>table</code>)
* '''column''' (<code>string</code>): name of the computed column
* '''column''' (<code>string</code>): name of the computed column


<span id="returns-34"></span>
<span id="returns-35"></span>
=== Returns ===
=== Returns ===


<code>bool</code>
* '''value''' (<code>bool</code>)


== invalidateorc ==
== invalidateorc ==
Line 692: Line 927:
'''Allowed number of parameters:''' 3–3
'''Allowed number of parameters:''' 3–3


<span id="parameters-35"></span>
<span id="parameters-36"></span>
=== Parameters ===
=== Parameters ===


* '''table''' (<code>table</code>):
* '''table''' (<code>table</code>)
* '''column''' (<code>string</code>): name of the ORC column
* '''column''' (<code>string</code>): name of the ORC column
* '''sortkeys''' (<code>list</code>): composite sort key values from which to invalidate
* '''sortkeys''' (<code>list</code>): composite sort key values from which to invalidate


<span id="returns-35"></span>
<span id="returns-36"></span>
=== Returns ===
=== Returns ===


<code>bool</code>
* '''value''' (<code>bool</code>)


== register_keytable_cleanup ==
== register_keytable_cleanup ==
Line 710: Line 945:
'''Allowed number of parameters:''' 4–4
'''Allowed number of parameters:''' 4–4


<span id="parameters-36"></span>
<span id="parameters-37"></span>
=== Parameters ===
=== Parameters ===


* '''base_table''' (<code>table</code>):
* '''base_table''' (<code>table</code>)
* '''kt_table''' (<code>table</code>):
* '''kt_table''' (<code>table</code>)
* '''tblvar''' (<code>string</code>): table alias used in scan column prefixes
* '''tblvar''' (<code>string</code>): table alias used in scan column prefixes
* '''key_pairs''' (<code>list</code>): list of (base_col kt_col) pairs
* '''key_pairs''' (<code>list</code>): list of (base_col kt_col) pairs


<span id="returns-36"></span>
<span id="returns-37"></span>
=== Returns ===
=== Returns ===


<code>bool</code>
* '''value''' (<code>bool</code>)


== initialize_cache_table ==
== initialize_cache_table ==
Line 729: Line 964:
'''Allowed number of parameters:''' 5–6
'''Allowed number of parameters:''' 5–6


<span id="parameters-37"></span>
<span id="parameters-38"></span>
=== Parameters ===
=== Parameters ===


* '''transaction''' (<code>any</code>): explicit transaction context carrying query-session ownership
* '''transaction''' (<code>any</code>): explicit transaction context carrying query-session ownership
* '''table''' (<code>table</code>):
* '''table''' (<code>table</code>)
* '''source_tables''' (<code>list</code>):
* '''source_tables''' (<code>list</code>)
* '''register_maintenance''' (<code>func</code>):
* '''register_maintenance''' (<code>func</code>)
* '''initializer''' (<code>func</code>):
** '''Returns'''
*** '''value''' (<code>any</code>)
* '''initializer''' (<code>func</code>)
** '''Returns'''
*** '''value''' (<code>any</code>)
* '''finalizer''' (<code>func</code>): optional zero-argument finalizer run under the same source-table locks after initialization ''(optional)''
* '''finalizer''' (<code>func</code>): optional zero-argument finalizer run under the same source-table locks after initialization ''(optional)''
** '''Returns'''
*** '''value''' (<code>any</code>)


<span id="returns-37"></span>
<span id="returns-38"></span>
=== Returns ===
=== Returns ===


<code>bool</code>
* '''value''' (<code>bool</code>)


== touch_keytable ==
== touch_keytable ==
Line 750: Line 991:
'''Allowed number of parameters:''' 1–1
'''Allowed number of parameters:''' 1–1


<span id="parameters-38"></span>
<span id="parameters-39"></span>
=== Parameters ===
=== Parameters ===


* '''table''' (<code>table</code>):
* '''table''' (<code>table</code>)


<span id="returns-38"></span>
<span id="returns-39"></span>
=== Returns ===
=== Returns ===


<code>bool</code>
* '''value''' (<code>bool</code>)


== locktables ==
== locktables ==
Line 766: Line 1,007:
'''Allowed number of parameters:''' 1–1
'''Allowed number of parameters:''' 1–1


<span id="parameters-39"></span>
<span id="parameters-40"></span>
=== Parameters ===
=== Parameters ===


* '''locks''' (<code>list</code>): flat list of schema, table, write? triples
* '''locks''' (<code>list</code>): flat list of schema, table, write? triples


<span id="returns-39"></span>
<span id="returns-40"></span>
=== Returns ===
=== Returns ===


<code>bool</code>
* '''value''' (<code>bool</code>)


== unlocktables ==
== unlocktables ==
Line 782: Line 1,023:
'''Allowed number of parameters:''' 0–0
'''Allowed number of parameters:''' 0–0


<span id="parameters-40"></span>
<span id="parameters-41"></span>
=== Parameters ===
=== Parameters ===


''This function has no parameters.''
''This function has no parameters.''


<span id="returns-40"></span>
<span id="returns-41"></span>
=== Returns ===
=== Returns ===


<code>bool</code>
* '''value''' (<code>bool</code>)


== get_fk_target ==
== get_fk_target ==
Line 798: Line 1,039:
'''Allowed number of parameters:''' 2–2
'''Allowed number of parameters:''' 2–2


<span id="parameters-41"></span>
<span id="parameters-42"></span>
=== Parameters ===
=== Parameters ===


* '''table''' (<code>table</code>):
* '''table''' (<code>table</code>)
* '''column''' (<code>string</code>): column name
* '''column''' (<code>string</code>): column name


<span id="returns-41"></span>
<span id="returns-42"></span>
=== Returns ===
=== Returns ===


<code>any</code>
* '''value''' (<code>any</code>)


== renametable ==
== renametable ==
Line 815: Line 1,056:
'''Allowed number of parameters:''' 3–3
'''Allowed number of parameters:''' 3–3


<span id="parameters-42"></span>
<span id="parameters-43"></span>
=== Parameters ===
=== Parameters ===


Line 822: Line 1,063:
* '''newname''' (<code>string</code>): new name of the table
* '''newname''' (<code>string</code>): new name of the table


<span id="returns-42"></span>
<span id="returns-43"></span>
=== Returns ===
=== Returns ===


<code>bool</code>
* '''value''' (<code>bool</code>)


== insert ==
== insert ==
Line 833: Line 1,074:
'''Allowed number of parameters:''' 3–7
'''Allowed number of parameters:''' 3–7


<span id="parameters-43"></span>
<span id="parameters-44"></span>
=== Parameters ===
=== Parameters ===


* '''table''' (<code>table</code>):
* '''table''' (<code>table</code>)
* '''columns''' (<code>list</code>): list of column names, e.g. '(&quot;ID&quot;, &quot;value&quot;)
* '''columns''' (<code>list</code>): list of column names, e.g. '(&quot;ID&quot;, &quot;value&quot;)
* '''datasets''' (<code>list</code>): list of list of column values, e.g. '('(1 10) '(2 15))
* '''datasets''' (<code>list</code>): list of list of column values, e.g. '('(1 10) '(2 15))
* '''onCollisionCols''' (<code>list</code>): list of columns of the old dataset that have to be passed to onCollision. Can also request $update, $set:<computed-column>, or NEW.<insert-column>. ''(optional)''
* '''onCollisionCols''' (<code>list</code>): list of columns of the old dataset that have to be passed to onCollision. Can also request $update, $set:<computed-column>, or NEW.<insert-column>. ''(optional)''
* '''onCollision''' (<code>func</code>): function called for each collision. Its positional parameters are the values requested by onCollisionCols, in the same order. If omitted, collisions raise an error. ''(optional)''
* '''onCollision''' (<code>func</code>): function called for each collision. Its positional parameters are the values requested by onCollisionCols, in the same order. If omitted, collisions raise an error. ''(optional)''
** '''Parameters'''
*** '''column values''' (<code>any</code>): one value for each onCollisionCols entry ''(variadic)''
** '''Returns'''
*** '''result''' (<code>any</code>)
* '''mergeNull''' (<code>bool</code>): if true, it will handle NULL values as equal according to SQL 2003's definition of DISTINCT (https://en.wikipedia.org/wiki/Null_(SQL)#When_two_nulls_are_equal:_grouping,_sorting,_and_some_set_operations) ''(optional)''
* '''mergeNull''' (<code>bool</code>): if true, it will handle NULL values as equal according to SQL 2003's definition of DISTINCT (https://en.wikipedia.org/wiki/Null_(SQL)#When_two_nulls_are_equal:_grouping,_sorting,_and_some_set_operations) ''(optional)''
* '''onInsertid''' (<code>func(id:number) -&gt; any</code>): (optional) callback (id)-&gt;any; called once with the first auto_increment id assigned for this INSERT ''(optional)''
* '''onInsertid''' (<code>func</code>): called once with the first auto_increment id assigned for this INSERT ''(optional)''
** '''Parameters'''
*** '''id''' (<code>number</code>): first assigned auto_increment id
** '''Returns'''
*** '''result''' (<code>any</code>): ignored callback result


<span id="returns-43"></span>
<span id="returns-44"></span>
=== Returns ===
=== Returns ===


<code>number</code>
* '''value''' (<code>number</code>)


== stat ==
== stat ==
Line 855: Line 1,104:
'''Allowed number of parameters:''' 0–2
'''Allowed number of parameters:''' 0–2


<span id="parameters-44"></span>
<span id="parameters-45"></span>
=== Parameters ===
=== Parameters ===


Line 861: Line 1,110:
* '''table''' (<code>string</code>): (optional) table name for detailed string output ''(optional)''
* '''table''' (<code>string</code>): (optional) table name for detailed string output ''(optional)''


<span id="returns-44"></span>
<span id="returns-45"></span>
=== Returns ===
=== Returns ===


<code>any</code>
* '''value''' (<code>any</code>)


== totalmem ==
== totalmem ==
Line 872: Line 1,121:
'''Allowed number of parameters:''' 0–0
'''Allowed number of parameters:''' 0–0


<span id="parameters-45"></span>
<span id="parameters-46"></span>
=== Parameters ===
=== Parameters ===


''This function has no parameters.''
''This function has no parameters.''


<span id="returns-45"></span>
<span id="returns-46"></span>
=== Returns ===
=== Returns ===


<code>number</code>
* '''value''' (<code>number</code>)


== resolve_column_name ==
== resolve_column_name ==
Line 888: Line 1,137:
'''Allowed number of parameters:''' 4–4
'''Allowed number of parameters:''' 4–4


<span id="parameters-46"></span>
<span id="parameters-47"></span>
=== Parameters ===
=== Parameters ===


Line 896: Line 1,145:
* '''ignorecase''' (<code>bool</code>): whether identifier case is ignored
* '''ignorecase''' (<code>bool</code>): whether identifier case is ignored


<span id="returns-46"></span>
<span id="returns-47"></span>
=== Returns ===
=== Returns ===


<code>string|nil</code>
* '''value''' (<code>string|nil</code>)


== show ==
== show ==
Line 905: Line 1,154:
show databases/tables/columns/shards
show databases/tables/columns/shards


(show) lists database names (show schema) lists table names (show table_handle) lists the memoized column defs (show table_handle true) returns table metadata (show table_handle &quot;statistics&quot;) returns index statistics (show schema true) lists tables with full info: [{name,engine,row_count,size_bytes,collation,comment},...] (show schema tbl) lists column defs (show schema tbl true) returns assoc {columns,meta,shards} (show schema tbl N) returns shard N overview assoc {shard,state,main_count,delta,deletions,size_bytes} (show schema tbl N true) returns shard N full assoc adding columns and indexes (show schema tbl &quot;statistics&quot;) returns index statistics (used by INFORMATION_SCHEMA)
(show) lists database names (show schema) lists table names (show table_handle) lists the memoized column defs (show table_handle true) returns table metadata (show table_handle &quot;statistics&quot;) returns index statistics (show schema true) lists tables with full info: [{name,engine,row_count,size_bytes,collation,comment},...] (show schema tbl) lists column defs (show schema tbl true) returns assoc {columns,meta,shards} (show schema tbl N) returns shard N overview assoc {shard,state,main_count,delta,deletions,size_bytes} (show schema tbl N true) returns shard N full assoc adding columns and indexes (show schema tbl &quot;statistics&quot;) returns INFORMATION_SCHEMA index statistics (show schema tbl &quot;indexes&quot;) returns MySQL SHOW INDEX rows


'''Allowed number of parameters:''' 0–4
'''Allowed number of parameters:''' 0–4


<span id="parameters-47"></span>
<span id="parameters-48"></span>
=== Parameters ===
=== Parameters ===


Line 917: Line 1,166:
* '''full''' (<code>bool</code>): (optional) true to include columns and indexes in shard detail ''(optional)''
* '''full''' (<code>bool</code>): (optional) true to include columns and indexes in shard detail ''(optional)''


<span id="returns-47"></span>
<span id="returns-48"></span>
=== Returns ===
=== Returns ===


<code>any</code>
* '''value''' (<code>any</code>)


== show_triggers ==
== show_triggers ==
Line 928: Line 1,177:
'''Allowed number of parameters:''' 1–2
'''Allowed number of parameters:''' 1–2


<span id="parameters-48"></span>
<span id="parameters-49"></span>
=== Parameters ===
=== Parameters ===


Line 934: Line 1,183:
* '''table''' (<code>string</code>): (optional) table name, if omitted shows all triggers in schema ''(optional)''
* '''table''' (<code>string</code>): (optional) table name, if omitted shows all triggers in schema ''(optional)''


<span id="returns-48"></span>
<span id="returns-49"></span>
=== Returns ===
=== Returns ===


<code>any</code>
* '''value''' (<code>any</code>)


== rebuild ==
== rebuild ==
Line 945: Line 1,194:
'''Allowed number of parameters:''' 0–3
'''Allowed number of parameters:''' 0–3


<span id="parameters-49"></span>
<span id="parameters-50"></span>
=== Parameters ===
=== Parameters ===


Line 952: Line 1,201:
* '''repartition''' (<code>bool</code>): with a table handle, whether to repartition that table (default: true) ''(optional)''
* '''repartition''' (<code>bool</code>): with a table handle, whether to repartition that table (default: true) ''(optional)''


<span id="returns-49"></span>
<span id="returns-50"></span>
=== Returns ===
=== Returns ===


<code>string</code>
* '''value''' (<code>string</code>)


<span id="loadcsv"></span>
<span id="loadcsv"></span>
Line 964: Line 1,213:
'''Allowed number of parameters:''' 3–5
'''Allowed number of parameters:''' 3–5


<span id="parameters-50"></span>
<span id="parameters-51"></span>
=== Parameters ===
=== Parameters ===


Line 973: Line 1,222:
* '''firstline''' (<code>bool</code>): (optional) if the first line contains the column names (otherwise, the tables column order is used) ''(optional)''
* '''firstline''' (<code>bool</code>): (optional) if the first line contains the column names (otherwise, the tables column order is used) ''(optional)''


<span id="returns-50"></span>
<span id="returns-51"></span>
=== Returns ===
=== Returns ===


<code>string</code>
* '''value''' (<code>string</code>)


<span id="loadjson"></span>
<span id="loadjson"></span>
Line 985: Line 1,234:
'''Allowed number of parameters:''' 2–2
'''Allowed number of parameters:''' 2–2


<span id="parameters-51"></span>
<span id="parameters-52"></span>
=== Parameters ===
=== Parameters ===


Line 991: Line 1,240:
* '''stream''' (<code>stream</code>): stream of the .jsonl file, read with: (stream filename)
* '''stream''' (<code>stream</code>): stream of the .jsonl file, read with: (stream filename)


<span id="returns-51"></span>
<span id="returns-52"></span>
=== Returns ===
=== Returns ===


<code>string</code>
* '''value''' (<code>string</code>)


== settings ==
== settings ==
Line 1,002: Line 1,251:
'''Allowed number of parameters:''' 0–2
'''Allowed number of parameters:''' 0–2


<span id="parameters-52"></span>
<span id="parameters-53"></span>
=== Parameters ===
=== Parameters ===


Line 1,008: Line 1,257:
* '''value''' (<code>any</code>): new value of that setting ''(optional)''
* '''value''' (<code>any</code>): new value of that setting ''(optional)''


<span id="returns-52"></span>
<span id="returns-53"></span>
=== Returns ===
=== Returns ===


<code>any</code>
* '''value''' (<code>any</code>)


== createcreatetabletrigger ==
== createcreatetabletrigger ==
Line 1,019: Line 1,268:
'''Allowed number of parameters:''' 6–6
'''Allowed number of parameters:''' 6–6


<span id="parameters-53"></span>
<span id="parameters-54"></span>
=== Parameters ===
=== Parameters ===


Line 1,029: Line 1,278:
* '''visible''' (<code>bool</code>): true = user trigger, false = internal trigger
* '''visible''' (<code>bool</code>): true = user trigger, false = internal trigger


<span id="returns-53"></span>
<span id="returns-54"></span>
=== Returns ===
=== Returns ===


<code>bool</code>
* '''value''' (<code>bool</code>)


== dropcreatetabletrigger ==
== dropcreatetabletrigger ==
Line 1,040: Line 1,289:
'''Allowed number of parameters:''' 4–4
'''Allowed number of parameters:''' 4–4


<span id="parameters-54"></span>
<span id="parameters-55"></span>
=== Parameters ===
=== Parameters ===


Line 1,048: Line 1,297:
* '''ifexists''' (<code>bool</code>): don't throw error if trigger doesn't exist
* '''ifexists''' (<code>bool</code>): don't throw error if trigger doesn't exist


<span id="returns-54"></span>
<span id="returns-55"></span>
=== Returns ===
=== Returns ===


<code>bool</code>
* '''value''' (<code>bool</code>)


== createtrigger ==
== createtrigger ==
Line 1,059: Line 1,308:
'''Allowed number of parameters:''' 6–6
'''Allowed number of parameters:''' 6–6


<span id="parameters-55"></span>
<span id="parameters-56"></span>
=== Parameters ===
=== Parameters ===


* '''table''' (<code>table</code>):
* '''table''' (<code>table</code>)
* '''name''' (<code>string</code>): name of the trigger
* '''name''' (<code>string</code>): name of the trigger
* '''timing''' (<code>string</code>): one of: before_insert, after_insert, before_update, after_update, before_delete, after_delete
* '''timing''' (<code>string</code>): one of: before_insert, after_insert, before_update, after_update, before_delete, after_delete
Line 1,069: Line 1,318:
* '''visible''' (<code>bool</code>): true = user trigger (shown in SHOW TRIGGERS), false = internal trigger (hidden)
* '''visible''' (<code>bool</code>): true = user trigger (shown in SHOW TRIGGERS), false = internal trigger (hidden)


<span id="returns-55"></span>
<span id="returns-56"></span>
=== Returns ===
=== Returns ===


<code>bool</code>
* '''value''' (<code>bool</code>)


== droptrigger ==
== droptrigger ==
Line 1,080: Line 1,329:
'''Allowed number of parameters:''' 3–3
'''Allowed number of parameters:''' 3–3


<span id="parameters-56"></span>
<span id="parameters-57"></span>
=== Parameters ===
=== Parameters ===


Line 1,087: Line 1,336:
* '''ifexists''' (<code>bool</code>): don't throw error if trigger doesn't exist
* '''ifexists''' (<code>bool</code>): don't throw error if trigger doesn't exist


<span id="returns-56"></span>
<span id="returns-57"></span>
=== Returns ===
=== Returns ===


<code>bool</code>
* '''value''' (<code>bool</code>)


== mysql_import ==
== mysql_import ==
Line 1,098: Line 1,347:
'''Allowed number of parameters:''' 4–8
'''Allowed number of parameters:''' 4–8


<span id="parameters-57"></span>
<span id="parameters-58"></span>
=== Parameters ===
=== Parameters ===


Line 1,110: Line 1,359:
* '''targettable''' (<code>string|nil</code>): target table (omit/nil =&gt; sourcetable) ''(optional)''
* '''targettable''' (<code>string|nil</code>): target table (omit/nil =&gt; sourcetable) ''(optional)''


<span id="returns-57"></span>
<span id="returns-58"></span>
=== Returns ===
=== Returns ===


<code>bool</code>
* '''value''' (<code>bool</code>)


== psql_import ==
== psql_import ==
Line 1,121: Line 1,370:
'''Allowed number of parameters:''' 4–9
'''Allowed number of parameters:''' 4–9


<span id="parameters-58"></span>
<span id="parameters-59"></span>
=== Parameters ===
=== Parameters ===


Line 1,134: Line 1,383:
* '''targettable''' (<code>string|nil</code>): target table (omit/nil =&gt; sourcetable) ''(optional)''
* '''targettable''' (<code>string|nil</code>): target table (omit/nil =&gt; sourcetable) ''(optional)''


<span id="returns-58"></span>
<span id="returns-59"></span>
=== Returns ===
=== Returns ===


<code>bool</code>
* '''value''' (<code>bool</code>)

Revision as of 11:59, 28 August 2026


Storage

Generated from MemCP commit c42e19eba on 27 August 2026. See Full SCM API documentation.

The Storage module is the low-level data interface used by generated SQL plans. It manages databases, tables, columns, constraints, scans, indexes, RecSets, computed data, persistence, and schema lifecycle operations.

Most application code should use SQL instead of calling these functions directly. Planner-generated calls rely on exact transaction, locking, callback, ownership, and quoting contracts. Functions that accept table handles or transaction contexts must not be replaced with direct access to shard internals.

For the physical model see Columnar Storage, Shards, RecordIDs, Main Storage, Delta Storage, and Persistency and Performance Guarantees.

table

resolves a schema+table name pair into a table handle

Allowed number of parameters: 2–2

Parameters

  • schema (string)
  • table (string)

Returns

  • value (table)

scan_estimate

estimate output row count for a table scan

Allowed number of parameters: 1–1

Parameters

  • table (table)

Returns

  • value (int)

table_planner_statistics

return the immutable O(1) planner-statistics snapshot for a table

Allowed number of parameters: 1–1

Parameters

  • table (table)

Returns

  • value (any)

scan_selectivity_estimate

bounded estimate of visible rows matching a table filter; stops at max_rows and does not log scan telemetry

Allowed number of parameters: 5–5

Parameters

  • tx (any): transaction context to use for visibility; usually ((context "session") "__memcp_tx")
  • table (table)
  • condition_cols (list<string>): columns passed to the selectivity predicate
    • column (string): column name passed to the corresponding callback parameter
  • condition (func): predicate sampled to estimate matching rows
    • Parameters
      • columns (any): one value for each entry in the matching column list, in the same order (variadic)
    • Returns
      • result (bool): true when the sampled row matches
  • max_rows (int)

Returns

  • value (list)

table_empty?

returns true if a table currently has no rows

Allowed number of parameters: 1–1

Parameters

  • table (table)

Returns

  • value (bool)

scan_recset

builds a query-local record-set handle from one table scan, or -- when given an existing recset instead of a table -- narrows that recset to the members which also satisfy filter, re-evaluating filter only over its existing membership. The latter is the cheap way to AND a further (possibly subscan-heavy) condition onto an already-narrowed recset without re-touching rows outside it (e.g. evaluating an expensive correlated check only over the rows a cheap selective filter already narrowed a table down to). The returned value is not persisted and can be scanned like a table

Allowed number of parameters: 4–4

Parameters

  • tx (any): transaction context to use for visibility; usually ((context "session") "__memcp_tx")
  • table (any): a table, or an existing recset to narrow further
  • filterColumns (list<string>): physical columns passed to filter before map/reduce; $recset_contains supplies a row-bound RecSet membership closure
    • column (string): column name passed to the corresponding callback parameter
  • filter (func): lambda function that decides whether a row enters the recset
    • Parameters
      • columns (any): one value for each entry in the matching column list, in the same order (variadic)
    • Returns
      • result (bool): true when the row belongs in the recset

Returns

  • value (recset)

recset_count

returns the number of currently stored recids in a query-local recset

Allowed number of parameters: 1–1

Parameters

  • recset (recset)

Returns

  • value (int)

recset_project_join

projects a source recset through key columns into a query-local target-table recset

Allowed number of parameters: 5–5

Parameters

  • tx (any): transaction context to use for visibility; usually ((context "session") "__memcp_tx")
  • source_recset (recset)
  • source_key_columns (list)
  • target_table (table)
  • target_key_columns (list)

Returns

  • value (recset)

recset_key_index

builds an immutable lookup function for key columns of the rows contained in a query-local recset

Allowed number of parameters: 3–3

Parameters

  • tx (any): transaction context used while reading source keys
  • source_recset (recset)
  • source_key_columns (list)

Returns

  • lookup (func): tests whether the recset contains a row with the supplied composite key
    • Parameters
      • key (any): one value for each source key column, in the same order (variadic)
    • Returns
      • present (bool): whether the composite key occurs in the recset

recset_union

combines query-local recsets from the same table and removes duplicate record IDs

Allowed number of parameters: 1–1

Parameters

  • recsets (list)

Returns

  • value (recset)

recset_intersect

intersects query-local recsets from the same table

Allowed number of parameters: 1–1

Parameters

  • recsets (list)

Returns

  • value (recset)

recset_difference

returns the records from the first query-local recset which occur in none of the following same-table recsets

Allowed number of parameters: 1–1

Parameters

  • recsets (list)

Returns

  • value (recset)

recset_not

returns the complement of a query-local recset relative to the currently visible rows of its base table

Allowed number of parameters: 1–1

Parameters

  • recset (recset)

Returns

  • value (recset)

scan_exists

returns true if a table contains at least one visible row matching the given filter; uses scan boundary analysis without map/reduce setup

Allowed number of parameters: 4–4

Parameters

  • tx (any): transaction context to use for visibility; usually ((context "session") "__memcp_tx")
  • table (table|list|recset)
  • filterColumns (list<string>): physical columns passed to filter before map/reduce; $recset_contains supplies a row-bound RecSet membership closure
    • column (string): column name passed to the corresponding callback parameter
  • filter (func): lambda function that decides whether a row exists
    • Parameters
      • columns (any): one value for each entry in the matching column list, in the same order (variadic)
    • Returns
      • result (bool): true when the row satisfies the existence test

Returns

  • value (bool)

scan

does an unordered parallel filter-map-reduce pass on a single table and returns the reduced result

Allowed number of parameters: 6–10

Parameters

  • tx (any): transaction context to use for visibility and mutations; usually ((context "session") "__memcp_tx")
  • table (table|list|recset): table handle, query-local recset, or a list for temporary data
  • filterColumns (list<string>): physical columns passed to filter before map/reduce; $recset_contains supplies a row-bound RecSet membership closure
    • column (string): column name passed to the corresponding callback parameter
  • filter (func): lambda function that decides whether a dataset is passed to the map phase. Equality and range comparisons may be translated into indexed scans
    • Parameters
      • columns (any): one value for each entry in the matching column list, in the same order (variadic)
    • Returns
      • result (bool): true when the row proceeds to map
  • mapColumns (list<string>): physical columns passed to map after filtering; pseudo columns are $update (update/delete current row), $recset_contains (row-bound RecSet membership), $set:<column>, $increment:<column>, and $invalidate:<column> (computed-column maintenance), plus NEW.<column> in trigger plans
    • column (string): column name passed to the corresponding callback parameter
  • map (func): lambda function that extracts or produces one value from the row; it may also use documented pseudo columns for mutations or result output
    • Parameters
      • columns (any): one value for each entry in the matching column list, in the same order (variadic)
    • Returns
      • result (any): value passed to reduce, or returned directly when no reducer is supplied
  • reduce (func): optional aggregation function used first within shards and then to combine shard results (optional)
    • Parameters
      • accumulator (any): current aggregate, initially the neutral value
      • value (any): next mapped or partially reduced value
    • Returns
      • accumulator (any): aggregate passed to the next reducer call or returned by the scan
  • neutral (any): (optional) neutral element for the reduce phase, otherwise nil is assumed (optional)
  • reduce2 (func): optional final reducer that combines the neutral value with the result produced by reduce (optional)
    • Parameters
      • accumulator (any): current aggregate, initially the neutral value
      • value (any): next mapped or partially reduced value
    • Returns
      • accumulator (any): aggregate passed to the next reducer call or returned by the scan
  • isOuter (bool): (optional) if true, in case of no hits, call map once anyway with NULL values (optional)

Returns

  • value (any)

scan_batch

does an unordered parallel filter-map-reduce pass on a single table using batchdata-backed #N pseudo columns and returns the reduced result

Allowed number of parameters: 8–12

Parameters

  • tx (any): transaction context to use for visibility and mutations; usually ((context "session") "__memcp_tx")
  • table (table|list|recset): table handle, query-local recset, or a list for temporary data
  • filterColumns (list<string>): columns passed to filter; #0, #1, ... address batchdata slots
    • column (string): column name passed to the corresponding callback parameter
  • filter (func): lambda function that decides whether a dataset is passed to the map phase
    • Parameters
      • columns (any): one value for each entry in the matching column list, in the same order (variadic)
    • Returns
      • result (bool): true when this table row and batch row proceed to map
  • mapColumns (list<string>): columns passed to map; #0, #1, ... address batchdata slots
    • column (string): column name passed to the corresponding callback parameter
  • map (func): lambda function that extracts data from the table row and batch row
    • Parameters
      • columns (any): one value for each entry in the matching column list, in the same order (variadic)
    • Returns
      • result (any): value passed to reduce or returned directly
  • stride (int): number of batchdata entries per batch row
  • batchdata (list<any>): flat batch buffer accessed via #N pseudo columns
    • slot (any): one batch value; every stride consecutive slots form a batch row
  • reduce (func): optional lambda function that aggregates mapped values (optional)
    • Parameters
      • accumulator (any): current aggregate, initially the neutral value
      • value (any): next mapped or partially reduced value
    • Returns
      • accumulator (any): aggregate passed to the next reducer call or returned by the scan
  • neutral (any): (optional) neutral element for the reduce phase, otherwise nil is assumed (optional)
  • reduce2 (func): optional final reducer that combines the neutral value with the result produced by reduce (optional)
    • Parameters
      • accumulator (any): current aggregate, initially the neutral value
      • value (any): next mapped or partially reduced value
    • Returns
      • accumulator (any): aggregate passed to the next reducer call or returned by the scan
  • isOuter (bool): (optional) if true, in case of no hits, call map once anyway with NULL values (optional)

Returns

  • value (any)

scan_order_batch_accept

incrementally scans a table or existing RecSet in scan_order order and applies a RecSet batch filter before OFFSET/LIMIT and map/reduce. The first candidate RecSet contains offset+limit rows; if too few rows are accepted, subsequent disjoint batches contain twice as many candidates until the accepted limit is satisfied or the input is exhausted. batchFilter is called as (batchFilter input_recset) and must return an exact subset RecSet of the same base table and transaction. A simple batchFilter may call (scan_recset tx input_recset filterColumns realFilter); complex filters may project input_recset to another table, apply search/ACL scans and project the result back to the input table. The returned RecSet is used only as a membership mask against the already ordered candidate vector, so output order is preserved without scanning the unordered RecSet again. For non-unique ORDER BY values, include an explicit unique tie-breaker. sortcols/sortdirs may both be empty; that path greedily collects candidates without sorting. limitPartitionCols is present for scan_order signature compatibility and currently must be 0

Allowed number of parameters: 10–14

Parameters

  • tx (any): transaction context used consistently by the candidate scan and every batch filter operation; usually ((context "session") "__memcp_tx")
  • table_or_recset (table|recset): base table or complete existing query-local RecSet from which ordered candidate batches are drawn
  • batchFilter (func): function (lambda (input_recset) accepted_recset). It may naively narrow input_recset with scan_recset, or run arbitrary RecSet projections/search/ACL operations and project back. It must return a same-table, same-transaction subset of input_recset
    • Parameters
      • input_recset (recset)
    • Returns
      • value (recset)
  • sortcols (list<string|func(columns:any...) -> any>): same as scan_order: columns or computed sort functions. Include a unique tie-breaker for a total repeatable order; use an empty list for greedy unsorted collection
    • sort column (string|func): a column name, or a function of row-column values that returns the sortable value
      • Parameters
        • columns (any): column values used to compute the sort key (variadic)
      • Returns
        • sort key (any): value compared at this sort position
  • sortdirs (list<func(left:any, right:any) -> bool>): same as scan_order: one relation per sort column; must also be empty when sortcols is empty
    • direction (func): strict ordering relation such as <, >, or a collate relation
      • Parameters
        • left (any): left sort value
        • right (any): right sort value
      • Returns
        • ordered (bool): true when left belongs before right
  • limitPartitionCols (number): reserved for scan_order signature compatibility; currently must be 0
  • offset (number): number of batch-filter-accepted rows to skip; it is not the number of driver candidates already examined
  • limit (number): finite maximum number of accepted rows passed to map; the initial candidate batch size is offset+limit and doubles for every subsequent batch
  • mapColumns (list<string>): physical columns passed to map after filtering; pseudo columns are $update (update/delete current row), $recset_contains (row-bound RecSet membership), $set:<column>, $increment:<column>, and $invalidate:<column> (computed-column maintenance), plus NEW.<column> in trigger plans; $break is reserved for internal ORC convergence and must not implement SQL OFFSET/LIMIT, which belong in the native offset and limit arguments
    • column (string): column name passed to the corresponding callback parameter
  • map (func): same map callback contract as scan_order; accepted record IDs are passed to its shard mapper in batches
    • Parameters
      • columns (any): one value for each entry in the matching column list, in the same order (variadic)
    • Returns
      • result (any): value passed to reduce or returned directly
  • reduce (func): optional serial reducer over mapped accepted rows, with the same accumulator contract as scan_order (optional)
    • Parameters
      • accumulator (any): current aggregate, initially the neutral value
      • value (any): next mapped or partially reduced value
    • Returns
      • accumulator (any): aggregate passed to the next reducer call or returned by the scan
  • neutral (any): optional neutral element for reduce; defaults to nil (optional)
  • isOuter (bool): optional scan_order-compatible outer behavior: map one NULL row when no accepted row reaches map (optional)
  • notFoundValue (any): optional result when no accepted row reaches map and isOuter is false; defaults to neutral (optional)

Returns

  • value (any)

scan_order

does an ordered parallel filter and serial map-reduce pass on a single table and returns the reduced result

Allowed number of parameters: 11–17

Parameters

  • tx (any): transaction context to use for visibility and mutations; usually ((context "session") "__memcp_tx")
  • table (table|list|recset): table handle, query-local RecSet, or a list for temporary data
  • filterColumns (list<string>): physical columns passed to filter before map/reduce; $recset_contains supplies a row-bound RecSet membership closure
    • column (string): column name passed to the corresponding callback parameter
  • filter (func): lambda function that decides whether a dataset is passed to the map phase. Equality and range comparisons may be translated into indexed scans
    • Parameters
      • columns (any): one value for each entry in the matching column list, in the same order (variadic)
    • Returns
      • result (bool): true when the row proceeds to ordering and map
  • sortcols (list<string|func(columns:any...) -> any>): columns used for ordering; each entry corresponds to one relation in sortdirs
    • sort column (string|func): a column name, or a function of row-column values that returns the sortable value
      • Parameters
        • columns (any): column values used to compute the sort key (variadic)
      • Returns
        • sort key (any): value compared at this sort position
  • sortdirs (list<func(left:any, right:any) -> bool>): one ordering relation per entry in sortcols; < is ascending and > is descending
    • direction (func): strict ordering relation such as <, >, or a collate relation
      • Parameters
        • left (any): left sort value
        • right (any): right sort value
      • Returns
        • ordered (bool): true when left belongs before right
  • limitPartitionCols (number): number of leading sort columns that form the partition key for per-partition offset/limit. 0 (default) means global offset/limit.
  • offset (number): number of globally ordered, filter-accepted items to skip before map; apply SQL OFFSET here rather than in map
  • limit (number): maximum globally ordered, filter-accepted items passed to map; -1 means unlimited; apply SQL LIMIT here so shard-local Top-K and the global merge can brake early
  • mapColumns (list<string>): physical columns passed to map after filtering; pseudo columns are $update (update/delete current row), $recset_contains (row-bound RecSet membership), $set:<column>, $increment:<column>, and $invalidate:<column> (computed-column maintenance), plus NEW.<column> in trigger plans; $break is reserved for internal ORC convergence and must not implement SQL OFFSET/LIMIT, which belong in the native offset and limit arguments
    • column (string): column name passed to the corresponding callback parameter
  • map (func): lambda function that extracts or produces one value from each accepted row
    • Parameters
      • columns (any): one value for each entry in the matching column list, in the same order (variadic)
    • Returns
      • result (any): value passed to reduce or returned directly
  • reduce (func): optional serial aggregation function over mapped values (optional)
    • Parameters
      • accumulator (any): current aggregate, initially the neutral value
      • value (any): next mapped or partially reduced value
    • Returns
      • accumulator (any): aggregate passed to the next reducer call or returned by the scan
  • neutral (any): (optional) neutral element for the reduce phase, otherwise nil is assumed (optional)
  • isOuter (bool): (optional) if true, in case of no hits, call map once anyway with NULL values (optional)
  • notFoundValue (any): (optional) result for no hits when isOuter is false; defaults to neutral (optional)
  • postOrderFilterColumns (list<string>): optional columns for a predicate evaluated in global order before OFFSET/LIMIT are counted; use for expensive acceptance checks that cannot participate in index boundaries (optional)
    • column (string): column name passed to the corresponding callback parameter
  • postOrderFilter (func): optional late acceptance predicate. Rejected rows do not count toward OFFSET/LIMIT and never reach map (optional)
    • Parameters
      • columns (any): one value for each entry in the matching column list, in the same order (variadic)
    • Returns
      • result (bool): true when the ordered row counts toward OFFSET/LIMIT and reaches map

Returns

  • value (any)

scan_order_multi

does an ordered parallel filter and serial map-reduce pass across multiple tables simultaneously, merging results into a single sorted stream

Allowed number of parameters: 13–17

Parameters

  • tx (any): transaction context
  • tables (list<table|recset>): scan sources; all per-table lists must have this length
    • source (table|recset): base table or query-local record set for one input stream
  • filterColumns (list<list<string>>): filter column lists, one per table
    • table filter columns (list<string>): columns supplied to the matching filterFns entry
      • column (string): column name in the corresponding table
  • filterFns (list<func(columns:any...) -> bool>): filter lambdas, one per table
    • table filter (func): predicate for the corresponding table and filterColumns entry
      • Parameters
        • columns (any): one value for each entry in the matching column list, in the same order (variadic)
      • Returns
        • result (bool): true when the row enters that table's ordered stream
  • sortcols (list<list<string|func(columns:any...) -> any>>): sort column lists, one per table; every inner list must match sortdirs in length and result domains
    • table sort columns (list<string|func(columns:any...) -> any>): sort expressions for the corresponding table
      • sort column (string|func): a column name, or a function of row-column values that returns the sortable value
        • Parameters
          • columns (any): column values used to compute the sort key (variadic)
        • Returns
          • sort key (any): value compared at this sort position
  • sortdirs (list<func(left:any, right:any) -> bool>): shared ordering relations used for every table stream and for the outer merge
    • direction (func): strict ordering relation such as <, >, or a collate relation
      • Parameters
        • left (any): left sort value
        • right (any): right sort value
      • Returns
        • ordered (bool): true when left belongs before right
  • perTableOffset (list<int>|nil): optional per-table offsets; nil disables all per-table offsets
    • offset (int): rows skipped in the corresponding table before the outer merge; -1 disables the offset
  • perTableLimit (list<int>|nil): optional per-table limits; nil disables all per-table limits
    • limit (int): maximum rows retained from the corresponding table before the outer merge; -1 disables the limit
  • limitPartitionCols (number): number of leading sort columns forming partition key
  • offset (number): number of items to skip (global)
  • limit (number): max number of items to read (global; -1 = unlimited)
  • mapColumns (list<list<string>>): map column lists, one per table
    • table map columns (list<string>): columns supplied to the matching mapFns entry
      • column (string): column name in the corresponding table
  • mapFns (list<func(columns:any...) -> any>): map lambdas, one per table
    • table map (func): mapper for the corresponding table and mapColumns entry
      • Parameters
        • columns (any): one value for each entry in the matching column list, in the same order (variadic)
      • Returns
        • result (any): value inserted into the merged stream and passed to reduce
  • reduce (func): optional aggregation function over mapped values from the merged stream (optional)
    • Parameters
      • accumulator (any): current aggregate, initially the neutral value
      • value (any): next mapped or partially reduced value
    • Returns
      • accumulator (any): aggregate passed to the next reducer call or returned by the scan
  • neutral (any): (optional) neutral element for reduce (optional)
  • isOuter (bool): (optional) if true, emit null row when no hits (optional)
  • notFoundValue (any): (optional) result for no hits when isOuter is false; defaults to neutral (optional)

Returns

  • value (any)

createdatabase

creates a new database

Allowed number of parameters: 1–2

Parameters

  • schema (string): name of the new database
  • ignoreexists (bool): if true, return false instead of throwing an error (optional)

Returns

  • value (bool)

dropdatabase

drops a database

Allowed number of parameters: 1–2

Parameters

  • schema (string): name of the database
  • ifexists (bool): if true, don't throw an error if it doesn't exist (optional)

Returns

  • value (bool)

checktablemaintenance

checks whether a user-initiated maintenance operation is allowed for a table

Allowed number of parameters: 3–3

Parameters

  • schema (string)
  • table (string)
  • operation (string)

Returns

  • value (bool)

maintenance_capabilities

returns the server-side maintenance capabilities for a database or table

Allowed number of parameters: 1–2

Parameters

  • schema (string)
  • table (string) (optional)

Returns

  • value (list)

createtable

creates a table, runs its oninit option and registered after-create-table lifecycle triggers synchronously, and returns only after initialization completes; concurrent if-not-exists callers wait for that same completion

Allowed number of parameters: 4–5

Parameters

  • schema (string): name of the existing database that will contain the table
  • table (string): name of the table to create
  • cols (list<list>): column and constraint definitions
    • definition (list): one of ("column" name type dimensions typeparams), ("unique" name columns), or ("foreign" name local_columns referenced_table referenced_columns update_mode delete_mode). Column lists contain strings; foreign-key modes are restrict, cascade, or set null. A column definition's dimensions contains integers and its typeparams uses the same fields documented by createcolumn options
  • options (list|assoc): table options as an alternating key/value list
    • auto_increment (int): first automatically assigned value; must be non-negative
    • charset (string): default character set name
    • collation (string): default collation name
    • comment (string): user-visible table comment
    • engine (string): storage engine: safe, logged, sloppy, memory, or cache
    • oninit (func): closed zero-argument initializer run synchronously once per data generation; concurrent if-not-exists callers wait for completion
      • Returns
        • result (any): ignored initializer result
  • ifnotexists (bool): when true, return false instead of failing if the table exists; if another caller is still creating it, wait for that caller's after-create-table initialization before returning false (optional)

Returns

  • value (bool): true when this call created and initialized the table, false when ifnotexists reused an initialized table

createcolumn

creates a new column in table

Allowed number of parameters: 5–7

Parameters

  • table (table)
  • colname (string): name of the new column
  • type (string): name of the basetype
  • dimensions (list<int>): dimensions of the type, for example precision and scale for decimal
    • dimension (int): one type-specific dimension
  • options (list|assoc): column properties and computed-column configuration as an alternating key/value list
    • auto_increment (bool): assign increasing values automatically
    • collate (string): collation used for this column
    • comment (string): user-visible column comment
    • default (any): literal value used when an insert omits the column
    • default_expression (string): expression evaluated when an insert omits the column
    • filter (func): predicate limiting which rows are computed
      • Parameters
        • columns (any): one value for each entry in the matching column list, in the same order (variadic)
      • Returns
        • result (bool): true when the row should be computed
    • filtercols (list<string>): columns supplied to filter before computing a value
      • column (string): column name passed to the corresponding callback parameter
    • mapcols (list<string>): columns supplied to mapfn for ordered-reduce computation
      • column (string): column name passed to the corresponding callback parameter
    • mapfn (func): maps one source row into a value for reducefn
      • Parameters
        • columns (any): one value for each entry in the matching column list, in the same order (variadic)
      • Returns
        • result (any): value passed to reducefn
    • null (bool): whether the column accepts nil values
    • partitioncount (int): number of leading sort columns that define independent reducer partitions
    • primary (bool): whether this column belongs to the primary key
    • reducefn (func): combines ordered mapped values into the computed-column aggregate (optional)
      • Parameters
        • accumulator (any): current aggregate, initially the neutral value
        • value (any): next mapped or partially reduced value
      • Returns
        • accumulator (any): aggregate passed to the next reducer call or returned by the scan
    • reduceinit (any): initial accumulator supplied to reducefn
    • sortcols (list<string|func(columns:any...) -> any>): columns or expressions defining ordered-reduce input order
      • sort column (string|func): a column name, or a function of row-column values that returns the sortable value
        • Parameters
          • columns (any): column values used to compute the sort key (variadic)
        • Returns
          • sort key (any): value compared at this sort position
    • sortdirs (list<func(left:any, right:any) -> bool>): one ordering relation for every sortcols entry
      • direction (func): strict ordering relation such as <, >, or a collate relation
        • Parameters
          • left (any): left sort value
          • right (any): right sort value
        • Returns
          • ordered (bool): true when left belongs before right
    • temp (bool): whether this is a query-local temporary computed column
    • unique (bool): whether values must be unique
    • update (any): expression evaluated when a row is updated
  • computorCols (list<string>): columns passed to computor in this order (optional)
    • column (string): column name passed to the corresponding callback parameter
  • computor (func): lambda expression that computes this column from the values selected by computorCols (optional)
    • Parameters
      • columns (any): one value for each entry in the matching column list, in the same order (variadic)
    • Returns
      • result (any): computed column value

Returns

  • value (bool)

createkey

creates a new key on a table

Allowed number of parameters: 4–4

Parameters

  • table (table)
  • keyname (string): name of the new key
  • unique (bool): whether the key is unique
  • columns (list): list of columns to include

Returns

  • value (bool)

dropkey

drops a named unique key from a table

Allowed number of parameters: 2–2

Parameters

  • table (table)
  • keyname (string): name of the unique key

Returns

  • value (bool)

createforeignkey

creates a new foreign key on a table

Allowed number of parameters: 7–7

Parameters

  • table1 (table)
  • keyname (string): name of the new key
  • columns1 (list): list of columns to include
  • table2 (table)
  • columns2 (list): list of columns to include
  • updatemode (string): restrict|cascade|set null
  • deletemode (string): restrict|cascade|set null

Returns

  • value (bool)

shardcolumn

tells us how it would partition a column according to their values. Returns a list of pivot elements.

Allowed number of parameters: 2–3

Parameters

  • table (table)
  • colname (string): name of the column
  • numpartitions (number): number of partitions; optional. leave 0 if you want to detect the partiton number automatically or copy the partition schema of the table (optional)

Returns

  • value (list)

partitiontable

suggests a partition scheme for a table. If the table has no partition scheme yet, it will immediately apply that scheme and return true. If the table already has a partition scheme, it will alter the partitioning score such that the partitioning scheme is considered in the next repartitioning and return false.

Allowed number of parameters: 2–2

Parameters

  • table (table)
  • columns (list): associative list of string -> list representing column name -> pivots. You can compute pivots by (shardcolumn ...)

Returns

  • value (bool)

altertable

alters a table

Allowed number of parameters: 3–3

Parameters

  • table (table)
  • operation (string): one of owner|drop|engine|collation|auto_increment
  • parameter (any): name of the column to drop or value of the parameter

Returns

  • value (bool)

altercolumn

alters a column

Allowed number of parameters: 4–4

Parameters

  • table (table)
  • column (string): name of the column
  • operation (string): one of drop|type|collation|auto_increment|comment
  • parameter (any): name of the column to drop or value of the parameter

Returns

  • value (bool)

droptable

removes a table

Allowed number of parameters: 2–3

Parameters

  • schema (string)
  • table (string)
  • ifexists (bool): if true, don't throw an error if it already exists (optional)

Returns

  • value (bool)

dropcolumn

drops a column from a table

Allowed number of parameters: 2–2

Parameters

  • table (table)
  • column (string): name of the column to drop

Returns

  • value (bool)

migratedropcolumn

drops a legacy system column during startup migration

Allowed number of parameters: 2–2

Parameters

  • table (table)
  • column (string): legacy column name

Returns

  • value (bool)

invalidatecolumn

marks all values of a computed column as stale

Allowed number of parameters: 2–2

Parameters

  • table (table)
  • column (string): name of the computed column

Returns

  • value (bool)

invalidateorc

invalidates ORC column rows from a sort key onwards via validMask scan

Allowed number of parameters: 3–3

Parameters

  • table (table)
  • column (string): name of the ORC column
  • sortkeys (list): composite sort key values from which to invalidate

Returns

  • value (bool)

register_keytable_cleanup

registers triggers on a base table to maintain keytable entries (insert/delete group keys)

Allowed number of parameters: 4–4

Parameters

  • base_table (table)
  • kt_table (table)
  • tblvar (string): table alias used in scan column prefixes
  • key_pairs (list): list of (base_col kt_col) pairs

Returns

  • value (bool)

initialize_cache_table

registers maintenance, locks source tables for a consistent snapshot, and runs a canonical planner-cache initializer exactly once

Allowed number of parameters: 5–6

Parameters

  • transaction (any): explicit transaction context carrying query-session ownership
  • table (table)
  • source_tables (list)
  • register_maintenance (func)
    • Returns
      • value (any)
  • initializer (func)
    • Returns
      • value (any)
  • finalizer (func): optional zero-argument finalizer run under the same source-table locks after initialization (optional)
    • Returns
      • value (any)

Returns

  • value (bool)

touch_keytable

extends the lease on a keytable so CacheManager defers eviction

Allowed number of parameters: 1–1

Parameters

  • table (table)

Returns

  • value (bool)

locktables

acquires WRITE or READ user-level locks on a list of tables (LOCK TABLES); implicitly releases any previously held locks

Allowed number of parameters: 1–1

Parameters

  • locks (list): flat list of schema, table, write? triples

Returns

  • value (bool)

unlocktables

releases all user-level table locks held by this session

Allowed number of parameters: 0–0

Parameters

This function has no parameters.

Returns

  • value (bool)

get_fk_target

returns (ref_table ref_column) if a single-column FK exists for the given column, nil otherwise

Allowed number of parameters: 2–2

Parameters

  • table (table)
  • column (string): column name

Returns

  • value (any)

renametable

renames a table

Allowed number of parameters: 3–3

Parameters

  • schema (string): name of the database
  • oldname (string): current name of the table
  • newname (string): new name of the table

Returns

  • value (bool)

insert

inserts a new dataset into table and returns the number of successful items

Allowed number of parameters: 3–7

Parameters

  • table (table)
  • columns (list): list of column names, e.g. '("ID", "value")
  • datasets (list): list of list of column values, e.g. '('(1 10) '(2 15))
  • onCollisionCols (list): list of columns of the old dataset that have to be passed to onCollision. Can also request $update, $set:<computed-column>, or NEW.<insert-column>. (optional)
  • onCollision (func): function called for each collision. Its positional parameters are the values requested by onCollisionCols, in the same order. If omitted, collisions raise an error. (optional)
    • Parameters
      • column values (any): one value for each onCollisionCols entry (variadic)
    • Returns
      • result (any)
  • mergeNull (bool): if true, it will handle NULL values as equal according to SQL 2003's definition of DISTINCT (https://en.wikipedia.org/wiki/Null_(SQL)#When_two_nulls_are_equal:_grouping,_sorting,_and_some_set_operations) (optional)
  • onInsertid (func): called once with the first auto_increment id assigned for this INSERT (optional)
    • Parameters
      • id (number): first assigned auto_increment id
    • Returns
      • result (any): ignored callback result

Returns

  • value (number)

stat

return system statistics as assoc: mem_available, mem_total, process_memory, shard_memory, shard_budget, persisted_memory, persisted_budget, cache_entry_count, cache_entry_size. (stat schema) and (stat schema tbl) return a string with detailed memory usage.

Allowed number of parameters: 0–2

Parameters

  • schema (string): (optional) database name for detailed string output (optional)
  • table (string): (optional) table name for detailed string output (optional)

Returns

  • value (any)

totalmem

Returns total physical memory in bytes (from /proc/meminfo)

Allowed number of parameters: 0–0

Parameters

This function has no parameters.

Returns

  • value (number)

resolve_column_name

resolve a physical column name from immutable table metadata

Allowed number of parameters: 4–4

Parameters

  • schema (string): database name
  • table (string): table name
  • column (string): column name
  • ignorecase (bool): whether identifier case is ignored

Returns

  • value (string|nil)

show

show databases/tables/columns/shards

(show) lists database names (show schema) lists table names (show table_handle) lists the memoized column defs (show table_handle true) returns table metadata (show table_handle "statistics") returns index statistics (show schema true) lists tables with full info: [{name,engine,row_count,size_bytes,collation,comment},...] (show schema tbl) lists column defs (show schema tbl true) returns assoc {columns,meta,shards} (show schema tbl N) returns shard N overview assoc {shard,state,main_count,delta,deletions,size_bytes} (show schema tbl N true) returns shard N full assoc adding columns and indexes (show schema tbl "statistics") returns INFORMATION_SCHEMA index statistics (show schema tbl "indexes") returns MySQL SHOW INDEX rows

Allowed number of parameters: 0–4

Parameters

  • schema_or_table (string|table|recset): (optional) database name or resolved table/recset handle (optional)
  • table_or_property (string|bool): (optional) table name, true for full info, or "statistics" for a handle (optional)
  • property (int|bool|string): (optional) shard index (int), true for full table info, or "statistics" (optional)
  • full (bool): (optional) true to include columns and indexes in shard detail (optional)

Returns

  • value (any)

show_triggers

show triggers for a given table

Allowed number of parameters: 1–2

Parameters

  • schema (string): database name
  • table (string): (optional) table name, if omitted shows all triggers in schema (optional)

Returns

  • value (any)

rebuild

rebuilds main storages and returns the amount of time it took; with a table handle, rebuilds only that table

Allowed number of parameters: 0–3

Parameters

  • table_or_all (bool|table): table handle for a table-local rebuild; otherwise whether to rebuild unchanged shards globally (default: false) (optional)
  • all_or_repartition (bool): with a table: whether to rebuild unchanged shards; globally: whether to repartition (default: true) (optional)
  • repartition (bool): with a table handle, whether to repartition that table (default: true) (optional)

Returns

  • value (string)

loadCSV

loads a CSV stream into a table and returns the amount of time it took. The first line of the file must be the headlines. The headlines must match the table's columns exactly.

Allowed number of parameters: 3–5

Parameters

  • schema (string): name of the database
  • table (string): name of the table
  • stream (stream): CSV file, load with: (stream filename)
  • delimiter (string): (optional) delimiter defaults to ";" (optional)
  • firstline (bool): (optional) if the first line contains the column names (otherwise, the tables column order is used) (optional)

Returns

  • value (string)

loadJSON

loads a .jsonl file from stream into a database and returns the amount of time it took. JSONL is a linebreak separated file of JSON objects. Each JSON object is one dataset in the database. Before you add rows, you must declare the table in a line '#table <tablename>'. All other lines starting with # are comments. Columns are created dynamically as soon as they occur in a json object.

Allowed number of parameters: 2–2

Parameters

  • schema (string): name of the database where you want to put the tables in
  • stream (stream): stream of the .jsonl file, read with: (stream filename)

Returns

  • value (string)

settings

reads or writes a global settings value. This modifies your data/settings.json.

Allowed number of parameters: 0–2

Parameters

  • key (string): name of the key to set or get (for reference, rts) (optional)
  • value (any): new value of that setting (optional)

Returns

  • value (any)

createcreatetabletrigger

registers a lifecycle trigger that fires synchronously after a future createtable for the given schema/table succeeds

Allowed number of parameters: 6–6

Parameters

  • schema (string): name of the database
  • table (string): name of the table to watch for creation
  • name (string): name of the trigger
  • source_sql (string): original SQL body text (for diagnostics)
  • body (any): trigger body (Scheme procedure or deferred trigger expression)
  • visible (bool): true = user trigger, false = internal trigger

Returns

  • value (bool)

dropcreatetabletrigger

removes a registered create-table lifecycle trigger

Allowed number of parameters: 4–4

Parameters

  • schema (string): name of the database
  • table (string): name of the table watched for creation
  • name (string): name of the trigger
  • ifexists (bool): don't throw error if trigger doesn't exist

Returns

  • value (bool)

createtrigger

creates a new trigger on a table

Allowed number of parameters: 6–6

Parameters

  • table (table)
  • name (string): name of the trigger
  • timing (string): one of: before_insert, after_insert, before_update, after_update, before_delete, after_delete
  • source_sql (string): original SQL body text (for SHOW TRIGGERS)
  • body (any): trigger body (parsed Scheme expression)
  • visible (bool): true = user trigger (shown in SHOW TRIGGERS), false = internal trigger (hidden)

Returns

  • value (bool)

droptrigger

removes a trigger from a table

Allowed number of parameters: 3–3

Parameters

  • schema (string): name of the database
  • name (string): name of the trigger
  • ifexists (bool): don't throw error if trigger doesn't exist

Returns

  • value (bool)

mysql_import

imports schema+data from a MySQL server into MemCP

Allowed number of parameters: 4–8

Parameters

  • host (string|nil): MySQL host (nil => 127.0.0.1)
  • port (int|nil): MySQL port (nil => 3306)
  • username (string): MySQL username
  • password (string): MySQL password
  • sourcedb (string|nil): source database (omit/nil => all non-system dbs) (optional)
  • targetdb (string|nil): target database (omit/nil => sourcedb) (optional)
  • sourcetable (string|nil): source table (omit/nil => all tables in sourcedb) (optional)
  • targettable (string|nil): target table (omit/nil => sourcetable) (optional)

Returns

  • value (bool)

psql_import

imports schema+data from a PostgreSQL server into MemCP

Allowed number of parameters: 4–9

Parameters

  • host (string|nil): PostgreSQL host (nil => 127.0.0.1)
  • port (int|nil): PostgreSQL port (nil => 5432)
  • username (string): PostgreSQL username
  • password (string): PostgreSQL password
  • sourcedb (string|nil): source database (omit/nil => all non-system dbs) (optional)
  • sourceschema (string|nil): source schema (omit/nil => all non-system schemas in sourcedb) (optional)
  • targetdb (string|nil): target database (omit/nil => sourcedb) (optional)
  • sourcetable (string|nil): source table (omit/nil => all tables in sourceschema) (optional)
  • targettable (string|nil): target table (omit/nil => sourcetable) (optional)

Returns

  • value (bool)
Retrieved from "https://www.memcp.org/index.php?title=Storage&oldid=358"