Wikiservice: Refresh MemCP documentation: accuracy, operational guidance, performance profile and maintained API reference

2026-08-28T10:14:01Z

Refresh MemCP documentation: accuracy, operational guidance, performance profile and maintained API reference

← Older revision		Revision as of 12:14, 28 August 2026
Line 16:		Line 16:
	! Workload !! Observed result !! Interpretation		! Workload !! Observed result !! Interpretation
	\|-		\|-
	\| OLAP, aggregation, and search-oriented workflows \|\| ~~Up to~~ '''10× ~~faster~~''' ~~than~~ MariaDB/PostgreSQL in measured cases \|\| RecSets, compressed column scans, batching, late materialization, and parallel execution match the workload		\| OLAP, aggregation, and search-oriented workflows \|\| '''Speedups of 10× and more''' over MariaDB/PostgreSQL in measured cases \|\| RecSets, compressed column scans, batching, late materialization, and parallel execution match the workload
	\|-		\|-
	\| Conventional OLTP application paths \|\| Currently about '''1.3–2.0× the elapsed time''' in measured cases \|\| Broader JIT coverage, compile cost, point paths, and connector compatibility remain optimization work		\| Conventional OLTP application paths \|\| Currently about '''1.3–2.0× the elapsed time''' in measured cases \|\| Broader JIT coverage, compile cost, point paths, and connector compatibility remain optimization work
Line 22:		Line 22:
	\| Complete WordPress- and wiki-style page builds \|\| '''No significant difference in overall page-loading time''' in the measured application workflows \|\| End-to-end impact depends on application work and on the number and mix of SQL queries		\| Complete WordPress- and wiki-style page builds \|\| '''No significant difference in overall page-loading time''' in the measured application workflows \|\| End-to-end impact depends on application work and on the number and mix of SQL queries
	\|-		\|-
	\| ~~PostgreSQL filtered lists~~ over roughly one million documents \|\| Around '''30 seconds''' in the ~~observed application workflow~~ \|\| The search/list path is the urgent migration candidate even if the smaller OLTP path is not yet faster		\| Filtered list over roughly one million documents \|\| Around '''30 seconds on PostgreSQL''' versus '''1.6 seconds on MemCP''' for the same query \|\| The search/list path is the urgent migration candidate even if the smaller OLTP path is not yet faster
	\|-		\|-
	\| Write-heavy tables using <code>ENGINE=logged</code> \|\| About '''10× the write throughput''' of <code>safe</code> in project measurements \|\| Avoids commit-time <code>fsync</code>, but recent committed writes can be lost on kernel crash or power failure		\| Write-heavy tables using <code>ENGINE=logged</code> \|\| About '''10× the write throughput''' of <code>safe</code> in project measurements \|\| Avoids commit-time <code>fsync</code>, but recent committed writes can be lost on kernel crash or power failure
Line 56:		Line 56:
	The difficult shape is not a plain LIMIT. It combines selective or correlated filters with an order that favors another access path:		The difficult shape is not a plain LIMIT. It combines selective or correlated filters with an order that favors another access path:

	<~~syntaxhighlight lang="sql"~~>SELECT id, title, changed_at		<pre>SELECT id, title, changed_at
	FROM documents		FROM documents
	WHERE tenant_id = ?		WHERE tenant_id = ?
Line 64:		Line 64:
	AND p.user_id = ?)		AND p.user_id = ?)
	ORDER BY changed_at DESC		ORDER BY changed_at DESC
	LIMIT 100;</~~syntaxhighlight~~>		LIMIT 100;</pre>

	A conventional optimizer may walk the ordering index and reject rows until it finds 100 matches, or materialize/filter a large intermediate result before sorting. MemCP can represent the permission/filter domain as a RecSet, intersect it with other boundaries, read ordered candidates in batches and stop when enough accepted rows have been emitted. See [[RecSets]], [[Scan]], and [[Advanced SQL Tutorial]].		A conventional optimizer may walk the ordering index and reject rows until it finds 100 matches, or materialize/filter a large intermediate result before sorting. MemCP can represent the permission/filter domain as a RecSet, intersect it with other boundaries, read ordered candidates in batches and stop when enough accepted rows have been emitted. See [[RecSets]], [[Scan]], and [[Advanced SQL Tutorial]].
Line 74:		Line 74:
	=== Search and filtered lists become slower as documents grow ===		=== Search and filtered lists become slower as documents grow ===

	Faceted lists combine permissions, tags, states, text/prefix filters, joins, counts, sorting and pagination. Their cost can grow with the candidate domain even though the user sees only 20 or 100 rows. This is the workflow in which project measurements have ~~reached up to~~ 10× MariaDB/PostgreSQL ~~performance and in which a PostgreSQL~~ list over roughly one million documents ~~was observed~~ around 30 seconds.		Faceted lists combine permissions, tags, states, text/prefix filters, joins, counts, sorting and pagination. Their cost can grow with the candidate domain even though the user sees only 20 or 100 rows. This is the workflow in which project measurements have produced speedups of 10× and more over MariaDB/PostgreSQL. In one concrete list over roughly one million documents, the same query took around 30 seconds on PostgreSQL and 1.6 seconds on MemCP.

	RecSets are designed to carry compact membership between those operators without copying wide result rows. Ordered batch acceptance and late materialization then avoid doing full output work for every candidate.		RecSets are designed to carry compact membership between those operators without copying wide result rows. Ordered batch acceptance and late materialization then avoid doing full output work for every candidate.
Line 112:		Line 112:
	Start an isolated instance with persistent storage:		Start an isolated instance with persistent storage:

	<~~syntaxhighlight lang="bash"~~>docker volume create memcp_eval_data		<pre>docker volume create memcp_eval_data
	docker run -d --name memcp-eval \		docker run -d --name memcp-eval \
	-e ROOT_PASSWORD='choose-a-password' \		-e ROOT_PASSWORD='choose-a-password' \
	-v memcp_eval_data:/data \		-v memcp_eval_data:/data \
	-p 127.0.0.1:4321:4321 -p 127.0.0.1:3307:3307 \		-p 127.0.0.1:4321:4321 -p 127.0.0.1:3307:3307 \
	carli2/memcp:latest</~~syntaxhighlight~~>		carli2/memcp:latest</pre>

	Then import a copy of the relevant schema and data, run the real query through the same client path, and inspect its physical plan:		Then import a copy of the relevant schema and data, run the real query through the same client path, and inspect its physical plan:

	<~~syntaxhighlight lang="sql"~~>EXPLAIN PHYSICAL		<pre>EXPLAIN PHYSICAL
	SELECT ...;		SELECT ...;
	</~~syntaxhighlight~~>		</pre>

	Do not stop at a synthetic <code>SELECT 1</code>. Test the query that users actually wait for, with representative cardinalities, concurrency, authentication, durability and cold/warm states. [[Migration from MySQL and PostgreSQL]] describes a staged evaluation; [[Advanced SQL Tutorial]] explains the optimizations visible in the plan.		Do not stop at a synthetic <code>SELECT 1</code>. Test the query that users actually wait for, with representative cardinalities, concurrency, authentication, durability and cold/warm states. [[Migration from MySQL and PostgreSQL]] describes a staged evaluation; [[Advanced SQL Tutorial]] explains the optimizations visible in the plan.

Wikiservice: Refresh MemCP documentation: accuracy, operational guidance, performance profile and maintained API reference

2026-08-28T09:59:33Z

Refresh MemCP documentation: accuracy, operational guidance, performance profile and maintained API reference

Show changes

Carli: Created page with " =MemCP — Faster alternative when "MySQL is too slow"= '''Short description:''' MemCP is an in-memory, column-oriented database optimized for high throughput OLAP and OLTP workloads. It is designed to deliver much faster aggregation and query performance than traditional row-based engines like MySQL. ==Overview== MemCP is a blazing-fast, in-memory, columnar database with built-in REST APIs and a lightweight SQL frontend. It is intended for modern workloads where MySQL..."

2025-09-04T12:40:13Z

Created page with " =MemCP — Faster alternative when "MySQL is too slow"= '''Short description:''' MemCP is an in-memory, column-oriented database optimized for high throughput OLAP and OLTP workloads. It is designed to deliver much faster aggregation and query performance than traditional row-based engines like MySQL. ==Overview== MemCP is a blazing-fast, in-memory, columnar database with built-in REST APIs and a lightweight SQL frontend. It is intended for modern workloads where MySQL..."

New page

=MemCP — Faster alternative when "MySQL is too slow"=
'''Short description:''' MemCP is an in-memory, column-oriented database optimized for high throughput OLAP and OLTP workloads. It is designed to deliver much faster aggregation and query performance than traditional row-based engines like MySQL.
==Overview==
MemCP is a blazing-fast, in-memory, columnar database with built-in REST APIs and a lightweight SQL frontend. It is intended for modern workloads where MySQL becomes a bottleneck for aggregation and analytic queries.
==Key benefits==
*'''In-memory, columnar storage''' — very fast for aggregations and analytics.
*'''MySQL-compatible SQL frontend''' — easy to try alongside existing MySQL deployments.
*'''Low memory footprint for base system''' — minimal system requirements for core system.
*'''Open source''' — source code and history available on GitHub.
==When to consider MemCP==
*Your MySQL instance struggles with large aggregations or analytics queries.
*You want an in-memory option that provides much faster group/aggregation performance.
*You need a developer-friendly, REST-accessible datastore for microservices.
==Quickstart / Installation==
A basic example (high level):<pre>
# 1) Clone the repository
git clone https://github.com/launix-de/memcp.git

# 2) Build / run according to README (see GitHub repo for exact commands)
cd memcp
# follow build/run instructions from repository README
</pre>
==Example usage==
*MemCP runs a small management service and exposes a MySQL-compatible port for clients (see documentation and examples on the project pages).
*For microservices, MemCP provides example code showing prepared statements and a simple key/value microservice using the built-in SQL frontend.
==Benchmarks & comparisons==
MemCP's documentation and comparison pages describe performance improvements versus MySQL for aggregation workloads (examples and benchmark summaries are available on the project site). Please consult the project's "Comparison: MemCP vs. MySQL" page and published benchmarks for details.
==Links / Resources==
*Official site: https://memcp.org
*Project & source: https://github.com/launix-de/memcp
==License==
MemCP is published under GPL 3 in the GitHub repository
==See also==
*[[Comparison: MemCP vs. MySQL]]
*[[Deployment]]
*[[MemCP_for_Microservices]]

MySQL is too slow - Revision history

Wikiservice: Refresh MemCP documentation: accuracy, operational guidance, performance profile and maintained API reference

Wikiservice: Refresh MemCP documentation: accuracy, operational guidance, performance profile and maintained API reference