Storage Backends: Difference between revisions
Wikiservice (talk | contribs) (Refresh MemCP documentation: accuracy, operational guidance, performance profile and maintained API reference) |
Wikiservice (talk | contribs) (Refresh MemCP documentation: accuracy, operational guidance, performance profile and maintained API reference) |
||
| Line 10: | Line 10: | ||
The local backend uses a directory below <code>-data</code>. For S3 or Ceph, create a JSON database configuration in the data directory. An S3 configuration names endpoint, region, bucket, prefix, credentials, and optional path-style access. Ceph names cluster, user, pool, prefix, and optional config file. | The local backend uses a directory below <code>-data</code>. For S3 or Ceph, create a JSON database configuration in the data directory. An S3 configuration names endpoint, region, bucket, prefix, credentials, and optional path-style access. Ceph names cluster, user, pool, prefix, and optional config file. | ||
< | <pre>{ | ||
"backend": "s3", | "backend": "s3", | ||
"access_key_id": "...", | "access_key_id": "...", | ||
| Line 19: | Line 19: | ||
"prefix": "databases", | "prefix": "databases", | ||
"force_path_style": true | "force_path_style": true | ||
}</ | }</pre> | ||
Ceph is built with <code>make ceph</code> and requires <code>librados-dev</code>. Keep secrets outside version control and restrict config-file permissions. Test schema writes, column reload, WAL replay, listing/deletion, outage behavior, and backups against the actual service. Object storage latency affects cold reads, rebuilds, and WAL handling even though hot query execution is memory-oriented. | Ceph is built with <code>make ceph</code> and requires <code>librados-dev</code>. Keep secrets outside version control and restrict config-file permissions. Test schema writes, column reload, WAL replay, listing/deletion, outage behavior, and backups against the actual service. Object storage latency affects cold reads, rebuilds, and WAL handling even though hot query execution is memory-oriented. | ||
Latest revision as of 12:13, 28 August 2026
Storage Backends
MemCP can persist a database on the local filesystem, S3-compatible object storage, or optional Ceph/RADOS. ENGINE durability semantics still apply on top of the selected backend.
The local backend uses a directory below -data. For S3 or Ceph, create a JSON database configuration in the data directory. An S3 configuration names endpoint, region, bucket, prefix, credentials, and optional path-style access. Ceph names cluster, user, pool, prefix, and optional config file.
{
"backend": "s3",
"access_key_id": "...",
"secret_access_key": "...",
"region": "us-east-1",
"endpoint": "http://minio:9000",
"bucket": "memcp",
"prefix": "databases",
"force_path_style": true
}
Ceph is built with make ceph and requires librados-dev. Keep secrets outside version control and restrict config-file permissions. Test schema writes, column reload, WAL replay, listing/deletion, outage behavior, and backups against the actual service. Object storage latency affects cold reads, rebuilds, and WAL handling even though hot query execution is memory-oriented.