CLI reference

The acid command-line tool runs the same engine as the Python API. Use it for one-off queries from a shell prompt, for catalog downloads and inspection, and for building margin caches.

This page lists every subcommand and every flag, with defaults read straight from src/acid_cli/. For the design rationale behind configuration discovery and --db resolution, see docs/archive/CONFIG-SYSTEM.md.

Synopsis

acid [--config FILE] {query|validate|config|search|list|download|inspect|hats|web} ...
acid --version

Global options

Flag	Description
--version	Print version, copyright, and license; exit.
--config FILE	Path to an acid.conf (overrides ACID_CONFIG and the default search).

acid query — run a SQL query

acid query [QUERY|-] [-f FILE]
           [--db ENTRIES] [--open PATH,RA,DEC]
           [--output PATH] [--format {hats,parquet,csv,fits}]
           [--workers N] [--threads N] [--ram-budget SIZE] [--tmpdir DIR]
           [--cone RA,DEC,RADIUS_DEG]
           [--progress {auto,on,off,plain}]

Reads the SQL query from a positional argument, from stdin (when the positional is -), or from a file (-f FILE). Exactly one of these is required.

Where to write results

--output	--format	Effect
omitted	(rejected if given)	Pretty-print to stdout (fixed-width on a TTY, TSV when piped).
directory or path with no extension	inferred / hats	Write a partitioned HATS tree at --output.
*.parquet / *.pq	inferred / parquet	Single Parquet file.
*.csv	inferred / csv	Single CSV file.
*.fits / *.fit	inferred / fits	Single FITS binary table.

If you set --format explicitly, it wins over the extension; an unrecognized extension with no --format is an error rather than a silent fallback to HATS. A global-reduce query (aggregate / top-K) cannot be written as a HATS tree — use single-file format, or omit --output to pretty-print.

Flags

Flag	Default	Description
-f, --file FILE	—	Read the SQL query from a file.
--db ENTRIES	$ACID_PATH → config path → ~/datasets	':'-separated list of HATS directories, registry YAMLs, or both. Leftmost entry wins on name collisions. A directory entry also exposes loose top-level <name>.parquet files as single-file (unsharded) tables — referenceable by bare name as a JOIN right-hand side (a same-named HATS catalog dir wins).
--open PATH,RA,DEC	—	Use a raw data file (.parquet/.csv/.tsv/.fits/.arrow/VOTable) as a named table in the query, alongside the --db catalogs (a virtual catalog: spilled once, queried like HATS). Two forms: positional PATH,RA,DEC (table name = file basename), or named NAME=PATH,ra=RA,dec=DEC. The RA/DEC coordinate column names are required (never guessed). Repeatable.
--output PATH	—	Output path; omit to pretty-print to stdout.
--format FMT	inferred from --output (no ext → hats)	One of hats, parquet, csv, fits. Rejected without --output.
--workers N	resolved from config / ACID_WORKERS / "auto" (cgroup-aware)	Process-pool size.
--threads N	cpu_cap // workers (cgroup-aware)	Per-worker Polars thread budget.
--ram-budget SIZE	25 % of available RAM (cgroup-aware)	Total RAM the planner budgets for; work tuples are sized to fit SIZE / workers each. Bytes or a human size (64GB, 512MiB, 32g). See Performance — RAM budget.
--tmpdir DIR	config / $TMPDIR	Base scratch directory; a per-run subdir is created and removed on exit.
--cone RA,DEC,RADIUS_DEG	—	Restrict the query to a sky cone (degrees, ICRS).
--progress MODE	auto	auto (animated bar on a TTY, silent when piped), on (force bar), off (silent), plain (one committed line per stage; good for logs).

Examples

# Pretty-print to stdout — no --output.
acid query "SELECT COUNT(*) FROM gaia_dr3" --db /data/hats

# Write a HATS tree (default --format when --output has no extension).
acid query -f xmatch.sql --db catalogs.yaml --output results/ --workers 32

# Write a single CSV — extension picks the format.
acid query "SELECT id, ra, dec FROM object LIMIT 1000" \
    --db /data/hats --output sample.csv

# Read SQL from stdin.
echo "SELECT * FROM object LIMIT 10" | acid query - --db /data/hats

# Restrict to a 2 deg cone around (50, -50).
acid query "SELECT g.id, t.designation FROM gaia g JOIN two_mass t \
            ON XMATCH(radius_arcsec => 1.0)" \
    --db /data/hats --cone 50,-50,2

# Crossmatch a raw CSV target list (named --open form) against a catalog.
acid query "SELECT t.id, g.source_id FROM t \
            JOIN gaia_dr3 ON XMATCH(radius_arcsec => 1.0)" \
    --db /data/hats --open t=candidates.csv,ra=RA,dec=DEC

# Cap the RAM the planner budgets for (work tuples shrink to fit).
acid query -f big.sql --db /data/hats --workers 32 --ram-budget 64GB

acid validate — parse + analyze, don't run

acid validate [QUERY|-] [-f FILE] [--db ENTRIES]

Same --db / query-source rules as acid query. Prints the analyzed plan summary (anchor, joins, projection, aggregation, ordering, footprint filters) and exits. Use it to catch ParseError / ValidationError before kicking off a long run.

acid config — show or edit acid.conf

acid config show [--effective]
acid config get  KEY [--effective]
acid config set  KEY VALUE
acid config unset KEY

show lists the file's settings. --effective shows the resolved values (with provenance: env > config > built-in) plus the search order ACID walked.

get KEY prints the file value (exits 1 silently if unset) — use --effective to print the resolved value instead.

set and unset rewrite the target file in place; comments and formatting are not preserved.

Valid keys: path, download_path, workers, reader_workers, mem_per_worker_gb, tmpdir, inmem_row_limit, workers_jemalloc_conf, ram_budget, margin_arcsec, skymap_style, skymap_cmap.

acid search — list downloadable catalogs

acid search [PATTERN]
            [--cache {use,refresh,off}]
            [--timeout SEC] [--insecure] [--no-color]

Crawls every root on the download path ($ACID_DOWNLOAD_PATH → config download_path → built-in default) over its native transport — local directories, ssh:// hosts, and http(s):// mirrors alike — and lists the HATS catalogs available to acid download. One line per catalog; the name is the token to hand to acid download.

Flag	Default	Description
PATTERN	—	Show only catalogs whose name contains this text (case-insensitive substring).
--cache {use,refresh,off}	use	Remote-listing cache mode: use serves a fresh-enough cached listing (remote roots are cached ~1h under $XDG_CACHE_HOME/acid/downloads), refresh forces a re-crawl and rewrites the cache, off bypasses it entirely (neither read nor write).
--timeout SEC	300	Per-request timeout in seconds (HTTP / SSH).
--insecure	off	Skip TLS cert verification (HTTPS only — self-signed mirrors).
--no-color	off	Disable ANSI color.

Output

Catalog rows go to stdout; the searched roots, the count, and any shadowing footnote go to stderr as quiet context (so acid search | … pipes only the data).

• On a terminal — an aligned, colored table, with a live braille spinner and a running catalog count while the crawl is in flight. Each row reads NAME <N> rows · margins: <widths> arcsec (or margins: none); the row count comes free from the catalog's hats_nrows property (omitted when absent), and the margin widths are the margin-cache radii (arcsec) available for the catalog. With more than one download root, the row also shows a compact root label; a catalog shadowed by an earlier root carries a trailing *.
• When piped — clean tab-separated lines, name⇥kind⇥nrows⇥margins⇥root⇥marker, where kind is hats_catalog, nrows is the row count (empty when unknown), and marker is shadowed (the resolved copy leaves it empty). This is stable for scripting.

Catalogs grouped under a namespace directory surface as namespace/child (e.g. wise/allwise) — the exact token acid download accepts. HATS collections are listed as plain catalog names; the collection structure is read internally only to find their margins.

Shadowing. Every occurrence of a name across the roots is shown, but acid download <name> resolves first-wins, so a same-named catalog at a later root is flagged shadowed (trailing * on a TTY; the shadowed 6th TSV column when piped). The summary reports the count, e.g. ✓ 12 catalogs (2 shadowed). Download a shadowed copy by its explicit URL/path, not by name.

Examples

# Everything available on the download path.
acid search

# Only catalogs whose name contains "gaia".
acid search gaia

# Force a fresh crawl past the ~1h cache.
acid search --cache refresh

# Names of only the catalogs `acid download` would actually fetch
# (drop the shadowed ones, whose 6th column is non-empty).
acid search | awk -F'\t' '$6 == "" { print $1 }'

acid list — list catalogs you can open

acid list [PATTERN]
          [--cache {use,refresh,off}]
          [--timeout SEC] [--insecure] [--no-color]

The local twin of acid search. It uses the same discovery engine, transports, flags, and output contract — but crawls the catalog path ($ACID_PATH → config path → built-in default ~/datasets) instead of the download path. So acid search answers what can I download, while acid list answers what's already here that I can open by name — the catalogs acid open / acid query resolve a bare name against, first-wins across the roots.

The flags, the TTY-table-vs-piped-TSV output, the namespace/child names, the margin-radii column, the row counts, and the shadowing rules are all exactly as documented under acid search, with two differences: the shadowing footnote names acid open (which resolves the first match) rather than acid download, and acid list also lists single-file (unsharded) tables.

Unsharded tables. A loose top-level <name>.parquet under a catalog-path root — the kind of file acid import --single-file writes — is an unsharded table: openable by name as a join-RHS lookup (it has no coordinates, no margins). acid list shows it alongside the catalogs, distinguished by kind: a TTY row reads NAME <N> rows · table (unsharded), and the piped kind column is parquet_table (vs hats_catalog). They're local-only and top-level-only — exactly the files acid open <name> resolves. acid search does not list them (a loose parquet isn't a downloadable archive). If a root has both a HATS catalog foo/ and a loose foo.parquet, the catalog wins resolution and the table is shown shadowed.

Examples

# Everything you can open by name — HATS catalogs and unsharded tables.
acid list

# Only the ones whose name contains "gaia".
acid list gaia

# Just the names (piped → TSV).
acid list | cut -f1

# Only the single-file tables (kind column = parquet_table).
acid list | awk -F'\t' '$2 == "parquet_table" { print $1 }'

acid download — fetch a HATS catalog

acid download SOURCE [DEST]
              [--cone RA,DEC,RADIUS_DEG]
              [--columns COL,COL,...]
              [--workers N] [--timeout SEC]
              [--estimate] [--prefetch-metadata]
              [--skip-margin] [--insecure]
              [--tmpdir DIR]

Flag	Default	Description
SOURCE	required	A catalog name to resolve against the download path, or an explicit source. A name is a bare token (gaia_dr3) or a nested namespace/child (wise/allwise, as acid search surfaces). An explicit source — a URL, an SSH user@host:path, or a local path with a leading ./ / / / ~ — is used verbatim. (To copy from a local relative directory, prefix it with ./; a bare token is always a name lookup.)
DEST	optional for a name	Destination directory. A path (containing a /) or URL is used as-is; a bare token resolves to <ACID_PATH-root>/<token>. Omit it for a name source — it defaults to <ACID_PATH-root>/<leaf> (a nested name's last segment, so wise/allwise → <ACID_PATH>/allwise). An explicit source requires an explicit dest.
--cone RA,DEC,RADIUS_DEG	—	Download only partitions overlapping this cone (degrees, ICRS).
--columns COL,COL,...	all	Download only the named columns (HATS-required columns like RA/Dec/_healpix_29 are always included).
--workers N	8	Parallel download workers.
--estimate	off	Print estimated bytes/files and exit without downloading.
--prefetch-metadata	off	Force fetching _metadata for exact sizes / bulk byte ranges.
--skip-margin	off	Don't download the catalog's margin cache.
--timeout SEC	300	Per-file timeout (HTTP only).
--insecure	off	Skip TLS cert verification (HTTPS only — self-signed certs, testing).
--tmpdir DIR	$TMPDIR	Scratch directory for the point_map.fits mmap build. Keep on fast local storage when DEST is networked.

Downloaded subsets are valid HATS catalogs with a rebuilt _metadata and a regenerated point_map.fits footprint. If any file fails after retries, the download exits non-zero — there is no "partial success" mode, on purpose (a half-downloaded catalog looks structurally valid).

The built-in download path is two roots, searched first-wins: https://data.lsdb.io/hats/ then ssh://slacd/sdf/home/m/mjuric/datasets. Override it with $ACID_DOWNLOAD_PATH or an acid.conf download_path = line. Use acid search to see what each root offers and which copy a name resolves to.

Examples

# By name — resolved against the download path; lands under <ACID_PATH>/gaia_dr3.
acid download gaia_dr3

# Nested name — lands under its leaf, <ACID_PATH>/allwise.
acid download wise/allwise

# Full catalog over an explicit HTTP URL (explicit source needs an explicit dest).
acid download https://data.lsdb.io/hats/two_mass/two_mass /data/two_mass

# 5-deg cone, column subset, over SSH.
acid download user@server:/hats/gaia /data/gaia \
    --cone 180,0,5 --columns ra,dec,phot_g_mean_mag

# Copy from a LOCAL directory: prefix ./ or / so it isn't a name lookup.
acid download ./mirror/two_mass /data/two_mass

# Estimate first.
acid download https://data.lsdb.io/hats/two_mass/two_mass /data/two_mass \
    --cone 50,-50,2 --estimate

acid import — build a HATS catalog

# Pick the source kind: a file format, a Butler dataset, or peek.
acid import {parquet,csv,fits,arrow} SOURCES... --out PATH|NAME --ra COL --dec COL
            [--schema FILE] [--columns A,B,...]
            [--storage-option KEY=VALUE]... [--insecure] [--from-list URL|PATH]
            [--mode {auto,inmem,shuffle}] [--single-file [--sort COL]]
            [--margin-arcsec R] [--no-margin]
            [--workers N] [--reader-workers N] [--ram-budget BYTES] [--tmpdir DIR]
            [--order K] [--max-order M] [--rows-per-partition N]
            [--skymap-style {quadrant,blocks,braille,off}]
            [--skymap-cmap {auto,gray,viridis,inferno,magma,plasma,none}]
            [--overwrite]
acid import csv    SOURCES... [--no-header] [--comment-char CHAR] [--delimiter CHAR] …
acid import butler --repo REPO --collection COLL --dataset DS [--where EXPR] --out … --ra … --dec … …
acid import peek   SOURCES... [--list] [--storage-option …] [--from-list URL|PATH] [--no-header] [--delimiter …]

Reads tabular files (FITS, CSV/TSV, parquet, Arrow) — local or on a remote fsspec store — or a Rubin Butler dataset, and writes a HATS catalog, building a margin cache afterward by default. The first word names the source kind: a file format (parquet/csv/fits/arrow), a Rubin butler dataset, or peek (inspect without importing). Naming the format keeps each format's options under its own verb and lets a file whose extension doesn't say its type still import (acid import csv survey.dat). The import path is chosen from the input size: in memory when it fits, else an out-of-core shuffle (remote inputs always shuffle). See the importing guide for the task-shaped walkthrough.

Flags shared by every file-format verb (parquet/csv/fits/arrow) and butler:

Flag	Default	Description
SOURCES	required (or --from-list)	One or more files, directories, or quoted globs — local paths or remote scheme:// URLs (http(s):// out of the box; s3:// / gs:// / az:// with the backend package installed). Read as the chosen format, with the same columns (order may differ). A directory/glob imports every data file inside; over http(s):// the server's directory index (autoindex) is scraped, non-data links ignored — for a mirror with no index, use --from-list. (Not for butler.)
--out PATH\|NAME	required	Output catalog. A bare NAME joins the catalog library (<first writable ACID_PATH root>/NAME); a path (has a /) is used verbatim.
--ra COL / --dec COL	required	RA / Dec column names. Never guessed — use acid import peek to find them.
--schema FILE	inferred	YAML/JSON declaring column types (and order). Recommended for CSV. Positional form (name: {at: N, type: …}) names headerless columns.
--columns A,B,...	all	Output columns + order. RA/Dec kept automatically. Slims wide catalogs (e.g. dropping FITS array columns).
--storage-option KEY=VALUE	—	fsspec backend option for a remote SOURCES (repeatable; true/false/null/integers coerced). E.g. anon=true, endpoint_url=https://…, key=…/secret=…. (Not for butler.)
--insecure	off	Skip TLS cert verification for a remote HTTPS SOURCES (self-signed certs / private mirrors only). (Not for butler.)
--from-list URL\|PATH	—	Read the input file list from a manifest instead of a directory index (for HTTP mirrors that serve none). A newline-delimited list of names/URLs, or a SHA256SUMS-style checksums file (auto-detected); # comments/blank lines skipped; relative names resolve against the manifest's dir. SOURCES may then only be glob pattern(s) that filter the list (or be omitted to import every listed file). (Not for butler.)
--mode {auto,inmem,shuffle}	auto	auto picks in-memory vs out-of-core by input size (remote inputs always shuffle). Force a path for testing. (File verbs only; butler is always shuffle.)
--single-file	off	Write one flat file instead of a HATS tree — a format converter / concatenator. No spatial index, so --ra/--dec are optional. Output format = the --out extension (.parquet/.csv/.fits), else parquet; a bare --out NAME lands under ACID_PATH as <NAME>.parquet, re-openable by name as an unsharded table. In-memory: the (sorted) output must fit RAM. Works for the file verbs and butler (concatenate every selected dataref — handy for non-spatial visit/exposure metadata tables).
--sort COL	—	With --single-file, write the output sorted ascending by COL (in memory; NULLs last).
--margin-arcsec R	10.0	Radius for the auto-built margin cache. Make it ≥ the widest XMATCH radius you'll run against the catalog.
--no-margin	off	Skip the margin build (do it later with acid hats build-margin).
--workers N	auto	Parallel workers (from RAM/cores).
--reader-workers N	same as --workers	Workers for the read + shuffle phase (Pass A) only; the merge + write phase stays at --workers. Raise for a read-bound source, lower to cap reader memory. Resolves --reader-workers → $ACID_READER_WORKERS → reader_workers (config) → --workers. Shuffle mode only.
--ram-budget BYTES	0.5 × available	RAM for the in-core shuffle buffer (e.g. 64GB).
--tmpdir DIR	$TMPDIR	Scratch base for out-of-core spills.
--order K / --max-order M / --rows-per-partition N	6 / 10 / 1e6	Shuffle-mode HEALPix-order and partition-size knobs.
--skymap-style STYLE	quadrant	Style of the end-of-import sky map (drawn on a TTY): quadrant (area-faithful), blocks (classic ░▒▓█ density ramp), braille (finer), or off. Resolves flag → $ACID_SKYMAP_STYLE → skymap_style (config).
--skymap-cmap CMAP	auto	Density colormap for the sky map: auto (magma on a dark terminal, gray on a light one — detected via OSC 11), explicit gray/viridis/inferno/magma/plasma, or none for monochrome. Resolves flag → $ACID_SKYMAP_CMAP → skymap_cmap (config).
--overwrite	off	Replace an existing catalog at --out.

acid import csv adds the CSV/TSV read options:

Flag	Default	Description
--no-header	off	CSV/TSV has no header row; needs a positional --schema.
--comment-char CHAR	—	Skip CSV/TSV lines starting with CHAR (e.g. #).
--delimiter CHAR	, / tab	CSV/TSV field separator (e.g. \|). Defaults: comma for .csv, tab for .tsv.

acid import butler (requires the LSST stack; always uses the shuffle):

Flag	Default	Description
--repo / --collection / --dataset	required (together)	Rubin Butler repo path/alias, collection, and dataset type to import. A glob --collection imports the UNION of the dataset type across all matching collections (deduped by data ID, newest wins); --dataset must be a single exact type.
--where EXPR	—	Butler query expression limiting which datasets are pulled (e.g. "band='r'").
--from-list PATH\|-	—	Import only the datarefs in a acid import peek butler … --list JSONL manifest (- = stdin). Each line is self-describing, so --collection/--dataset/--where don't apply (and are rejected); only --repo is needed. The manifest must list a single dataset type.

acid import peek SOURCES... summarizes the input (format, rows, column schema with each column's estimated byte size, a Bytes/row total, candidate RA/Dec, and the first 10 matched files) and exits without writing — auto-detecting the format, reading metadata + a small sample only. The byte sizes are measured where it's cheap (FITS column formats, the parquet footer, the CSV/Arrow sample) and a bucket default otherwise, so a list/array column shows its real per-row size; multiply Bytes/row by the row count to size --workers / --ram-budget. With --list it instead prints the full matched-file list (one per line — local paths relative to the cwd, remote URLs verbatim; plain when redirected, a count header on a terminal), for checking a fat glob / remote listing before committing. It takes --storage-option / --insecure / --from-list for remote sources and the CSV read options so its sample matches a real CSV import.

Like acid download, an import either completes or fails loudly — it never leaves a half-written catalog. A bare-name --out lands under ACID_PATH in both the in-memory and out-of-core paths.

Examples

# Inspect an input before importing — what are the coordinate columns?
acid import peek tractor-1126p222.fits

# Check exactly which files a directory / glob matched (scriptable when piped).
acid import peek '/data/legacy/tractor/**/*.fits' --list | wc -l

# One FITS file → a named catalog in the library (margin built by default).
acid import fits tractor-1126p222.fits --out legacy_dr10 --ra ra --dec dec

# A directory of FITS bricks → an explicit path, slimmed to a few columns.
acid import fits /data/legacy/tractor/ --out /data/hats/legacy --ra ra --dec dec \
    --columns ra,dec,type,flux_g,flux_r,flux_z

# Gzipped, '#'-commented CSV with a declared schema.
acid import csv gaia_chunk.csv.gz --out gaia_chunk --ra ra --dec dec \
    --comment-char '#' --schema gaia.schema.yaml

# Pipe-delimited catalog, no header, positional schema.
acid import csv psc.tbl --out two_mass_psc --ra ra --dec dec \
    --delimiter '|' --no-header --schema psc.schema.yaml

# Remote: a quoted S3 glob from a public bucket (peek it first, then import).
acid import peek 's3://open-data/cat/*.parquet' --storage-option anon=true
acid import parquet 's3://open-data/cat/*.parquet' --out cat --ra ra --dec dec \
    --storage-option anon=true

# Remote http directory (autoindex scraped) and, for an index-less mirror, a manifest.
acid import parquet https://example.org/dr1/ --out dr1 --ra ra --dec dec
acid import parquet --from-list https://example.org/dr1/SHA256SUMS --out dr1 --ra ra --dec dec

# From a Rubin Butler.
acid import butler --repo /repo/main \
    --collection LSSTComCam/runs/DRP/DP1 --dataset object \
    --out dp1_object --ra coord_ra --dec coord_dec --where "band='r'"

# Curate a set of datarefs with `peek … --list`, then import exactly those.
acid import peek butler --repo /repo/main \
    --collection 'LSSTComCam/runs/DRP/*' --dataset object --list \
  | acid import butler --repo /repo/main --from-list - \
        --out dp1_object --ra coord_ra --dec coord_dec

acid inspect — show catalog info

acid inspect [MODE] SOURCE

MODE is one of summary (default), schema, or properties. SOURCE is a bare catalog name (resolved against the catalog path, like acid query), a local path, an HTTP URL, or a user@host:path SSH path.

acid inspect /data/two_mass                       # summary
acid inspect schema /data/two_mass                # column schema
acid inspect properties /data/two_mass            # HATS properties
acid inspect two_mass                             # bare name → catalog path
acid inspect https://data.lsdb.io/hats/two_mass/two_mass

It also inspects single-file (unsharded) tables — a loose <name>.parquet (the output of acid import --single-file, openable by name as a join-RHS lookup). acid inspect visit and acid inspect schema visit render a table summary / column schema from the parquet footer; properties reports that a flat table has no HATS properties. An explicit path to a raw file (acid inspect ./visits.parquet) works the same way.

acid inspect visit                                # single-file table summary
acid inspect schema visit                         # its column schema

acid hats — HATS catalog operations

acid hats build-margin CATALOG
                       [--margin-arcsec ARCSEC]
                       [--workers N] [--output DIR]
                       [--overwrite] [--mem-limit GB]
                       [--tmpdir DIR]

Builds a HATS margin cache for CATALOG. Output passes hats.read_hats(...) and matches the upstream hats-import tool.

Flag	Default	Description
CATALOG	required	Path to a local HATS catalog.
--margin-arcsec ARCSEC	10.0	Margin threshold in arcseconds. Must be at least as large as the largest XMATCH radius you'll ever run against this catalog (see margin caches).
--workers N	cgroup-aware cpu_cap	Parallel workers.
--output DIR	<catalog>_<margin>arcsec	Output directory.
--overwrite	off	Overwrite an existing margin cache at --output.
--mem-limit GB	10 % of RAM	Memory limit before the accumulator spills to disk.
--tmpdir DIR	spill under the output directory	Base directory for the spill scratch (must already exist). Point at fast local storage when --output is on a slow networked filesystem.

Example

acid hats build-margin /data/two_mass --margin-arcsec 10.0 --workers 16

acid web — publish a catalog info page

Builds and serves a self-contained, static "should I use this catalog?" website from a catalog's HATS metadata. Three verbs — see the publishing guide for the full workflow.

acid web init  [DIR] [--title TEXT] [--subtitle TEXT] [--url URL]
acid web add   CATALOG [--into DIR] [--meta FILE]
acid web rm    CATALOG [--from DIR] [--purge]
acid web list  [DIR]
acid web serve [DIR] [--host HOST] [--port PORT]

Verb	Description
init [DIR]	Scaffold a servable web directory (index.html + empty datasets/ + an index.json manifest + the .acid.webdir marker). DIR defaults to the current directory and must be new or empty.
add CATALOG	Resolve CATALOG (a name via ACID_PATH, or a local path), generate <catalog>/acid.web/{card.json,skymap.png} from its metadata, symlink the catalog into <webdir>/datasets/ (relative), and record it in the manifest. --into is the web directory (default: current dir).
rm CATALOG	The inverse of add: drop the catalog's manifest entry and its datasets/ symlink. CATALOG is the id shown in the site. The catalog's data is never touched; --purge additionally deletes the catalog's generated acid.web/ (its card + sky map). --from is the web directory (default: current dir).
list [DIR]	List the catalogs linked into the web directory (manifest order): id, display name, row count, column count, and the symlink target. Aligned on a terminal, TSV when piped; a dangling entry is flagged. DIR defaults to the current directory.
serve [DIR]	Serve the web directory over HTTP and print the URL (no browser is opened). DIR defaults to the current directory. Default bind is 127.0.0.1 (loopback only); pass --host 0.0.0.0 to reach it from other hosts.

The technical fields (rows, partitions, schema, sky coverage, the density sky map) come straight from the catalog. Editorial fields (blurb, license, citation, source, download URL) come from an optional acid-web.yaml sidecar next to the catalog (or --meta FILE); when none exists, add writes a starter acid-web.yaml you can fill in and re-run.

Examples

# Build a site, add two catalogs, serve it.
acid web init ./catalog-site
acid web add gaia_dr3 --into ./catalog-site
acid web add two_mass --into ./catalog-site
acid web serve ./catalog-site            # → http://localhost:8000/

# From inside the web directory, --into / DIR default to the current dir.
cd ./catalog-site
acid web add sdss_dr17
acid web list                            # what's in the site
acid web rm  sdss_dr17                    # take it back out
acid web serve

Environment variables

A few env vars influence CLI behavior; the full list (including the allocator and worker-startup knobs) is in MEMORY-TUNING.md. The ones a user is most likely to set:

Variable	Effect
ACID_PATH	Default value for --db (overrides config path).
ACID_WORKERS	Default for --workers.
ACID_MEM_PER_WORKER_GB	RAM/worker bound for --workers auto.
ACID_RAM_BUDGET	Default for --ram-budget (bytes or 64GB/512MiB).
ACID_TMPDIR	Default for --tmpdir.
ACID_INMEM_ROW_LIMIT	Spill threshold (rows).
ACID_CONFIG	Path to a specific acid.conf.
ACID_PROGRESS	0 to silence, 1 to force the bar, plain for committed lines per stage.
ACID_PROFILE	1 to print a per-step profile to stderr.
ACID_PROFILE_OUT	Path for the full per-worker profile JSON.