dlake v0.5.43

Backups arrive: dlake backup makes, lists, inspects and downloads restorable copies of your Data Lake's DLO schema, weekly by default and kept 30 days, with connector-fed and platform log tables backed up as schema only unless you choose otherwise. Every command after sign-in now goes through the MCP host, so the CLI needs no other Commercient host and no sign-in token is cached. Backups can also be portable (.bacpac), scoped per table, and consistent to a moment on servers that support snapshots.

New: dlake backup. Restorable backups of your Data Lake’s DLO schema: a native SQL Server backup file of a database holding only DLO — tables and their data, views, functions, procedures, triggers, row-level security, sequences and synonyms. backup run makes one now, backup list and backup show <id> list and inspect them (object counts, warnings and the restore statement), backup download <id> [--out <path>] downloads one and checks its SHA-256, and backup settings [--enabled on|off] [--schedule weekly|daily|off] [--include-sync-tables on|off] chooses whether and how often they are made, and what they hold. Weekly by default; backups are kept 30 days.
On shared SQL servers the temporary backup database is created, backed up and removed by a dedicated backup creator login that may create databases and nothing else; your Data Lake’s own login still makes the copy of your data, so the creator never reads it. An “insufficient rights” result now says which of the two logins is missing a right.
Connector-fed tables are backed up as schema only by default. Tables a connector sync writes are rebuilt from their source systems and are usually most of a Data Lake’s volume, so a backup now keeps their structure (columns, keys, indexes, constraints) without their rows; your own tables, views and procedures are backed up in full. backup show lists the schema-only tables with the rows each held. backup settings --include-sync-tables on includes their rows for a self-contained copy.
Platform log tables are always backed up as schema only. The platform’s own audit logs, notifications, run history and event queues keep their structure in the backup but not their rows, whatever --include-sync-tables says: they record what happened rather than hold your data, and they start empty after a restore. backup show lists them separately.
The pre-backup space check now counts only the data that will be copied, and a refusal shows what the room is for: the temporary database, the backup file and headroom, with the connector data and the platform log data left out reported as separate figures.
Every command after sign-in now goes through the MCP host. keys, projects, query, export, docs, s3, view, login and status no longer need any other Commercient host, so the CLI works from any network. dlake login checks your key with one call before it saves anything: a key that is valid but not a full-scope Admin key is saved for the data plane, and a rejected key saves nothing. No sign-in token is cached any more; the old token cache is removed on first run.
dlake s3 and dlake view show|create|alter|drop now need a full-scope key owned by an Admin user, as dlake keys and dlake projects do. Scoped keys and non-admin users can still use query, export, docs, tool and view list. This is a temporary restriction for s3 and the view changes, until they have data-plane equivalents.
dlake view list now lists the views exposed through the Data API that your key can see, with each one’s type and field count; dlake admin list_views still lists every view for admins.
Document uploads are capped at 10 MB. dlake docs upload refuses a larger file before sending anything.
dlake s3 put and dlake s3 get move files over 8 MB straight between your machine and your bucket through a short-lived link. dlake s3 ls pages with --token.
Changed JSON output. dlake status --json, dlake login --json and dlake profiles list --json no longer carry authUrl or apiUrl. status --json reports the MCP host, the Registration host and whether your key was accepted and on which planes; profiles list shows each profile’s MCP URL. --auth-url and --api-url on dlake login are accepted and ignored for this release.
dlake watch checks the MCP host, the sign-up catalog and the published downloads (plus, with --profile, a live query); --tenant-host is accepted and ignored for this release.
Fixed: -o <file> on dlake export and dlake docs get now writes to the file you name.
Portable backups. backup settings --format portable|native|both adds a portable package (a .bacpac) that restores onto any SQL Server 2016 or later, including Express and Azure SQL, while the native file stays exact for same-or-newer servers; backup show reports the formats a backup holds and the oldest server each restores to, and backup download --format picks one. With both, a portable half that cannot be produced does not fail the backup: the native file is kept and the reason is recorded.
Choose what a backup holds. backup scope sets, per table, full data, schema only or excluded, and switches code objects (views, procedures, functions, triggers) on or off; platform tables keep a protected floor. Portable packages take the tables their foreign keys need.
Backups are consistent to a moment. On SQL Server editions that support it, the copy is taken from a database snapshot, so every table reflects the same instant; backup show reports point-in-time or per-table and why. Only one backup runs per Data Lake at a time.
Upgrade: npm install -g @commercient/dlake (or npm install -g datalake), then run dlake skills install to refresh the bundled agent skills.