Previously, a bug was introduced that added a `LIST` operation
on every replica sync which significantly increased the cost of
running Litestream against S3. This changes the behavior to only
issue the `LIST` operation when the generation has changed.
Checksum mismatch can regularly occur now that write locks have
been removed during WAL sync. This does not pose any corruption
risk but does sound scary to end users. Moving this to trace
logging instead.
This commit adds the ability to run a subcommand through Litestream.
Shutting down the subcommand will cause Litestream to gracefully
shutdown. Litestream will forward interrupt signals and wait for
the subprocess to shutdown.
This commit adds a flag to `litestream restore` to skip the restore
if the database file already exists. It is useful when using the
official Litestream Docker image and you don't have the ability to
add a script to check for the existence of the file.
Previously, S3 would default to sync every 10s when using a config
file but only every 1s when using the replica URL on the command
line. Obviously, this is confusing.
A sync interval of 1s could incur a cost of $1.30/month, however,
in practice applications are not receiving a constant stream of
writes so the cost is typically only a few pennies. This lower
sync interval will provide a smaller window for data loss in the
event of a catastrophic failure at a neglible cost.
This commit fixes a bug introduced by parallel restore (03831e2)
where snapshot-only restores were not being handled correctly and
Litestream would hang indefinitely. Now the restore will check
explicitly for snapshot-only restores and exit the restore process
early to avoid WAL handling completely.
This commit removes the uncompressed binary from the release page.
It was originally added to simplify Docker but it turns out that
having to chmod the binary will double its size.
This commit reattempts a change to remove the write lock that was
previously tried in 998e831. This change will reduce the number of
locks on the database which should help reduce error messages that
applications see when they do not have busy_timeout set.
In addition to the lock removal, a passive checkpoint is issued
immediately before the read lock is obtained to prevent additional
checkpoints by the application itself. SQLite does not support
checkpoints from an active transaction so it cannot be done afterward.
This commit adds optional `LITESTREAM_ACCESS_KEY_ID` and
`LITESTREAM_SECRET_ACCESS_KEY` environment variables that can
be used instead of their `AWS` counterparts. The AWS-prefixed
variables have caused some confusion with users who were not
using AWS S3.
This commit changes the restore to download multiple WAL files to
the local disk in parallel while another goroutine applies those
files in order. Downloading & applying the WAL files in serial
reduces the total throughput as WAL files are typically made up of
multiple small files.
This commit fixes a bug where restoring to a specific index will
incorrectly choose the latest snapshot instead of choosing the
latest snapshot that occurred before the given index.
This commit changes the `-index` flag on the `restore` command by
parsing it as a hex number instead of a decimal number. This is
done because the index is represented in hex form everywhere else
in the application.
