Commit Graph

58 Commits

Author SHA1 Message Date
Ben Johnson
4a17c81b91 Add -if-replica-exists flag to restore
This commit adds a flag to change the exit code when restoring
from a replica where there is no existing backup. When set,
finding no backup will return a `0` exit code. The command will
still fail if other errors occur.
2021-03-21 08:08:11 -06:00
Ben Johnson
929a66314c Default to force path style if endpoint set
This commit changes the replica configuration behavior to default
the `force-path-style` field to `true` when an `endpoint` is set.
This works because the only service that does not use the path
style is AWS S3 which does not use an endpoint.
2021-03-11 15:26:41 -07:00
Ben Johnson
8947adc312 Expose additional DB configuration settings
This commit exposes the monitor interval, checkpoint interval,
minimum checkpoint page count, and maximum checkpoint page count
via the YAML configuration file.
2021-03-06 08:33:19 -07:00
Ben Johnson
b211e82ed2 Fix logged hostport for metrics endpoint
This commit fixes a bug where the bind address is not reported
correctly in the log if a hostname is specified. Previously it
would always report the host as "localhost" even if a host was
specified (such as "0.0.0.0:9090").

This commit also adds validation to require the port to be
specified and only specifying a hostname will return an error.
2021-03-06 07:23:09 -07:00
Ben Johnson
325482a97c Prevent user from specifying replica URL & config flag
Previously, if a replica URL was specified then the `-config` flag
was silently ignored. This commit changes this behavior so that
specifying both the URL & config flag will now return an error.
2021-02-28 08:09:24 -07:00
Ben Johnson
afb8731ead Add snapshot interval
This commit adds the ability to periodically perform snapshots on
an interval that is separate from retention. For example, this lets
you retain backups for 24 hours but you can snapshot your database
every six hours to improve recovery time.
2021-02-25 15:34:13 -07:00
Ben Johnson
622ba82ebb Fix global settings propagation
This commit fixes an issue caused by a refactor where setting global
or local AWS credentials in a config file fails.
2021-02-22 06:37:40 -07:00
Ben Johnson
ad9ce43127 Add support for S3-compatible object storage.
This commits adds support for non-AWS S3-compatible storage such as
MinIO, Backblaze B2, & Google Cloud Storage (GCS). Other backends
should also work but some code has been added to make URL-based
configurations work more easily.
2021-02-21 09:40:48 -07:00
Ben Johnson
c5390dec1d Add Windows Service & MSI builds 2021-02-19 16:21:04 -07:00
Ben Johnson
962a2a894b Fix tabwriter 2021-02-08 15:55:15 -07:00
Ben Johnson
267b140fab Reduce s3 sync interval when using replica URL
This commit changes the default sync interval from 10s to 1s
when replicating using the inline replica URL. This approach is
used when users are first testing the software so a faster
replication interval makes it easier to see results.
2021-02-08 14:09:01 -07:00
Ben Johnson
58a6c765fe Add trace file to replicate command
This commit removes the verbose flag (`-v`) and replaces it with
the trace flag (`-trace PATH`). This moves tracing to a separate
file instead of writing to STDOUT.
2021-02-06 07:31:19 -07:00
Ben Johnson
ad7bf7f974 Reduce logging output
Previously, there were excessive log messages for checkpoints and
retention. These have been removed or combined into a single log
message where appropriate.
2021-01-31 08:12:18 -07:00
Ben Johnson
7ca2e193b9 Allow replica URLs for generations command 2021-01-27 07:48:56 -07:00
Ben Johnson
67eeb49101 Allow replica URL to be used for commands
This commit refactors the commands to allow a replica URL when
restoring a database. If the first CLI arg is a URL with a scheme,
the it is treated as a replica URL.
2021-01-26 16:33:16 -07:00
Ben Johnson
f7213ed35c Allow replication without config file.
This commit changes `litestream replicate` to accept a database
path and a replica URL instead of using the config file. This allows
people to quickly try out the tool instead of learning the config
file syntax.
2021-01-25 10:33:50 -07:00
Ben Johnson
8f5761ee13 Disable prometheus metrics by default
The HTTP server should only be enabled if a user explicitly sets a
port for it.
2021-01-24 09:16:23 -07:00
Ben Johnson
7fb98df240 cleanup 2021-01-18 15:58:49 -07:00
Ben Johnson
78563f821d Do not require databases when starting replication 2021-01-16 09:15:16 -07:00
Ben Johnson
1b8cfc8a41 Add validation interval 2021-01-15 16:37:04 -07:00
Ben Johnson
a7ec05ad7a Allow global AWS settings in config. 2021-01-15 12:27:41 -07:00
Ben Johnson
28dd7b564e Lookup s3 bucket region if not specified 2021-01-15 12:18:07 -07:00
Ben Johnson
43dda4315f Allow URLs for replica config path 2021-01-15 12:04:23 -07:00
Ben Johnson
e1c9e09161 Update wal segment naming 2021-01-14 15:26:29 -07:00
Ben Johnson
1e4e9633cc Add s3 sync interval 2021-01-14 15:04:26 -07:00
Ben Johnson
a42f83f3cb Add LITESTREAM_CONFIG env var 2021-01-13 13:17:38 -07:00
Ben Johnson
57a02a8628 S3 replica 2021-01-13 10:14:54 -07:00
Ben Johnson
faa5765745 Add retention policy, remove WAL subdir 2021-01-12 15:22:37 -07:00
Ben Johnson
1fa1313b0b Add trace logging. 2021-01-11 11:04:29 -07:00
Ben Johnson
9828b4c1dd Rename 'databases' to 'dbs' in config 2021-01-10 10:07:07 -07:00
Ben Johnson
dde9d1042d Update generation lag calc 2021-01-10 09:54:05 -07:00
Ben Johnson
8f30ff7d93 Fix negative duration truncation. 2021-01-10 09:52:04 -07:00
Ben Johnson
aa136a17ee Fix duration truncation. 2021-01-10 09:46:39 -07:00
Ben Johnson
9d0e79c2cf Add db metrics 2020-12-31 16:30:56 -07:00
Ben Johnson
3b9275488d Add 'validate' command 2020-12-31 10:36:48 -07:00
Ben Johnson
cff778464e Add 'databases' command. 2020-12-30 16:10:48 -07:00
Ben Johnson
11d7d22383 Add 'wal' command 2020-12-30 16:03:23 -07:00
Ben Johnson
8a7d8175fc Restrict generations command to single db 2020-12-30 15:44:21 -07:00
Ben Johnson
ffc25e2654 Add 'snapshots' command. 2020-12-30 15:31:35 -07:00
Ben Johnson
ddf85f7150 Add release workflow 2020-12-29 13:44:58 -07:00
Ben Johnson
6f604e29b0 Fix unit tests. 2020-12-29 12:56:50 -07:00
Ben Johnson
41448ceb89 Rename replicator to replica 2020-12-29 12:49:23 -07:00
Ben Johnson
81e99c8035 Add 'restore' command. 2020-12-29 12:37:42 -07:00
Ben Johnson
44973dbbbc intermediate 2020-12-27 09:56:23 -07:00
Ben Johnson
98014f4e49 Add generations command 2020-12-26 09:39:21 -07:00
Ben Johnson
22d44fda6d Rename binary to litestream; add replicate command 2020-12-24 17:05:25 -07:00
Ben Johnson
8d7f5b28a9 Rename replicator config to replica 2020-12-23 16:02:38 -07:00
Ben Johnson
311f47fe98 Fix wal rollover 2020-12-22 16:56:23 -07:00
Ben Johnson
6aceb5553e Rename daemon to litestreamd 2020-12-22 13:28:30 -07:00
Ben Johnson
a8387ed6f9 Add read lock 2020-12-19 08:56:44 -07:00