Commit Graph

257 Commits

Author SHA1 Message Date
Ben Johnson 976df182c0 Fix Windows build 2021-01-31 10:08:28 -07:00
Ben Johnson 0e28a650e6 Merge pull request #33 from benbjohnson/log-wal-checksum-mismatch
Log WAL frame checksum mismatch
2021-01-31 08:57:51 -07:00
Ben Johnson f17768e830 Log WAL frame checksum mismatch
Currently, the WAL copy function can encounter a checksum mismatch in a
WAL frame and it will return an error. This can occur for partial writes
and is recovered from moments later. This commit changes the error to a
log write instead.
2021-01-31 08:52:12 -07:00
Ben Johnson 2c142d3a0c Merge pull request #32 from benbjohnson/persist-mismatch-validation-data
Persist primary/replica copies after validation mismatch
2021-01-31 08:50:15 -07:00
Ben Johnson 4e469f8b02 Persist primary/replica copies after validation mismatch
This commit changes `ValidateReplica()` to persist copies of the
primary & replica databases for inspection if a validation mismatch
occurs.
2021-01-31 08:47:06 -07:00
Ben Johnson 3f268b70f8 Merge pull request #31 from benbjohnson/adjust-logging
Reduce logging output
2021-01-31 08:14:57 -07:00
Ben Johnson ad7bf7f974 Reduce logging output
Previously, there were excessive log messages for checkpoints and
retention. These have been removed or combined into a single log
message where appropriate.
2021-01-31 08:12:18 -07:00
Ben Johnson 778451f09f CONTRIBUTING 2021-01-28 13:31:28 -07:00
Ben Johnson 8e9a15933b README 2021-01-27 08:01:48 -07:00
Ben Johnson da1d7c3183 README 2021-01-27 07:59:12 -07:00
Ben Johnson a178ef4714 Merge pull request #27 from benbjohnson/generations-replica-url
Allow replica URLs for generations command
v0.3.1
2021-01-27 07:50:29 -07:00
Ben Johnson 7ca2e193b9 Allow replica URLs for generations command 2021-01-27 07:48:56 -07:00
Ben Johnson 39a6fabb9f Fix restore logging. 2021-01-26 17:01:00 -07:00
Ben Johnson 0249b4e4f5 Merge pull request #25 from benbjohnson/replica-url 2021-01-26 16:40:17 -07:00
Ben Johnson 67eeb49101 Allow replica URL to be used for commands
This commit refactors the commands to allow a replica URL when
restoring a database. If the first CLI arg is a URL with a scheme,
the it is treated as a replica URL.
2021-01-26 16:33:16 -07:00
Ben Johnson f7213ed35c Allow replication without config file.
This commit changes `litestream replicate` to accept a database
path and a replica URL instead of using the config file. This allows
people to quickly try out the tool instead of learning the config
file syntax.
2021-01-25 10:33:50 -07:00
Ben Johnson a532a0198e README 2021-01-24 10:09:54 -07:00
Ben Johnson 16f79e5814 Merge pull request #24 from benbjohnson/document-retention-period
Document retention period configuration
2021-01-24 09:32:31 -07:00
Ben Johnson 39aefc2c02 Document retention period configuration 2021-01-24 09:28:57 -07:00
Ben Johnson 0b08669bca Merge pull request #23 from benbjohnson/disable-metrics-by-default
Disable prometheus metrics by default
2021-01-24 09:18:37 -07:00
Ben Johnson 8f5761ee13 Disable prometheus metrics by default
The HTTP server should only be enabled if a user explicitly sets a
port for it.
2021-01-24 09:16:23 -07:00
Ben Johnson d2eb4fa5ba Remove PR action 2021-01-24 08:54:29 -07:00
Ben Johnson ca489c5e73 Merge pull request #22 from benbjohnson/notorize
Add signed homebrew install
2021-01-24 08:50:01 -07:00
Ben Johnson f0ae48af4c Add signed homebrew install 2021-01-24 08:47:16 -07:00
Ben Johnson 9eae39e2fa README 2021-01-21 15:01:30 -07:00
Ben Johnson 42ab293ffb README 2021-01-21 14:53:21 -07:00
Ben Johnson c8b72bf16b Fix release action v0.3.0 2021-01-21 14:39:23 -07:00
Ben Johnson 9c4de6c520 Debian release action 2021-01-21 14:34:42 -07:00
Ben Johnson 94411923a7 Fix unit test 2021-01-21 13:52:35 -07:00
Ben Johnson e92db9ef4b Enforce stricter validation on restart.
Previously, the sync would validate the last page written to ensure
that replication picked up from the last position. However, a large
WAL file followed by a series of shorter checkpointed WAL files means
that the last page could be the same even if multiple checkpoints
have occurred.

To fix this, the WAL header must match the shadow WAL header when
starting litestream since there are no guarantees about checkpoints.
2021-01-21 13:44:05 -07:00
Ben Johnson 031a526b9a Only copy committed WAL pages 2021-01-21 12:44:11 -07:00
Ben Johnson 2244be885d README 2021-01-20 17:05:47 -07:00
Ben Johnson 95bcaa5927 Fix file replica compression 2021-01-19 09:25:38 -07:00
Ben Johnson 1935ebd6f0 Fix S3 GET bytes metric 2021-01-19 06:46:13 -07:00
Ben Johnson 7fb98df240 cleanup 2021-01-18 15:58:49 -07:00
Ben Johnson f31c22af62 Remove s3 bucket lookup log 2021-01-18 15:27:16 -07:00
Ben Johnson 139d836d7a Fix file/dir mode 2021-01-18 15:23:28 -07:00
Ben Johnson 14dad1fd5a Switch from gzip to lz4 2021-01-18 14:45:12 -07:00
Ben Johnson 35d755e7f2 Remove debugging code 2021-01-18 10:33:30 -07:00
Ben Johnson 358dcd4650 Copy shadow WAL immediately after init 2021-01-18 10:01:16 -07:00
Ben Johnson 2ce4052300 Remove write lock during db checksum 2021-01-18 07:05:27 -07:00
Ben Johnson 44af75fa98 Fix missing WAL reader error 2021-01-18 07:05:17 -07:00
Ben Johnson 3c4fd152c9 Add more checksum logging 2021-01-18 06:38:03 -07:00
Ben Johnson d259d9b9e3 Fix checksum logging 2021-01-17 10:19:39 -07:00
Ben Johnson 90a1d959d4 Remove size from s3 filenames 2021-01-17 10:02:06 -07:00
Ben Johnson 04d75507e3 Fix checksum hex padding 2021-01-17 09:52:09 -07:00
Ben Johnson 4b65e6a88f Log validation position 2021-01-17 07:38:13 -07:00
Ben Johnson 07a65cbac7 Fix crc64 unit test 2021-01-16 10:04:03 -07:00
Ben Johnson 6ac6a8536d Obtain write lock during validation. 2021-01-16 09:27:43 -07:00
Ben Johnson 71ab15e50a Fix S3 GET stats 2021-01-16 09:22:33 -07:00