Commit Graph

14 Commits

Author SHA1 Message Date
Ben Johnson
a090706421 Implement live read replication
This commit adds an http server and client for streaming snapshots
and WAL pages from an upstream Litestream primary to a read-only
replica.
2022-02-19 09:06:49 -07:00
Ben Johnson
006e4b7155 Update index & offset encoding
Previously, the index & offsets were encoded as 8-character hex
strings, however, this limits the maximum value to a `uint32`. This
is normally not an issue, however, indices could go over the maximum
value of 4 billion over time and the offset could exceed this value
for an especially large WAL update. For safety, these encodings have
been updated to 16-character hex encodings.
2022-02-08 13:14:49 -07:00
Ben Johnson
762c7ae531 Implement FileWatcher 2022-02-06 09:51:04 -07:00
Ben Johnson
8009bcf654 Remove Windows support
Unfortunately, I don't have the expertise or bandwidth to maintain
the Windows support in Litestream. I'm open to re-adding support in
the future but right now it is hindering development and is not
well-tested or well-used.
2022-02-05 08:19:31 -07:00
Ben Johnson
5d811f2e39 Fix golangci-lint issues 2022-01-31 09:21:20 -07:00
Ben Johnson
f6c859061b Fix CodeQL warnings 2022-01-31 08:53:21 -07:00
Ben Johnson
f308e0b154 CLI test coverage 2022-01-11 13:29:20 -07:00
Ben Johnson
3f0ec9fa9f Refactor Restore()
This commit refactors out the complexity of downloading ordered WAL
files in parallel to a type called `WALDownloader`. This makes it
easier to test the restore separately from the download.
2022-01-04 15:03:59 -07:00
Ben Johnson
77274abf81 Refactor shadow WAL to use segments 2021-07-23 07:46:21 -06:00
Ben Johnson
fc897b481f Group replica wal segments by index
This commit changes the replica path format to group segments within
a single index in the same directory. This is to eventually add the
ability to seek to a record on file-based systems without having
to iterate over the records. The DB shadow WAL will also be changed
to this same format to support live replicas.
2021-06-14 15:24:05 -06:00
Ben Johnson
1c0c69a5ab Unify replica client metrics 2021-06-01 18:19:56 -06:00
Ben Johnson
ac32e8e089 Add Google Cloud Storage replica 2021-05-22 07:16:14 -06:00
Ben Johnson
fb80bc10ae Refactor replica system 2021-05-21 07:44:36 -06:00
Ben Johnson
a42f83f3cb Add LITESTREAM_CONFIG env var 2021-01-13 13:17:38 -07:00