286 Commits

Author SHA1 Message Date
Ben Johnson
dd8fdd8c8c Reduce binary size 2021-04-17 09:12:38 -06:00
Ben Johnson
dfd1b1b92d Merge pull request #158 from benbjohnson/sqlite-flags
Omit load extensions for static builds
2021-04-17 09:08:36 -06:00
Ben Johnson
c4c30e394d Omit load extensions for static builds 2021-04-17 09:06:54 -06:00
Ben Johnson
28673aeb01 Merge pull request #157 from benbjohnson/expand-env
Configuration file environment variable expansion
2021-04-16 09:32:11 -06:00
Ben Johnson
04ae010378 Configuration file environment variable expansion
This commit adds simple variable expansion using either `$FOO`
or `${FOO}` when evaluating the config file. This can be disabled
by any command by using the `-no-expand-env` flag.
2021-04-16 09:28:01 -06:00
Ben Johnson
cefbcb0460 Merge pull request #156 from benbjohnson/docker
Docker
2021-04-16 08:37:30 -06:00
Ben Johnson
01407c3c25 Add Docker image 2021-04-16 08:35:22 -06:00
Ben Johnson
66fdb208c7 Merge pull request #154 from benbjohnson/fix-not-database-error-message
Remove reference to "wal" in first db init command
2021-04-15 11:59:28 -06:00
Ben Johnson
247896b8b7 Remove reference to "wal" in first db init command
This commit changes the error message of the first SQL command
executed during initialization. Typically, it wraps the error with
a message of "enable wal" since it is enabling the WAL mode but
that can be confusing if the DB connection or file is invalid.

Instead, the error is returned as-is and we can determine the
source of the error since it is the only unwrapped DB-related error.
2021-04-15 11:51:22 -06:00
Ben Johnson
1e6e741f55 Merge pull request #151 from benbjohnson/arm6
Add arm6 release builds
2021-04-12 09:47:54 -06:00
Ben Johnson
b31daabf52 Add arm6 release builds 2021-04-11 10:27:45 -06:00
Ben Johnson
1ccb4ef922 Merge pull request #149 from benbjohnson/arm-cgo
Enable CGO for cross compilation
2021-04-10 12:49:00 -06:00
Ben Johnson
54f0659c7b Enable CGO for cross compilation 2021-04-10 12:35:43 -06:00
Ben Johnson
5c0b8536f0 Merge pull request #148 from benbjohnson/arm
Support ARM release builds
2021-04-10 08:48:32 -06:00
Ben Johnson
462330ead6 Support ARM release builds 2021-04-10 08:39:10 -06:00
Ben Johnson
178cf836b1 Move pull request template 2021-04-07 16:24:49 -06:00
Ben Johnson
f45c0d8560 Create pull_request_template.md 2021-04-07 16:23:11 -06:00
Ben Johnson
bb146e2c09 Merge pull request #141 from benbjohnson/skip-verify
Add skip-verify flag for using self-signed certificates
2021-03-27 08:02:55 -06:00
Ben Johnson
f1d2df3e73 Add skip-verify flag for using self-signed certificates
This commit adds a `skip-verify` flag to the replica configuration
so that it can be used with self-signed certificates. This is useful
when running a local instance of MinIO with TLS for testing.
2021-03-27 08:00:09 -06:00
Ben Johnson
ef39987cc7 Merge pull request #133 from benbjohnson/sigterm
Catch sigterm & add shutdown logging
2021-03-21 09:38:38 -06:00
Ben Johnson
ee0c4c62d8 Catch sigterm & add shutdown logging
This commit changes the signal handler for `replicate` to catch
`syscall.SIGTERM` for non-Windows installations. It also adds some
logging to indicat when a shutdown has been initiated and when it
has finished.
2021-03-21 09:36:13 -06:00
Ben Johnson
e2de7e852c Merge pull request #132 from benbjohnson/sync-on-close
Sync on close
2021-03-21 08:45:26 -06:00
Ben Johnson
0529ce74b7 Sync on close
This commit changes the `replicate` command so that it performs a
final DB sync & replica sync before it exits to ensure it has
backed up all WAL frames at the time of exit.
2021-03-21 08:43:55 -06:00
Ben Johnson
421693130c Merge pull request #131 from benbjohnson/if-replica-exists
Add `-if-replica-exists` flag to restore
2021-03-21 08:09:57 -06:00
Ben Johnson
4a17c81b91 Add -if-replica-exists flag to restore
This commit adds a flag to change the exit code when restoring
from a replica where there is no existing backup. When set,
finding no backup will return a `0` exit code. The command will
still fail if other errors occur.
2021-03-21 08:08:11 -06:00
Ben Johnson
ba068ea3f8 Upload raw release binaries 2021-03-20 09:50:48 -06:00
Ben Johnson
085974fe1d Merge pull request #130 from benbjohnson/static
Add static release builds
2021-03-20 09:42:53 -06:00
Ben Johnson
18598a10e6 Add static release builds 2021-03-20 09:41:19 -06:00
Ben Johnson
16c50d1d2e Merge pull request #120 from benbjohnson/default-force-path-style
Default to force path style if endpoint set
v0.3.3
2021-03-11 15:29:18 -07:00
Ben Johnson
929a66314c Default to force path style if endpoint set
This commit changes the replica configuration behavior to default
the `force-path-style` field to `true` when an `endpoint` is set.
This works because the only service that does not use the path
style is AWS S3 which does not use an endpoint.
2021-03-11 15:26:41 -07:00
Ben Johnson
2e7a6ae715 Merge pull request #118 from benbjohnson/default-region 2021-03-09 15:42:04 -07:00
Ben Johnson
896aef070c Default region if endpoint specified 2021-03-09 15:38:32 -07:00
Ben Johnson
3598d8b572 Merge pull request #111 from benbjohnson/linode
Add support for Linode Object Storage replica URLs
2021-03-07 08:55:29 -07:00
Ben Johnson
3183cf0e2e Add support for Linode Object Storage replica URLs
This commit adds the ability to specify Linode Object Storage
as replica URLs in the command line and configuration file:

	s3://MYBKT.us-east-1.linodeobjects.com/MYPATH
2021-03-07 08:47:24 -07:00
Ben Johnson
a59ee6ed63 Merge pull request #110 from benbjohnson/digitalocean
Add support for DigitalOcean Spaces replica URLs
2021-03-07 08:29:23 -07:00
Ben Johnson
e4c1a82eb2 Add support for DigitalOcean Spaces replica URLs
This commit adds the ability to specify DigitalOcean Spaces as
replica URLs in the command line and configuration file:

	s3://mybkt.nyc3.digitaloceanspaces.com/mypath
2021-03-07 08:25:26 -07:00
Ben Johnson
aa54e4698d Merge pull request #109 from benbjohnson/wal-mismatch-validation-info
Add WAL validation debug information
2021-03-07 07:55:02 -07:00
Ben Johnson
43e40ce8d3 Merge pull request #108 from benbjohnson/revert-lock-removal
Revert sync lock removal
2021-03-07 07:52:49 -07:00
Ben Johnson
0bd1b13b94 Add wal validation debug information on error
This commit adds the WAL header and shadow path to "wal header mismatch"
errors to help debug issues. The mismatch seems to happen more often
than I would expect on restart. This error doesn't cause any corruption;
it simply causes a generation to restart which requires a snapshot.
2021-03-07 07:48:43 -07:00
Ben Johnson
1c16aae550 Revert sync lock removal
This commit reverts the removal of the SQLite write lock during
WAL sync (998e831c5c). The change
caused validation mismatch errors during the long-running test
although the restored database did not appear to be corrupted so
perhaps it's simply a locking issue during validation.
2021-03-07 07:30:25 -07:00
Ben Johnson
49f47ea87f Merge pull request #105 from benbjohnson/db-config-fields
Expose additional DB configuration settings
2021-03-06 08:37:02 -07:00
Ben Johnson
8947adc312 Expose additional DB configuration settings
This commit exposes the monitor interval, checkpoint interval,
minimum checkpoint page count, and maximum checkpoint page count
via the YAML configuration file.
2021-03-06 08:33:19 -07:00
Ben Johnson
9341863bdb Merge pull request #104 from benbjohnson/remove-sync-lock
Remove SQLite write lock during WAL sync
2021-03-06 08:08:23 -07:00
Ben Johnson
998e831c5c Remove SQLite write lock during WAL sync
Originally, Litestream relied on a SQLite write lock to ensure
transactions were atomically replicated. However, this was changed
so that Litestream itself now validates the transaction boundaries.
As such, the write lock on the database is no longer needed. The
read lock is sufficient to prevent WAL rollover and the WAL is
append only so it is safe to read up to a known position calculated
via fstat().

WAL validation change was made in 031a526b9a

The locking code, however, was moved in this commit to the
post-checkpoint copy to ensure the end-of-file is not overwritten
by an aggressive writers.
2021-03-06 07:51:35 -07:00
Ben Johnson
b2ca113fb5 Merge pull request #103 from benbjohnson/fix-addr-log
Fix logged hostport for metrics endpoint
2021-03-06 07:30:36 -07:00
Ben Johnson
b211e82ed2 Fix logged hostport for metrics endpoint
This commit fixes a bug where the bind address is not reported
correctly in the log if a hostname is specified. Previously it
would always report the host as "localhost" even if a host was
specified (such as "0.0.0.0:9090").

This commit also adds validation to require the port to be
specified and only specifying a hostname will return an error.
2021-03-06 07:23:09 -07:00
Ben Johnson
e2779169a0 README 2021-03-02 08:11:51 -07:00
Ben Johnson
ec2f9c84d5 Merge pull request #96 from benbjohnson/acknowledgements
Acknowledgments
2021-02-28 09:19:44 -07:00
Ben Johnson
78eb8dcc53 Acknowledgments 2021-02-28 09:16:35 -07:00
Ben Johnson
cafa0f5942 Merge pull request #94 from benbjohnson/prevent-config-and-replica-url
Prevent user from specifying replica URL & config flag
2021-02-28 08:31:29 -07:00