Compare commits

...
Sign in to create a new pull request.

591 commits

Author SHA1 Message Date
Alexander Neumann
686f24b578 doc: Clarify B2 application keys 2018-08-02 21:14:05 +02:00
Alexander Neumann
247d2b7215 Merge pull request #1921 from salbertson/patch-1
Add a "Reviewed by Hound" badge
2018-08-02 20:03:43 +02:00
Alexander Neumann
017cd113d3 Merge pull request #1922 from salbertson/patch-2
Use https when linking to chris.beams.io
2018-08-02 20:03:40 +02:00
Scott Albertson
f744c2553e
Use https when linking to chris.beams.io
Why not link to How to [Write a Git Commit Message](https://chris.beams.io/posts/git-commit/) using HTTPS, it's going to redirect anyway.
2018-08-01 14:59:06 -07:00
Alexander Neumann
56cd6bd495 Merge pull request #1919 from restic/update-deps
Update dependencies
2018-08-01 23:56:55 +02:00
Alexander Neumann
bff635bc5f Update dependencies, enable pruning for vendor/
So, `dep` got an nice new feature to remove tests and non-go files from
`vendor/`, and this brings the size of the vendor directory from ~300MiB
down to ~20MiB. We don that now.
2018-08-01 21:32:15 +02:00
Alexander Neumann
3422c1ca83 Merge pull request #1729 from mholt/stats
Implement `restic stats` command to get more info about a repository
2018-07-31 23:24:36 +02:00
Matthew Holt
f6b2731aa5 stats: Add manual doc, improve -h doc
Also rename files-by-content to files-by-contents, once and for all
2018-07-31 22:54:10 +02:00
Scott Albertson
3eb5b45b41
Add a "Reviewed by Hound" badge 2018-07-31 13:53:24 -07:00
Alexander Neumann
01aacf41b5 Merge pull request #1915 from mlissner/patch-2
Adds warning re performance of prune
2018-07-31 22:42:20 +02:00
Mike Lissner
2caf8edc55 Add warning of the performance of prune
I went pretty loud with this, but I think the performance is bad enough
that it's really worth highlighting, especially since it locks the index
during the prune.
2018-07-31 22:41:40 +02:00
Alexander Neumann
3151978f58 Fix changelog type 2018-07-31 21:57:27 +02:00
Alexander Neumann
ab4ef432ff Add entry to changelog 2018-07-31 21:29:47 +02:00
Alexander Neumann
be4f54b603 Merge pull request #1913 from restic/restic-password-stdin-message
Print message for password being read from stdin
2018-07-31 21:28:12 +02:00
Alexander Neumann
7260110c27 Merge pull request #1914 from restic/update-blazer
Add support for B2 application keys
2018-07-31 21:27:50 +02:00
Alexander Neumann
2437f11af7 Update github.com/kurin/blazer to 0.5.1
This adds support for B2 application keys.
2018-07-31 20:51:36 +02:00
Alexander Neumann
57873502f8 Add note about B2 application keys to the docs 2018-07-31 20:49:54 +02:00
Alexander Neumann
3678ec9ad8 Print message for password being read from stdin
Closes #1911
2018-07-31 20:21:18 +02:00
Alexander Neumann
a717e9e6f7 Improve message for number of bytes newly added 2018-07-31 19:08:43 +02:00
Alexander Neumann
12c797700e make statsWalkSnapshot return a function 2018-07-27 21:44:59 +02:00
Matthew Holt
daca9d6815 Consolidate mode flags; use new Walk function 2018-07-27 21:27:40 +02:00
Matthew Holt
930602a444 Update comment now that question was answered 2018-07-27 21:27:39 +02:00
Matthew Holt
acb05e7855 Fix filepath uniqueness bug for blobs-per-file mode 2018-07-27 21:27:39 +02:00
Matthew Holt
a7b95d716a Implement four counting modes 2018-07-27 21:27:39 +02:00
Matthew Holt
925b542eb0 Count unique files by blob sequence rather than tree ID 2018-07-27 21:27:39 +02:00
Matthew Holt
f7659bd8b0 stats: Initial implementation of stats command 2018-07-27 21:27:39 +02:00
Alexander Neumann
8c124a2b75 Merge pull request #1902 from mlissner/patch-1
b2 bucket names need to be unique
2018-07-23 22:58:42 +02:00
Mike Lissner
d3ad63a4ec
b2 bucket names need to be unique
Adds a small warning indicating that b2 bucket names need to be unique. It's an easy mistake to make, and it's surprising to get the following error if you're not accustomed to the way B2 works:

    Fatal: create repository at b2:postgres failed: NewBucket: b2_create_bucket: 400: Bucket name is already in use
2018-07-23 11:48:59 -07:00
Alexander Neumann
271c50cf5c Add entry to changelog 2018-07-23 20:15:55 +02:00
Alexander Neumann
1aeb193fd9 Merge pull request #1900 from restic/fix-1870
restorer: Add test for restore with include filter
2018-07-23 20:15:50 +02:00
Alexander Neumann
f715bef82f Merge pull request #1899 from garrmcnu/check-cache-dir
check: Use --cache-dir argument
2018-07-22 21:03:52 +02:00
Alexander Neumann
4fc00d4120 Merge pull request #1901 from restic/update-blazer
Update github.com/kurin/blazer
2018-07-22 20:59:52 +02:00
Garry McNulty
7603ab7ac1 check: Update --cache-dir argument handling based on code review comments
The temporary cache directory is created in the specified directory, or
if not specified in the default temporary directory.
2018-07-22 18:24:11 +01:00
Alexander Neumann
36fa1f8c20 Merge pull request #1894 from restic/fix1893
Return error when exclude file cannot be read
2018-07-22 14:34:27 +02:00
Alexander Neumann
445fb23b6d Rework issue templates for Bug reports and Features 2018-07-22 14:26:23 +02:00
Alexander Neumann
5f79b4cb6c Update issue template again 2018-07-22 14:21:08 +02:00
Alexander Neumann
8e15b59347 Use underline style markup for issue/PR templates 2018-07-22 14:17:53 +02:00
Alexander Neumann
6e2e957332 Add entry to changelog 2018-07-22 14:16:08 +02:00
Alexander Neumann
7ffc03ff8f Update github.com/kurin/blazer to 0.5.0
This includes support for the upcoming B2 application keys feature.
2018-07-22 14:12:02 +02:00
Alexander Neumann
44924ba043 restorer: Fix traverseTree
traverseTree() was meant to call enterDir() whenever a directory is
selected for restore, either explicitly or implicitly (=contains a file
which is to be restored). After restoring a file, leaveDir() is called
in reverse order for all intermediate directories so that the metadata
can be restored.

When a directory is selected implicitly, the metadata for it is
restored. This is different from the previous restorer behavior, which
created implicitly selected intermediate directories with permissions
0700 (only user can read/write it).

This commit changes the behavior back to the old one. Only a directory
is explicitly selected for restore, enterDir()/leaveDir() are called for
it. Otherwise, only visitNode() is called, so visitNode() needs to make
sure the parent directory exists. If the directory is explicitly
included, leaveDir() will then restore the metadata correctly.

When we decide to change the behavior (restore metadata for all
intermediate directories, even if selected implicitly), we should do
that in the selection functions, not here.

This finally resolves #1870
2018-07-21 23:24:40 +02:00
Alexander Neumann
ce19f26948 restorer: Add tests for traverseTree 2018-07-21 23:24:40 +02:00
Alexander Neumann
74016d5981 restorer: Fix return of saveSnapshot 2018-07-21 23:24:40 +02:00
Alexander Neumann
57636a4573 restorer: Run tests in the same package 2018-07-21 23:24:40 +02:00
Alexander Neumann
4f6d2502f7 restorer: Add test for restore with include filter 2018-07-21 23:24:40 +02:00
Garry McNulty
f1f69bc648 check: Use --cache-dir argument
Closes #1880
2018-07-20 20:51:20 +01:00
Alexander Neumann
d7551d7b0c Add entry to changelog 2018-07-18 21:41:20 +02:00
Alexander Neumann
fb74de6360 Return an error when exclude files cannot be read 2018-07-18 21:39:07 +02:00
Alexander Neumann
67535e00a8 Merge pull request #1889 from ProactiveServices/patch-3
doc: Minor grammar, RST syntax fixes
2018-07-18 21:22:10 +02:00
Alexander Neumann
19592285eb Merge pull request #1888 from ProactiveServices/patch-2
doc: Minor grammar fixes
2018-07-18 21:21:52 +02:00
Alexander Neumann
f64862722a Merge pull request #1887 from restic/disable-error-size
checker: Disable size check for now
2018-07-18 21:19:54 +02:00
Adam Piggott
254239c2a9
doc: Minor grammar, RST syntax fixes
Fix unescaped backslash
Fix wording of Windows installation
2018-07-18 02:28:23 +01:00
Adam Piggott
cce1a1f768
doc: Minor grammar fixes 2018-07-18 02:25:31 +01:00
Alexander Neumann
754482fe6c checker: Disable size check for now 2018-07-15 21:52:38 +02:00
Alexander Neumann
73153dbd3f Merge pull request #1885 from restic/create-restore-target
restore: Make sure the target directory exists
2018-07-15 16:28:25 +02:00
Alexander Neumann
92421ec47f restore: Make sure target directory exists 2018-07-15 16:02:04 +02:00
Alexander Neumann
9acc9243ba Add test for not-existing top-level dir and top-level file 2018-07-15 16:00:26 +02:00
Alexander Neumann
df64998649 Merge pull request #1882 from duzvik/aws-credentials-priority
Change AWS credentials priority, to accept AWS_SESSION_TOKEN
2018-07-14 20:48:42 +02:00
Alexander Neumann
64d27eed86 doc: Improve dump to stdout
Closes #1884
2018-07-14 20:45:52 +02:00
Alexander Neumann
abb18a830c Fix test 2018-07-14 11:51:34 +02:00
denis.uzvik
1e42f4f300 S3 backend: accept AWS_SESSION_TOKEN 2018-07-12 16:18:19 +03:00
Alexander Neumann
bd742ddb69 cache: Don't recreate CACHEDIR.TAG 2018-07-08 12:05:12 +02:00
Alexander Neumann
b511f4dce2 Improve help message for check 2018-07-05 22:19:08 +02:00
Alexander Neumann
7961740dcc Fix link 2018-07-05 21:03:40 +02:00
Alexander Neumann
dc3032c360 Mention that AppsCode is sponsoring backend tests 2018-07-05 21:01:57 +02:00
Alexander Neumann
44fb2a860f Merge pull request #1861 from McKael/fix_find_ignore_case
Fix find -i (case-insensitive search)
2018-06-24 21:08:03 +02:00
Mikael Berthe
fbf8073dfc Fix find -i (case-insensitive search) 2018-06-24 19:35:11 +02:00
Alexander Neumann
7ddf91b65c Merge pull request #1772 from ifedorenko/restore-verify
Restore verify
2018-06-23 14:50:29 +02:00
Alexander Neumann
8dae2de2ce Merge pull request #1858 from restic/fix-1857
Allow excluding files with $ via --exclude-file
2018-06-22 21:22:23 +02:00
Alexander Neumann
03a0377410 Merge pull request #1856 from stkw0/master
Ignore ExcludeOtherFs if Stdin is true
2018-06-22 20:49:34 +02:00
Alexander Neumann
025ec9dff5 Allow excluding files with $ via --exclude-file
Previously it wasn't possbile to exclude files with a literal dollar
sign (`$`) via exclude files, now users can write `$$` for that.
2018-06-22 20:46:04 +02:00
David Roman
2384c1cee7
Ignore ExcludeOtherFs if Stdin is true
Closes: #1807
2018-06-21 23:12:38 +02:00
Alexander Neumann
bb2ad76833 Merge pull request #1855 from restic/fix-1854
Allows saving files/dirs on different file systems together with `--one-file-system`.
2018-06-21 20:48:33 +02:00
Alexander Neumann
30cfd13328 Add changelog 2018-06-20 23:05:09 +02:00
Alexander Neumann
9ffc26883a archiver: Unroll tree 2018-06-20 22:56:41 +02:00
Alexander Neumann
83c51db903 fs: Add helper functions ReadDir/ReadDirNames 2018-06-20 22:53:53 +02:00
Alexander Neumann
d30d5d4473 Merge pull request #1853 from skriss/list-keys-json-output
add JSON output support for restic key list cmd
2018-06-19 21:28:45 +02:00
Steve Kriss
5088905502 add JSON output support for restic key list cmd
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-18 15:26:26 -07:00
Alexander Neumann
ae72b438b0 doc: Fix typo 2018-06-18 22:19:01 +02:00
Alexander Neumann
ddf2065ce2 doc: Split references out into smaller files
Closes #1852
2018-06-18 22:17:48 +02:00
Alexander Neumann
228a970540 doc: Add paragraph about finding path in file 2018-06-17 20:49:11 +02:00
Alexander Neumann
c7a8086c19 Merge pull request #1851 from restic/fix-1849
walker: Don't ignore empty trees by default
2018-06-17 12:25:33 +02:00
Alexander Neumann
c2c06ae2c9 walker: Don't ignore empty trees by default
Closes #1849
2018-06-17 09:49:03 +02:00
Alexander Neumann
1824168aa3 Merge pull request #1848 from iquiw/netbsd-support
Support NetBSD without fuse
2018-06-16 17:12:20 +02:00
Iku Iwasa
350761f1ba Add cross-compilation for netbsd/386 and netbsd/amd64 2018-06-16 21:05:26 +09:00
Alexander Neumann
3231945a85 Merge pull request #1846 from cbergmann/patch-1
Clarify live status display
2018-06-16 11:19:58 +02:00
Iku Iwasa
f080142137 Support NetBSD without fuse 2018-06-16 15:55:04 +09:00
Clemens Bergmann
ff785924de
Clarify live status display
When I backup one of my filesystems which has a lot of Hard Links (Backup directory of burp) the live status shows me 4.5 TB but it only takes up 1.2 TB of space in the repository. This is confusing because my repo is on S3 and I feared a huge Bill. This change should clarify this.
2018-06-15 12:41:48 +02:00
Alexander Neumann
393a7266c9 Merge pull request #1845 from dimejo/patch-2
doc: enhance FAQ about AV software on Windows
2018-06-14 20:48:15 +02:00
Alex JOST
cb8d2d3df5
doc: enhance FAQ about AV software on Windows
As discussed in #1840, realtime protection of AV software can cause serious performance degradation on Windows OS.
2018-06-14 19:39:04 +02:00
Alexander Neumann
a884ce1566 Merge pull request #1841 from jernst/master
Added documentation about what happens if you run out of space during a backup
2018-06-14 19:13:31 +02:00
Alexander Neumann
5ae8316c24 Merge pull request #1844 from gsanchietti/patch-1
Documentation: minor RST syntax fixes
2018-06-14 19:12:57 +02:00
Giacomo Sanchietti
85eca1b5e9 Documentation: minor RST syntax fixes
- Fix bad link syntax
- Fix compilation warning
2018-06-14 15:01:49 +02:00
Johannes Ernst
a1536f38fa Fixed typo and grammar per feedback on PR. 2018-06-13 00:24:42 +00:00
Johannes Ernst
888f52afd1 Added documentation about what happens if you run out of space
during a backup, based on conversation on the forum:
https://forum.restic.net/t/limited-backup-destination-space/733/6
2018-06-11 21:20:08 +00:00
Igor Fedorenko
e206680947 restore: New --verify flag to verify restored files content
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-06-11 07:57:22 -04:00
Igor Fedorenko
5fa6dc53cb Refactor: introduced restorer tree visitor
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-06-11 07:57:22 -04:00
Igor Fedorenko
26be094f28 Refactor: moved restorer to separate package
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-06-11 07:57:22 -04:00
Alexander Neumann
e4c0d77bdd Add VERSION for 0.9.1 2018-06-10 11:31:03 +02:00
Alexander Neumann
1dd655dad2 Generate CHANGELOG.md for 0.9.1 2018-06-10 11:30:53 +02:00
Alexander Neumann
581d0984fe Fix changelog entry 2018-06-10 11:29:57 +02:00
Alexander Neumann
e62add84bc Move changelog files for 0.9.1 2018-06-10 11:22:32 +02:00
Alexander Neumann
63779c1eb4 Merge pull request #1839 from restic/fix-find
Fix find, do not skip some snapshots
2018-06-10 10:08:47 +02:00
Alexander Neumann
c204382ea9 Revert "Fix integration tests on Windows"
This reverts commit 33dbd0ba5c.
2018-06-10 00:01:28 +02:00
Alexander Neumann
321efec60c Fix integration tests on Windows 2018-06-10 00:00:22 +02:00
Alexander Neumann
33dbd0ba5c Fix integration tests on Windows 2018-06-09 23:58:44 +02:00
Alexander Neumann
9a73869c27 Update docs for RHEL/CentOS 2018-06-09 23:41:40 +02:00
Alexander Neumann
8f26fe271c ls: Use walker for ls 2018-06-09 23:35:20 +02:00
Alexander Neumann
251335f124 Add entry to changelog 2018-06-09 23:35:20 +02:00
Alexander Neumann
081743d0a5 find: Use walker.Walk 2018-06-09 23:35:20 +02:00
Alexander Neumann
3a86f4852b Add walker for trees in the repo 2018-06-09 23:35:20 +02:00
Alexander Neumann
14aead94b3 filter: Allow double wildcard in ChildMatch 2018-06-09 23:18:13 +02:00
Alexander Neumann
ce01ca30d6 find: Correct tree pruning optimization
The `find` command will now take care to only mark trees as "not found"
when the pattern couldn't be found within any subtree.

Closes #1825, #1823
2018-06-09 18:59:13 +02:00
Alexander Neumann
e2d347a698 find: Use OS independent slash-based format 2018-06-09 18:58:13 +02:00
Alexander Neumann
42ebb0a0a6 backup: Parse timestamp earlier 2018-06-09 18:21:12 +02:00
Alexander Neumann
419acad3c3 Merge pull request #1837 from restic/fix-1833
cache: Ensure failed downloads are retried
2018-06-09 18:20:21 +02:00
Alexander Neumann
810b5ea076 Add entry to changelog 2018-06-09 17:55:51 +02:00
Alexander Neumann
fc5439a37a cache: Ensure failed downloads are retried
This fixes #1833, which consists of two different bugs:

 * The `defer` in `cacheFile()` may remove a channel from the
   `inProgress` map although it is not responsible for downloading the
   file

 * If the download fails, goroutines waiting for the file to be cached
   assumed that the file was there, there was no way to signal the
   error.
2018-06-09 17:50:56 +02:00
Alexander Neumann
48aab8bd65 Merge pull request #1836 from restic/update-blazer
Update github.com/kurin/blazer
2018-06-09 14:31:12 +02:00
Alexander Neumann
6fbcd1694b Add entry to changelog 2018-06-09 14:31:02 +02:00
Alexander Neumann
494fe2a8b5 Merge pull request #1835 from restic/fix-1834
backup: Fix deadlock
2018-06-09 14:28:16 +02:00
Alexander Neumann
f761068f4e Update github.com/kurin/blazer 2018-06-09 12:32:18 +02:00
Alexander Neumann
c44e808aa5 backup: Fix deadlock
When the archiver is faster than the scanner, restic deadlocks. This
commit adds a `finished` channel to the struct in `ui/backup.go` so that
scanner results are ignored when the archiver is already finished.

Closes #1834
2018-06-09 12:15:19 +02:00
Alexander Neumann
ab37c6095a Merge pull request #1821 from michaelkoetter/fix-1795
#1795 use unix.IoctlGetWinsize to get terminal size
2018-06-07 20:20:06 +02:00
Michael Kötter
d6fd94e49d Don't run Solaris build for go1.9 2018-06-04 15:04:50 +02:00
Michael Kötter
53040a2e34
add "solaris/amd64" to cross-compile archs 2018-06-04 12:51:34 +02:00
Alexander Neumann
cfc19b4582 Merge pull request #1828 from restic/handle-s3-list-errors
s3: Pass list errors up to the caller
2018-06-02 10:34:49 +02:00
Alexander Neumann
141fabdd09 s3: Pass list errors up to the caller 2018-06-01 22:15:23 +02:00
Alexander Neumann
d49ca42771 Merge pull request #1827 from restic/azure-large-files
azure: Support uploading large files
2018-06-01 18:37:26 +02:00
Alexander Neumann
f6fded729d Add entry to changelog 2018-06-01 14:52:16 +02:00
Alexander Neumann
465700595c azure: Support uploading large files
Closes #1822
2018-06-01 14:52:16 +02:00
Alexander Neumann
0fcd9d6926 Merge pull request #1824 from rfjakob/ssh_command_exited
sftp: persist "ssh command exited" error
2018-05-31 21:26:39 +02:00
Jakob Unterwurzacher
dd3b9910ee sftp: persist "ssh command exited" error
If our ssh process has died, not only the next, but all subsequent
calls to clientError() should indicate the error.

restic output when the ssh process is killed with "kill -9":

  Save(<data/afb68adbf9>) returned error, retrying after 253.661803ms: Write: failed to send packet header: write |1: file already closed
  Save(<data/afb68adbf9>) returned error, retrying after 580.752212ms: ssh command exited: signal: killed
  Save(<data/afb68adbf9>) returned error, retrying after 790.150468ms: ssh command exited: signal: killed
  Save(<data/afb68adbf9>) returned error, retrying after 1.769595051s: ssh command exited: signal: killed
  [...]
  error in cleanup handler: ssh command exited: signal: killed

Before this patch:

  Save(<data/de698d934f>) returned error, retrying after 252.84163ms: Write: failed to send packet header: write |1: file already closed
  Save(<data/de698d934f>) returned error, retrying after 660.236963ms: OpenFile: failed to send packet header: write |1: file already closed
  Save(<data/de698d934f>) returned error, retrying after 568.049909ms: OpenFile: failed to send packet header: write |1: file already closed
  Save(<data/de698d934f>) returned error, retrying after 2.428813824s: OpenFile: failed to send packet header: write |1: file already closed
  [...]
  error in cleanup handler: failed to send packet header: write |1: file already closed
2018-05-30 19:28:14 +02:00
Alexander Neumann
185b60c22b Document project governance 2018-05-28 22:29:06 +02:00
Michael Kötter
589c23dc23 #1795 use unix.IoctlGetWinsize to get terminal size 2018-05-27 23:44:48 +02:00
Alexander Neumann
0183fea926 Merge pull request #1820 from restic/fix-1803
termstatus: Fix panic for non-terminal runs
2018-05-27 13:08:25 +02:00
Alexander Neumann
7d9642523b termstatus: Fix panic for non-terminal runs
Closes #1803
2018-05-27 12:52:01 +02:00
Alexander Neumann
4bf07a74a0 Merge pull request #1806 from mholt/patch-1
doc: Clarify multiple forget policies get ORed
2018-05-26 11:03:22 +02:00
Alexander Neumann
2a976d795f b2: Remove extra error check 2018-05-26 10:12:30 +02:00
Alexander Neumann
1892b314f8 Merge pull request #1815 from restic/update-blazer
Update github.com/kurin/blazer
2018-05-25 20:46:35 +02:00
Alexander Neumann
b7bed406b9 Update github.com/kurin/blazer 2018-05-25 20:26:26 +02:00
Matt Holt
ee4202f7c3
doc: Clarify multiple forget policies get ORed 2018-05-23 17:28:02 -06:00
Alexander Neumann
4cd28713b6 Merge pull request #1802 from restic/rclone-add-limits
rclone: Add limiting bandwidth to the rclone backend
2018-05-22 21:19:52 +02:00
Alexander Neumann
e3fe87f269 Remove superseded feature from the CHANGELOG
For a discussion please see https://github.com/restic/restic/issues/1796
2018-05-22 20:57:07 +02:00
Alexander Neumann
a02698fcdd Add entry to changelog 2018-05-22 20:48:29 +02:00
Alexander Neumann
bfd923e81e rclone: Respect bandwith limits 2018-05-22 20:48:17 +02:00
Alexander Neumann
20bfed5985 Update build.go 2018-05-21 20:31:19 +02:00
Alexander Neumann
e40191942d Add VERSION for 0.9.0 2018-05-21 15:57:32 +02:00
Alexander Neumann
abd34ab76f Update manpages and auto-completion 2018-05-21 15:57:32 +02:00
Alexander Neumann
4b43a269ee Generate CHANGELOG.md for 0.9.0 2018-05-21 15:57:22 +02:00
Alexander Neumann
e2b7dc6528 Move changelog files for the 0.9.0 release 2018-05-21 15:57:02 +02:00
Alexander Neumann
d2431b667f Merge pull request #1794 from restic/fix-1789
Update docs, clarify --cacert
2018-05-21 12:35:06 +02:00
Alexander Neumann
b70fdf61c4 Merge pull request #1791 from restic/fix-1790
archiver: Fix backup from stdin
2018-05-21 12:07:55 +02:00
Alexander Neumann
e6f25c4811 Update docs, clarify --cacert
Closes #1789
2018-05-21 12:06:30 +02:00
Alexander Neumann
adb682bc43 archiver: Don't open files with O_NONBLOCK
This is not necessary any more, we're doing an lstat() before opening
an item, so we already known it's a file and not a pipe.
2018-05-20 16:11:51 +02:00
Alexander Neumann
1e9744c9a4 archiver: Refuse to save an empty snapshot 2018-05-20 16:11:51 +02:00
Alexander Neumann
9a02f17cc2 archiver: Add tests for Save() for fs.Reader 2018-05-20 16:11:36 +02:00
Alexander Neumann
c284712cae Merge pull request #1784 from restic/improve-error-list
list: Improve error message
2018-05-18 21:53:09 +02:00
Alexander Neumann
2dbdf381b2 Merge pull request #1782 from skriss/add-s3-file-creds
Add S3 file creds and reorder creds chain
2018-05-18 21:52:54 +02:00
Alexander Neumann
a1a49ce211 Merge pull request #1787 from tyll/patch-1
Use more descriptive long option
2018-05-18 21:51:24 +02:00
Till Maas
3252e4200c
Use more descriptive long option
-r could also mean recursive so use --repo to be clear.
2018-05-18 19:25:56 +02:00
Alexander Neumann
8d9d218d1c list: Improve error message
Before:

    $ restic list
    Fatal: type not specified

After:

    $ restic list
    Fatal: type not specified, usage: list [blobs|packs|index|snapshots|keys|locks]

Closes #1783
2018-05-17 19:41:56 +02:00
Steve Kriss
0785fbd418 changelog entry
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-05-17 10:09:38 -07:00
Steve Kriss
b358dd369b S3: rearrange credentials chain to be standard
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-05-16 16:49:33 -07:00
Steve Kriss
d67b9a32c6 S3: add file credentials to chain
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-05-16 16:35:14 -07:00
Alexander Neumann
ecfe59235e Merge pull request #1781 from McKael/fix_restick_backup_rejectByDevice
Fix restic backup --one-file-system /
2018-05-16 21:58:58 +02:00
Mikael Berthe
a868a30f4d Fix restic backup --one-file-system /
This patch should fix the following panic when trying to backup the
root filesystem with thre --one-file-system flag:

    % restic backup --one-file-system /
    (...)
    panic: item /, device id 2082 not found, allowedDevs: map[/:2082]
2018-05-16 13:44:14 +02:00
Alexander Neumann
347a645450 Fix double error message 2018-05-15 11:03:33 +02:00
Alexander Neumann
9f5565b0fc Merge pull request #1735 from mholt/forget-max-age
forget: Add --max-age policy to set hard cutoff for removing snapshots
2018-05-14 21:18:06 +02:00
Alexander Neumann
fd979ab4c5 Merge pull request #1779 from restic/improve-error
Improve error message for readdir/readdirnames
2018-05-14 21:17:57 +02:00
Alexander Neumann
375868edcf Add documentation 2018-05-13 12:54:23 +02:00
Alexander Neumann
060d8b57e0 Restructure TestApplyPolicy 2018-05-13 12:48:42 +02:00
Alexander Neumann
cc627e832b Add custom Duration type 2018-05-13 12:48:42 +02:00
Alexander Neumann
5a0f0e3faa Add support for keeping a range of snapshots 2018-05-13 12:48:42 +02:00
Matthew Holt
b52f2aa9a4 forget: Add policy to keep snapshots before a date 2018-05-13 12:48:10 +02:00
Alexander Neumann
60ea2435be Improve error message for readdir/readdirnames
As mentioned in the forum[1], restic does not include the dir name when
readdir/readdirnames fails.

[1] https://forum.restic.net/t/readdirnames-readdirent-no-such-file-or-directory/653
2018-05-13 10:34:50 +02:00
Alexander Neumann
159badf5ba Merge pull request #1778 from restic/fix-1771
archiver: Improve error handling
2018-05-13 00:13:54 +02:00
Alexander Neumann
903a3a31dc Merge pull request #1776 from restic/fix-1775
Always use cleaned path for excludes
2018-05-13 00:13:52 +02:00
Alexander Neumann
548227e6df Merge pull request #1773 from ExoUNX/patch-1
Fix small typo
2018-05-13 00:13:49 +02:00
Alexander Neumann
cd03275005 Merge pull request #1774 from ExoUNX/patch-2
Fix formatting
2018-05-13 00:13:46 +02:00
Alexander Neumann
e43c9202a6 archiver: Make sure backend error is passed up 2018-05-12 23:55:59 +02:00
Alexander Neumann
c5e75d1c98 archiver: Add test for early abort on unhandled error 2018-05-12 23:55:59 +02:00
Alexander Neumann
526956af35 archiver: Read files/dirs in order 2018-05-12 23:55:54 +02:00
Alexander Neumann
256104111d archiver: Clarify names 2018-05-12 23:55:54 +02:00
Alexander Neumann
21c83b1725 archiver: Add high-level documentation 2018-05-12 23:55:54 +02:00
Alexander Neumann
581c62ee72 archiver: Improve error handling
This commit changes how the worker goroutines for saving e.g. blobs
interact. Before, it was possible to get stuck sending an instruction to
archive a file or dir when no worker goroutines were available any more.
This commit introduces a `done` channel for each of the worker pools,
which is set to the channel returned by `tomb.Dying()`, so it is closed
when the first worker returned an error.
2018-05-12 23:55:54 +02:00
Alexander Neumann
ef7747313d backup: Use absolute paths for allowed devices 2018-05-11 21:32:44 +02:00
Alexander Neumann
18d4ac2fd9 backup: Always use cleaned path for excludes 2018-05-11 21:32:41 +02:00
Alexander Neumann
9180e2c48a Remove unneeded file excludes 2018-05-11 21:26:10 +02:00
Gaige Lama
a63989afcd
Ran gofmt on backend.go
I ran gofmt on backend.go, this appears to valid edit.
2018-05-11 11:07:16 -04:00
Gaige Lama
d3c0bd6d0e
Fix small typo
"explicitely" should be "explicitly"
2018-05-11 10:55:12 -04:00
Alexander Neumann
fcfa6f0355 build: Add option to enable PIE build mode 2018-05-11 09:50:10 +02:00
Alexander Neumann
580f90d745 Merge pull request #1770 from restic/handle-blob-save-errors
archiver: Correct error handling
2018-05-10 22:36:00 +02:00
Alexander Neumann
c7b624ba0d Merge pull request #1769 from McKael/cat_snapshot_stacktrace
cat snapshot: Do not display a stack trace with invalid IDs
2018-05-10 21:41:04 +02:00
Alexander Neumann
ca4af43c03 archiver: Return low-level errors
This commit changes the archiver so that low-level errors saving data to
the repo are returned to the caller (instead of being handled by the
error callback function). This correctly bubbles up errors like a full
temp file system and makes restic abort early and makes all other worker
goroutines exit.
2018-05-10 21:30:09 +02:00
Alexander Neumann
1f2463f42e archiver: Return correct error 2018-05-10 20:48:00 +02:00
Alexander Neumann
157c854d04 backup: Remove errored files from status display 2018-05-10 20:48:00 +02:00
Mikael Berthe
ffc276a603 cat: Do not display a stack trace
Don't show a stack trace when "restic cat snapshot" is invoked with
invalid/nonexistent IDs.
2018-05-10 19:22:46 +02:00
Alexander Neumann
e42b7db008 Merge pull request #1767 from restic/update-blazer
Update github.com/kurin/blazer
2018-05-08 22:48:54 +02:00
Alexander Neumann
024148cac9 Update github.com/kurin/blazer 2018-05-08 22:33:55 +02:00
Alexander Neumann
8943741a0b Fix code block 2018-05-08 20:36:58 +02:00
Alexander Neumann
95c5517c35 Merge pull request #1765 from restic/update-deps
Update golang.org/x/{sys,crypto}
2018-05-08 19:34:30 +02:00
Alexander Neumann
06179a7e81 Change URL for Travis 2018-05-08 19:13:39 +02:00
Alexander Neumann
cf1fb50f9c Merge pull request #1764 from restic/fix-display
Fix display
2018-05-07 23:30:50 +02:00
Alexander Neumann
6793300850 Raise minimum Go version to 1.9
The golang.org/sys/crypto library uses math/bits, which was only
introduced in Go 1.9.
2018-05-07 23:28:48 +02:00
Alexander Neumann
2cbdfbf652 Update golang.org/x/{sys,crypto} 2018-05-07 22:47:39 +02:00
Alexander Neumann
b2208bb9c2 Rework termstatus
This now keeps the cursor at the first column of the first status line
so that messages printed to stdout or stderr by some other part of the
progarm will still be visible. The message will overwrite the status
lines, but those are easily reprinted on the next status update.
2018-05-07 21:40:07 +02:00
Alexander Neumann
4c25495d68 backup: Hide percent until total size is known 2018-05-06 20:20:25 +02:00
Alexander Neumann
abdd59ea1b Merge pull request #1757 from restic/fix-debug-1755
Fix panic for nil-pointer dereference
2018-05-03 21:11:37 +02:00
Alexander Neumann
05ca903d48 Mention that first backup will take longer 2018-05-03 20:57:30 +02:00
Alexander Neumann
fd77646f8b Fix panic for nil-pointer dereference
Closes #1755
2018-05-03 20:49:30 +02:00
Alexander Neumann
2a67258867 Merge pull request #1754 from restic/fix-quiet
backup: Disable status output for --quiet
2018-05-02 21:42:17 +02:00
Alexander Neumann
fca4fe4459 backup: Disable status output for --quiet 2018-05-02 21:24:18 +02:00
Alexander Neumann
26757ae2e5 Merge pull request #1750 from restic/archiver-fix-small-things
backup: Tune user interface a bit
2018-05-01 23:15:54 +02:00
Alexander Neumann
9d6890a236 Merge pull request #1751 from restic/fix-archiver
archiver: Use lstat before open/fstat
2018-05-01 23:15:32 +02:00
Alexander Neumann
2218ecd049 archiver: Use lstat before open/fstat
The previous code tried to be as efficient as possible and only do a
single open() on an item to save, and then fstat() on the fd to find out
what the item is (file, dir, other). For normal files, it would then
start reading the data without opening the file again, so it could not
be exchanged for e.g. a symlink.

This behavior starts the watchdog on my machine when /dev is saved
with restic, and after a few seconds, the machine reboots.

This commit reverts the behavior to the strategy the old archiver code
used: run lstat(), then decide what to do. For normal files, open the
file and then run fstat() on the fd to verify it's still a normal file,
then start reading the data.

The downside is that for normal files we now do two stat() calls
(lstat+fstat) instead of only one. On the upside, this does not start
the watchdog. :)
2018-05-01 23:05:50 +02:00
Alexander Neumann
d0974c155d backup: Tune user interface a bit
Make non-verbose mode a bit more verbose.
2018-05-01 22:18:32 +02:00
Alexander Neumann
8026e6fdfb Merge pull request #1749 from restic/add-cache-cmd
Add 'cache' command to list and cleanup cache dirs
2018-05-01 21:25:52 +02:00
Alexander Neumann
01f9662614 Merge pull request #1748 from restic/detect-bom
Respect Encoding and Byte Order Mark when reading text files
2018-05-01 21:25:46 +02:00
Alexander Neumann
f928aeec34 Add 'cache' command 2018-05-01 16:27:44 +02:00
Alexander Neumann
f77bc0fae8 Use textfile.Read() to read files
This converts the text to UTF-8 and removes a Byte Order Mark.
2018-05-01 16:25:53 +02:00
Alexander Neumann
eb6650b201 Add textfile 2018-05-01 14:38:59 +02:00
Alexander Neumann
bc68d55e94 fs: Add TestTempFile 2018-05-01 14:38:41 +02:00
Alexander Neumann
ecbbd851a1 Merge pull request #1744 from restic/fix-parent-detect
backup: Clean target paths before finding parent
2018-05-01 11:13:19 +02:00
Alexander Neumann
336719b058 Merge pull request #1746 from brycied00d/fix_tls-client-cert_argument_parsing
http backend: Parse the correct argument when loading --tls-client-cert
2018-05-01 10:22:10 +02:00
Bryce Chidester
e9f1721678 http backend: Parse the correct argument when loading --tls-client-cert
Previously, the function read from ARGV[1] (hardcoded) rather than the
value passed to it, the command-line argument as it exists in globalOptions.

Resolves #1745
2018-04-30 15:21:09 -07:00
Alexander Neumann
64d98945a6 backup: Clean target paths before finding parent
This resolves an issue described in the forum where restic could not
find a parent snapshot if the target path ends with a slash:
https://forum.restic.net/t/new-archiver-code-please-test/623/23
2018-04-30 22:03:11 +02:00
Alexander Neumann
84f82dae1a Correct typo
Closes #1743
2018-04-30 21:41:30 +02:00
Alexander Neumann
6bfd9f833b Merge pull request #1742 from tbm/typos
Fix typos in section on rclone
2018-04-30 19:21:19 +02:00
Martin Michlmayr
bb1a22d1e6 Fix typos in section on rclone 2018-04-30 18:32:10 +02:00
Alexander Neumann
438719f269 Merge pull request #1741 from restic/improve-archiver
Improve archiver, fix hang on fifos
2018-04-30 16:00:10 +02:00
Alexander Neumann
c83c03ed63 archiver: Fix blocking on pipes 2018-04-30 15:34:58 +02:00
Alexander Neumann
19b9c881ca fs: Add O_NONBLOCK 2018-04-30 15:34:58 +02:00
Alexander Neumann
4e34325035 archiver: Process dirs concurrently 2018-04-30 15:13:28 +02:00
Alexander Neumann
78bd591c7c archiver: Improve buffer pool 2018-04-30 15:13:28 +02:00
Alexander Neumann
39ac12f6ea archiver: Correct comment 2018-04-30 14:19:07 +02:00
Alexander Neumann
400730afca archiver: Improve memory usage, tune buffer pool 2018-04-30 14:19:07 +02:00
Alexander Neumann
d80e108b03 backup: Clear status lines on finish 2018-04-30 14:19:07 +02:00
Alexander Neumann
846c2b6869 backup: Fix ETA calculation for >= 100% 2018-04-30 14:19:07 +02:00
Alexander Neumann
d8bbe5dc84 Print repository ID after opening 2018-04-30 14:19:07 +02:00
Alexander Neumann
d926b9fd80 Add profile build flag 2018-04-30 14:19:04 +02:00
Alexander Neumann
4ba8d40282 Add block profile option 2018-04-30 14:18:45 +02:00
Alexander Neumann
4fb1401266 Fix --cacert help text 2018-04-29 13:17:33 +02:00
Alexander Neumann
6d4c40f8d0 Add section about file read concurrency 2018-04-28 22:39:56 +02:00
Alexander Neumann
56e394ac33 Merge pull request #1494 from restic/new-archiver
New archiver code
2018-04-28 22:24:39 +02:00
Alexander Neumann
c3cc5d7cee Update docs 2018-04-28 22:08:11 +02:00
Alexander Neumann
6b12b92339 Add entry to changelog 2018-04-28 22:08:11 +02:00
Alexander Neumann
16c314ab7f termstatus: Don't print status if in background 2018-04-28 22:08:11 +02:00
Alexander Neumann
1449d7dc29 Remove background checking code 2018-04-28 22:08:11 +02:00
Alexander Neumann
0e78ac92d8 Use new archiver code for backup 2018-04-28 22:08:11 +02:00
Alexander Neumann
c703d21d55 Vendor gopkg.in/tomb.v2 2018-04-27 21:42:15 +02:00
Alexander Neumann
1af96fc6dd Add termstatus 2018-04-27 21:42:15 +02:00
Alexander Neumann
9fac2ca832 Add flags to set verbosity 2018-04-25 14:42:45 +02:00
Alexander Neumann
a5c0cf2324 Add workaround for symlinked temp dir on darwin
Chdir to the tempdir, then use os.Getwd() to get the name that
filepath.Abs() uses (and stores in the Snapshot).
2018-04-25 14:42:45 +02:00
Alexander Neumann
38926d8576 Use new archiver code in tests 2018-04-25 14:42:45 +02:00
Alexander Neumann
f279731168 Add new archiver code 2018-04-25 14:42:45 +02:00
Alexander Neumann
76b616451f Remove unneeded code 2018-04-23 21:40:33 +02:00
Alexander Neumann
fd12a3af20 Remove old archiver code 2018-04-23 21:40:33 +02:00
Alexander Neumann
3cd92efdcf Vendor github.com/mattn/go-isatty 2018-04-22 11:37:05 +02:00
Alexander Neumann
b804279fe8 integration tests: Don't print anything to stdout 2018-04-22 11:37:05 +02:00
Alexander Neumann
a56b8fad87 repository: Improve buffer pooling 2018-04-22 11:37:05 +02:00
Alexander Neumann
4c00efd4bf Vendor go-cmp 2018-04-22 11:37:05 +02:00
Alexander Neumann
b6f98bdb02 node: Fill minimal info 2018-04-22 11:37:05 +02:00
Alexander Neumann
c4b2486b7c fs: Add interface and FS implementations
This adds two implementations of the new `FS` interface: One for the local
file system (`Local`) and one for a single file read from an
`io.Reader` (`Reader`).
2018-04-22 11:37:05 +02:00
Alexander Neumann
83ca08245b checker: Check metadata size and blob sizes 2018-04-22 11:37:05 +02:00
Alexander Neumann
a069467e72 ls: Improve output 2018-04-22 11:37:05 +02:00
Alexander Neumann
6a7c23d2ae tree: Add convenience functions 2018-04-22 11:37:05 +02:00
Alexander Neumann
cc847a3d6d tree: Improve error for pre-existing node 2018-04-22 11:37:05 +02:00
Alexander Neumann
baebf45e2e FindLatestSnapshot: Use absolute paths 2018-04-22 11:37:05 +02:00
Alexander Neumann
fa4f438bc1 snapshot: Do not modify slice of paths 2018-04-22 11:37:05 +02:00
Alexander Neumann
4e0b2a8e3a snapshot: correct error handling for filepath.Abs 2018-04-22 11:37:05 +02:00
Alexander Neumann
0532f08048 Add test.Helper, also works with Go 1.8 2018-04-22 11:37:05 +02:00
Alexander Neumann
a472868e06 fs: Add TestChdir() 2018-04-22 11:37:05 +02:00
Alexander Neumann
e4fdc5eb76 fs: Add IsRegularFile() 2018-04-22 11:37:05 +02:00
Alexander Neumann
09365cc4ea Add --trace-profile 2018-04-22 11:37:05 +02:00
Eri Bastos
2aa6b49651 Return exit code 130 when SIGINT is received 2018-04-20 21:09:50 +02:00
Alexander Neumann
7877797c7e Merge pull request #1720 from mholt/new-password-flag
key: Add --new-password flag for non-interactive password changes
2018-04-20 14:52:41 +02:00
Matthew Holt
1a26355dbe Add changelog file 2018-04-20 14:29:39 +02:00
Matthew Holt
c5829e9ffc key: Add flag for non-interactive password changes 2018-04-20 14:29:39 +02:00
Alexander Neumann
b5b246edd5 Add entry to changelog 2018-04-20 14:23:37 +02:00
Alexander Neumann
ee5e14d536 Merge pull request #1731 from restic/fix-1730
Do not restore sockets, correct error handling
2018-04-20 14:21:01 +02:00
Alexander Neumann
09bd924710 Do not restore sockets, correct error handling
Closes #1730
2018-04-20 13:53:11 +02:00
Alexander Neumann
a9c2e84ccd Merge pull request #1718 from ardichoke/patch-1
Update Autocompletion  Generation Documentation
2018-04-13 22:23:37 +02:00
Ryan DeShone
a144b81c4a
Update Autocompletion Generation Documentation
The autocompletion command has been changed to generate. Update documentation to reflect this change.
2018-04-12 11:50:26 -04:00
Alexander Neumann
3c453a4217 Merge pull request #1715 from rawtaz/patch-1
Add "Including and Exluding Files" heading in docs
2018-04-11 21:27:22 +02:00
rawtaz
1e2f23d77a
Add "Including and Exluding Files" heading in docs
Adds "Including and Exluding Files" heading in the backup section in the docs.

[ci skip]
2018-04-11 21:14:26 +02:00
Alexander Neumann
2c76e724ab Merge pull request #1712 from restic/list-ignore-non-existing-dirs
Improve handling non-existing dirs in List()
2018-04-10 21:54:00 +02:00
Alexander Neumann
577faa7570 local/sftp: Handling non-existing dirs in List() 2018-04-10 21:35:30 +02:00
Alexander Neumann
6a34e0d10f Merge pull request #1709 from restic/improve-check-errors
Improve error message for orphaned pack files
2018-04-07 13:38:41 +02:00
Alexander Neumann
b08f21cdc6 Add entry to changelog 2018-04-07 13:05:44 +02:00
Alexander Neumann
1c1fede399 Improve error message for orphaned pack files 2018-04-07 10:07:54 +02:00
Alexander Neumann
63a0913e6e Merge pull request #1705 from alirazeen/doc-add-scripting
Add scripting section to docs
2018-04-07 09:47:17 +02:00
Ali Razeen
325957443e Add scripting section to docs 2018-04-04 10:43:20 -04:00
Alexander Neumann
4b2d3b15a2 Add entry to changelog 2018-04-03 20:42:06 +02:00
Alexander Neumann
4e2a87c920 Merge pull request #1703 from ebastos/issue1608
Fixed issue #1608 - Use --time argument properly
2018-04-03 20:40:41 +02:00
Eri Bastos
901e1b129c Fixed issue #1608 - Use --time argument properly
Backups via stdin will now handle --time argument and pass it down as
expected
2018-04-03 14:40:42 -03:00
Alexander Neumann
4478d633e2 Merge pull request #1702 from restic/update-simple-scrypt
Update github.com/elithrar/simple-scrypt
2018-04-02 20:07:18 +02:00
Alexander Neumann
92f516b1d4 Update github.com/elithrar/simple-scrypt
For details see #1697
2018-04-02 19:48:25 +02:00
Alexander Neumann
03193e6d92 Fix changelog entries 2018-04-02 12:42:48 +02:00
Alexander Neumann
01fe719aff check: Make sure temp cache dir is removed 2018-04-01 18:09:53 +02:00
Alexander Neumann
2c964df3e2 Merge pull request #1699 from restic/fix-incremental-backup-test
Slightly increment size for TestIncrementalBackup
2018-04-01 14:33:12 +02:00
Alexander Neumann
8919125b0b Merge pull request #1696 from restic/fix-check-cache
check: Improve cache handling
2018-04-01 14:33:00 +02:00
Alexander Neumann
1f5137aa70 Add entry to CHANGELOG 2018-04-01 13:59:27 +02:00
Alexander Neumann
a95eb33616 check: Use cache in temporary directory if possible
Closes #1694
2018-04-01 13:59:27 +02:00
Alexander Neumann
e68a7fea8a check: Allow filling the cache during check
Closes #1665
2018-04-01 13:59:27 +02:00
Alexander Neumann
2e7ec717c1 repository: Move cache preparation into function 2018-04-01 13:59:27 +02:00
Alexander Neumann
22d5061df2 Merge pull request #1698 from restic/reduce-backend-tests-travis
Travis: Skip cloud backend tests for most Go versions
2018-04-01 13:58:38 +02:00
Alexander Neumann
4544a77172 Slightly increment size for TestIncrementalBackup
This should make the test more reliable, it should hit the accidental
"repo is has grown too much" way less often.
2018-04-01 13:49:42 +02:00
Alexander Neumann
b3a073e066 Travis: Skip cloud backend tests for most Go versions 2018-04-01 13:13:38 +02:00
Alexander Neumann
b077a1227b Merge pull request #1657 from restic/rclone-backend
Rclone backend
2018-04-01 10:56:10 +02:00
Alexander Neumann
3f48e0e0f4 Add extra options to rclone
For details see https://github.com/restic/restic/pull/1657#issuecomment-377707486
2018-04-01 10:34:30 +02:00
Alexander Neumann
86f4b03730 Remove unneeded byte counters 2018-04-01 10:18:38 +02:00
Alexander Neumann
c43c94776b rclone: Make concurrent connections configurable 2018-04-01 10:18:38 +02:00
Alexander Neumann
0b776e63e7 backend/rclone: Request random file name
When `/` is requested, rclone returns the list of all files in the
remote, which is not what we want (and it can take quite some time).
2018-04-01 10:18:38 +02:00
Alexander Neumann
360ff1806a doc: Fix instructions for rclone backend 2018-04-01 10:18:38 +02:00
Alexander Neumann
1beeb7d0dd doc/REST: Make documentation match reality 2018-04-01 10:18:38 +02:00
Alexander Neumann
e978b36713 doc: Add hint how to debug rclone 2018-04-01 10:18:38 +02:00
Alexander Neumann
737d93860a Extend first timeout to 60 seconds. 2018-04-01 10:18:38 +02:00
Alexander Neumann
011217e4bf backend/rclone: Improve documentation and README 2018-04-01 10:18:38 +02:00
Alexander Neumann
362d5afec4 Add entry to changelog 2018-04-01 10:18:38 +02:00
Alexander Neumann
4172fcd167 doc: Add rclone backend 2018-04-01 10:18:38 +02:00
Alexander Neumann
518bf4e5f6 doc: Correct verbatim text in the manual 2018-04-01 10:18:38 +02:00
Alexander Neumann
17312d3a98 backend/rest: Ensure base URL ends with slash
This makes it easier for rclone.
2018-04-01 10:18:38 +02:00
Alexander Neumann
4d5c7a8749 backend/rclone: Make sure rclone terminates 2018-04-01 10:18:38 +02:00
Alexander Neumann
fc0295016a Address code review comments 2018-04-01 10:18:38 +02:00
Alexander Neumann
99b62c11b8 backend/rclone: Stop rclone in case of errors 2018-04-01 10:18:38 +02:00
Alexander Neumann
6d9a029e09 backend/rclone: Prefix all error messages 2018-04-01 10:18:38 +02:00
Alexander Neumann
20352886f3 Update Gopkg.lock 2018-04-01 10:18:38 +02:00
Alexander Neumann
3622b60c13 CI: Check that rclone backend test isn't skipped 2018-04-01 10:16:31 +02:00
Alexander Neumann
065fe1e54f backend/rclone: Skip test if binary is unavailable 2018-04-01 10:16:31 +02:00
Alexander Neumann
4dc0f24b38 backend/tests: Drain reader before returning error 2018-04-01 10:16:31 +02:00
Alexander Neumann
fe99340e40 Add rclone backend 2018-04-01 10:16:31 +02:00
Alexander Neumann
e377759c81 rest: Export Backend struct 2018-04-01 10:16:31 +02:00
Alexander Neumann
61f6db25f4 CI: install rclone 2018-04-01 10:16:31 +02:00
Alexander Neumann
cabbbd2b14 backend/rest: Export Content-Types 2018-04-01 10:16:31 +02:00
Alexander Neumann
cf4cf94418 Move backend/sftp.StartForeground to backend/ 2018-04-01 10:16:31 +02:00
Alexander Neumann
34f27edc03 Refactor SplitShellStrings 2018-04-01 10:16:31 +02:00
Alexander Neumann
345b6c4694 Move backend/sftp.SplitShellArgs to backend/ 2018-04-01 10:16:31 +02:00
Alexander Neumann
e4a39e02d2 Merge pull request #1695 from xulongwu4/patch-2
Add instructions for installation on Solus
2018-03-31 09:14:25 +02:00
xulongwu4
432e167255
Add instructions for installation on Solus 2018-03-30 17:36:29 -04:00
Alexander Neumann
594256bfa4 Merge pull request #1693 from restic/update-deps
Update dependencies
2018-03-30 17:19:08 +02:00
Alexander Neumann
0fcb1e6b7a Merge pull request #1692 from restic/print-forget-policy
forget: Print policy
2018-03-30 17:19:05 +02:00
Alexander Neumann
38795c66c9 Update vendored library gopkg.in/yaml.v2 2018-03-30 12:53:13 +02:00
Alexander Neumann
c0960f538f Update vendored library google.golang.org/api 2018-03-30 12:51:18 +02:00
Alexander Neumann
5b6568875c Update vendored library golang.org/x/text 2018-03-30 12:50:04 +02:00
Alexander Neumann
d8dd79eb0b Update vendored library golang.org/x/sys 2018-03-30 12:48:49 +02:00
Alexander Neumann
2bdeb645b9 Update vendored library golang.org/x/sync 2018-03-30 12:47:30 +02:00
Alexander Neumann
9f2ffa3e50 Update vendored library golang.org/x/oauth2 2018-03-30 12:46:26 +02:00
Alexander Neumann
d4bab5c133 Update vendored library golang.org/x/net 2018-03-30 12:45:07 +02:00
Alexander Neumann
3473d73d0c Update vendored library github.com/spf13/cobra 2018-03-30 12:43:03 +02:00
Alexander Neumann
917cc542c9 Update vendored library github.com/sirupsen/logrus 2018-03-30 12:41:46 +02:00
Alexander Neumann
a9cf5d482a Update vendored library github.com/russross/blackfriday 2018-03-30 12:40:05 +02:00
Alexander Neumann
75946e7c58 Update vendored library github.com/pkg/xattr 2018-03-30 12:38:37 +02:00
Alexander Neumann
19035e977b Update vendored library github.com/pkg/sftp 2018-03-30 12:37:16 +02:00
Alexander Neumann
d9ba9279e0 Update vendored library github.com/ncw/swift 2018-03-30 12:35:13 +02:00
Alexander Neumann
31e156c666 Update vendored library github.com/minio/minio-go 2018-03-30 12:33:40 +02:00
Alexander Neumann
7e6fff324c Update vendored library golang.org/x/crypto 2018-03-30 12:26:26 +02:00
Alexander Neumann
e94d2da890 Update vendored library github.com/golang/protobuf 2018-03-30 11:52:18 +02:00
Alexander Neumann
874b3dbbd9 Update vendored library github.com/go-ini/ini 2018-03-30 11:51:01 +02:00
Alexander Neumann
0d01c27c9e Update vendored library github.com/dgrijalva/jwt-go 2018-03-30 11:49:18 +02:00
Alexander Neumann
30110fcfc2 Update vendored library github.com/cpuguy83/go-md2man 2018-03-30 11:48:16 +02:00
Alexander Neumann
673f0bbd6c Update vendored library github.com/cenkalti/backoff 2018-03-30 11:45:27 +02:00
Alexander Neumann
5a77b2ab49 Update vendored library github.com/Azure/azure-sdk-for-go 2018-03-30 11:42:11 +02:00
Alexander Neumann
a951e7b126 Update vendored library cloud.google.com/go 2018-03-30 11:41:12 +02:00
Alexander Neumann
d3f9c8b362 forget: Print policy 2018-03-30 10:24:26 +02:00
Alexander Neumann
a4ff591165 Update URL to template 2018-03-25 11:55:37 +02:00
Alexander Neumann
49dd70c771 Merge pull request #1686 from gtrafimenkov/minor-spelling-fixes
Correct spelling mistakes
2018-03-25 11:54:53 +02:00
Gennady Trafimenkov
a64f24029b
Correct spelling mistakes 2018-03-25 00:42:33 +03:00
Alexander Neumann
0886738d24 Add entry to CHANGELOG 2018-03-24 18:40:49 +01:00
Alexander Neumann
9fc38803e0 Merge pull request #1684 from restic/fix-rest-tests
Fix rest-server tests
2018-03-24 18:39:44 +01:00
Alexander Neumann
e5c929b793 Fix rest-server tests
Since today, the rest-server needs to be explicitly told (via
`--no-auth`) that authentication is not necessary.
2018-03-24 18:06:21 +01:00
Alexander Neumann
0e0fee9c8f Update changelog template 2018-03-24 17:31:21 +01:00
Alexander Neumann
26769a39eb Merge pull request #1679 from rawtaz/version-oneliner
Make version output one line.
2018-03-23 21:14:37 +01:00
Leo R. Lundgren
923be90906 Make version output one line. 2018-03-21 21:49:03 +01:00
Alexander Neumann
84a22eac92 Merge pull request #1675 from oliver/doc-fixes
Minor doc fixes/improvements
2018-03-21 20:54:51 +01:00
Alexander Neumann
6eb1be0be4 Fix changelog files 2018-03-21 20:53:11 +01:00
Alexander Neumann
f31bbcf1a9 CI: Return error when calens fails 2018-03-21 20:53:01 +01:00
Alexander Neumann
5d09fca6a2 Merge pull request #1676 from bowensong/quiet-skip-scan
Skip archiver.Scan before backup when --quiet is set
2018-03-21 20:45:34 +01:00
Bowen Song
34671d7c9b Skip archiver.Scan before backup when --quiet is set 2018-03-20 20:44:10 +00:00
Alexander Neumann
4a524da736 FAQ: Add suggestion on how to solve sftp error 2018-03-18 21:39:11 +01:00
Oliver Gerlich
e361cc3807 040_backup.rst: add note regarding paths in --files-from argument 2018-03-18 20:51:04 +01:00
Alexander Neumann
3cd8a7bc96 Fix small things left open by #1552 2018-03-18 19:54:12 +01:00
Alexander Neumann
8206f85d2e Merge pull request #1552 from lawrencejones/use-auto-auth
Automatically load Google auth
2018-03-18 19:53:30 +01:00
Alexander Neumann
7022144e0f Merge pull request #1673 from rawtaz/doc-precompiled
doc: Add more info about using pre-compiled builds.
2018-03-18 19:50:25 +01:00
Oliver Gerlich
1bee3e01fa 040_backup.rst: fix typo 2018-03-18 17:06:21 +01:00
Oliver Gerlich
624a2d8305 040_backup.rst: quote wildcard characters
This doesn't really matter for --exclude in most cases, but it avoids
spreading bad ideas.
2018-03-18 17:05:53 +01:00
Lawrence Jones
57c6233982
dep ensure 2018-03-16 10:31:30 +00:00
Leo R. Lundgren
c161aba084 doc: Add more info about using pre-compiled builds. 2018-03-15 21:23:07 +01:00
Alexander Neumann
0279fd7212 Merge pull request #1669 from restic/make-tests-faster
Reduce test runtime
2018-03-13 19:50:34 +01:00
Alexander Neumann
dedf17f5e8 Merge pull request #1650 from copart/patch-1
Added copr repositories for RHEL/Centos/Fedora
2018-03-11 21:52:18 +01:00
Alexander Neumann
817890794d Merge pull request #1668 from restic/fix-1663
Return the first password/key which works
2018-03-11 21:51:54 +01:00
Alexander Neumann
b9ada91054 Reduce data for TestCreateSnapshot 2018-03-11 21:42:39 +01:00
Alexander Neumann
dfb6d0fced Reduce data for TestIncrementalBackup 2018-03-11 21:17:27 +01:00
Alexander Neumann
c6c1dccc53 Reduce data set for TestRestore 2018-03-11 21:10:37 +01:00
Alexander Neumann
279566bafe Reduce dataset for integration tests 2018-03-11 21:07:47 +01:00
Alexander Neumann
c67a8452f7 Disable polynomial check for chunker for tests 2018-03-11 20:59:40 +01:00
Alexander Neumann
5253ef218c Remove TestParallelSaveWithDuplication 2018-03-11 19:49:48 +01:00
Alexander Neumann
0923976909 Remove TestArchiverDuplication 2018-03-11 19:44:25 +01:00
Lawrence Jones
492baf991f
Update docs and add changelog entry: Google auth
Add documentation around using default Google application credentials,
along with a changelog extra that describes the feature and the
potential impact on existing restic uses (read: none).
2018-03-11 17:12:30 +00:00
Lawrence Jones
0dfdc11ed9
Automatically load Google auth
This change removes the hardcoded Google auth mechanism for the GCS
backend, instead using Google's provided client library to discover and
generate credential material.

Google recommend that client libraries use their common auth mechanism
in order to authorise requests against Google services. Doing so means
you automatically support various types of authentication, from the
standard GOOGLE_APPLICATION_CREDENTIALS environment variable to making
use of Google's metadata API if running within Google Container Engine.
2018-03-11 17:11:25 +00:00
Alexander Neumann
54c6837ec4 Merge pull request #1651 from qbit/obsd_inst
doc: Add install instructions for OpenBSD
2018-03-11 14:49:40 +01:00
Alexander Neumann
e085713b35 Return the first password/key which works
Closes #1663
2018-03-11 14:12:21 +01:00
Alexander Neumann
e77d8c64a7 Merge pull request #1661 from restic/fix-rest-content-length
rest: Really set Content-Length HTTP header
2018-03-10 20:34:30 +01:00
Alexander Neumann
a410fa16a1 Merge pull request #1667 from restic/improve-error-config
Return error message for config decryption failure
2018-03-10 20:34:25 +01:00
Alexander Neumann
b3e1089cf9 Return error message for config decryption failure
See #1663
2018-03-09 21:05:35 +01:00
Alexander Neumann
7f8e269891 Merge pull request #1662 from ebastos/version_password
Skip checking for password file issue #1632
2018-03-09 20:42:32 +01:00
Alexander Neumann
fcc9ce81ba rest: Really set Content-Length HTTP header 2018-03-09 20:21:34 +01:00
Eri Bastos
b9d643358a Skip checking for password file existence if command is 'version' - Issue #1632 2018-03-08 17:55:03 -04:00
Alexander Neumann
ab5ef600a2 Merge pull request #1660 from ncw/rest-fix
backend/rest: check HTTP error response for List
2018-03-08 21:55:58 +01:00
Nick Craig-Wood
04c4033695 backend/rest: check HTTP error response for List
Before this change restic would attempt to JSON decode the error
message resulting in confusing `Decode: invalid character 'B' looking
for beginning of value` messages.  Afterwards it will return `List
failed, server response: 400 Bad Request (400)`
2018-03-08 10:22:43 +00:00
Alexander Neumann
de37b68baa Move all unreleased changelog entries 2018-03-05 21:31:52 +01:00
Alexander Neumann
bdc206d440 Remove unneeded mkdir 2018-03-05 21:17:57 +01:00
Alexander Neumann
efe2e792b3 Correct changelog entries 2018-03-05 21:17:52 +01:00
Alexander Neumann
6f3c23eba7 Merge pull request #1653 from restic/fix-1652
lock: Ignore invalid lock file
2018-03-05 20:49:03 +01:00
Alexander Neumann
4b34bc3210 Reformat changelog entry 2018-03-05 20:40:40 +01:00
Alexander Neumann
6ed9100aa1 Add version number hint for the changelog entry 2018-03-05 20:34:07 +01:00
Alexander Neumann
c63b02d0f1 Move changelog entries to unreleased version 2018-03-05 20:32:10 +01:00
Alexander Neumann
d0205ec889 Add entry to changelog 2018-03-05 20:22:45 +01:00
Alexander Neumann
d8dcbc89d1 lock: Ignore invalid lock file
This commit fixes a bug introduced in
e9ea268847: When an invalid lock is
encountered (e.g. if the file is empty), the code used to ignore that,
but now returns the error.

Now, invalid files are ignored for the normal lock check, and removed
when `restic unlock --remove-all` is run.

Closes #1652
2018-03-05 20:19:57 +01:00
Alexander Neumann
be0a5b7f06 Merge pull request #1649 from jasperla/solaris
Minimal set of patches to get restic working on Solaris
2018-03-05 20:00:17 +01:00
Aaron Bieber
24ce08e122
doc: Add install instructions for OpenBSD 2018-03-05 06:31:57 -07:00
copart
864eaeab7c
Added copr repositories for RHEL/Centos/Fedora 2018-03-04 15:50:30 -05:00
Jasper Lievisse Adriaanse
96311d1a2b Add support for illumos/Solaris
This does come without xattr/fuse support at this point.

NB: not hooking up the integration tests as restic won't compile without
    cgo with Go < 1.10.
2018-03-04 20:11:29 +00:00
Alexander Neumann
da77f4a2e2 Merge pull request #1647 from duzvik/aws-session-token
Change priority of AWS credential providers to accept AWS_SESSION_TOKEN
2018-03-04 20:54:56 +01:00
denis.uzvik
6bb1bcce03 Change priority of AWS credential providers to accept AWS_SESSION_TOKEN 2018-03-04 19:58:27 +02:00
Alexander Neumann
6edf28d1e1 Merge pull request #1639 from restic/improve-backend-save
backend: Improve/Cleanup Save()
2018-03-04 13:35:50 +01:00
Alexander Neumann
929afc63d5 Use int64 for the length in the RewindReader 2018-03-04 10:40:42 +01:00
Alexander Neumann
99f7fd74e3 backend: Improve Save()
As mentioned in issue [#1560](https://github.com/restic/restic/pull/1560#issuecomment-364689346)
this changes the signature for `backend.Save()`. It now takes a
parameter of interface type `RewindReader`, so that the backend
implementations or our `RetryBackend` middleware can reset the reader to
the beginning and then retry an upload operation.

The `RewindReader` interface also provides a `Length()` method, which is
used in the backend to get the size of the data to be saved. This
removes several ugly hacks we had to do to pull the size back out of the
`io.Reader` passed to `Save()` before. In the `s3` and `rest` backend
this is actively used.
2018-03-03 15:49:44 +01:00
Alexander Neumann
58306bfabb Merge pull request #1648 from duzvik/s3-bucketexists
Ignore s3 AccessDenied error, during creation of repository
2018-03-02 22:38:33 +01:00
denis.uzvik
f6890210aa Add entry to changelog 2018-03-02 11:06:06 +02:00
denis.uzvik
5873ab4031 Ignore s3 AccessDenied error, during creation of repository 2018-03-02 10:47:20 +02:00
Alexander Neumann
ab7a3a803d Update build.go from github.com/fd0/build-go 2018-02-28 21:19:28 +01:00
Alexander Neumann
1e868933c5 build.go: Allow specifying the temp dir to use 2018-02-27 21:56:42 +01:00
Alexander Neumann
21f67a0a13 Fix linebreak in GitHub changelog template 2018-02-26 21:41:37 +01:00
Alexander Neumann
272ccec7e1 Add VERSION for 0.8.3 2018-02-26 21:32:16 +01:00
Alexander Neumann
68bf1509bd Update manpages and auto-completion 2018-02-26 21:32:16 +01:00
Alexander Neumann
cfccd67600 Generate CHANGELOG.md for 0.8.3 2018-02-26 21:32:07 +01:00
Alexander Neumann
bc461d32e0 Add release date for 0.8.3 2018-02-26 21:31:50 +01:00
Alexander Neumann
ee4bfdf954 changelog: Fix spelling 2018-02-26 21:27:32 +01:00
Alexander Neumann
3037894f62 Add entry to changelog 2018-02-26 21:20:41 +01:00
Alexander Neumann
89075bdf6d Merge pull request #1643 from restic/fix-1641
Ignore files in the repo with invalid names
2018-02-26 21:18:05 +01:00
Alexander Neumann
c323f73bf9 Ignore files in the repo with invalid names
Closes #1641
2018-02-26 20:53:38 +01:00
Alexander Neumann
aef5e03731 Merge pull request #1638 from restic/fix-list-retry
backend/retry: return worker function error and abort
2018-02-25 21:20:08 +01:00
Alexander Neumann
fc1f74d32d Merge pull request #1640 from restic/fix-1637
mount: Ignore non-existing locks
2018-02-25 14:00:04 +01:00
Alexander Neumann
7d59df1ab8 mount: Ignore non-existing locks
Closes #1637
2018-02-25 13:11:03 +01:00
Alexander Neumann
2866f3f31c Add pull request to changelog entry 2018-02-24 14:53:46 +01:00
Alexander Neumann
dc1154c8ad Merge pull request #1556 from ifedorenko/check-subset
Add --read-data-subset flag to check command
2018-02-24 14:53:20 +01:00
Alexander Neumann
35a816e8ab Add entry to changelog 2018-02-24 13:34:42 +01:00
Alexander Neumann
93210614f4 backend/retry: return worker function error and abort
This is a bug fix: Before, when the worker function fn in List() of the
RetryBackend returned an error, the operation is retried with the next
file. This is not consistent with the documentation, the intention was
that when fn returns an error, this is passed on to the caller and the
List() operation is aborted. Only errors happening on the underlying
backend are retried.

The error leads to restic ignoring exclusive locks that are present in
the repo, so it may happen that a new backup is written which references
data that is going to be removed by a concurrently running `prune`
operation.

The bug was reported by a user here:
https://forum.restic.net/t/restic-backup-returns-0-exit-code-when-already-locked/484
2018-02-24 13:26:13 +01:00
Alexander Neumann
dfd37afee2 Merge pull request #1636 from kurin/pack-header
Refactor the eager-header reads for readability.
2018-02-23 17:36:45 +01:00
Toby Burress
08a5281bd4 Incorporate PR review comments. 2018-02-22 17:37:10 +00:00
Toby Burress
cdb48a8970 Add tests for the eager-header refactor. 2018-02-22 01:14:04 +00:00
Toby Burress
4fd5f0b8a9 Refactor the eager-header reads for readability.
This pulls the header reads into a function that works in terms of the
number of records requested.  This preserves the existing logic of
initially reading 15 records and then falling back if that fails.

In the event of a header with more than 15 records, it will read all
records, including the already-seen final 15 records.
2018-02-22 00:45:40 +00:00
Alexander Neumann
92ad6bf74f Add pull request to changelog 2018-02-21 19:52:16 +01:00
Alexander Neumann
2c7dd3edf4 Merge pull request #1635 from ifedorenko/1633-negative-load-offset
Fixed unexpected 'pack file cannot be listed' error
2018-02-21 19:51:38 +01:00
Igor Fedorenko
19e7803ac6 Fixed unexpected 'pack file cannot be listed' error
Fixes #1633

Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-02-20 21:28:57 -05:00
Alexander Neumann
9f0605766c Add entry to changelog 2018-02-20 22:10:52 +01:00
Alexander Neumann
1a5d7a9965 Merge pull request #1634 from restic/update-blazer
Update github.com/kurin/blazer to 0.3.0
2018-02-20 22:01:30 +01:00
Alexander Neumann
296769355d Update github.com/kurin/blazer to 0.3.0
This commit will reduce the number of HTTP requests per file uploaded
from two to one.
2018-02-20 21:01:21 +01:00
Igor Fedorenko
07d080830e Add --read-data-subset flag to check command
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-02-18 23:31:27 -05:00
Alexander Neumann
c99eabfb37 Merge pull request #1625 from restic/update-blazer
Update B2 client library (github.com/kurin/blazer)
2018-02-18 19:18:52 +01:00
Alexander Neumann
842fe43590 Update github.com/kurin/blazer to 0.2.2 2018-02-18 14:53:23 +01:00
Alexander Neumann
be02008025 Merge pull request #1611 from qbit/master
On OpenBSD only root can set sticky bit
2018-02-18 12:57:54 +01:00
Alexander Neumann
29da86b473 Merge pull request #1623 from restic/backend-relax-restrictions
backend: Relax requirement for new files
2018-02-18 12:56:52 +01:00
Alexander Neumann
bad7215696 Add entry to CHANGELOG 2018-02-18 12:04:44 +01:00
Alexander Neumann
881ff5e554 Move changelog file, improve text 2018-02-18 11:51:45 +01:00
Alexander Neumann
86b7fd0335 Merge pull request #1624 from rawtaz/patch-1
Attempt to make issue template a bit clearer
2018-02-18 11:50:11 +01:00
rawtaz
70209d7d1d
End both list items with a dot.
Changed my mind after checking other lists in the project's files, ending with a dot seems to be preferred here, and I like that better too.
2018-02-17 23:47:40 +01:00
rawtaz
f07552161c
Remove potentially excessive dot
Some people like list items to not end with a comma or dot, some like it when they do. To keep things like and coherent I removed them in this case.
2018-02-17 23:40:30 +01:00
rawtaz
856f3a9135
Add forum URL back 2018-02-17 23:36:47 +01:00
Alexander Neumann
49e9bcadb7 Merge pull request #1560 from ifedorenko/1559-load-error-handling
Retry all repository file download errors
2018-02-17 23:25:28 +01:00
rawtaz
1b8823ef2e
Attempt to make issue template a bit clearer 2018-02-17 23:19:58 +01:00
Alexander Neumann
b5062959c8 backend: Relax requirement for new files
Before, all backend implementations were required to return an error if
the file that is to be written already exists in the backend. For most
backends, that means making a request (e.g. via HTTP) and returning an
error when the file already exists.

This is not accurate, the file could have been created between the HTTP
request testing for it, and when writing starts. In addition, apart from
the `config` file in the repo, all other file names have pseudo-random
names with a very very low probability of a collision. And even if a
file name is written again, the way the restic repo is structured this
just means that the same content is placed there again. Which is not a
problem, just not very efficient.

So, this commit relaxes the requirement to return an error when the file
in the backend already exists, which allows reducing the number of API
requests and thereby the latency for remote backends.
2018-02-17 22:39:18 +01:00
Alexander Neumann
2fb4d44a4d Add VERSION for 0.8.2 2018-02-17 20:27:28 +01:00
Alexander Neumann
3c08f956a3 Update manpages and auto-completion 2018-02-17 20:27:28 +01:00
Alexander Neumann
1463a7aaaa Generate CHANGELOG.md for 0.8.2 2018-02-17 20:27:17 +01:00
Alexander Neumann
76b8e080c3 changelog: Add release date for 0.8.2 2018-02-17 20:26:43 +01:00
Alexander Neumann
a9f9243afc scripts/release: Fix usage for restic/builder
The restic/builder Docker container now has a slightly different usage.
2018-02-17 19:10:32 +01:00
Alexander Neumann
34bf9de339 Fix Go versions for Travis 2018-02-17 18:48:15 +01:00
Alexander Neumann
597f6be4e4 Disable CI tests on Go 1.8 on OS X 2018-02-17 18:39:16 +01:00
Alexander Neumann
aeeec0ff94 Update Go version for CI tests 2018-02-17 18:37:37 +01:00
Igor Fedorenko
ab040d8811 Introduced repository.DownloadAndHash helper
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-02-16 21:13:11 -05:00
Igor Fedorenko
d58ae43317 Reworked Backend.Load API to retry errors during ongoing download
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-02-16 21:12:14 -05:00
Alexander Neumann
b723094739 scripts/release: Check changelog subdir with date 2018-02-16 23:13:36 +01:00
Alexander Neumann
84dcc3c396 Update changelog/ for new version of calens
We've reworked calens[1] a bit, the changelog/releases file is not
needed any more. Insteady, the release date is stored in the dir name
which contains the changelog entries.

[1] https://github.com/restic/calens
2018-02-16 23:05:58 +01:00
Alexander Neumann
21747bc219 Remove punctuation from changelog entry title 2018-02-16 22:49:24 +01:00
Alexander Neumann
28b00891d2 scripts/release: Check Go version in container 2018-02-16 22:35:53 +01:00
Alexander Neumann
b4e00c324f scripts/release: Check release date first 2018-02-16 21:52:34 +01:00
Alexander Neumann
4eb9df63cf Merge pull request #1616 from rawtaz/fix-backup-doc
Restructure and complement exclude options doc
2018-02-14 22:11:14 +01:00
Alexander Neumann
fde674d214 Merge pull request #1615 from rawtaz/fix-forget-doc
Fix #1519 and formatting in `forget` doc
2018-02-14 22:11:10 +01:00
rawtaz
fbb08c14de
Note that multiple --exclude-file may be used. 2018-02-14 22:05:52 +01:00
Leo R. Lundgren
8b95b48bae Fix grammar and formatting. 2018-02-14 02:59:47 +01:00
Leo R. Lundgren
71a4ccea71 Document refusal to process empty forget policy. 2018-02-14 02:59:41 +01:00
Leo R. Lundgren
5a9c7c51e6 Fix --keep-* options list formatting. 2018-02-14 02:59:33 +01:00
rawtaz
5abd133e26
Restructure and complement exclude options doc 2018-02-14 02:03:04 +01:00
Alexander Neumann
2debb5c0e0 Merge pull request #1613 from restic/strip-space-files-from
Strip spaces from items read via --files-from
2018-02-11 21:35:23 +01:00
Alexander Neumann
54c0794cf3 Merge pull request #1603 from jfparis/master
Add support for GOARM parameter when cross compiling
2018-02-11 21:03:29 +01:00
Alexander Neumann
cff3f3dc0d Add entry to CHANGELOG 2018-02-11 21:00:38 +01:00
Alexander Neumann
c5575c7ed9 Strip spaces from items read via --files-from
In #1590, it was mentioned that while lines read from exclude files via
`--exclude-file` have leading and trailing spaces stripped, this is not
the case for lines read via `--files-from`. This commit fixes that,
spaces are always stripped.
2018-02-11 20:56:11 +01:00
Aaron Bieber
99d88ad297
Disable the 'testSticky' test on OpenBSD. Only root can set sticky. 2018-02-11 07:46:31 -07:00
Alexander Neumann
7f0d964a05 Update build.go 2018-02-11 11:34:03 +01:00
Alexander Neumann
514f1b8917 Relax timeout backend test 2018-02-10 12:53:38 +01:00
Alexander Neumann
a7a99be460 Correct forum host name in issue template
Closes #1609
2018-02-10 11:13:22 +01:00
Alexander Neumann
e17f65b23a Update logo 2018-02-05 20:48:48 +01:00
Jean-François Paris
e86fb5eedd Add support for GOARM parameter when cross compiling
Go can cross compile to older ARM architectures (e.g. v5). In order to
do so one needs to use the GOARM parameter as documented in
https://github.com/golang/go/wiki/GoArm
2018-02-04 14:15:12 +00:00
Alexander Neumann
c8cb9a9509 Merge pull request #1579 from ifedorenko/RetryBackend-List
Implement RetryBackend.List()
2018-01-30 21:39:21 +01:00
Igor Fedorenko
aa333f4d49 Implement RetryBackend.List()
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-01-29 22:14:12 -05:00
Alexander Neumann
c40002246d Merge pull request #1598 from Phaeilo/s3-tutorial
Update AWS S3 tutorial, fixing #1593
2018-01-29 21:52:32 +01:00
Philip Huppert
77a6054d64 Update AWS S3 tutorial, fixing #1593 2018-01-29 20:47:27 +01:00
Alexander Neumann
94ec55ff2c Merge pull request #1595 from restic/remove-read-bandwidth
Remove archiver progress "data processed" bandwith
2018-01-28 10:15:53 +01:00
Alexander Neumann
2369da158f Merge pull request #1592 from ncw/helpful-tests
Make backend tests more helpful
2018-01-28 10:09:35 +01:00
Alexander Neumann
5379950c64 Add entry to changelog 2018-01-28 09:36:55 +01:00
Alexander Neumann
cbfa516575 Remove archiver progress "data processed" bandwith
This commit removes the bandwidth displayed during backup process. It is
misleading and seldomly correct, because it's neither the "read
bandwidth" (only for the very first backup) nor the "upload bandwidth".
Many users are confused about (and rightly so), c.f. #1581, #1033, #1591

We'll eventually replace this display with something more relevant when
 #1494 is done.
2018-01-28 09:30:26 +01:00
Nick Craig-Wood
fb62da1748 Make backend tests more helpful
* In TestList check that backend is empty first
  * Improve error message in TestBackend
2018-01-27 21:36:35 +00:00
Alexander Neumann
b1b1f6e04f Merge pull request #1594 from restic/gcs-use-http-transport
GS: Use generic http transport
2018-01-27 22:16:49 +01:00
Alexander Neumann
05f826a4fc Update Gopkg.lock 2018-01-27 21:48:41 +01:00
Alexander Neumann
649afb1198 Add entry to changelog 2018-01-27 20:16:51 +01:00
Alexander Neumann
5dc8d3588d GS: Use generic http transport
During the development of #1524 I discovered that the Google Cloud
Storage backend did not yet use the HTTP transport, so things such as
bandwidth limiting did not work. This commit does the necessary magic to
make the GS library use our HTTP transport.
2018-01-27 20:12:34 +01:00
Alexander Neumann
9d2aa0a71c Merge pull request #1524 from brycied00d/https_client_certificate
Support for TLS client certificate authentication
2018-01-27 20:12:09 +01:00
Alexander Neumann
c34db983d8 Read TLS client cert and key from the same file 2018-01-27 14:02:01 +01:00
Bryce Chidester
e805b968b1 Support for TLS client certificate authentication
This adds --tls-client-cert and --tls-client-key parameters and enables use
of that certificate/key pair when connecting to https servers.
2018-01-27 13:18:22 +01:00
Alexander Neumann
e706f1a8d1 Merge pull request #1589 from restic/fix-intermediate-index-upload
archiver: Fix intermediate index upload
2018-01-27 12:22:11 +01:00
Alexander Neumann
4219bfbcc9 Add entry to changelog 2018-01-26 22:05:53 +01:00
Alexander Neumann
f99c95c766 archiver: Fix intermediate index upload
A user discovered[1] that when the backup finishes during the upload of
an intermediate index, the upload is cancelled and the index never fully
saved, but the snapshot is saved and the backup finalizes without an
error. This lead to a situation where a snapshot references data that is
contained in the repo, but not referenced in any index, leading to
strange error messages.

This commit uses a dedicated context to signal the intermediate index
uploading routine to terminate after the last index has been uploaded.
This way, an upload running when the backup finishes is completed before
the routine terminates and the snapshot is saved.

[1] https://forum.restic.net/t/error-loading-tree-check-prune-and-forget-gives-error-b2-backend/406
2018-01-26 22:01:07 +01:00
Alexander Neumann
cccb2fc7e7 Merge pull request #1583 from restic/close-open-backend-files
Close backend files in case of errors
2018-01-26 21:57:28 +01:00
Alexander Neumann
ff3de66ddf Merge pull request #1582 from restic/optimize-debug-log
Optimize debug logs
2018-01-26 21:57:18 +01:00
Alexander Neumann
fb0d01ef1f Add pull request URL to changelog 2018-01-26 21:56:59 +01:00
Alexander Neumann
e11a183578 Merge pull request #1588 from restic/fix-sftp-without-tty
sftp: Allow running ssh without a tty
2018-01-26 21:56:41 +01:00
Alexander Neumann
ee6732d2c6 Add entry to changelog 2018-01-26 21:56:29 +01:00
Alexander Neumann
4b2f2b542d Merge pull request #1584 from restic/limit-index-file-size
Limit index file size
2018-01-26 21:53:12 +01:00
Alexander Neumann
6eb2d76435 index: Lower parallel load to 4 2018-01-26 21:10:38 +01:00
Alexander Neumann
f63c7048c7 index: Limit index files to 3000 pack files 2018-01-26 21:07:16 +01:00
Alexander Neumann
ca3c566a99 index: Check structure after saving a new index 2018-01-26 21:07:16 +01:00
Alexander Neumann
7719cf88d9 b2: Check timeout 2018-01-26 21:07:05 +01:00
Alexander Neumann
00e905ebe6 sftp: Allow running ssh without a tty 2018-01-26 19:21:14 +01:00
Alexander Neumann
164acceb55 Appveyor: Update Go version and download URL 2018-01-25 21:09:23 +01:00
Alexander Neumann
909d9273cc Close backend files in case of errors 2018-01-25 21:05:57 +01:00
Alexander Neumann
d62a664bb7 Merge pull request #1580 from ifedorenko/tests-errors-cause
Use errors.Cause in backend TestListCancel
2018-01-25 20:55:43 +01:00
Alexander Neumann
663c57ab4d debug: Remove manual Str() call Log() 2018-01-25 20:49:41 +01:00
Alexander Neumann
ed99f53786 debug: Auto-shorten IDs passed as parameters 2018-01-25 20:49:26 +01:00
Igor Fedorenko
abc4027083 Use errors.Cause in backend TestListCancel
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-01-25 08:53:50 -05:00
Alexander Neumann
d6212ee2d9 Fix entry to changelog 2018-01-24 20:54:51 +01:00
Alexander Neumann
9c55e8d69c Merge pull request #1549 from MJDSys/more_index_lookup_avoids
More optimizations to avoid calling Index.Lookup()
2018-01-24 20:53:30 +01:00
Alexander Neumann
1c9fca6d14 Combine changelog files for #1574 and #1575 2018-01-24 20:07:22 +01:00
Alexander Neumann
be205563be Merge pull request #1575 from ifedorenko/1567_optimize-repository-ListPack
Optimize Repository.ListPack()
2018-01-24 20:01:15 +01:00
Alexander Neumann
fc6c341b26 Merge pull request #1574 from ifedorenko/1567_optimize-pack-readHeader
Optimize pack readHeader() implementation
2018-01-24 19:54:14 +01:00
Alexander Neumann
f86c141b83 Merge pull request #1577 from EdwardBetts/patch-1
Correct a spelling mistake.
2018-01-24 19:44:49 +01:00
Alexander Neumann
a6d4888d48 Merge pull request #1573 from restic/update-dps
Update dependencies
2018-01-24 19:43:32 +01:00
Alexander Neumann
44550a88a0 Merge pull request #1571 from restic/rework-backend-list
Rework backend list
2018-01-24 19:43:07 +01:00
Edward Betts
b5c23695c9
Correct a spelling mistake. 2018-01-24 12:09:41 +00:00
Igor Fedorenko
0084e42cb6 Optimize Repository.ListPack()
Use pack file size returned by Backend.List() to avoid extra per-pack
Backend.Stat() requests

Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-01-23 22:39:51 -05:00
Matthew Dawson
fe33c05a20
debug/log: Add benchmarks for calling the logging function
Add some benchmarks for calling Log, both with a static string
along with calling the ID.Str and ID.String functions.
2018-01-23 22:30:00 -05:00
Matthew Dawson
3789e55e20
repostiory/index: Remove logging from Lookup function.
The logging in these functions double the time they take to execute.
However, it is only really useful on failures, which are better
reported by the calling functions.

benchmark                                            old ns/op     new ns/op     delta
BenchmarkMasterIndexLookupSingleIndex-6              897           395           -55.96%
BenchmarkMasterIndexLookupMultipleIndex-6            2001          1090          -45.53%
BenchmarkMasterIndexLookupSingleIndexUnknown-6       492           215           -56.30%
BenchmarkMasterIndexLookupMultipleIndexUnknown-6     1649          912           -44.69%

benchmark                                            old allocs     new allocs     delta
BenchmarkMasterIndexLookupSingleIndex-6              9              1              -88.89%
BenchmarkMasterIndexLookupMultipleIndex-6            19             1              -94.74%
BenchmarkMasterIndexLookupSingleIndexUnknown-6       6              0              -100.00%
BenchmarkMasterIndexLookupMultipleIndexUnknown-6     16             0              -100.00%

benchmark                                            old bytes     new bytes     delta
BenchmarkMasterIndexLookupSingleIndex-6              160           96            -40.00%
BenchmarkMasterIndexLookupMultipleIndex-6            240           96            -60.00%
BenchmarkMasterIndexLookupSingleIndexUnknown-6       48            0             -100.00%
BenchmarkMasterIndexLookupMultipleIndexUnknown-6     128           0             -100.00%
2018-01-23 22:28:38 -05:00
Matthew Dawson
4cec7e236a
Add Changelog 2018-01-23 22:28:32 -05:00
Matthew Dawson
3a16148447
archiver/archiver: Use Index.Has() instead of Index.Lookup() in isKnownBlob
Index.Has() is a faster then Index.Lookup() for checking if a blob exists
in the index.  As the returned data is never used, this avoids a ton
of allocations.
2018-01-23 22:26:10 -05:00
Matthew Dawson
df2c03a6a4
repository/master_index: Optimize Index.Lookup()
When looking up a blob in the master index, with several
indexes present in the master index, a significant amount of time
is spent generating errors for each failed lookup.  However, these
errors are often used to check if a blob is present, but the contents
are not inspected making the overhead of the error not useful.

Instead, change Index.Lookup (and Index.LookupSize) to instead return
a boolean denoting if the blob was found instead of an error.  Also change
all the calls to these functions to handle the new function signature.

benchmark                                            old ns/op     new ns/op     delta
BenchmarkMasterIndexLookupSingleIndex-6              820           897           +9.39%
BenchmarkMasterIndexLookupMultipleIndex-6            12821         2001          -84.39%
BenchmarkMasterIndexLookupSingleIndexUnknown-6       5378          492           -90.85%
BenchmarkMasterIndexLookupMultipleIndexUnknown-6     17026         1649          -90.31%

benchmark                                            old allocs     new allocs     delta
BenchmarkMasterIndexLookupSingleIndex-6              9              9              +0.00%
BenchmarkMasterIndexLookupMultipleIndex-6            59             19             -67.80%
BenchmarkMasterIndexLookupSingleIndexUnknown-6       22             6              -72.73%
BenchmarkMasterIndexLookupMultipleIndexUnknown-6     72             16             -77.78%

benchmark                                            old bytes     new bytes     delta
BenchmarkMasterIndexLookupSingleIndex-6              160           160           +0.00%
BenchmarkMasterIndexLookupMultipleIndex-6            3200          240           -92.50%
BenchmarkMasterIndexLookupSingleIndexUnknown-6       1232          48            -96.10%
BenchmarkMasterIndexLookupMultipleIndexUnknown-6     4272          128           -97.00%
2018-01-23 22:25:56 -05:00
Matthew Dawson
ebce4b2581
repository/index: Speed up benchmarks and tests
When setting up the index used for benchmarking, use math/rand instead of
crypto/rand since the generated ids don't need to be evenly distributed,
and not be secure against guessing.  As such, use a different random id
function (only available during tests) that uses math/rand instead.
2018-01-23 22:21:05 -05:00
Igor Fedorenko
953f3d55ee Optimize pack readHeader() implementation
Load pack header length and 15 header entries with single backend
request. This eliminates separate header Load() request for most pack
files and significantly improves index.New() performance.

Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-01-23 22:02:25 -05:00
Alexander Neumann
7e6bfdae79 backend/rest: Implement REST API v2 2018-01-23 23:15:26 +01:00
Alexander Neumann
0f4cbea27d Document the REST v2 protocol 2018-01-23 23:15:26 +01:00
Alexander Neumann
59782e347c repository: Fix repack test
This reduces the chance of duplicate blobs, otherwise the tests fail
(make the contents of a blob depend on a pseudo-random number instead of
the size, sizes may be duplicate).
2018-01-23 23:14:05 +01:00
Alexander Neumann
e835abeceb backend/test: Reliably trigger timeout error 2018-01-23 23:14:05 +01:00
Alexander Neumann
8e812b7ac0 errors: Make Cause() unwrap *url.Error 2018-01-23 23:14:05 +01:00
Alexander Neumann
685ce719ad Add comments for List() 2018-01-23 23:14:05 +01:00
Alexander Neumann
2b39f9f4b2 Update dependencies
Among others, this updates minio-go, so that the new "eu-west-3" zone
for AWS is supported.
2018-01-23 19:40:42 +01:00
Alexander Neumann
b63de7c798 script/release: Tag versioned docker image 2018-01-21 21:41:14 +01:00
Alexander Neumann
b0c6e53241 Fix calls to repo/backend.List() everywhere 2018-01-21 21:15:09 +01:00
Alexander Neumann
e9ea268847 Change List() implementation for all backends 2018-01-21 21:15:09 +01:00
Alexander Neumann
dd91b13ff3 Vendor golang.org/x/sync 2018-01-21 18:35:37 +01:00
Alexander Neumann
c4e9d5d11e backend: Add tests for new List() function 2018-01-21 18:35:37 +01:00
Alexander Neumann
52230b8f07 backend: Rework List()
For a discussion see #1567
2018-01-21 18:35:37 +01:00
Alexander Neumann
a3d43a92b3 Merge pull request #1569 from restic/ext-rest-backend-test
REST: refactor backend tests
2018-01-21 18:02:09 +01:00
Alexander Neumann
2130897ce0 rest: Add test for external server 2018-01-20 10:25:47 +01:00
Alexander Neumann
67da240068 rest: Refactor backend tests 2018-01-20 10:25:37 +01:00
Alexander Neumann
1046eabf95 rest: Remove unneeded tempdir 2018-01-20 10:13:04 +01:00
Alexander Neumann
663e8284b2 init: Correct text (backend vs. repository)
Closes #1565
2018-01-20 09:51:49 +01:00
Alexander Neumann
5a43ca1ccb Add entry to CHANGELOG 2018-01-20 09:50:31 +01:00
Alexander Neumann
200415e0a1 Merge pull request #1564 from restic/sftp-password-prompt
sftp: Prompt for password, don't terminate on SIGINT
2018-01-20 09:48:17 +01:00
Alexander Neumann
ea50687e8b Merge pull request #1568 from martinseener/patch-1
Added hint where to place restic.exe on windows
2018-01-20 09:47:50 +01:00
Alexander Neumann
7335a2492f Add PR to changelog 2018-01-19 13:05:55 +01:00
Alexander Neumann
bcb6881ffb Merge pull request #1548 from ifedorenko/checker-backend-Test
checker: Optimize checker.Packs()
2018-01-19 13:05:35 +01:00
Alexander Neumann
37df829b3d CONTRIBUTING: clarify wording 2018-01-19 13:00:49 +01:00
Martin
39f1ba1897
Added hint where to place restic.exe on windows
I've added a hint for pre-compiled binaries on windows where to place them for easier use in the shell or in scripts without the hassle for absolute paths.
2018-01-19 12:24:09 +01:00
Igor Fedorenko
231076fa4a checker: Optimize checker.Packs()
Use result of single repository.List() to find both missing and
orphaned data packs. For 500GB repository this eliminates ~100K
repository.Test() calls and improves check time by >30M in my
environment (~45min before this change and ~7min after).

Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-01-18 20:50:39 -05:00
Alexander Neumann
0bdb131521 Remove SuspendSignalHandler 2018-01-17 23:14:47 +01:00
Alexander Neumann
c31a5e7e5c Add argument to Skipf() 2018-01-17 23:14:37 +01:00
Alexander Neumann
05958caf6e sftp: Prompt for password, don't terminate on SIGINT
This is a follow-up on fb9729fdb9, which
runs the `ssh` in its own process group and selects that process group
as the foreground group. After the sftp connection is established,
restic switches back to the previous foreground process group.

This allows `ssh` to prompt for the password, but it won't receive
the interrupt signal (SIGINT, ^C) later on, because it is not in the
foreground process group any more, allowing a clean tear down.
2018-01-17 23:02:47 +01:00
Alexander Neumann
87e31799f2 Merge pull request #1483 from dstosberg/master
document how to create a full backup without running restic as root
2018-01-16 17:21:55 +01:00
Alexander Neumann
35528506a6 Merge pull request #1554 from restic/fix-988
fuse: Correct behavior for reading after EOF, add snapshot template string
2018-01-16 17:17:55 +01:00
Alexander Neumann
2544978aaf Add entry to changelog 2018-01-15 14:47:17 +01:00
Alexander Neumann
97565d68a2 fuse/mount: Add option for snapshot template 2018-01-15 14:47:17 +01:00
Dennis Stosberg
60a7a9d019 manual: document how to create a full backup without running restic as root 2018-01-08 19:59:15 +01:00
6194 changed files with 216924 additions and 3479202 deletions

View file

@ -1,63 +0,0 @@
<!--
NOTE: Not filling out the issue template needs a good reason, otherwise it may
take a lot longer to find the problem! Please take the time to help us
debugging the problem by collecting information, even if it seems irrelevant to
you. Thanks!
If you have a question, the forum at https://discourse.restic.net is a better place.
Please do not create issues for usage or documentation questions! We're using
the GitHub issue tracker mainly for tracking bugs and feature requests.
-->
## Output of `restic version`
## How did you run restic exactly?
<!--
This section should include at least:
* The complete command line and any environment variables you used to
configure restic's backend access. Make sure to replace sensitive values!
* The output of the commands, what restic prints gives may give us much
information to diagnose the problem!
-->
## What backend/server/service did you use to store the repository?
## Expected behavior
<!--
Describe what you'd like restic to do differently.
-->
## Actual behavior
<!--
In this section, please try to concentrate on observations, so only describe
what you observed directly.
-->
## Steps to reproduce the behavior
<!--
The more time you spend describing an easy way to reproduce the behavior (if
this is possible), the easier it is for the project developers to fix it!
-->
## Do you have any idea what may have caused this?
## Do you have an idea how to solve the issue?
## Did restic help you or made you happy in any way?
<!--
Answering this question is not required, but if you have anything positive to share, please do so here!
Sometimes we get tired of reading bug reports all day and a little positive end note does wonders.
Idea by Joey Hess, https://joeyh.name/blog/entry/two_holiday_stories/
-->

93
.github/ISSUE_TEMPLATE/Bug.md vendored Normal file
View file

@ -0,0 +1,93 @@
---
name: Bug report
about: Report a problem with restic to help us resolve it and improve
---
<!--
Welcome! - We kindly ask that you:
1. Fill out the issue template below - not doing so needs a good reason.
2. Use the forum if you have a question rather than a bug or feature request.
The forum is at: https://forum.restic.net
NOTE: Not filling out the issue template needs a good reason, as otherwise it
may take a lot longer to find the problem, not to mention it can take up a lot
more time which can otherwise be spent on development. Please also take the
time to help us debug the issue by collecting relevant information, even if
it doesn't seem to be relevant to you. Thanks!
The forum is a better place for questions about restic or general suggestions
and topics, e.g. usage or documentation questions! This issue tracker is mainly
for tracking bugs and feature requests directly relating to the development of
the software itself, rather than the project.
Thanks for understanding, and for contributing to the project!
-->
Output of `restic version`
--------------------------
How did you run restic exactly?
-------------------------------
<!--
This section should include at least:
* The complete command line and any environment variables you used to
configure restic's backend access. Make sure to replace sensitive values!
* The output of the commands, what restic prints gives may give us much
information to diagnose the problem!
-->
What backend/server/service did you use to store the repository?
----------------------------------------------------------------
Expected behavior
-----------------
<!--
Describe what you'd like restic to do differently.
-->
Actual behavior
---------------
<!--
In this section, please try to concentrate on observations, so only describe
what you observed directly.
-->
Steps to reproduce the behavior
-------------------------------
<!--
The more time you spend describing an easy way to reproduce the behavior (if
this is possible), the easier it is for the project developers to fix it!
-->
Do you have any idea what may have caused this?
-----------------------------------------------
Do you have an idea how to solve the issue?
-------------------------------------------
Did restic help you or made you happy in any way?
-------------------------------------------------
<!--
Answering this question is not required, but if you have anything positive to share, please do so here!
Sometimes we get tired of reading bug reports all day and a little positive end note does wonders.
Idea by Joey Hess, https://joeyh.name/blog/entry/two_holiday_stories/
-->

57
.github/ISSUE_TEMPLATE/Feature.md vendored Normal file
View file

@ -0,0 +1,57 @@
---
name: Feature request
about: Suggest a new feature or enhancement for restic
---
<!--
Welcome! - We kindly ask that you:
1. Fill out the issue template below - not doing so needs a good reason.
2. Use the forum if you have a question rather than a bug or feature request.
The forum is at: https://forum.restic.net
The forum is a better place for questions about restic or general suggestions
and topics, e.g. usage or documentation questions! This issue tracker is mainly
for tracking bugs and feature requests directly relating to the development of
the software itself, rather than the project.
Thanks for understanding, and for contributing to the project!
-->
Output of `restic version`
--------------------------
<!--
Please add the version of restic you're currently using here, this helps us
later to see what has changed in restic when we revisit this issue after some
time.
-->
What should restic do differently? Which functionality do you think we should add?
----------------------------------------------------------------------------------
<!--
Please describe the feature you'd like us to add here.
-->
What are you trying to do?
--------------------------
<!--
This section should contain a brief description what you're trying to do, which
would be possible after implementing the new feature.
-->
Did restic help you or made you happy in any way?
-------------------------------------------------
<!--
Answering this question is not required, but if you have anything positive to share, please do so here!
Sometimes we get tired of reading bug reports all day and a little positive end note does wonders.
Idea by Joey Hess, https://joeyh.name/blog/entry/two_holiday_stories/
-->

View file

@ -1,3 +1,5 @@
<!--
Thank you very much for contributing code or documentation to restic! Please
fill out the following questions to make it easier for us to review your
@ -8,24 +10,27 @@ your time and add more commits. If you're done and ready for review, please
check the last box.
-->
### What is the purpose of this change? What does it change?
What is the purpose of this change? What does it change?
--------------------------------------------------------
<!--
Describe the changes here, as detailed as needed.
-->
### Was the change discussed in an issue or in the forum before?
Was the change discussed in an issue or in the forum before?
------------------------------------------------------------
<!--
Link issues and relevant forum posts here.
-->
### Checklist
Checklist
---------
- [ ] I have read the [Contribution Guidelines](https://github.com/restic/restic/blob/master/CONTRIBUTING.md#providing-patches)
- [ ] I have added tests for all changes in this PR
- [ ] I have added documentation for the changes (in the manual)
- [ ] There's a new file in a subdir of `changelog/x.y.z` that describe the changes for our users (template [here](https://github.com/restic/restic/blob/master/changelog/changelog-entry.tmpl))
- [ ] There's a new file in `changelog/unreleased/` that describes the changes for our users (template [here](https://github.com/restic/restic/blob/master/changelog/TEMPLATE))
- [ ] I have run `gofmt` on the code in all commits
- [ ] All commit messages are formatted in the same style as [the other commits in the repo](https://github.com/restic/restic/blob/master/CONTRIBUTING.md#git-commits)
- [ ] I'm done, this Pull Request is ready for review

View file

@ -1,30 +1,20 @@
language: go
sudo: false
go:
- 1.8.x
- 1.9.x
os:
- linux
- osx
env:
matrix:
RESTIC_TEST_FUSE=0
matrix:
exclude:
- os: osx
go: 1.8.x
- os: linux
go: 1.9.x
include:
- os: linux
go: 1.9.x
go: "1.9.x"
env: RESTIC_TEST_FUSE=0 RESTIC_TEST_CLOUD_BACKENDS=0 RESTIC_BUILD_SOLARIS=0
# only run fuse and cloud backends tests on Travis for the latest Go on Linux
- os: linux
go: "1.10.x"
sudo: true
env:
RESTIC_TEST_FUSE=1
- os: osx
go: "1.10.x"
env: RESTIC_TEST_FUSE=0 RESTIC_TEST_CLOUD_BACKENDS=0
branches:
only:

View file

@ -1,3 +1,583 @@
Changelog for restic 0.9.1 (2018-06-10)
=======================================
The following sections list the changes in restic 0.9.1 relevant to
restic users. The changes are ordered by importance.
Summary
-------
* Fix #1801: Add limiting bandwidth to the rclone backend
* Fix #1822: Allow uploading large files to MS Azure
* Fix #1825: Correct `find` to not skip snapshots
* Fix #1833: Fix caching files on error
* Fix #1834: Resolve deadlock
Details
-------
* Bugfix #1801: Add limiting bandwidth to the rclone backend
The rclone backend did not respect `--limit-upload` or `--limit-download`. Oftentimes it's
not necessary to use this, as the limiting in rclone itself should be used because it gives much
better results, but in case a remote instance of rclone is used (e.g. called via ssh), it is still
relevant to limit the bandwidth from restic to rclone.
https://github.com/restic/restic/issues/1801
* Bugfix #1822: Allow uploading large files to MS Azure
Sometimes, restic creates files to be uploaded to the repository which are quite large, e.g.
when saving directories with many entries or very large files. The MS Azure API does not allow
uploading files larger that 256MiB directly, rather restic needs to upload them in blocks of
100MiB. This is now implemented.
https://github.com/restic/restic/issues/1822
* Bugfix #1825: Correct `find` to not skip snapshots
Under certain circumstances, the `find` command was found to skip snapshots containing
directories with files to look for when the directories haven't been modified at all, and were
already printed as part of a different snapshot. This is now corrected.
In addition, we've switched to our own matching/pattern implementation, so now things like
`restic find "/home/user/foo/**/main.go"` are possible.
https://github.com/restic/restic/issues/1825
https://github.com/restic/restic/issues/1823
* Bugfix #1833: Fix caching files on error
During `check` it may happen that different threads access the same file in the backend, which
is then downloaded into the cache only once. When that fails, only the thread which is
responsible for downloading the file signals the correct error. The other threads just assume
that the file has been downloaded successfully and then get an error when they try to access the
cached file.
https://github.com/restic/restic/issues/1833
* Bugfix #1834: Resolve deadlock
When the "scanning" process restic runs to find out how much data there is does not finish before
the backup itself is done, restic stops doing anything. This is resolved now.
https://github.com/restic/restic/issues/1834
https://github.com/restic/restic/pull/1835
Changelog for restic 0.9.0 (2018-05-21)
=======================================
The following sections list the changes in restic 0.9.0 relevant to
restic users. The changes are ordered by importance.
Summary
-------
* Fix #1608: Respect time stamp for new backup when reading from stdin
* Fix #1652: Ignore/remove invalid lock files
* Fix #1730: Ignore sockets for restore
* Fix #1684: Fix backend tests for rest-server
* Fix #1745: Correctly parse the argument to --tls-client-cert
* Enh #1433: Support UTF-16 encoding and process Byte Order Mark
* Enh #1561: Allow using rclone to access other services
* Enh #1665: Improve cache handling for `restic check`
* Enh #1721: Add `cache` command to list cache dirs
* Enh #1758: Allow saving OneDrive folders in Windows
* Enh #549: Rework archiver code
* Enh #1552: Use Google Application Default credentials
* Enh #1477: Accept AWS_SESSION_TOKEN for the s3 backend
* Enh #1648: Ignore AWS permission denied error when creating a repository
* Enh #1649: Add illumos/Solaris support
* Enh #1709: Improve messages `restic check` prints
* Enh #827: Add --new-password-file flag for non-interactive password changes
* Enh #1735: Allow keeping a time range of snaphots
* Enh #1782: Use default AWS credentials chain for S3 backend
Details
-------
* Bugfix #1608: Respect time stamp for new backup when reading from stdin
When reading backups from stdin (via `restic backup --stdin`), restic now uses the time stamp
for the new backup passed in `--time`.
https://github.com/restic/restic/issues/1608
https://github.com/restic/restic/pull/1703
* Bugfix #1652: Ignore/remove invalid lock files
This corrects a bug introduced recently: When an invalid lock file in the repo is encountered
(e.g. if the file is empty), the code used to ignore that, but now returns the error. Now, invalid
files are ignored for the normal lock check, and removed when `restic unlock --remove-all` is
run.
https://github.com/restic/restic/issues/1652
https://github.com/restic/restic/pull/1653
* Bugfix #1730: Ignore sockets for restore
We've received a report and correct the behavior in which the restore code aborted restoring a
directory when a socket was encountered. Unix domain socket files cannot be restored (they are
created on the fly once a process starts listening). The error handling was corrected, and in
addition we're now ignoring sockets during restore.
https://github.com/restic/restic/issues/1730
https://github.com/restic/restic/pull/1731
* Bugfix #1684: Fix backend tests for rest-server
The REST server for restic now requires an explicit parameter (`--no-auth`) if no
authentication should be allowed. This is fixed in the tests.
https://github.com/restic/restic/pull/1684
* Bugfix #1745: Correctly parse the argument to --tls-client-cert
Previously, the --tls-client-cert method attempt to read ARGV[1] (hardcoded) instead of the
argument that was passed to it. This has been corrected.
https://github.com/restic/restic/issues/1745
https://github.com/restic/restic/pull/1746
* Enhancement #1433: Support UTF-16 encoding and process Byte Order Mark
On Windows, text editors commonly leave a Byte Order Mark at the beginning of the file to define
which encoding is used (oftentimes UTF-16). We've added code to support processing the BOMs in
text files, like the exclude files, the password file and the file passed via `--files-from`.
This does not apply to any file being saved in a backup, those are not touched and archived as they
are.
https://github.com/restic/restic/issues/1433
https://github.com/restic/restic/issues/1738
https://github.com/restic/restic/pull/1748
* Enhancement #1561: Allow using rclone to access other services
We've added the ability to use rclone to store backup data on all backends that it supports. This
was done in collaboration with Nick, the author of rclone. You can now use it to first configure a
service, then restic manages the rest (starting and stopping rclone). For details, please see
the manual.
https://github.com/restic/restic/issues/1561
https://github.com/restic/restic/pull/1657
https://rclone.org
* Enhancement #1665: Improve cache handling for `restic check`
For safety reasons, restic does not use a local metadata cache for the `restic check` command,
so that data is loaded from the repository and restic can check it's in good condition. When the
cache is disabled, restic will fetch each tiny blob needed for checking the integrity using a
separate backend request. For non-local backends, that will take a long time, and depending on
the backend (e.g. B2) may also be much more expensive.
This PR adds a few commits which will change the behavior as follows:
* When `restic check` is called without any additional parameters, it will build a new cache in a
temporary directory, which is removed at the end of the check. This way, we'll get readahead for
metadata files (so restic will fetch the whole file when the first blob from the file is
requested), but all data is freshly fetched from the storage backend. This is the default
behavior and will work for almost all users.
* When `restic check` is called with `--with-cache`, the default on-disc cache is used. This
behavior hasn't changed since the cache was introduced.
* When `--no-cache` is specified, restic falls back to the old behavior, and read all tiny blobs
in separate requests.
https://github.com/restic/restic/issues/1665
https://github.com/restic/restic/issues/1694
https://github.com/restic/restic/pull/1696
* Enhancement #1721: Add `cache` command to list cache dirs
The command `cache` was added, it allows listing restic's cache directoriers together with
the last usage. It also allows removing old cache dirs without having to access a repo, via
`restic cache --cleanup`
https://github.com/restic/restic/issues/1721
https://github.com/restic/restic/pull/1749
* Enhancement #1758: Allow saving OneDrive folders in Windows
Restic now contains a bugfix to two libraries, which allows saving OneDrive folders in
Windows. In order to use the newer versions of the libraries, the minimal version required to
compile restic is now Go 1.9.
https://github.com/restic/restic/issues/1758
https://github.com/restic/restic/pull/1765
* Enhancement #549: Rework archiver code
The core archiver code and the complementary code for the `backup` command was rewritten
completely. This resolves very annoying issues such as 549. The first backup with this release
of restic will likely result in all files being re-read locally, so it will take a lot longer. The
next backup after that will be fast again.
Basically, with the old code, restic took the last path component of each to-be-saved file or
directory as the top-level file/directory within the snapshot. This meant that when called as
`restic backup /home/user/foo`, the snapshot would contain the files in the directory
`/home/user/foo` as `/foo`.
This is not the case any more with the new archiver code. Now, restic works very similar to what
`tar` does: When restic is called with an absolute path to save, then it'll preserve the
directory structure within the snapshot. For the example above, the snapshot would contain
the files in the directory within `/home/user/foo` in the snapshot. For relative
directories, it only preserves the relative path components. So `restic backup user/foo`
will save the files as `/user/foo` in the snapshot.
While we were at it, the status display and notification system was completely rewritten. By
default, restic now shows which files are currently read (unless `--quiet` is specified) in a
multi-line status display.
The `backup` command also gained a new option: `--verbose`. It can be specified once (which
prints a bit more detail what restic is doing) or twice (which prints a line for each
file/directory restic encountered, together with some statistics).
Another issue that was resolved is the new code only reads two files at most. The old code would
read way too many files in parallel, thereby slowing down the backup process on spinning discs a
lot.
https://github.com/restic/restic/issues/549
https://github.com/restic/restic/issues/1286
https://github.com/restic/restic/issues/446
https://github.com/restic/restic/issues/1344
https://github.com/restic/restic/issues/1416
https://github.com/restic/restic/issues/1456
https://github.com/restic/restic/issues/1145
https://github.com/restic/restic/issues/1160
https://github.com/restic/restic/pull/1494
* Enhancement #1552: Use Google Application Default credentials
Google provide libraries to generate appropriate credentials with various fallback
sources. This change uses the library to generate our GCS client, which allows us to make use of
these extra methods.
This should be backward compatible with previous restic behaviour while adding the
additional capabilities to auth from Google's internal metadata endpoints. For users
running restic in GCP this can make authentication far easier than it was before.
https://github.com/restic/restic/pull/1552
https://developers.google.com/identity/protocols/application-default-credentials
* Enhancement #1477: Accept AWS_SESSION_TOKEN for the s3 backend
Before, it was not possible to use s3 backend with AWS temporary security credentials(with
AWS_SESSION_TOKEN). This change gives higher priority to credentials.EnvAWS credentials
provider.
https://github.com/restic/restic/issues/1477
https://github.com/restic/restic/pull/1479
https://github.com/restic/restic/pull/1647
* Enhancement #1648: Ignore AWS permission denied error when creating a repository
It's not possible to use s3 backend scoped to a subdirectory(with specific permissions).
Restic doesn't try to create repository in a subdirectory, when 'bucket exists' of parent
directory check fails due to permission issues.
https://github.com/restic/restic/pull/1648
* Enhancement #1649: Add illumos/Solaris support
https://github.com/restic/restic/pull/1649
* Enhancement #1709: Improve messages `restic check` prints
Some messages `restic check` prints are not really errors, so from now on restic does not treat
them as errors any more and exits cleanly.
https://github.com/restic/restic/pull/1709
https://forum.restic.net/t/what-is-the-standard-procedure-to-follow-if-a-backup-or-restore-is-interrupted/571/2
* Enhancement #827: Add --new-password-file flag for non-interactive password changes
This makes it possible to change a repository password without being prompted.
https://github.com/restic/restic/issues/827
https://github.com/restic/restic/pull/1720
https://forum.restic.net/t/changing-repo-password-without-prompt/591
* Enhancement #1735: Allow keeping a time range of snaphots
We've added the `--keep-within` option to the `forget` command. It instructs restic to keep
all snapshots within the given duration since the newest snapshot. For example, running
`restic forget --keep-within 5m7d` will keep all snapshots which have been made in the five
months and seven days since the latest snapshot.
https://github.com/restic/restic/pull/1735
* Enhancement #1782: Use default AWS credentials chain for S3 backend
Adds support for file credentials to the S3 backend (e.g. ~/.aws/credentials), and reorders
the credentials chain for the S3 backend to match AWS's standard, which is static credentials,
env vars, credentials file, and finally remote.
https://github.com/restic/restic/pull/1782
Changelog for restic 0.8.3 (2018-02-26)
=======================================
The following sections list the changes in restic 0.8.3 relevant to
restic users. The changes are ordered by importance.
Summary
-------
* Fix #1633: Fixed unexpected 'pack file cannot be listed' error
* Fix #1641: Ignore files with invalid names in the repo
* Fix #1638: Handle errors listing files in the backend
* Enh #1497: Add --read-data-subset flag to check command
* Enh #1560: Retry all repository file download errors
* Enh #1623: Don't check for presence of files in the backend before writing
* Enh #1634: Upgrade B2 client library, reduce HTTP requests
Details
-------
* Bugfix #1633: Fixed unexpected 'pack file cannot be listed' error
Due to a regression introduced in 0.8.2, the `rebuild-index` and `prune` commands failed to
read pack files with size of 587, 588, 589 or 590 bytes.
https://github.com/restic/restic/issues/1633
https://github.com/restic/restic/pull/1635
* Bugfix #1641: Ignore files with invalid names in the repo
The release 0.8.2 introduced a bug: when restic encounters files in the repo which do not have a
valid name, it tries to load a file with a name of lots of zeroes instead of ignoring it. This is now
resolved, invalid file names are just ignored.
https://github.com/restic/restic/issues/1641
https://github.com/restic/restic/pull/1643
https://forum.restic.net/t/help-fixing-repo-no-such-file/485/3
* Bugfix #1638: Handle errors listing files in the backend
A user reported in the forum that restic completes a backup although a concurrent `prune`
operation was running. A few error messages were printed, but the backup was attempted and
completed successfully. No error code was returned.
This should not happen: The repository is exclusively locked during `prune`, so when `restic
backup` is run in parallel, it should abort and return an error code instead.
It was found that the bug was in the code introduced only recently, which retries a List()
operation on the backend should that fail. It is now corrected.
https://github.com/restic/restic/pull/1638
https://forum.restic.net/t/restic-backup-returns-0-exit-code-when-already-locked/484
* Enhancement #1497: Add --read-data-subset flag to check command
This change introduces ability to check integrity of a subset of repository data packs. This
can be used to spread integrity check of larger repositories over a period of time.
https://github.com/restic/restic/issues/1497
https://github.com/restic/restic/pull/1556
* Enhancement #1560: Retry all repository file download errors
Restic will now retry failed downloads, similar to other operations.
https://github.com/restic/restic/pull/1560
* Enhancement #1623: Don't check for presence of files in the backend before writing
Before, all backend implementations were required to return an error if the file that is to be
written already exists in the backend. For most backends, that means making a request (e.g. via
HTTP) and returning an error when the file already exists.
This is not accurate, the file could have been created between the HTTP request testing for it,
and when writing starts, so we've relaxed this requeriment, which saves one additional HTTP
request per newly added file.
https://github.com/restic/restic/pull/1623
* Enhancement #1634: Upgrade B2 client library, reduce HTTP requests
We've upgraded the B2 client library restic uses to access BackBlaze B2. This reduces the
number of HTTP requests needed to upload a new file from two to one, which should improve
throughput to B2.
https://github.com/restic/restic/pull/1634
Changelog for restic 0.8.2 (2018-02-17)
=======================================
The following sections list the changes in restic 0.8.2 relevant to
restic users. The changes are ordered by importance.
Summary
-------
* Fix #1506: Limit bandwith at the http.RoundTripper for HTTP based backends
* Fix #1512: Restore directory permissions as the last step
* Fix #1528: Correctly create missing subdirs in data/
* Fix #1590: Strip spaces for lines read via --files-from
* Fix #1589: Complete intermediate index upload
* Fix #1594: Google Cloud Storage: Use generic HTTP transport
* Fix #1595: Backup: Remove bandwidth display
* Enh #1522: Add support for TLS client certificate authentication
* Enh #1541: Reduce number of remote requests during repository check
* Enh #1567: Reduce number of backend requests for rebuild-index and prune
* Enh #1507: Only reload snapshots once per minute for fuse mount
* Enh #1538: Reduce memory allocations for querying the index
* Enh #1549: Speed up querying across indices and scanning existing files
* Enh #1554: Fuse/mount: Correctly handle EOF, add template option
* Enh #1564: Don't terminate ssh on SIGINT
* Enh #1579: Retry Backend.List() in case of errors
* Enh #1584: Limit index file size
Details
-------
* Bugfix #1506: Limit bandwith at the http.RoundTripper for HTTP based backends
https://github.com/restic/restic/issues/1506
https://github.com/restic/restic/pull/1511
* Bugfix #1512: Restore directory permissions as the last step
This change allows restoring into directories that were not writable during backup. Before,
restic created the directory, set the read-only mode and then failed to create files in the
directory. This change now restores the directory (with its permissions) as the very last
step.
https://github.com/restic/restic/issues/1512
https://github.com/restic/restic/pull/1536
* Bugfix #1528: Correctly create missing subdirs in data/
https://github.com/restic/restic/issues/1528
https://github.com/restic/restic/pull/1529
* Bugfix #1590: Strip spaces for lines read via --files-from
Leading and trailing spaces in lines read via `--files-from` are now stripped, so it behaves
the same as with lines read via `--exclude-file`.
https://github.com/restic/restic/issues/1590
https://github.com/restic/restic/pull/1613
* Bugfix #1589: Complete intermediate index upload
After a user posted a comprehensive report of what he observed, we were able to find a bug and
correct it: During backup, restic uploads so-called "intermediate" index files. When the
backup finishes during a transfer of such an intermediate index, the upload is cancelled, but
the backup is finished without an error. This leads to an inconsistent state, where the
snapshot references data that is contained in the repo, but is not referenced in any index.
The situation can be resolved by building a new index with `rebuild-index`, but looks very
confusing at first. Since all the data got uploaded to the repo successfully, there was no risk
of data loss, just minor inconvenience for our users.
https://github.com/restic/restic/pull/1589
https://forum.restic.net/t/error-loading-tree-check-prune-and-forget-gives-error-b2-backend/406
* Bugfix #1594: Google Cloud Storage: Use generic HTTP transport
It was discovered that the Google Cloud Storage backend did not use the generic HTTP transport,
so things such as bandwidth limiting with `--limit-upload` did not work. This is resolved now.
https://github.com/restic/restic/pull/1594
* Bugfix #1595: Backup: Remove bandwidth display
This commit removes the bandwidth displayed during backup process. It is misleading and
seldomly correct, because it's neither the "read bandwidth" (only for the very first backup)
nor the "upload bandwidth". Many users are confused about (and rightly so), c.f. #1581, #1033,
#1591
We'll eventually replace this display with something more relevant when the new archiver code
is ready.
https://github.com/restic/restic/pull/1595
* Enhancement #1522: Add support for TLS client certificate authentication
Support has been added for using a TLS client certificate for authentication to HTTP based
backend. A file containing the PEM encoded private key and certificate can be set using the
`--tls-client-cert` option.
https://github.com/restic/restic/issues/1522
https://github.com/restic/restic/pull/1524
* Enhancement #1541: Reduce number of remote requests during repository check
This change eliminates redundant remote repository calls and significantly improves
repository check time.
https://github.com/restic/restic/issues/1541
https://github.com/restic/restic/pull/1548
* Enhancement #1567: Reduce number of backend requests for rebuild-index and prune
We've found a way to reduce then number of backend requests for the `rebuild-index` and `prune`
operations. This significantly speeds up the operations for high-latency backends.
https://github.com/restic/restic/issues/1567
https://github.com/restic/restic/pull/1574
https://github.com/restic/restic/pull/1575
* Enhancement #1507: Only reload snapshots once per minute for fuse mount
https://github.com/restic/restic/pull/1507
* Enhancement #1538: Reduce memory allocations for querying the index
This change reduces the internal memory allocations when the index data structures in memory
are queried if a blob (part of a file) already exists in the repo. It should speed up backup a bit,
and maybe even reduce RAM usage.
https://github.com/restic/restic/pull/1538
* Enhancement #1549: Speed up querying across indices and scanning existing files
This change increases the whenever a blob (part of a file) is searched for in a restic
repository. This will reduce cpu usage some when backing up files already backed up by restic.
Cpu usage is further decreased when scanning files.
https://github.com/restic/restic/pull/1549
* Enhancement #1554: Fuse/mount: Correctly handle EOF, add template option
We've added the `--snapshot-template` string, which can be used to specify a template for a
snapshot directory. In addition, accessing data after the end of a file via the fuse mount is now
handled correctly.
https://github.com/restic/restic/pull/1554
* Enhancement #1564: Don't terminate ssh on SIGINT
We've reworked the code which runs the `ssh` login for the sftp backend so that it can prompt for a
password (if needed) but does not exit when the user presses CTRL+C (SIGINT) e.g. during
backup. This allows restic to properly shut down when it receives SIGINT and remove the lock
file from the repo, afterwards exiting the `ssh` process.
https://github.com/restic/restic/pull/1564
https://github.com/restic/restic/pull/1588
* Enhancement #1579: Retry Backend.List() in case of errors
https://github.com/restic/restic/pull/1579
* Enhancement #1584: Limit index file size
Before, restic would create a single new index file on `prune` or `rebuild-index`, this may
lead to memory problems when this huge index is created and loaded again. We're now limiting the
size of the index file, and split newly created index files into several smaller ones. This
allows restic to be more memory-efficient.
https://github.com/restic/restic/issues/1412
https://github.com/restic/restic/issues/979
https://github.com/restic/restic/issues/526
https://github.com/restic/restic/pull/1584
Changelog for restic 0.8.1 (2017-12-27)
=======================================
@ -327,7 +907,7 @@ Summary
* Enh #1061: Add Dockerfile and official Docker image
* Enh #1126: Use the standard Go git repository layout, use `dep` for vendoring
* Enh #1134: Add support for storing backups on Google Cloud Storage
* Enh #1144: Properly report errors when reading files with exclude patterns.
* Enh #1144: Properly report errors when reading files with exclude patterns
* Enh #1149: Add support for storing backups on Microsoft Azure Blob Storage
* Enh #1196: Add `--group-by` to `forget` command for flexible grouping
* Enh #1203: Print stats on all BSD systems when SIGINFO (ctrl+t) is received
@ -415,7 +995,7 @@ Details
https://github.com/restic/restic/pull/1134
https://github.com/restic/restic/pull/1052
* Enhancement #1144: Properly report errors when reading files with exclude patterns.
* Enhancement #1144: Properly report errors when reading files with exclude patterns
https://github.com/restic/restic/pull/1144
@ -435,7 +1015,7 @@ Details
* Enhancement #1203: Print stats on all BSD systems when SIGINFO (ctrl+t) is received
https://github.com/restic/restic/pull/1203
https://github.com/restic/restic/pull/1082
https://github.com/restic/restic/pull/1082#issuecomment-326279920
* Enhancement #1205: Allow specifying time/date for a backup with `--time`
@ -475,12 +1055,12 @@ Details
* Enhancement #1055: Create subdirs below `data/` for local/sftp backends
The local and sftp backends now create the subdirs below `data/` on open/init. This way, restic
makes sure that they always exist. This is connected to an issue for the sftp server:
makes sure that they always exist. This is connected to an issue for the sftp server.
https://github.com/restic/restic/issues/1055
https://github.com/restic/rest-server/pull/11#issuecomment-309879710
https://github.com/restic/restic/pull/1077
https://github.com/restic/restic/pull/1105
https://github.com/restic/rest-server/pull/11#issuecomment-309879710
* Enhancement #1067: Allow loading credentials for s3 from IAM
@ -492,7 +1072,7 @@ Details
* Enhancement #1073: Add `migrate` cmd to migrate from `s3legacy` to `default` layout
The `migrate` command for chaning the `s3legacy` layout to the `default` layout for s3
The `migrate` command for changing the `s3legacy` layout to the `default` layout for s3
backends has been improved: It can now be restarted with `restic migrate --force s3_layout`
and automatically retries operations on error.

View file

@ -120,7 +120,8 @@ down to the following steps:
6. You will receive comments on your code and the feature or bug that they
address. Maybe you need to rework some minor things, in this case push new
commits to the branch you created for the pull request, they will be
commits to the branch you created for the pull request (or amend the
existing commit, use common sense to decide which is better), they will be
automatically added to the pull request.
7. If your pull request changes anything that users should be aware of (a
@ -163,7 +164,7 @@ history and triaging bugs much easier.
Git commit messages have a very terse summary in the first line of the commit
message, followed by an empty line, followed by a more verbose description or a
List of changed things. For examples, please refer to the excellent [How to
Write a Git Commit Message](http://chris.beams.io/posts/git-commit/).
Write a Git Commit Message](https://chris.beams.io/posts/git-commit/).
If you change/add multiple different things that aren't related at all, try to
make several smaller commits. This is much easier to review. Using `git add -p`

27
GOVERNANCE.md Normal file
View file

@ -0,0 +1,27 @@
# restic project governance
## Overview
The restic project uses a governance model commonly described as Benevolent
Dictator For Life (BDFL). This document outlines our understanding of what this
means. It is derived from the [i3 window manager project
governance](https://raw.githubusercontent.com/i3/i3/next/.github/GOVERNANCE.md).
## Roles
* user: anyone who interacts with the restic project
* core contributor: a handful of people who have contributed significantly to
the project by any means (issue triage, support, documentation, code, etc.).
Core contributors are recognizable via GitHubs "Member" badge.
* Benevolent Dictator For Life (BDFL): a single individual who makes decisions
when consensus cannot be reached. restic's current BDFL is [@fd0](https://github.com/fd0).
## Decision making process
In general, we try to reach consensus in discussions. In case consensus cannot
be reached, the BDFL makes a decision.
## Contribution process
The contribution process is described in a separate document called
[CONTRIBUTING](CONTRIBUTING.md).

499
Gopkg.lock generated
View file

@ -3,217 +3,472 @@
[[projects]]
branch = "master"
digest = "1:94e9caf404409a2990cfd22aca37d758494c098eff3e2c37fda1abed862e74dd"
name = "bazil.org/fuse"
packages = [".","fs","fuseutil"]
revision = "371fbbdaa8987b715bdd21d6adc4c9b20155f748"
packages = [
".",
"fs",
"fuseutil",
]
pruneopts = "UT"
revision = "65cc252bf6691cb3c7014bcb2c8dc29de91e3a7e"
[[projects]]
digest = "1:5c3894b2aa4d6bead0ceeea6831b305d62879c871780e7b76296ded1b004bc57"
name = "cloud.google.com/go"
packages = ["compute/metadata"]
revision = "2d3a6656c17a60b0815b7e06ab0be04eacb6e613"
version = "v0.16.0"
pruneopts = "UT"
revision = "aad3f485ee528456e0768f20397b4d9dd941e755"
version = "v0.25.0"
[[projects]]
digest = "1:46ea9487304f4b3c787f54483ecb13a338d686dcd670db0ab1a112ed0ae2128e"
name = "github.com/Azure/azure-sdk-for-go"
packages = ["storage"]
revision = "7692b0cef22674113fcf71cc17ac3ccc1a7fef48"
version = "v11.2.2-beta"
packages = [
"storage",
"version",
]
pruneopts = "UT"
revision = "4e8cbbfb1aeab140cd0fa97fd16b64ee18c3ca6a"
version = "v19.1.0"
[[projects]]
digest = "1:27d0cd1a78fc836f7c0f07749d029a5f7895c84ad066187b08b70e9d1830098e"
name = "github.com/Azure/go-autorest"
packages = ["autorest","autorest/adal","autorest/azure","autorest/date"]
revision = "c67b24a8e30d876542a85022ebbdecf0e5a935e8"
version = "v9.4.1"
packages = [
"autorest",
"autorest/adal",
"autorest/azure",
"autorest/date",
"logger",
"version",
]
pruneopts = "UT"
revision = "dd94e014aaf16d1df746762e392aa201c1b4c461"
version = "v10.15.0"
[[projects]]
digest = "1:2209584c0f7c9b68c23374e659357ab546e1b70eec2761f03280f69a8fd23d77"
name = "github.com/cenkalti/backoff"
packages = ["."]
revision = "61153c768f31ee5f130071d08fc82b85208528de"
pruneopts = "UT"
revision = "2ea60e5f094469f9e65adb9cd103795b73ae743e"
version = "v2.0.0"
[[projects]]
digest = "1:7cb4fdca4c251b3ef8027c90ea35f70c7b661a593b9eeae34753c65499098bb1"
name = "github.com/cpuguy83/go-md2man"
packages = ["md2man"]
pruneopts = "UT"
revision = "20f5889cbdc3c73dbd2862796665e7c465ade7d1"
version = "v1.0.8"
[[projects]]
digest = "1:76dc72490af7174349349838f2fe118996381b31ea83243812a97e5a0fd5ed55"
name = "github.com/dgrijalva/jwt-go"
packages = ["."]
pruneopts = "UT"
revision = "06ea1031745cb8b3dab3f6a236daf2b0aa468b7e"
version = "v3.2.0"
[[projects]]
branch = "master"
digest = "1:6f9339c912bbdda81302633ad7e99a28dfa5a639c864061f1929510a9a64aa74"
name = "github.com/dustin/go-humanize"
packages = ["."]
pruneopts = "UT"
revision = "9f541cc9db5d55bce703bd99987c9d5cb8eea45e"
[[projects]]
digest = "1:c7edfbb6320d6a93240d663dc52bca92bed4c116abe54c35679eec4e7cc2bd77"
name = "github.com/elithrar/simple-scrypt"
packages = ["."]
pruneopts = "UT"
revision = "d150773194090feb6c897805a7bcea8d49544e2c"
version = "v1.3.0"
[[projects]]
digest = "1:fe8a03a8222d5b913f256972933d26d24ad7c8286692a42943bc01633cc8fce3"
name = "github.com/go-ini/ini"
packages = ["."]
pruneopts = "UT"
revision = "358ee7663966325963d4e8b2e1fbd570c5195153"
version = "v1.38.1"
[[projects]]
digest = "1:15042ad3498153684d09f393bbaec6b216c8eec6d61f63dff711de7d64ed8861"
name = "github.com/golang/protobuf"
packages = ["proto"]
pruneopts = "UT"
revision = "b4deda0973fb4c70b50d226b1af49f3da59f5265"
version = "v1.1.0"
[[projects]]
name = "github.com/cpuguy83/go-md2man"
packages = ["md2man"]
revision = "1d903dcb749992f3741d744c0f8376b4bd7eb3e1"
version = "v1.0.7"
[[projects]]
name = "github.com/dgrijalva/jwt-go"
packages = ["."]
revision = "dbeaa9332f19a944acb5736b4456cfcc02140e29"
version = "v3.1.0"
[[projects]]
branch = "master"
name = "github.com/dustin/go-humanize"
packages = ["."]
revision = "bb3d318650d48840a39aa21a027c6630e198e626"
[[projects]]
name = "github.com/elithrar/simple-scrypt"
packages = ["."]
revision = "2325946f714c95de4a6088202c402fbdfa64163b"
version = "v1.2.0"
[[projects]]
name = "github.com/go-ini/ini"
packages = ["."]
revision = "32e4c1e6bc4e7d0d8451aa6b75200d19e37a536a"
version = "v1.32.0"
[[projects]]
branch = "master"
name = "github.com/golang/protobuf"
packages = ["proto"]
revision = "1e59b77b52bf8e4b449a57e6f79f21226d571845"
digest = "1:2e3c336fc7fde5c984d2841455a658a6d626450b1754a854b3b32e7a8f49a07a"
name = "github.com/google/go-cmp"
packages = [
"cmp",
"cmp/internal/diff",
"cmp/internal/function",
"cmp/internal/value",
]
pruneopts = "UT"
revision = "3af367b6b30c263d47e8895973edcca9a49cf029"
version = "v0.2.0"
[[projects]]
digest = "1:870d441fe217b8e689d7949fef6e43efbc787e50f200cb1e70dbca9204a1d6be"
name = "github.com/inconshreveable/mousetrap"
packages = ["."]
pruneopts = "UT"
revision = "76626ae9c91c4f2a10f34cad8ce83ea42c93bb75"
version = "v1.0"
[[projects]]
branch = "master"
digest = "1:190ff84d9b2ed6589088f178cba8edb4b8ecb334df4572421fb016be1ac20463"
name = "github.com/juju/ratelimit"
packages = ["."]
pruneopts = "UT"
revision = "59fac5042749a5afb9af70e813da1dd5474f0167"
version = "1.0.1"
[[projects]]
branch = "master"
digest = "1:9cedee824c21326bd26950bd9e1ffe9dc4e7ca03dc8634d0e6f954ee6a383172"
name = "github.com/kr/fs"
packages = ["."]
revision = "2788f0dbd16903de03cb8186e5c7d97b69ad387b"
pruneopts = "UT"
revision = "1455def202f6e05b95cc7bfc7e8ae67ae5141eba"
version = "v0.1.0"
[[projects]]
digest = "1:1faa76bd9bffce9c25eaca0597afb67bd05a21ae57fe4378154ce8855ef163d1"
name = "github.com/kurin/blazer"
packages = ["b2","base","internal/b2types","internal/blog"]
revision = "e269a1a17bb6aec278c06a57cb7e8f8d0d333e04"
version = "v0.2.1"
packages = [
"b2",
"base",
"internal/b2assets",
"internal/b2types",
"internal/blog",
"x/window",
]
pruneopts = "UT"
revision = "caf65aa76491dc533bac68ad3243ce72fa4e0a0a"
version = "v0.5.1"
[[projects]]
branch = "master"
name = "github.com/minio/go-homedir"
digest = "1:4e878df5f4e9fd625bf9c9aac77ef7cbfa4a74c01265505527c23470c0e40300"
name = "github.com/marstr/guid"
packages = ["."]
revision = "21304a94172ae3a09dee2cd86a12fb6f842138c7"
pruneopts = "UT"
revision = "8bd9a64bf37eb297b492a4101fb28e80ac0b290f"
version = "v1.1.0"
[[projects]]
digest = "1:d4d17353dbd05cb52a2a52b7fe1771883b682806f68db442b436294926bbfafb"
name = "github.com/mattn/go-isatty"
packages = ["."]
pruneopts = "UT"
revision = "0360b2af4f38e8d38c7fce2a9f4e702702d73a39"
version = "v0.0.3"
[[projects]]
digest = "1:95c73c666919be2843b955eafc83f58c136312b74f79c703152f4c4a95fd64dc"
name = "github.com/minio/minio-go"
packages = [".","pkg/credentials","pkg/encrypt","pkg/policy","pkg/s3signer","pkg/s3utils","pkg/set"]
revision = "57a8ae886b49af6eb0d2c27c2d007ed2f71e1da5"
version = "4.0.3"
packages = [
".",
"pkg/credentials",
"pkg/encrypt",
"pkg/s3signer",
"pkg/s3utils",
"pkg/set",
]
pruneopts = "UT"
revision = "70799fe8dae6ecfb6c7d7e9e048fce27f23a1992"
version = "v6.0.5"
[[projects]]
branch = "master"
digest = "1:8eb17c2ec4df79193ae65b621cd1c0c4697db3bc317fe6afdc76d7f2746abd05"
name = "github.com/mitchellh/go-homedir"
packages = ["."]
pruneopts = "UT"
revision = "3864e76763d94a6df2f9960b16a20a33da9f9a66"
[[projects]]
digest = "1:928de5172dd3563964d1b88a4ee3775cf72e16f1efabb482ab6d0e0bab86ee69"
name = "github.com/ncw/swift"
packages = ["."]
revision = "c95c6e5c2d1a3d37fc44c8c6dc9e231c7500667d"
pruneopts = "UT"
revision = "b2a7479cf26fa841ff90dd932d0221cb5c50782d"
version = "v1.0.39"
[[projects]]
digest = "1:40e195917a951a8bf867cd05de2a46aaf1806c50cf92eebf4c16f78cd196f747"
name = "github.com/pkg/errors"
packages = ["."]
pruneopts = "UT"
revision = "645ef00459ed84a119197bfb8d8205042c6df63d"
version = "v0.8.0"
[[projects]]
digest = "1:cfa0d7741863a0e1d30e0ccdd4b48a96a471cdb47892303de8b92c3713af3e77"
name = "github.com/pkg/profile"
packages = ["."]
pruneopts = "UT"
revision = "5b67d428864e92711fcbd2f8629456121a56d91f"
version = "v1.2.1"
[[projects]]
digest = "1:23ed92ba5d90a2dfe817f3895027ccef796e79c30be5125d48e17afdcc395d73"
name = "github.com/pkg/sftp"
packages = ["."]
revision = "98203f5a8333288eb3163b7c667d4260fe1333e9"
version = "1.0.0"
pruneopts = "UT"
revision = "57673e38ea946592a59c26592b7e6fbda646975b"
version = "v1.8.0"
[[projects]]
digest = "1:0d67664e93e366f072ac9672feea29bfc63c9f90f005e9e8a0df1954153f5a14"
name = "github.com/pkg/xattr"
packages = ["."]
revision = "23c75e3f6c1d8b13b3dd905b011a7f38a06044b7"
version = "v0.2.1"
pruneopts = "UT"
revision = "ae385d07bb53f092fcc7daaf738d8513df084931"
version = "v0.3.1"
[[projects]]
digest = "1:13ecc4000f49cf0aa3ee56fffcc93119c8edffacfff08674c80d2757d8c33a83"
name = "github.com/restic/chunker"
packages = ["."]
pruneopts = "UT"
revision = "db83917be3b88cc307464b7d8a221c173e34a0db"
version = "v0.2.0"
[[projects]]
digest = "1:8bc629776d035c003c7814d4369521afe67fdb8efc4b5f66540d29343b98cf23"
name = "github.com/russross/blackfriday"
packages = ["."]
revision = "4048872b16cc0fc2c5fd9eacf0ed2c2fedaa0c8c"
version = "v1.5"
pruneopts = "UT"
revision = "55d61fa8aa702f59229e6cff85793c22e580eaf5"
version = "v1.5.1"
[[projects]]
name = "github.com/satori/uuid"
digest = "1:274f67cb6fed9588ea2521ecdac05a6d62a8c51c074c1fccc6a49a40ba80e925"
name = "github.com/satori/go.uuid"
packages = ["."]
revision = "879c5887cd475cd7864858769793b2ceb0d44feb"
pruneopts = "UT"
revision = "f58768cc1a7a7e77a3bd49e98cdd21419399b6a3"
version = "v1.2.0"
[[projects]]
digest = "1:d867dfa6751c8d7a435821ad3b736310c2ed68945d05b50fb9d23aee0540c8cc"
name = "github.com/sirupsen/logrus"
packages = ["."]
pruneopts = "UT"
revision = "3e01752db0189b9157070a0e1668a620f9a85da2"
version = "v1.0.6"
[[projects]]
digest = "1:e01b05ba901239c783dfe56450bcde607fc858908529868259c9a8765dc176d0"
name = "github.com/spf13/cobra"
packages = [
".",
"doc",
]
pruneopts = "UT"
revision = "ef82de70bb3f60c65fb8eebacbb2d122ef517385"
version = "v0.0.3"
[[projects]]
digest = "1:9424f440bba8f7508b69414634aef3b2b3a877e522d8a4624692412805407bb7"
name = "github.com/spf13/pflag"
packages = ["."]
pruneopts = "UT"
revision = "583c0c0531f06d5278b7d917446061adc344b5cd"
version = "v1.0.1"
[[projects]]
branch = "master"
digest = "1:eefb1f49ec07e71206d4c9ea1a3e634cad331c2180733e4121b8ae39e8e92ecb"
name = "golang.org/x/crypto"
packages = [
"argon2",
"blake2b",
"curve25519",
"ed25519",
"ed25519/internal/edwards25519",
"internal/chacha20",
"internal/subtle",
"pbkdf2",
"poly1305",
"scrypt",
"ssh",
"ssh/terminal",
]
pruneopts = "UT"
revision = "c126467f60eb25f8f27e5a981f32a87e3965053f"
[[projects]]
branch = "master"
digest = "1:8356aa7bdcb10a210b814b64ff76d61de7c36ac4cb6263de3af5e3e2e546956d"
name = "golang.org/x/net"
packages = [
"context",
"context/ctxhttp",
"http/httpguts",
"http2",
"http2/hpack",
"idna",
]
pruneopts = "UT"
revision = "32f9bdbd7df18e8641d215e7ea68be88b971feb0"
[[projects]]
branch = "master"
digest = "1:bea0314c10bd362ab623af4880d853b5bad3b63d0ab9945c47e461b8d04203ed"
name = "golang.org/x/oauth2"
packages = [
".",
"google",
"internal",
"jws",
"jwt",
]
pruneopts = "UT"
revision = "3d292e4d0cdc3a0113e6d207bb137145ef1de42f"
[[projects]]
branch = "master"
digest = "1:39ebcc2b11457b703ae9ee2e8cca0f68df21969c6102cb3b705f76cca0ea0239"
name = "golang.org/x/sync"
packages = ["errgroup"]
pruneopts = "UT"
revision = "1d60e4601c6fd243af51cc01ddf169918a5407ca"
[[projects]]
branch = "master"
digest = "1:a220a85c72a6cb7339c412cb2b117019a7fd94007cdfffb3b5b1d058227a2bf8"
name = "golang.org/x/sys"
packages = [
"cpu",
"unix",
"windows",
]
pruneopts = "UT"
revision = "bd9dbc187b6e1dacfdd2722a87e83093c2d7bd6e"
[[projects]]
digest = "1:e5a8511f063c38c51ab9ab80e718e9149f692652aeb4e393a8c020dd1bf38ca2"
name = "golang.org/x/text"
packages = [
"collate",
"collate/build",
"encoding",
"encoding/internal",
"encoding/internal/identifier",
"encoding/unicode",
"internal/colltab",
"internal/gen",
"internal/tag",
"internal/triegen",
"internal/ucd",
"internal/utf8internal",
"language",
"runes",
"secure/bidirule",
"transform",
"unicode/bidi",
"unicode/cldr",
"unicode/norm",
"unicode/rangetable",
]
pruneopts = "UT"
revision = "f21a4dfb5e38f5895301dc265a8def02365cc3d0"
version = "v0.3.0"
[[projects]]
branch = "master"
digest = "1:fb983acae7bd9c3ed9aadc1b1241d9e559ed21dbf84c17a0dda663ca169ccd69"
name = "google.golang.org/api"
packages = [
"gensupport",
"googleapi",
"googleapi/internal/uritemplates",
"storage/v1",
]
pruneopts = "UT"
revision = "31ca0e01cd791f07750cb23fc99327721f753290"
[[projects]]
digest = "1:c8907869850adaa8bd7631887948d0684f3787d0912f1c01ab72581a6c34432e"
name = "google.golang.org/appengine"
packages = [
".",
"internal",
"internal/app_identity",
"internal/base",
"internal/datastore",
"internal/log",
"internal/modules",
"internal/remote_api",
"internal/urlfetch",
"urlfetch",
]
pruneopts = "UT"
revision = "b1f26356af11148e710935ed1ac8a7f5702c7612"
version = "v1.1.0"
[[projects]]
name = "github.com/sirupsen/logrus"
packages = ["."]
revision = "f006c2ac4710855cf0f916dd6b77acf6b048dc6e"
version = "v1.0.3"
[[projects]]
name = "github.com/spf13/cobra"
packages = [".","doc"]
revision = "7b2c5ac9fc04fc5efafb60700713d4fa609b777b"
version = "v0.0.1"
[[projects]]
name = "github.com/spf13/pflag"
packages = ["."]
revision = "e57e3eeb33f795204c1ca35f56c44f83227c6e66"
version = "v1.0.0"
[[projects]]
branch = "master"
name = "golang.org/x/crypto"
packages = ["curve25519","ed25519","ed25519/internal/edwards25519","pbkdf2","poly1305","scrypt","ssh","ssh/terminal"]
revision = "94eea52f7b742c7cbe0b03b22f0c4c8631ece122"
[[projects]]
branch = "master"
name = "golang.org/x/net"
packages = ["context","context/ctxhttp"]
revision = "a8b9294777976932365dabb6640cf1468d95c70f"
[[projects]]
branch = "master"
name = "golang.org/x/oauth2"
packages = [".","google","internal","jws","jwt"]
revision = "f95fa95eaa936d9d87489b15d1d18b97c1ba9c28"
[[projects]]
branch = "master"
name = "golang.org/x/sys"
packages = ["unix","windows"]
revision = "8b4580aae2a0dd0c231a45d3ccb8434ff533b840"
[[projects]]
branch = "master"
name = "google.golang.org/api"
packages = ["gensupport","googleapi","googleapi/internal/uritemplates","storage/v1"]
revision = "3a1d936b7575b82197a1fea0632218dd07b1e65c"
[[projects]]
name = "google.golang.org/appengine"
packages = [".","internal","internal/app_identity","internal/base","internal/datastore","internal/log","internal/modules","internal/remote_api","internal/urlfetch","urlfetch"]
revision = "150dc57a1b433e64154302bdc40b6bb8aefa313a"
version = "v1.0.0"
[[projects]]
branch = "v2"
digest = "1:5bb148b78468350091db2ffbb2370f35cc6dcd74d9378a31b1c7b86ff7528f08"
name = "gopkg.in/tomb.v2"
packages = ["."]
pruneopts = "UT"
revision = "d5d1b5820637886def9eef33e03a27a9f166942c"
[[projects]]
digest = "1:342378ac4dcb378a5448dd723f0784ae519383532f5e70ade24132c4c8693202"
name = "gopkg.in/yaml.v2"
packages = ["."]
revision = "287cf08546ab5e7e37d55a84f7ed3fd1db036de5"
pruneopts = "UT"
revision = "5420a8b6744d3b0345ab293f6fcba19c978f1183"
version = "v2.2.1"
[solve-meta]
analyzer-name = "dep"
analyzer-version = 1
inputs-digest = "f0a207197cb502238ac87ca8e07b2640c02ec380a50b036e09ef87e40e31ca2d"
input-imports = [
"bazil.org/fuse",
"bazil.org/fuse/fs",
"github.com/Azure/azure-sdk-for-go/storage",
"github.com/cenkalti/backoff",
"github.com/elithrar/simple-scrypt",
"github.com/google/go-cmp/cmp",
"github.com/juju/ratelimit",
"github.com/kurin/blazer/b2",
"github.com/mattn/go-isatty",
"github.com/minio/minio-go",
"github.com/minio/minio-go/pkg/credentials",
"github.com/ncw/swift",
"github.com/pkg/errors",
"github.com/pkg/profile",
"github.com/pkg/sftp",
"github.com/pkg/xattr",
"github.com/restic/chunker",
"github.com/spf13/cobra",
"github.com/spf13/cobra/doc",
"github.com/spf13/pflag",
"golang.org/x/crypto/poly1305",
"golang.org/x/crypto/scrypt",
"golang.org/x/crypto/ssh/terminal",
"golang.org/x/net/context",
"golang.org/x/net/context/ctxhttp",
"golang.org/x/net/http2",
"golang.org/x/oauth2/google",
"golang.org/x/sync/errgroup",
"golang.org/x/sys/unix",
"golang.org/x/text/encoding/unicode",
"google.golang.org/api/googleapi",
"google.golang.org/api/storage/v1",
"gopkg.in/tomb.v2",
]
solver-name = "gps-cdcl"
solver-version = 1

View file

@ -19,3 +19,7 @@
# [[override]]
# name = "github.com/x/y"
# version = "2.4.0"
[prune]
unused-packages = true
go-tests = true

View file

@ -1,4 +1,4 @@
|Documentation| |Build Status| |Build status| |Report Card| |Say Thanks| |TestCoverage|
|Documentation| |Build Status| |Build status| |Report Card| |Say Thanks| |TestCoverage| |Reviewed by Hound|
Introduction
------------
@ -29,7 +29,7 @@ and add some data:
.. code-block:: console
$ restic -r /tmp/backup backup ~/work
$ restic --repo /tmp/backup backup ~/work
enter password for repository:
scan [/home/user/work]
scanned 764 directories, 1816 files in 0:00
@ -57,6 +57,7 @@ Therefore, restic supports the following backends for storing backups natively:
- `BackBlaze B2 <https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#backblaze-b2>`__
- `Microsoft Azure Blob Storage <https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#microsoft-azure-blob-storage>`__
- `Google Cloud Storage <https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#google-cloud-storage>`__
- And many other services via the `rclone <https://rclone.org>`__ `Backend <https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#other-services-via-rclone>`__
Design Principles
-----------------
@ -110,10 +111,18 @@ License
Restic is licensed under `BSD 2-Clause License <https://opensource.org/licenses/BSD-2-Clause>`__. You can find the
complete text in ``LICENSE``.
Sponsorship
-----------
Backend integration tests for Google Cloud Storage and Microsoft Azure Blob
Storage are sponsored by `AppsCode <https://appscode.com>`__!
|AppsCode|
.. |Documentation| image:: https://readthedocs.org/projects/restic/badge/?version=latest
:target: https://restic.readthedocs.io/en/latest/?badge=latest
.. |Build Status| image:: https://travis-ci.org/restic/restic.svg?branch=master
:target: https://travis-ci.org/restic/restic
.. |Build Status| image:: https://travis-ci.com/restic/restic.svg?branch=master
:target: https://travis-ci.com/restic/restic
.. |Build status| image:: https://ci.appveyor.com/api/projects/status/nuy4lfbgfbytw92q/branch/master?svg=true
:target: https://ci.appveyor.com/project/fd0/restic/branch/master
.. |Report Card| image:: https://goreportcard.com/badge/github.com/restic/restic
@ -122,3 +131,7 @@ complete text in ``LICENSE``.
:target: https://saythanks.io/to/restic
.. |TestCoverage| image:: https://codecov.io/gh/restic/restic/branch/master/graph/badge.svg
:target: https://codecov.io/gh/restic/restic
.. |AppsCode| image:: https://cdn.appscode.com/images/logo/appscode/ac-logo-color.png
:target: https://appscode.com
.. |Reviewed by Hound| image:: https://img.shields.io/badge/Reviewed_by-Hound-8E64B0.svg
:target: https://houndci.com

View file

@ -1 +1 @@
0.8.1
0.9.1

View file

@ -17,8 +17,8 @@ init:
install:
- rmdir c:\go /s /q
- appveyor DownloadFile https://storage.googleapis.com/golang/go1.9.windows-amd64.msi
- msiexec /i go1.9.windows-amd64.msi /q
- appveyor DownloadFile https://dl.google.com/go/go1.10.windows-amd64.msi
- msiexec /i go1.10.windows-amd64.msi /q
- go version
- go env
- appveyor DownloadFile http://sourceforge.netcologne.de/project/gnuwin32/tar/1.13-1/tar-1.13-1-bin.zip -FileName tar.zip

155
build.go
View file

@ -1,4 +1,33 @@
// +build ignore
// BSD 2-Clause License
//
// Copyright (c) 2016-2018, Alexander Neumann <alexander@bumpern.de>
// All rights reserved.
//
// This file has been copied from the repository at:
// https://github.com/fd0/build-go
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are met:
//
// * Redistributions of source code must retain the above copyright notice, this
// list of conditions and the following disclaimer.
//
// * Redistributions in binary form must reproduce the above copyright notice,
// this list of conditions and the following disclaimer in the documentation
// and/or other materials provided with the distribution.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
// AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
// DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
// FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
// DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
// SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
// CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
// OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
// +build ignore_build_go
package main
@ -15,29 +44,35 @@ import (
"strings"
)
var (
verbose bool
keepGopath bool
runTests bool
enableCGO bool
)
var config = struct {
Name string
Namespace string
Main string
Tests []string
MinVersion GoVersion
}{
// config contains the configuration for the program to build.
var config = Config{
Name: "restic", // name of the program executable and directory
Namespace: "github.com/restic/restic", // subdir of GOPATH, e.g. "github.com/foo/bar"
Main: "github.com/restic/restic/cmd/restic", // package name for the main package
Tests: []string{ // tests to run
"github.com/restic/restic/internal/...",
"github.com/restic/restic/cmd/..."},
MinVersion: GoVersion{Major: 1, Minor: 8, Patch: 0}, // minimum Go version supported
"github.com/restic/restic/cmd/...",
},
MinVersion: GoVersion{Major: 1, Minor: 9, Patch: 0}, // minimum Go version supported
}
// Config configures the build.
type Config struct {
Name string
Namespace string
Main string
Tests []string
MinVersion GoVersion
}
var (
verbose bool
keepGopath bool
runTests bool
enableCGO bool
enablePIE bool
)
// specialDir returns true if the file begins with a special character ('.' or '_').
func specialDir(name string) bool {
if name == "." {
@ -191,8 +226,11 @@ func showUsage(output io.Writer) {
fmt.Fprintf(output, " -T --test run tests\n")
fmt.Fprintf(output, " -o --output set output file name\n")
fmt.Fprintf(output, " --enable-cgo use CGO to link against libc\n")
fmt.Fprintf(output, " --enable-pie use PIE buildmode\n")
fmt.Fprintf(output, " --goos value set GOOS for cross-compilation\n")
fmt.Fprintf(output, " --goarch value set GOARCH for cross-compilation\n")
fmt.Fprintf(output, " --goarm value set GOARM for cross-compilation\n")
fmt.Fprintf(output, " --tempdir dir use a specific directory for compilation\n")
}
func verbosePrintf(message string, args ...interface{}) {
@ -218,13 +256,28 @@ func cleanEnv() (env []string) {
}
// build runs "go build args..." with GOPATH set to gopath.
func build(cwd, goos, goarch, gopath string, args ...string) error {
func build(cwd string, ver GoVersion, goos, goarch, goarm, gopath string, args ...string) error {
a := []string{"build"}
a = append(a, "-asmflags", fmt.Sprintf("-trimpath=%s", gopath))
a = append(a, "-gcflags", fmt.Sprintf("-trimpath=%s", gopath))
if ver.AtLeast(GoVersion{1, 10, 0}) {
verbosePrintf("Go version is at least 1.10, using new syntax for -gcflags\n")
// use new prefix
a = append(a, "-asmflags", fmt.Sprintf("all=-trimpath=%s", gopath))
a = append(a, "-gcflags", fmt.Sprintf("all=-trimpath=%s", gopath))
} else {
a = append(a, "-asmflags", fmt.Sprintf("-trimpath=%s", gopath))
a = append(a, "-gcflags", fmt.Sprintf("-trimpath=%s", gopath))
}
if enablePIE {
a = append(a, "-buildmode=pie")
}
a = append(a, args...)
cmd := exec.Command("go", a...)
cmd.Env = append(cleanEnv(), "GOPATH="+gopath, "GOARCH="+goarch, "GOOS="+goos)
if goarm != "" {
cmd.Env = append(cmd.Env, "GOARM="+goarm)
}
if !enableCGO {
cmd.Env = append(cmd.Env, "CGO_ENABLED=0")
}
@ -232,7 +285,7 @@ func build(cwd, goos, goarch, gopath string, args ...string) error {
cmd.Dir = cwd
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
verbosePrintf("go %s\n", args)
verbosePrintf("go %s\n", a)
return cmd.Run()
}
@ -328,30 +381,39 @@ func ParseGoVersion(s string) (v GoVersion) {
s = s[2:]
data := strings.Split(s, ".")
if len(data) != 3 {
return
if len(data) < 2 || len(data) > 3 {
// invalid version
return GoVersion{}
}
major, err := strconv.Atoi(data[0])
var err error
v.Major, err = strconv.Atoi(data[0])
if err != nil {
return
return GoVersion{}
}
minor, err := strconv.Atoi(data[1])
if err != nil {
return
// try to parse the minor version while removing an eventual suffix (like
// "rc2" or so)
for s := data[1]; s != ""; s = s[:len(s)-1] {
v.Minor, err = strconv.Atoi(s)
if err == nil {
break
}
}
patch, err := strconv.Atoi(data[2])
if err != nil {
return
if v.Minor == 0 {
// no minor version found
return GoVersion{}
}
v = GoVersion{
Major: major,
Minor: minor,
Patch: patch,
if len(data) >= 3 {
v.Patch, err = strconv.Atoi(data[2])
if err != nil {
return GoVersion{}
}
}
return
}
@ -397,6 +459,9 @@ func main() {
targetGOOS := runtime.GOOS
targetGOARCH := runtime.GOARCH
targetGOARM := ""
gopath := ""
var outputFilename string
@ -420,16 +485,24 @@ func main() {
case "-o", "--output":
skipNext = true
outputFilename = params[i+1]
case "--tempdir":
skipNext = true
gopath = params[i+1]
case "-T", "--test":
runTests = true
case "--enable-cgo":
enableCGO = true
case "--enable-pie":
enablePIE = true
case "--goos":
skipNext = true
targetGOOS = params[i+1]
case "--goarch":
skipNext = true
targetGOARCH = params[i+1]
case "--goarm":
skipNext = true
targetGOARM = params[i+1]
case "-h":
showUsage(os.Stdout)
return
@ -440,6 +513,8 @@ func main() {
}
}
verbosePrintf("detected Go version %v\n", ver)
if len(buildTags) == 0 {
verbosePrintf("adding build-tag release\n")
buildTags = []string{"release"}
@ -456,9 +531,11 @@ func main() {
die("Getwd(): %v\n", err)
}
gopath, err := ioutil.TempDir("", fmt.Sprintf("%v-build-", config.Name))
if err != nil {
die("TempDir(): %v\n", err)
if gopath == "" {
gopath, err = ioutil.TempDir("", fmt.Sprintf("%v-build-", config.Name))
if err != nil {
die("TempDir(): %v\n", err)
}
}
verbosePrintf("create GOPATH at %v\n", gopath)
@ -514,7 +591,7 @@ func main() {
"-o", output, config.Main,
}
err = build(filepath.Join(gopath, "src"), targetGOOS, targetGOARCH, gopath, args...)
err = build(filepath.Join(gopath, "src"), ver, targetGOOS, targetGOARCH, targetGOARM, gopath, args...)
if err != nil {
die("build failed: %v\n", err)
}

View file

@ -2,9 +2,9 @@ Enhancement: Create subdirs below `data/` for local/sftp backends
The local and sftp backends now create the subdirs below `data/` on
open/init. This way, restic makes sure that they always exist. This is
connected to an issue for the sftp server:
connected to an issue for the sftp server.
https://github.com/restic/rest-server/pull/11#issuecomment-309879710
https://github.com/restic/restic/issues/1055
https://github.com/restic/rest-server/pull/11#issuecomment-309879710
https://github.com/restic/restic/pull/1077
https://github.com/restic/restic/pull/1105

View file

@ -1,6 +1,6 @@
Enhancement: Add `migrate` cmd to migrate from `s3legacy` to `default` layout
The `migrate` command for chaning the `s3legacy` layout to the `default` layout
The `migrate` command for changing the `s3legacy` layout to the `default` layout
for s3 backends has been improved: It can now be restarted with `restic migrate
--force s3_layout` and automatically retries operations on error.

View file

@ -1,3 +1,3 @@
Enhancement: Properly report errors when reading files with exclude patterns.
Enhancement: Properly report errors when reading files with exclude patterns
https://github.com/restic/restic/pull/1144

View file

@ -0,0 +1,8 @@
Enhancement: Add support for TLS client certificate authentication
Support has been added for using a TLS client certificate for authentication to
HTTP based backend. A file containing the PEM encoded private key and
certificate can be set using the `--tls-client-cert` option.
https://github.com/restic/restic/issues/1522
https://github.com/restic/restic/pull/1524

View file

@ -0,0 +1,7 @@
Enhancement: Reduce number of remote requests during repository check
This change eliminates redundant remote repository calls and significantly
improves repository check time.
https://github.com/restic/restic/issues/1541
https://github.com/restic/restic/pull/1548

View file

@ -0,0 +1,9 @@
Enhancement: Reduce number of backend requests for rebuild-index and prune
We've found a way to reduce then number of backend requests for the
`rebuild-index` and `prune` operations. This significantly speeds up the
operations for high-latency backends.
https://github.com/restic/restic/issues/1567
https://github.com/restic/restic/pull/1574
https://github.com/restic/restic/pull/1575

View file

@ -0,0 +1,7 @@
Bugfix: Strip spaces for lines read via --files-from
Leading and trailing spaces in lines read via `--files-from` are now stripped,
so it behaves the same as with lines read via `--exclude-file`.
https://github.com/restic/restic/issues/1590
https://github.com/restic/restic/pull/1613

View file

@ -0,0 +1,7 @@
Enhancement: Speed up querying across indices and scanning existing files
This change increases the whenever a blob (part of a file) is searched for in a
restic repository. This will reduce cpu usage some when backing up files already
backed up by restic. Cpu usage is further decreased when scanning files.
https://github.com/restic/restic/pull/1549

View file

@ -0,0 +1,7 @@
Enhancement: fuse/mount: Correctly handle EOF, add template option
We've added the `--snapshot-template` string, which can be used to specify a
template for a snapshot directory. In addition, accessing data after the end of
a file via the fuse mount is now handled correctly.
https://github.com/restic/restic/pull/1554

View file

@ -0,0 +1,10 @@
Enhancement: Don't terminate ssh on SIGINT
We've reworked the code which runs the `ssh` login for the sftp backend so that
it can prompt for a password (if needed) but does not exit when the user
presses CTRL+C (SIGINT) e.g. during backup. This allows restic to properly shut
down when it receives SIGINT and remove the lock file from the repo, afterwards
exiting the `ssh` process.
https://github.com/restic/restic/pull/1564
https://github.com/restic/restic/pull/1588

View file

@ -0,0 +1,3 @@
Enhancement: Retry Backend.List() in case of errors
https://github.com/restic/restic/pull/1579

View file

@ -0,0 +1,12 @@
Enhancement: Limit index file size
Before, restic would create a single new index file on `prune` or
`rebuild-index`, this may lead to memory problems when this huge index is
created and loaded again. We're now limiting the size of the index file, and
split newly created index files into several smaller ones. This allows restic
to be more memory-efficient.
https://github.com/restic/restic/pull/1584
https://github.com/restic/restic/issues/1412
https://github.com/restic/restic/issues/979
https://github.com/restic/restic/issues/526

View file

@ -0,0 +1,17 @@
Bugfix: Complete intermediate index upload
After a user posted a comprehensive report of what he observed, we were able to
find a bug and correct it: During backup, restic uploads so-called
"intermediate" index files. When the backup finishes during a transfer of such
an intermediate index, the upload is cancelled, but the backup is finished
without an error. This leads to an inconsistent state, where the snapshot
references data that is contained in the repo, but is not referenced in any
index.
The situation can be resolved by building a new index with `rebuild-index`, but
looks very confusing at first. Since all the data got uploaded to the repo
successfully, there was no risk of data loss, just minor inconvenience for our
users.
https://github.com/restic/restic/pull/1589
https://forum.restic.net/t/error-loading-tree-check-prune-and-forget-gives-error-b2-backend/406

View file

@ -0,0 +1,7 @@
Bugfix: Google Cloud Storage: Use generic HTTP transport
It was discovered that the Google Cloud Storage backend did not use the generic
HTTP transport, so things such as bandwidth limiting with `--limit-upload` did
not work. This is resolved now.
https://github.com/restic/restic/pull/1594

View file

@ -0,0 +1,11 @@
Bugfix: backup: Remove bandwidth display
This commit removes the bandwidth displayed during backup process. It is
misleading and seldomly correct, because it's neither the "read
bandwidth" (only for the very first backup) nor the "upload bandwidth".
Many users are confused about (and rightly so), c.f. #1581, #1033, #1591
We'll eventually replace this display with something more relevant when
the new archiver code is ready.
https://github.com/restic/restic/pull/1595

View file

@ -0,0 +1,8 @@
Enhancement: Add --read-data-subset flag to check command
This change introduces ability to check integrity of a subset of repository
data packs. This can be used to spread integrity check of larger repositories
over a period of time.
https://github.com/restic/restic/issues/1497
https://github.com/restic/restic/pull/1556

View file

@ -0,0 +1,7 @@
Bugfix: Fixed unexpected 'pack file cannot be listed' error
Due to a regression introduced in 0.8.2, the `rebuild-index` and `prune`
commands failed to read pack files with size of 587, 588, 589 or 590 bytes.
https://github.com/restic/restic/issues/1633
https://github.com/restic/restic/pull/1635

View file

@ -0,0 +1,10 @@
Bugfix: Ignore files with invalid names in the repo
The release 0.8.2 introduced a bug: when restic encounters files in the repo
which do not have a valid name, it tries to load a file with a name of lots of
zeroes instead of ignoring it. This is now resolved, invalid file names are
just ignored.
https://github.com/restic/restic/issues/1641
https://github.com/restic/restic/pull/1643
https://forum.restic.net/t/help-fixing-repo-no-such-file/485/3

Some files were not shown because too many files have changed in this diff Show more