David Mazieres 7 years ago
parent
commit
7791ca2a75
3 changed files with 225 additions and 0 deletions
  1. 1
    0
      .gitignore
  2. 6
    0
      Makefile.am
  3. 218
    0
      muchsync.1.md

+ 1
- 0
.gitignore View File

@@ -13,5 +13,6 @@
13 13
 /install-sh
14 14
 /missing
15 15
 /muchsync
16
+/muchsync.1
16 17
 Makefile
17 18
 Makefile.in

+ 6
- 0
Makefile.am View File

@@ -12,3 +12,9 @@ CLEANFILES = *~
12 12
 maintainer-clean-local:
13 13
 	+@echo rm -rf `sed -ne 's!^/!!p' .gitignore` Makefile.in
14 14
 	rm -rf `sed -ne 's!^/!!p' .gitignore` Makefile.in
15
+
16
+muchsync.1: muchsync.1.md
17
+	pandoc -s -w man muchsync.1.md -o muchsync.1
18
+
19
+man_MANS = muchsync.1
20
+EXTRA_DIST = muchsync.1.md $(man_MANS)

+ 218
- 0
muchsync.1.md View File

@@ -0,0 +1,218 @@
1
+% muchsync(1)
2
+% David Mazieres
3
+% 
4
+
5
+# NAME
6
+
7
+inktool - synchronize maildirs and notmuch databases
8
+
9
+# SYNOPSIS
10
+
11
+muchsync _options_ \
12
+muchsync _options_ _server-name_ _server-options_ \
13
+muchsync _options_ --init _maildir_ _server-name_ _server-options_
14
+
15
+# DESCRIPTION
16
+
17
+muchsync synchronizes the contents of maildirs and notmuch tags across
18
+machines.  Any given execution runs pairwise between two replicas, but
19
+the system scales to an arbitrary number of replicas synchronizing in
20
+arbitrary pairs.  For efficiency, version vectors and logical
21
+timestamps are used to limit synchronization to items a peer may not
22
+yet know about.
23
+
24
+Since muchsync replicates the tags in the notmuch database itself, it
25
+is recommended that you disable maildir flag synchronization by
26
+executing:
27
+
28
+    notmuch config set maildir.synchronize_flags=false
29
+
30
+One reason is that muchsync itself makes no effort to rename files to
31
+reflect tags.  Another reason is that the synchronize_flags feature
32
+only works on a small subset of pre-defined flags and so is not all
33
+that useful.  Moreover it marks flags by renaming files, which is not
34
+particularly efficient.  muchsync was largely motivated by the need
35
+for better flag synchronization.  If you are satisfied with the
36
+synchronize_flags feature, you might also be able to use a tool such
37
+as offlineimap instead of muchsync.
38
+
39
+muchsync separately synchronizes two classes of information:  the
40
+message-to-directory mapping (henceforth link counts) and the
41
+message-id-to-tag mapping (henceforth tags).  Using logical
42
+timestamps, it can detect update conflicts for each type of
43
+information.  We describe link count and tag synchronization in turn.
44
+
45
+Link count synchronization consists of ensuring that any given message
46
+(identified by its collision-resistant content hash) appears the same
47
+number of times in the same subdirectories on each replica.  Generally
48
+a message will appear only once in a single subdirectory.  However, if
49
+the message is moved or deleted on one replica, this will propagate to
50
+other replicas.
51
+
52
+If two replicas move or copy the same file between synchronization
53
+events (or one moves the file and the other deletes it), this
54
+constitutes an update conflict.  Update conflicts are resolved by
55
+storing in each subdirectory a number of copies equal to the maximum
56
+of the number of copies in that subdirectory on the two replicas.
57
+This is conservative, in the sense that a file will never be deleted
58
+after an update, but you may get extra copies of files.
59
+
60
+For example, if one replica moves a message to subdirectory .box1/cur
61
+and another moves the same message to subdirectory .box2/cur, the
62
+conflict will be resolved by placing two links to the message on each
63
+replica, one in .box1/cur and one in .box2/cur.  To respect the
64
+structure of maildirs, subdirectories ending `new` and `cur` are
65
+special-cased; conflicts between sibling `new` and `cur`
66
+subdirectories are resolved in favor of `cur` without creating
67
+additional copies of messages.
68
+
69
+Message tags are synchronized based on notmuch's message-ID (usually
70
+the Message-ID header of a message), rather than message contents.  On
71
+conflict, tags are combined combined as follows.  Any tag in the
72
+notmuch configuration parameter `new.tags` is removed from the message
73
+unless it appears on both replicas.  Any other tag is added if it
74
+appears on any replica.  In other words, tags in `new.tags` are
75
+logically anded, while all other flags are logically ored.  (This
76
+approach will give the most predictable results if `new.tags` has the
77
+same value in all your replicas.  The `--init` option ensures this
78
+initially, but subsequent changes to `new.tags` must be manually
79
+propagated.)
80
+
81
+
82
+# OPTIONS
83
+
84
+\-C _file_, \--config _file_
85
+:   Specify the path of the notmuch configuration file to use.  If
86
+    none is specified, the default is to use the contents of the
87
+    environment variable \$NOTMUCH_CONFIG, or if that variable is
88
+    unset, the value \$HOME/.notmuch-config.  (These are the same
89
+    defaults as the notmuch command itself.)
90
+
91
+\-F
92
+:   Check for modified files.  Without this option, muchsync assumes
93
+    that files in a maildir are never edited.  -F disables certain
94
+    optimizations so as to make muchsync at least check the timestamp
95
+    on every file, which will detect modified files at the cost of a
96
+    longer startup time.
97
+
98
+\-r /path/to/muchsync
99
+:   Specifies the path to muchsync on the server.  Ordinarily, muchsync
100
+    should be in the default path on the server and this option should
101
+    not be required.  However, this option is useful if you have to
102
+    install muchsync in a non-standard place, or which to test
103
+    development versions of the code.
104
+
105
+\-s ssh-cmd
106
+:   Specifies a command line to pass to /bin/sh to execute a command on
107
+    another machine.  The default value is "ssh -CTaxq".  Note that
108
+    because this string is passed to the shell, special characters
109
+    including spaces may need to be escaped.
110
+
111
+\-v
112
+:   The -v option increases verbosity.  The more times it is specified,
113
+    the more verbose muchsync will become.
114
+
115
+\--help
116
+:   Print a brief summary of muchsync's command-line options.
117
+
118
+\--init _maildir_
119
+:   This option clones an existing mailbox on a remote server into
120
+    _maildir_ on the local machine.  Neither _maildir_ nor your
121
+    notmuch configuration file (see ```--config``` above) should exist
122
+    when you run this command, as both will be created.  The
123
+    configuration file is copied from the server (adjusted to reflect
124
+    the local maildir), while _maildir_ is created as a replica of the
125
+    maildir you have on the server.
126
+
127
+\--new
128
+:   This command says to run "notmuch new" before starting the muchsync
129
+    operation.  It can be passed as either a client or a server
130
+    option.  For example:  The command "```muchsync myserver --new```"
131
+    will first run "notmuch new" on myserver, then synchronize the
132
+    local maildir with the maildir on the server.
133
+
134
+\--noup, \--noupload
135
+:   Transfer files from the server to the client, but not vice versa.
136
+
137
+\--upbg
138
+:   Transfer files from the server to the client in the foreground.
139
+    Then fork into the background to upload any new files from the
140
+    client to the server.  This option is useful when checking new
141
+    mail, if you want to begin reading your mail as soon as it has
142
+    been downloaded while the upload continues.
143
+
144
+\--version
145
+:   Report on the muchsync version number
146
+
147
+# EXAMPLES
148
+
149
+    muchsync -vv --new
150
+
151
+Run "notmuch new", then build the initial muchsync database on a
152
+server.  This command may take several minutes the first time it is
153
+run, as it must compute a content hash of every message in the
154
+database.
155
+
156
+    muchsync --init ~/maildir myserver --new
157
+
158
+First run "notmuch new" on myserver, then create a replica of your
159
+mailbox on myserver on the local machine in the directory ~/maildir.
160
+Note that neither your configuration file (by default
161
+~/.notmuch-config) nor ~/maildir should exist before running this
162
+command, as both will be created.  Moreover, it is recommended that
163
+you set maildir.synchronize_flags=false on the server before running
164
+this command, since this setting will be copied to the client.
165
+
166
+# FILES
167
+
168
+muchsync keeps all of its state in a subdirectory of your top maildir
169
+called ```.notmuch/muchsync```.
170
+
171
+# SEE ALSO
172
+
173
+notmuch(1).
174
+
175
+# BUGS
176
+
177
+muchsync never deletes directories.  If you want to remove a
178
+subdirectory completely, you must manually execute rmdir on all
179
+replicas.  Even if you manually delete a subdirectory, it will live on
180
+in the notmuch database.
181
+
182
+To synchronize deletions and re-creations properly, muchsync never
183
+deletes content hashes and their message IDs from its database, even
184
+after the last copy of a message has disappeared.  Such stale hashes
185
+should not consume an inordinate amount of disk space, but could
186
+conceivably pose a privacy risk if users believe deleting a message
187
+removes all traces of it.
188
+
189
+Message tags are synchronized based on notmuch's message-ID (usually
190
+the Message-ID header of a message), rather than based on message
191
+contents.  This is slightly strange because very different messages
192
+can have the same Message-ID header, meaning the user will likely only
193
+read one of many messages bearing the same Message-ID header.  It is
194
+conceivable that an attacker could suppress a message from a mailing
195
+list by sending another message with the same Message-ID.  This bug is
196
+in the design of notmuch, and hence not something that muchsync can
197
+work around.  muchsync does ensure that all versions of a message are
198
+propagated everywhere, so any tools used to work around the problem
199
+should work on all replicas.
200
+
201
+Because notmuch and Xapian do not keep any kind of modification time
202
+on database entries, every invocation of muchsync requires a complete
203
+scan of all tags in the Xapian database to detect any changed tags.
204
+Fortunately muchsync heavily optimizes the scan so that it should take
205
+well under a second for 100,000 mail messages.  However, this means
206
+that interfaces such as those used by notmuch-dump are not efficient
207
+enough (see the next paragraph).
208
+
209
+muchsync makes certain assumptions about the structure of notmuch's
210
+private types `notmuch_message_t` and `notmuch_directory_t`.  In
211
+particular, it assumes that the Xapian document ID is the second field
212
+of these the data structures.  Sadly, there is no efficient and clean
213
+way to extract this information from the notmuch library interface.
214
+muchsync also makes other assumptions about how tokens are named in
215
+the Xapian database.  These assumptions are necessary because the
216
+notmuch library interface and the notmuch dump utility are too slow to
217
+support synchronization every time you check mail.
218
+

Loading…
Cancel
Save