On Wed, 07 Jun 2006 09:49:13 +1200, "Roland Pope" <rpope@xxxxxxxxxxxxx> said: > David Korpiewski wrote: > > > You cannot replicate from one master to more than one replica. > However, you should be able to replicate from master to replica, then > from replica to another replica. > > Ie. Master->Replica1->Replica2 > > I realise this will introduce a small delay in the replication from > Master to Replica2, and will rely on replica1 being available, but it > should still be a valid option. Unless sync_server is logging all its actions to sync_client then this is a bit pointless, Replica2 will never have anything automatically sent to it. A quick grep of my 2.3.6 source directory suggests that it doesn't. There's no technical reason for sync_client not to be extended to support syncing to two different backend servers though (other than it being a mess of global communications variables, but I was already thinking about writing a patch to fix that). You would want to make sure that a failure on either replica didn't stop the other one running, and possibly do something like copy the log file twice, once for each backend - then spawn separate processes managing the actual sync... My problem at the moment is that if you can't contact the replica then the master doesn't even [expletive] start, which is just insane. I've attached a patch that fixes that in a pretty hacky way - but still has global connection objects. I don't think it's worthy of inclusion upstream until I make it more flexible. This patch also includes a command line option '-o' to sync_client which allows you to run it from the command line even if the replica isn't running and it will only attempt to connect once rather than spend 1000 seconds backing off trying to get a connection. I'm using that option in the master shutdown script - after closing down the server I run: if [ -f /var/cyrus/$HOST/sync/log ]; then sudo -u cyrus /usr/cyrus/bin/sync_client -C /etc/imapd-$HOST.conf \ -o -r -f /var/cyrus/$HOST/sync/log fi (equivalent to the more simplistic: sync_client -o -r -f /var/imap/sync/log) and depending on the exit code of sync_client above I know if I can consider the two copies "identical" when starting up again or if I need to force the next master start to be on the same host. This assumes replication is 100% reliable, which isn't really true, sync_client is still bailing out far too often. It's a start though. The cyrus init scripts also start up IP addresses (using Heartbeat's IPAddr2) which cyrus binds to. This allows us to run multiple master servers on the same host if necessary, just by switching over a replica, and also to keep contacting a particular named server via its IP address regardless of which replica host is actually running the master at that time. (NOTE: none of this is actually fully working yet, but it's starting to look pretty good. I got it up and running with heartbeat management yesterday on my test bed!) Bron. -- Bron Gondwana brong@xxxxxxxxxxx
diff -ur cyrus-imapd-2.3.6.orig/imap/sync_client.c cyrus-imapd-2.3.6/imap/sync_client.c --- cyrus-imapd-2.3.6.orig/imap/sync_client.c 2006-04-04 14:10:54.000000000 -0400 +++ cyrus-imapd-2.3.6/imap/sync_client.c 2006-06-06 03:25:59.000000000 -0400 @@ -110,15 +110,16 @@ extern char *optarg; extern int optind; -static struct protstream *toserver = NULL; -static struct protstream *fromserver = NULL; - /* List/Hash of messageIDs that are available on server */ static struct sync_msgid_list *msgid_onserver = NULL; static struct namespace sync_namespace; static struct auth_state *sync_authstate = NULL; +static struct backend *be = NULL; +static const char *servername = NULL; +static int connect_once = 0; + static int verbose = 0; static int verbose_logging = 0; @@ -155,10 +156,10 @@ static int send_lock() { - prot_printf(toserver, "LOCK\r\n"); - prot_flush(toserver); + prot_printf(be->out, "LOCK\r\n"); + prot_flush(be->out); - return(sync_parse_code("LOCK", fromserver, SYNC_PARSE_EAT_OKLINE, NULL)); + return(sync_parse_code("LOCK", be->in, SYNC_PARSE_EAT_OKLINE, NULL)); } static int send_unlock() @@ -167,18 +168,18 @@ int c = ' '; static struct buf token; /* BSS */ - prot_printf(toserver, "UNLOCK\r\n"); - prot_flush(toserver); + prot_printf(be->out, "UNLOCK\r\n"); + prot_flush(be->out); - r = sync_parse_code("UNLOCK", fromserver, SYNC_PARSE_NOEAT_OKLINE, NULL); + r = sync_parse_code("UNLOCK", be->in, SYNC_PARSE_NOEAT_OKLINE, NULL); if (r) return(r); - if ((c = getword(fromserver, &token)) != ' ') { - eatline(fromserver, c); + if ((c = getword(be->in, &token)) != ' ') { + eatline(be->in, c); syslog(LOG_ERR, "Garbage on Unlock response"); return(IMAP_PROTOCOL_ERROR); } - eatline(fromserver, c); + eatline(be->in, c); /* Clear out msgid_on_server list if server restarted */ if (!strcmp(token.s, "[RESTART]")) { @@ -309,42 +310,42 @@ static struct buf arg; int r = 0, unsolicited, c; - prot_printf(toserver, "RESERVE "); - sync_printastring(toserver, folder->name); + prot_printf(be->out, "RESERVE "); + sync_printastring(be->out, folder->name); for (msg = folder->msglist->head ; msg ; msg = msg->next) { msgid = sync_msgid_lookup(reserve_msgid_list, &msg->uuid); if (msgid && !msgid->reserved) { /* Attempt to Reserve message in this folder */ - prot_printf(toserver, " "); - sync_printastring(toserver, message_uuid_text(&msgid->uuid)); + prot_printf(be->out, " "); + sync_printastring(be->out, message_uuid_text(&msgid->uuid)); } } - prot_printf(toserver, "\r\n"); - prot_flush(toserver); + prot_printf(be->out, "\r\n"); + prot_flush(be->out); - r = sync_parse_code("RESERVE", fromserver, + r = sync_parse_code("RESERVE", be->in, SYNC_PARSE_EAT_OKLINE, &unsolicited); /* Parse response to record successfully reserved messages */ while (!r && unsolicited) { struct message_uuid tmp_uuid; - c = getword(fromserver, &arg); + c = getword(be->in, &arg); if (c == '\r') - c = prot_getc(fromserver); + c = prot_getc(be->in); if (c != '\n') { syslog(LOG_ERR, "Illegal response to RESERVE: %s", arg.s); - sync_eatlines_unsolicited(fromserver, c); + sync_eatlines_unsolicited(be->in, c); return(IMAP_PROTOCOL_ERROR); } if (!message_uuid_from_text(&tmp_uuid, arg.s)) { syslog(LOG_ERR, "Illegal response to RESERVE: %s", arg.s); - sync_eatlines_unsolicited(fromserver, c); + sync_eatlines_unsolicited(be->in, c); return(IMAP_PROTOCOL_ERROR); } @@ -356,7 +357,7 @@ syslog(LOG_ERR, "RESERVE: Unexpected response MessageID %s in %s", arg.s, folder->name); - r = sync_parse_code("RESERVE", fromserver, + r = sync_parse_code("RESERVE", be->in, SYNC_PARSE_EAT_OKLINE, &unsolicited); } return(r); @@ -581,12 +582,12 @@ static int user_reset(char *user) { - prot_printf(toserver, "RESET "); - sync_printastring(toserver, user); - prot_printf(toserver, "\r\n"); - prot_flush(toserver); + prot_printf(be->out, "RESET "); + sync_printastring(be->out, user); + prot_printf(be->out, "\r\n"); + prot_flush(be->out); - return(sync_parse_code("RESET", fromserver, SYNC_PARSE_EAT_OKLINE, NULL)); + return(sync_parse_code("RESET", be->in, SYNC_PARSE_EAT_OKLINE, NULL)); } /* ====================================================================== */ @@ -598,24 +599,24 @@ static struct buf uniqueid; static struct buf lastuid; - prot_printf(toserver, "SELECT "); - sync_printastring(toserver, name); - prot_printf(toserver, "\r\n"); - prot_flush(toserver); + prot_printf(be->out, "SELECT "); + sync_printastring(be->out, name); + prot_printf(be->out, "\r\n"); + prot_flush(be->out); - r = sync_parse_code("SELECT", fromserver, SYNC_PARSE_NOEAT_OKLINE, NULL); + r = sync_parse_code("SELECT", be->in, SYNC_PARSE_NOEAT_OKLINE, NULL); if (r) return(r); - if ((c = getword(fromserver, &uniqueid)) != ' ') { - eatline(fromserver, c); + if ((c = getword(be->in, &uniqueid)) != ' ') { + eatline(be->in, c); syslog(LOG_ERR, "Garbage on Select response"); return(IMAP_PROTOCOL_ERROR); } - c = getword(fromserver, &lastuid); - if (c == '\r') c = prot_getc(fromserver); + c = getword(be->in, &lastuid); + if (c == '\r') c = prot_getc(be->in); if (c != '\n') { - eatline(fromserver, c); + eatline(be->in, c); syslog(LOG_ERR, "Garbage on Select response"); return(IMAP_PROTOCOL_ERROR); } @@ -631,43 +632,43 @@ static int folder_create(char *name, char *part, char *uniqueid, char *acl, unsigned long uidvalidity) { - prot_printf(toserver, "CREATE "); - sync_printastring(toserver, name); - prot_printf(toserver, " "); - sync_printastring(toserver, part); - prot_printf(toserver, " "); - sync_printastring(toserver, uniqueid); - prot_printf(toserver, " "); - sync_printastring(toserver, acl); - prot_printf(toserver, " "); - prot_printf(toserver, "0"); - prot_printf(toserver, " "); - prot_printf(toserver, "%lu\r\n", uidvalidity); - prot_flush(toserver); + prot_printf(be->out, "CREATE "); + sync_printastring(be->out, name); + prot_printf(be->out, " "); + sync_printastring(be->out, part); + prot_printf(be->out, " "); + sync_printastring(be->out, uniqueid); + prot_printf(be->out, " "); + sync_printastring(be->out, acl); + prot_printf(be->out, " "); + prot_printf(be->out, "0"); + prot_printf(be->out, " "); + prot_printf(be->out, "%lu\r\n", uidvalidity); + prot_flush(be->out); - return(sync_parse_code("CREATE", fromserver, SYNC_PARSE_EAT_OKLINE, NULL)); + return(sync_parse_code("CREATE", be->in, SYNC_PARSE_EAT_OKLINE, NULL)); } static int folder_rename(char *oldname, char *newname) { - prot_printf(toserver, "RENAME "); - sync_printastring(toserver, oldname); - prot_printf(toserver, " "); - sync_printastring(toserver, newname); - prot_printf(toserver, "\r\n"); - prot_flush(toserver); + prot_printf(be->out, "RENAME "); + sync_printastring(be->out, oldname); + prot_printf(be->out, " "); + sync_printastring(be->out, newname); + prot_printf(be->out, "\r\n"); + prot_flush(be->out); - return(sync_parse_code("RENAME", fromserver, SYNC_PARSE_EAT_OKLINE, NULL)); + return(sync_parse_code("RENAME", be->in, SYNC_PARSE_EAT_OKLINE, NULL)); } static int folder_delete(char *name) { - prot_printf(toserver, "DELETE "); - sync_printastring(toserver, name); - prot_printf(toserver, "\r\n"); - prot_flush(toserver); + prot_printf(be->out, "DELETE "); + sync_printastring(be->out, name); + prot_printf(be->out, "\r\n"); + prot_flush(be->out); - return(sync_parse_code("DELETE", fromserver, SYNC_PARSE_EAT_OKLINE, NULL)); + return(sync_parse_code("DELETE", be->in, SYNC_PARSE_EAT_OKLINE, NULL)); } static int user_addsub(char *user, char *name) @@ -678,14 +679,14 @@ if (verbose_logging) syslog(LOG_INFO, "ADDSUB %s %s", user, name); - prot_printf(toserver, "ADDSUB "); - sync_printastring(toserver, user); - prot_printf(toserver, " "); - sync_printastring(toserver, name); - prot_printf(toserver, "\r\n"); - prot_flush(toserver); + prot_printf(be->out, "ADDSUB "); + sync_printastring(be->out, user); + prot_printf(be->out, " "); + sync_printastring(be->out, name); + prot_printf(be->out, "\r\n"); + prot_flush(be->out); - return(sync_parse_code("ADDSUB", fromserver, SYNC_PARSE_EAT_OKLINE, NULL)); + return(sync_parse_code("ADDSUB", be->in, SYNC_PARSE_EAT_OKLINE, NULL)); } static int user_delsub(char *user, char *name) @@ -696,44 +697,44 @@ if (verbose_logging) syslog(LOG_INFO, "DELSUB %s %s", user, name); - prot_printf(toserver, "DELSUB "); - sync_printastring(toserver, user); - prot_printf(toserver, " "); - sync_printastring(toserver, name); - prot_printf(toserver, "\r\n"); - prot_flush(toserver); + prot_printf(be->out, "DELSUB "); + sync_printastring(be->out, user); + prot_printf(be->out, " "); + sync_printastring(be->out, name); + prot_printf(be->out, "\r\n"); + prot_flush(be->out); - return(sync_parse_code("DELSUB", fromserver, SYNC_PARSE_EAT_OKLINE, NULL)); + return(sync_parse_code("DELSUB", be->in, SYNC_PARSE_EAT_OKLINE, NULL)); } static int folder_setacl(char *name, char *acl) { - prot_printf(toserver, "SETACL "); - sync_printastring(toserver, name); - prot_printf(toserver, " "); - sync_printastring(toserver, acl); - prot_printf(toserver, "\r\n"); - prot_flush(toserver); + prot_printf(be->out, "SETACL "); + sync_printastring(be->out, name); + prot_printf(be->out, " "); + sync_printastring(be->out, acl); + prot_printf(be->out, "\r\n"); + prot_flush(be->out); - return(sync_parse_code("SETACL", fromserver, SYNC_PARSE_EAT_OKLINE, NULL)); + return(sync_parse_code("SETACL", be->in, SYNC_PARSE_EAT_OKLINE, NULL)); } static int folder_setannotation(char *name, char *entry, char *userid, char *value) { - prot_printf(toserver, "SETANNOTATION "); - sync_printastring(toserver, name); - prot_printf(toserver, " "); - sync_printastring(toserver, entry); - prot_printf(toserver, " "); - sync_printastring(toserver, userid); - prot_printf(toserver, " "); - if (value) sync_printastring(toserver, value); - else prot_printf(toserver, "NIL"); - prot_printf(toserver, "\r\n"); - prot_flush(toserver); + prot_printf(be->out, "SETANNOTATION "); + sync_printastring(be->out, name); + prot_printf(be->out, " "); + sync_printastring(be->out, entry); + prot_printf(be->out, " "); + sync_printastring(be->out, userid); + prot_printf(be->out, " "); + if (value) sync_printastring(be->out, value); + else prot_printf(be->out, "NIL"); + prot_printf(be->out, "\r\n"); + prot_flush(be->out); - return(sync_parse_code("SETANNOTATION", fromserver, + return(sync_parse_code("SETANNOTATION", be->in, SYNC_PARSE_EAT_OKLINE, NULL)); } @@ -748,63 +749,63 @@ return(IMAP_IOERROR); } - prot_printf(toserver, "UPLOAD_SIEVE "); - sync_printastring(toserver, user); - prot_printf(toserver, " "); - sync_printastring(toserver, name); - prot_printf(toserver, " %lu {%lu+}\r\n", last_update, size); + prot_printf(be->out, "UPLOAD_SIEVE "); + sync_printastring(be->out, user); + prot_printf(be->out, " "); + sync_printastring(be->out, name); + prot_printf(be->out, " %lu {%lu+}\r\n", last_update, size); s = sieve; while (size) { - prot_putc(*s, toserver); + prot_putc(*s, be->out); s++; size--; } - prot_printf(toserver,"\r\n"); + prot_printf(be->out,"\r\n"); free(sieve); - prot_flush(toserver); + prot_flush(be->out); return(sync_parse_code("UPLOAD_SIEVE", - fromserver, SYNC_PARSE_EAT_OKLINE, NULL)); + be->in, SYNC_PARSE_EAT_OKLINE, NULL)); return(1); } static int sieve_delete(char *user, char *name) { - prot_printf(toserver, "DELETE_SIEVE "); - sync_printastring(toserver, user); - prot_printf(toserver, " "); - sync_printastring(toserver, name); - prot_printf(toserver, "\r\n"); - prot_flush(toserver); + prot_printf(be->out, "DELETE_SIEVE "); + sync_printastring(be->out, user); + prot_printf(be->out, " "); + sync_printastring(be->out, name); + prot_printf(be->out, "\r\n"); + prot_flush(be->out); return(sync_parse_code("DELETE_SIEVE", - fromserver, SYNC_PARSE_EAT_OKLINE, NULL)); + be->in, SYNC_PARSE_EAT_OKLINE, NULL)); } static int sieve_activate(char *user, char *name) { - prot_printf(toserver, "ACTIVATE_SIEVE "); - sync_printastring(toserver, user); - prot_printf(toserver, " "); - sync_printastring(toserver, name); - prot_printf(toserver, "\r\n"); - prot_flush(toserver); + prot_printf(be->out, "ACTIVATE_SIEVE "); + sync_printastring(be->out, user); + prot_printf(be->out, " "); + sync_printastring(be->out, name); + prot_printf(be->out, "\r\n"); + prot_flush(be->out); return(sync_parse_code("ACTIVATE_SIEVE", - fromserver, SYNC_PARSE_EAT_OKLINE, NULL)); + be->in, SYNC_PARSE_EAT_OKLINE, NULL)); } static int sieve_deactivate(char *user) { - prot_printf(toserver, "DEACTIVATE_SIEVE "); - sync_printastring(toserver, user); - prot_printf(toserver, "\r\n"); - prot_flush(toserver); + prot_printf(be->out, "DEACTIVATE_SIEVE "); + sync_printastring(be->out, user); + prot_printf(be->out, "\r\n"); + prot_flush(be->out); return(sync_parse_code("DEACTIVATE_SIEVE", - fromserver, SYNC_PARSE_EAT_OKLINE, NULL)); + be->in, SYNC_PARSE_EAT_OKLINE, NULL)); } /* ====================================================================== */ @@ -822,13 +823,13 @@ if (server && (client->limit == server->limit)) return(0); - prot_printf(toserver, "SETQUOTA "); - sync_printastring(toserver, client->root); + prot_printf(be->out, "SETQUOTA "); + sync_printastring(be->out, client->root); - prot_printf(toserver, " %d\r\n", client->limit); - prot_flush(toserver); + prot_printf(be->out, " %d\r\n", client->limit); + prot_flush(be->out); - return(sync_parse_code("SETQUOTA",fromserver,SYNC_PARSE_EAT_OKLINE,NULL)); + return(sync_parse_code("SETQUOTA",be->in,SYNC_PARSE_EAT_OKLINE,NULL)); } /* ====================================================================== */ @@ -959,38 +960,38 @@ continue; if (!have_update) { - prot_printf(toserver, "SETFLAGS"); + prot_printf(be->out, "SETFLAGS"); have_update = 1; } - prot_printf(toserver, " %lu (", record.uid); + prot_printf(be->out, " %lu (", record.uid); flags_printed = 0; if (record.system_flags & FLAG_DELETED) - sync_flag_print(toserver, &flags_printed,"\\deleted"); + sync_flag_print(be->out, &flags_printed,"\\deleted"); if (record.system_flags & FLAG_ANSWERED) - sync_flag_print(toserver, &flags_printed,"\\answered"); + sync_flag_print(be->out, &flags_printed,"\\answered"); if (record.system_flags & FLAG_FLAGGED) - sync_flag_print(toserver,&flags_printed, "\\flagged"); + sync_flag_print(be->out,&flags_printed, "\\flagged"); if (record.system_flags & FLAG_DRAFT) - sync_flag_print(toserver,&flags_printed, "\\draft"); + sync_flag_print(be->out,&flags_printed, "\\draft"); for (flag = 0 ; flag < MAX_USER_FLAGS ; flag++) { if (mailbox->flagname[flag] && (record.user_flags[flag/32] & (1<<(flag&31)) )) - sync_flag_print(toserver, &flags_printed, + sync_flag_print(be->out, &flags_printed, mailbox->flagname[flag]); } - prot_printf(toserver, ")"); + prot_printf(be->out, ")"); } if (!have_update) return(0); - prot_printf(toserver, "\r\n"); - prot_flush(toserver); + prot_printf(be->out, "\r\n"); + prot_flush(be->out); - return(sync_parse_code("SETFLAGS",fromserver,SYNC_PARSE_EAT_OKLINE,NULL)); + return(sync_parse_code("SETFLAGS",be->in,SYNC_PARSE_EAT_OKLINE,NULL)); } /* ====================================================================== */ @@ -1035,9 +1036,9 @@ /* Expunge messages on server which do not exist on client */ while (msg && (record.uid > msg->uid)) { if (count++ == 0) - prot_printf(toserver, "EXPUNGE"); + prot_printf(be->out, "EXPUNGE"); - prot_printf(toserver, " %lu", msg->uid); + prot_printf(be->out, " %lu", msg->uid); msg = msg->next; } @@ -1049,9 +1050,9 @@ /* Expunge messages on server which do not exist on client */ while (msg) { if (count++ == 0) - prot_printf(toserver, "EXPUNGE"); + prot_printf(be->out, "EXPUNGE"); - prot_printf(toserver, " %lu", msg->uid); + prot_printf(be->out, " %lu", msg->uid); msg = msg->next; } @@ -1059,9 +1060,9 @@ if (count == 0) return(0); - prot_printf(toserver, "\r\n"); - prot_flush(toserver); - return(sync_parse_code("EXPUNGE",fromserver,SYNC_PARSE_EAT_OKLINE,NULL)); + prot_printf(be->out, "\r\n"); + prot_flush(be->out); + return(sync_parse_code("EXPUNGE",be->in,SYNC_PARSE_EAT_OKLINE,NULL)); } /* ====================================================================== */ @@ -1131,15 +1132,15 @@ */ if (sync_msgid_lookup(msgid_onserver, &record->uuid)) { - prot_printf(toserver, " COPY"); + prot_printf(be->out, " COPY"); need_body = 0; } else { sync_msgid_add(msgid_onserver, &record->uuid); - prot_printf(toserver, " PARSED"); + prot_printf(be->out, " PARSED"); need_body = 1; } - prot_printf(toserver, " %s %lu %lu %lu %lu (", + prot_printf(be->out, " %s %lu %lu %lu %lu (", message_uuid_text(&record->uuid), record->uid, record->internaldate, record->sentdate, record->last_updated); @@ -1147,21 +1148,21 @@ flags_printed = 0; if (record->system_flags & FLAG_DELETED) - sync_flag_print(toserver, &flags_printed, "\\deleted"); + sync_flag_print(be->out, &flags_printed, "\\deleted"); if (record->system_flags & FLAG_ANSWERED) - sync_flag_print(toserver, &flags_printed, "\\answered"); + sync_flag_print(be->out, &flags_printed, "\\answered"); if (record->system_flags & FLAG_FLAGGED) - sync_flag_print(toserver, &flags_printed, "\\flagged"); + sync_flag_print(be->out, &flags_printed, "\\flagged"); if (record->system_flags & FLAG_DRAFT) - sync_flag_print(toserver, &flags_printed, "\\draft"); + sync_flag_print(be->out, &flags_printed, "\\draft"); for (flag = 0 ; flag < MAX_USER_FLAGS ; flag++) { if (mailbox->flagname[flag] && (record->user_flags[flag/32] & (1<<(flag&31)) )) - sync_flag_print(toserver, + sync_flag_print(be->out, &flags_printed, mailbox->flagname[flag]); } - prot_printf(toserver, ")"); + prot_printf(be->out, ")"); if (need_body) { /* Server doesn't have this message yet */ @@ -1182,16 +1183,16 @@ return(IMAP_IOERROR); } - prot_printf(toserver, " %lu %lu %lu {%lu+}\r\n", + prot_printf(be->out, " %lu %lu %lu {%lu+}\r\n", record->header_size, record->content_lines, record->cache_version, cache_size); - prot_write(toserver, + prot_write(be->out, (char *)(mailbox->cache_base + record->cache_offset), cache_size); - prot_printf(toserver, "{%lu+}\r\n", record->size); - prot_write(toserver, (char *)msg_base, record->size); + prot_printf(be->out, "{%lu+}\r\n", record->size); + prot_write(be->out, (char *)msg_base, record->size); mailbox_unmap_message(mailbox, record->uid, &msg_base, &msg_size); sequence++; } @@ -1238,7 +1239,7 @@ } if (count++ == 0) - prot_printf(toserver, "UPLOAD %lu %lu", + prot_printf(be->out, "UPLOAD %lu %lu", mailbox->last_uid, mailbox->last_appenddate); /* Message with this UUID exists on client but not server */ @@ -1252,18 +1253,18 @@ if (count == 0) return(r); - prot_printf(toserver, "\r\n"); - prot_flush(toserver); + prot_printf(be->out, "\r\n"); + prot_flush(be->out); - r = sync_parse_code("UPLOAD", fromserver, SYNC_PARSE_NOEAT_OKLINE, NULL); + r = sync_parse_code("UPLOAD", be->in, SYNC_PARSE_NOEAT_OKLINE, NULL); if (r) return(r); - if ((c = getword(fromserver, &token)) != ' ') { - eatline(fromserver, c); + if ((c = getword(be->in, &token)) != ' ') { + eatline(be->in, c); syslog(LOG_ERR, "Garbage on Upload response"); return(IMAP_PROTOCOL_ERROR); } - eatline(fromserver, c); + eatline(be->in, c); /* Clear out msgid_on_server list if server restarted */ if (!strcmp(token.s, "[RESTART]")) { @@ -1308,7 +1309,7 @@ continue; if (count++ == 0) - prot_printf(toserver, "UPLOAD %lu %lu", + prot_printf(be->out, "UPLOAD %lu %lu", mailbox->last_uid, mailbox->last_appenddate); if ((r=upload_message_work(mailbox, msgno, &record))) @@ -1318,18 +1319,18 @@ if (count == 0) return(r); - prot_printf(toserver, "\r\n"); - prot_flush(toserver); + prot_printf(be->out, "\r\n"); + prot_flush(be->out); - r = sync_parse_code("UPLOAD", fromserver, SYNC_PARSE_NOEAT_OKLINE, NULL); + r = sync_parse_code("UPLOAD", be->in, SYNC_PARSE_NOEAT_OKLINE, NULL); if (r) return(r); - if ((c = getword(fromserver, &token)) != ' ') { - eatline(fromserver, c); + if ((c = getword(be->in, &token)) != ' ') { + eatline(be->in, c); syslog(LOG_ERR, "Garbage on Upload response"); return(IMAP_PROTOCOL_ERROR); } - eatline(fromserver, c); + eatline(be->in, c); /* Clear out msgid_on_server list if server restarted */ if (!strcmp(token.s, "[RESTART]")) { @@ -1348,10 +1349,10 @@ static int update_uidlast(struct mailbox *mailbox) { - prot_printf(toserver, "UIDLAST %lu %lu\r\n", + prot_printf(be->out, "UIDLAST %lu %lu\r\n", mailbox->last_uid, mailbox->last_appenddate); - prot_flush(toserver); - return(sync_parse_code("UIDLAST",fromserver, SYNC_PARSE_EAT_OKLINE, NULL)); + prot_flush(be->out); + return(sync_parse_code("UIDLAST",be->in, SYNC_PARSE_EAT_OKLINE, NULL)); } @@ -1389,16 +1390,16 @@ } /* Update seen list */ - prot_printf(toserver, "SETSEEN "); - sync_printastring(toserver, user); - prot_printf(toserver, " "); - sync_printastring(toserver, m.name); - prot_printf(toserver, " %lu %lu %lu ", + prot_printf(be->out, "SETSEEN "); + sync_printastring(be->out, user); + prot_printf(be->out, " "); + sync_printastring(be->out, m.name); + prot_printf(be->out, " %lu %lu %lu ", lastread, last_recent_uid, lastchange); - sync_printastring(toserver, seenuid); - prot_printf(toserver, "\r\n"); - prot_flush(toserver); - r = sync_parse_code("SETSEEN",fromserver,SYNC_PARSE_EAT_OKLINE,NULL); + sync_printastring(be->out, seenuid); + prot_printf(be->out, "\r\n"); + prot_flush(be->out); + r = sync_parse_code("SETSEEN",be->in,SYNC_PARSE_EAT_OKLINE,NULL); bail: mailbox_close(&m); @@ -1505,45 +1506,45 @@ static struct buf entry, userid, value; struct sync_annot_list *server_list = sync_annot_list_create(); - prot_printf(toserver, "LIST_ANNOTATIONS "); - sync_printastring(toserver, name); - prot_printf(toserver, "\r\n", name); - prot_flush(toserver); - r=sync_parse_code("LIST_ANNOTATIONS", fromserver, + prot_printf(be->out, "LIST_ANNOTATIONS "); + sync_printastring(be->out, name); + prot_printf(be->out, "\r\n", name); + prot_flush(be->out); + r=sync_parse_code("LIST_ANNOTATIONS", be->in, SYNC_PARSE_EAT_OKLINE, &unsolicited); while (!r && unsolicited) { - if ((c = getastring(fromserver, toserver, &entry)) != ' ') { + if ((c = getastring(be->in, be->out, &entry)) != ' ') { syslog(LOG_ERR, "LIST_ANNOTATIONS: Invalid type %d response from server: %s", unsolicited, entry.s); - sync_eatlines_unsolicited(fromserver, c); + sync_eatlines_unsolicited(be->in, c); r = IMAP_PROTOCOL_ERROR; break; } - if ((c = getastring(fromserver, toserver, &userid)) != ' ') { + if ((c = getastring(be->in, be->out, &userid)) != ' ') { syslog(LOG_ERR, "LIST_ANNOTATIONS: Invalid type %d response from server: %s", unsolicited, userid.s); - sync_eatlines_unsolicited(fromserver, c); + sync_eatlines_unsolicited(be->in, c); r = IMAP_PROTOCOL_ERROR; break; } - c = getastring(fromserver, toserver, &value); - if (c == '\r') c = prot_getc(fromserver); + c = getastring(be->in, be->out, &value); + if (c == '\r') c = prot_getc(be->in); if (c != '\n') { syslog(LOG_ERR, "LIST_ANNOTATIONS: Invalid type %d response from server: %s", unsolicited, value.s); - sync_eatlines_unsolicited(fromserver, c); + sync_eatlines_unsolicited(be->in, c); r = IMAP_PROTOCOL_ERROR; break; } sync_annot_list_add(server_list, entry.s, userid.s, value.s); - r = sync_parse_code("LIST_ANNOTATIONS", fromserver, + r = sync_parse_code("LIST_ANNOTATIONS", be->in, SYNC_PARSE_EAT_OKLINE, &unsolicited); } @@ -1890,44 +1891,44 @@ static struct buf arg; struct quota quota, *quotap; - prot_printf(toserver, "MAILBOXES"); + prot_printf(be->out, "MAILBOXES"); for (folder = client_list->head ; folder; folder = folder->next) { /* Quietly skip over folders that have already been processed */ if (folder->mark) continue; - prot_printf(toserver, " "); - sync_printastring(toserver, folder->name); + prot_printf(be->out, " "); + sync_printastring(be->out, folder->name); } - prot_printf(toserver, "\r\n"); - prot_flush(toserver); + prot_printf(be->out, "\r\n"); + prot_flush(be->out); - r = sync_parse_code("MAILBOXES", fromserver, + r = sync_parse_code("MAILBOXES", be->in, SYNC_PARSE_EAT_OKLINE, &unsolicited_type); while (!r && (unsolicited_type > 0)) { switch (unsolicited_type) { case 2: /* New folder */ - if ((c = getword(fromserver, &id)) != ' ') + if ((c = getword(be->in, &id)) != ' ') goto parse_err; - if ((c = getastring(fromserver, toserver, &name)) != ' ') + if ((c = getastring(be->in, be->out, &name)) != ' ') goto parse_err; - if ((c = getastring(fromserver, toserver, &acl)) != ' ') + if ((c = getastring(be->in, be->out, &acl)) != ' ') goto parse_err; - c = getastring(fromserver, toserver, &lastuid); + c = getastring(be->in, be->out, &lastuid); quotap = NULL; if (c == ' ') { - c = getword(fromserver, &arg); + c = getword(be->in, &arg); quota.limit = atoi(arg.s); quotap = "a; } - if (c == '\r') c = prot_getc(fromserver); + if (c == '\r') c = prot_getc(be->in); if (c != '\n') goto parse_err; if (!imparse_isnumber(lastuid.s)) goto parse_err; @@ -1940,22 +1941,22 @@ if (folder == NULL) goto parse_err; /* No current folder */ msg = sync_msg_list_add(folder->msglist); - if (((c = getword(fromserver, &arg)) != ' ') || + if (((c = getword(be->in, &arg)) != ' ') || ((msg->uid = sync_atoul(arg.s)) == 0)) goto parse_err; - if (((c = getword(fromserver, &arg)) != ' ')) goto parse_err; + if (((c = getword(be->in, &arg)) != ' ')) goto parse_err; if (!message_uuid_from_text(&msg->uuid, arg.s)) goto parse_err; - c = sync_getflags(fromserver, &msg->flags, &folder->msglist->meta); - if (c == '\r') c = prot_getc(fromserver); + c = sync_getflags(be->in, &msg->flags, &folder->msglist->meta); + if (c == '\r') c = prot_getc(be->in); if (c != '\n') goto parse_err; break; default: goto parse_err; } - r = sync_parse_code("MAILBOXES", fromserver, + r = sync_parse_code("MAILBOXES", be->in, SYNC_PARSE_EAT_OKLINE, &unsolicited_type); } return(r); @@ -1964,7 +1965,7 @@ syslog(LOG_ERR, "MAILBOXES: Invalid unsolicited response type %d from server: %s", unsolicited_type, arg.s); - sync_eatlines_unsolicited(fromserver, c); + sync_eatlines_unsolicited(be->in, c); return(IMAP_PROTOCOL_ERROR); } @@ -2214,17 +2215,17 @@ free(seen_file); /* Update seen db */ - prot_printf(toserver, "SETSEEN_ALL "); - sync_printastring(toserver, user); - prot_printf(toserver, " {%lu+}\r\n", len); + prot_printf(be->out, "SETSEEN_ALL "); + sync_printastring(be->out, user); + prot_printf(be->out, " {%lu+}\r\n", len); - prot_write(toserver, base, len); + prot_write(be->out, base, len); map_free(&base, &len); - prot_printf(toserver, "\r\n"); - prot_flush(toserver); + prot_printf(be->out, "\r\n"); + prot_flush(be->out); - return(sync_parse_code("SETSEEN_ALL",fromserver,SYNC_PARSE_EAT_OKLINE,NULL)); + return(sync_parse_code("SETSEEN_ALL",be->in,SYNC_PARSE_EAT_OKLINE,NULL)); } int do_user_sieve(char *user, struct sync_sieve_list *server_list) @@ -2290,10 +2291,10 @@ int do_user_start(char *user) { - prot_printf(toserver, "USER "); - sync_printastring(toserver, user); - prot_printf(toserver, "\r\n"); - prot_flush(toserver); + prot_printf(be->out, "USER "); + sync_printastring(be->out, user); + prot_printf(be->out, "\r\n"); + prot_flush(be->out); return(0); } @@ -2317,7 +2318,7 @@ struct sync_msg *msg = NULL; struct quota quota, *quotap; - r = sync_parse_code("USER", fromserver, + r = sync_parse_code("USER", be->in, SYNC_PARSE_NOEAT_OKLINE, &unsolicited_type); /* Unpleasant: translate remote access error into "please reset me" */ @@ -2328,49 +2329,49 @@ switch (unsolicited_type) { case 4: /* New Sieve script */ - c = getastring(fromserver, toserver, &name); + c = getastring(be->in, be->out, &name); if (c != ' ') goto parse_err; - c = getastring(fromserver, toserver, &time); + c = getastring(be->in, be->out, &time); if (c == ' ') { - c = getastring(fromserver, toserver, &flag); + c = getastring(be->in, be->out, &flag); if (!strcmp(flag.s, "*")) active = 1; } else active = 0; - if (c == '\r') c = prot_getc(fromserver); + if (c == '\r') c = prot_getc(be->in); if (c != '\n') goto parse_err; sync_sieve_list_add(server_sieve_list, name.s, atoi(time.s), active); break; case 3: /* New subscription */ - c = getastring(fromserver, toserver, &name); - if (c == '\r') c = prot_getc(fromserver); + c = getastring(be->in, be->out, &name); + if (c == '\r') c = prot_getc(be->in); if (c != '\n') goto parse_err; sync_folder_list_add(server_sub_list, name.s, name.s, NULL, NULL); break; case 2: /* New folder */ - if ((c = getword(fromserver, &id)) != ' ') + if ((c = getword(be->in, &id)) != ' ') goto parse_err; - if ((c = getastring(fromserver, toserver, &name)) != ' ') + if ((c = getastring(be->in, be->out, &name)) != ' ') goto parse_err; - if ((c = getastring(fromserver, toserver, &acl)) != ' ') + if ((c = getastring(be->in, be->out, &acl)) != ' ') goto parse_err; - c = getastring(fromserver, toserver, &lastuid); + c = getastring(be->in, be->out, &lastuid); quotap = NULL; if (c == ' ') { - c = getword(fromserver, &arg); + c = getword(be->in, &arg); quota.limit = atoi(arg.s); quotap = "a; } - if (c == '\r') c = prot_getc(fromserver); + if (c == '\r') c = prot_getc(be->in); if (c != '\n') goto parse_err; if (!imparse_isnumber(lastuid.s)) goto parse_err; @@ -2383,23 +2384,23 @@ if (folder == NULL) goto parse_err; /* No current folder */ msg = sync_msg_list_add(folder->msglist); - if (((c = getword(fromserver, &arg)) != ' ') || + if (((c = getword(be->in, &arg)) != ' ') || ((msg->uid = sync_atoul(arg.s)) == 0)) goto parse_err; - if (((c = getword(fromserver, &arg)) != ' ')) goto parse_err; + if (((c = getword(be->in, &arg)) != ' ')) goto parse_err; if (!message_uuid_from_text(&msg->uuid, arg.s)) goto parse_err; - c = sync_getflags(fromserver, &msg->flags, &folder->msglist->meta); - if (c == '\r') c = prot_getc(fromserver); + c = sync_getflags(be->in, &msg->flags, &folder->msglist->meta); + if (c == '\r') c = prot_getc(be->in); if (c != '\n') goto parse_err; break; default: goto parse_err; } - r = sync_parse_code("USER", fromserver, + r = sync_parse_code("USER", be->in, SYNC_PARSE_EAT_OKLINE, &unsolicited_type); } @@ -2408,7 +2409,7 @@ parse_err: syslog(LOG_ERR, "USER: Invalid type %d response from server", unsolicited_type); - sync_eatlines_unsolicited(fromserver, c); + sync_eatlines_unsolicited(be->in, c); return(IMAP_PROTOCOL_ERROR); } @@ -2538,26 +2539,26 @@ static struct buf name; struct sync_folder_list *server_list = sync_folder_list_create(); - prot_printf(toserver, "LSUB "); - sync_printastring(toserver, user); - prot_printf(toserver, "\r\n"); - prot_flush(toserver); - r=sync_parse_code("LSUB",fromserver, SYNC_PARSE_EAT_OKLINE, &unsolicited); + prot_printf(be->out, "LSUB "); + sync_printastring(be->out, user); + prot_printf(be->out, "\r\n"); + prot_flush(be->out); + r=sync_parse_code("LSUB",be->in, SYNC_PARSE_EAT_OKLINE, &unsolicited); while (!r && unsolicited) { - c = getastring(fromserver, toserver, &name); + c = getastring(be->in, be->out, &name); - if (c == '\r') c = prot_getc(fromserver); + if (c == '\r') c = prot_getc(be->in); if (c != '\n') { syslog(LOG_ERR, "LSUB: Invalid type %d response from server: %s", unsolicited, name.s); - sync_eatlines_unsolicited(fromserver, c); + sync_eatlines_unsolicited(be->in, c); r = IMAP_PROTOCOL_ERROR; break; } sync_folder_list_add(server_list, name.s, name.s, NULL, NULL); - r = sync_parse_code("LSUB", fromserver, + r = sync_parse_code("LSUB", be->in, SYNC_PARSE_EAT_OKLINE, &unsolicited); } @@ -2576,45 +2577,45 @@ struct sync_sieve_list *server_list = sync_sieve_list_create(); int active = 0; - prot_printf(toserver, "LIST_SIEVE "); - sync_printastring(toserver, user); - prot_printf(toserver, "\r\n"); - prot_flush(toserver); + prot_printf(be->out, "LIST_SIEVE "); + sync_printastring(be->out, user); + prot_printf(be->out, "\r\n"); + prot_flush(be->out); r=sync_parse_code("LIST_SIEVE", - fromserver, SYNC_PARSE_EAT_OKLINE, &unsolicited); + be->in, SYNC_PARSE_EAT_OKLINE, &unsolicited); while (!r && unsolicited) { - c = getastring(fromserver, toserver, &name); + c = getastring(be->in, be->out, &name); if (c != ' ') { syslog(LOG_ERR, "LIST_SIEVE: Invalid name response from server: %s", name.s); - sync_eatlines_unsolicited(fromserver, c); + sync_eatlines_unsolicited(be->in, c); r = IMAP_PROTOCOL_ERROR; break; } - c = getastring(fromserver, toserver, &time); + c = getastring(be->in, be->out, &time); if (c == ' ') { - c = getastring(fromserver, toserver, &flag); + c = getastring(be->in, be->out, &flag); if (!strcmp(flag.s, "*")) active = 1; } else active = 0; - if (c == '\r') c = prot_getc(fromserver); + if (c == '\r') c = prot_getc(be->in); if (c != '\n') { syslog(LOG_ERR, "LIST_SIEVE: Invalid flag response from server: %s", flag.s); - sync_eatlines_unsolicited(fromserver, c); + sync_eatlines_unsolicited(be->in, c); r = IMAP_PROTOCOL_ERROR; break; } sync_sieve_list_add(server_list, name.s, atoi(time.s), active); - r = sync_parse_code("LIST_SIEVE", fromserver, + r = sync_parse_code("LIST_SIEVE", be->in, SYNC_PARSE_EAT_OKLINE, &unsolicited); } if (r) { @@ -3124,10 +3125,10 @@ if (*restartp == 0) return(0); - prot_printf(toserver, "RESTART\r\n"); - prot_flush(toserver); + prot_printf(be->out, "RESTART\r\n"); + prot_flush(be->out); - r = sync_parse_code("RESTART", fromserver, SYNC_PARSE_EAT_OKLINE, NULL); + r = sync_parse_code("RESTART", be->in, SYNC_PARSE_EAT_OKLINE, NULL); if (r) syslog(LOG_ERR, "sync_client RESTART failed"); @@ -3137,8 +3138,7 @@ return(r); } -struct backend *replica_connect(struct backend *be, const char *servername, - sasl_callback_t *cb) +struct backend *replica_connect(struct backend *be, sasl_callback_t *cb) { int wait; @@ -3146,7 +3146,7 @@ be = backend_connect(be, servername, &protocol[PROTOCOL_CSYNC], "", cb, NULL); - if (be || wait > 1000) break; + if (be || connect_once || wait > 1000) break; fprintf(stderr, "Can not connect to server '%s', retrying in %d seconds\n", @@ -3154,6 +3154,12 @@ sleep(wait); } + if (!be) { + fprintf(stderr, "Can not connect to server '%s'\n", + servername); + _exit(1); + } + return be; } @@ -3189,17 +3195,7 @@ if (pid == 0) { if (be->sock == -1) { /* Reopen up connection to server */ - be = replica_connect(be, be->hostname, cb); - - if (!be) { - fprintf(stderr, "Can not connect to server '%s'\n", - be->hostname); - _exit(1); - } - - /* XXX hack. should just pass 'be' around */ - fromserver = be->in; - toserver = be->out; + be = replica_connect(be, cb); } r = do_daemon_work(sync_log_file, sync_shutdown_file, @@ -3236,7 +3232,6 @@ int opt, i = 0; char *alt_config = NULL; char *input_filename = NULL; - const char *servername = NULL; int r = 0; int exit_rc = 0; int mode = MODE_UNKNOWN; @@ -3249,7 +3244,6 @@ char buf[512]; FILE *file; int len; - struct backend *be = NULL; sasl_callback_t *cb; /* Global list */ @@ -3260,12 +3254,16 @@ setbuf(stdout, NULL); - while ((opt = getopt(argc, argv, "C:vlS:F:f:w:t:d:rums")) != EOF) { + while ((opt = getopt(argc, argv, "C:vlS:F:f:w:t:d:rumso")) != EOF) { switch (opt) { case 'C': /* alt config file */ alt_config = optarg; break; + case 'o': /* only try to connect once */ + connect_once = 1; + break; + case 'v': /* verbose */ verbose++; break; @@ -3373,20 +3371,10 @@ config_getstring(IMAPOPT_SYNC_REALM), config_getstring(IMAPOPT_SYNC_PASSWORD)); - /* Open up connection to server */ - be = replica_connect(NULL, servername, cb); - - if (!be) { - fprintf(stderr, "Can not connect to server '%s'\n", servername); - exit(1); - } - - /* XXX hack. should just pass 'be' around */ - fromserver = be->in; - toserver = be->out; - switch (mode) { case MODE_USER: + /* Open up connection to server */ + be = replica_connect(NULL, cb); if (input_filename) { if ((file=fopen(input_filename, "r")) == NULL) { syslog(LOG_NOTICE, "Unable to open %s: %m", input_filename); @@ -3443,6 +3431,8 @@ break; case MODE_MAILBOX: + /* Open up connection to server */ + be = replica_connect(NULL, cb); { struct sync_folder_list *folder_list = sync_folder_list_create(); struct sync_user *user; @@ -3495,6 +3485,8 @@ break; case MODE_SIEVE: + /* Open up connection to server */ + be = replica_connect(NULL, cb); for (i = optind; !r && i < argc; i++) { if ((r = send_lock())) { if (verbose) { @@ -3522,6 +3514,8 @@ case MODE_REPEAT: if (input_filename) { + /* Open up connection to server */ + be = replica_connect(NULL, cb); exit_rc = do_sync(input_filename); } else { diff -ur cyrus-imapd-2.3.6.orig/man/sync_client.8 cyrus-imapd-2.3.6/man/sync_client.8 --- cyrus-imapd-2.3.6.orig/man/sync_client.8 2005-03-31 15:27:53.000000000 -0500 +++ cyrus-imapd-2.3.6/man/sync_client.8 2006-06-06 03:29:57.000000000 -0400 @@ -109,6 +109,10 @@ .BI \-l Verbose logging mode. .TP +.BI \-o +Only attempt to connect to the backend server once rather than waiting +up to 1000 seconds before giving up. +.TP .BI \-C " config-file" Read configuration options from \fIconfig-file\fR. .TP
---- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html