[Act] Act services moving on a new server

[Philippe Bruhat (BooK)]

On Thu, Apr 07, 2016 at 02:47:53AM +0200, Salve J Nilsen wrote:
> Sébastien Aperghis-Tramoni said:
> >
> >Steps:
> >- shutdown all services on spectre
> >- dump all PostgreSQL databases
> >- rsync everything a last time from spectre to phantom
> >- reimport all PostgreSQL databases
> >- startup all services on phantom
> >
> >Executing these steps should roughly take an hour, given that
> >data have been rsync'd every 2 hours since weeks. But as said,
> >DNS propagation will make the disruption longer for people still
> >trying to reach spectre. This can be mitigated to some degree
> >by reducing the TTL of the records.
> >
> >Unless I oversaw something, migrating over the next days should
> >be an appropriate time.
> >
> >Opinions?
> 
> 0) Make a cronological task list, with ALL necessary steps written down.
> Even the minor ones!
> 
> 1) Do the $NEW_TTL change right now, so people with $CURRENT_TTL can have
> time to update their DNS cache. Then wait with any migration until
> $CURRENT_TTL time has passed. :)

This only works for the domains we control.

I know of a few that point to the address of spectre, and still will
after the move until we can reach the owners and let them know the
new address. If we don't know the affected domains, then I think the
only solution is to put up a static page with some contact info asking
specifically for the URL used to reach the now-broken site. Then we can
use whois to try and contact the domain owner, so that they point to
the real site.

We can find some of those domains by looking in the Act config for the
"main" url of each conference, and get a list of all the vanity domains
that we don't control.

One such domain is www.aconferencetoolkit.com, which is apparently owned
by Robert Blackwell from Pittsburgh.

To mitigate the downtime caused by DNS propagation, what about:
- setup the Act database on phantom
- dump and shutdown the Act database on spectre
- restore that last copy of the spectre db on phantom
- point the spectre Act config to the db on phantom (this might add some
  latency, but that's only for people connected to spectre)
- at this stage, we have two Act servers on different boxes pointing to
  the same database instance
- reconfigure the DNS to point to phantom
- wait for at least $CURRENT_TTL, until every domain points to phantom
- shut down spectre

This way, both the old and new IP point to a working Act instance.

> 2) Make a "this site is under maintainance" static webpage, that one can put
> up during the migration. Doesn't have to be more than a catch-all
> mod_rewrite rule pointing to a static webpage containing a few words of
> apology, a smiley and an ETA for when the sites are expected to be up again.

As explained below, this should probably contain some contact info and
instructions asking "through which URL did you reach this page?". This
should remain on spectre as long as it's still connected to the network.


-- 
 Philippe Bruhat (BooK)

 Only a fool follows orders without knowing why.
                                    (Moral from Groo The Wanderer #86 (Epic))