Monday, July 12, 2010

Gratia Scheduled Maintenance 7/14

All FNAL-based Gratia services will be down on 2010/07/14 for OS and service
upgrades. In addition, the previously announced decommissioning of the legacy
redirector service will also take place at this time.

Gratia release notes for v1.06.16:

* Principal improvement is to the housekeeping feature: this is expected to greatly reduce and hopefully eliminate the instances of significant data lag in reporting.

Outage details:

* At about 09:00 CDT (14:00 UTC), incoming data to the GRATIA-OSG-PROD, GRATIA-OSG-ITB, GRATIA-OSG-TRANSFER and GRATIA-OSG-DAILY services will be stopped in such a way as to hopefully eliminate the possibility of probes becoming "stuck" as has happened in the past (note: probe release currently in VDT test cycle will eliminate this completely).

* There will be one data-collection outage of about 20 minutes, with a shorter
outage in reporting as services are shuffled between highly-available servers and the reporting services are upgraded.

* There will be another outage in data collection of much longer duration as
collector services are upgraded. Because a DB schema upgrade is involved, this could be several hours in duration. During this period, reporting services will be available but data will of course be stale.

* When all collectors upgrades are complete, data collection will resume. This is not expected to be later than 16:00 CDT (21:00 UTC), but given the uncertainty in the time required to upgrade each schema service may be resumed in actuality some time earlier or later than this estimate. In any event, the MyOSG downtime page will have the latest details.

* During any data collection outage, data are retained on probes and re-sent when collector service resumes.

* At some point during this period, the legacy redirector service which has up to now redirected probe data sent to obsolete addresses and port numbers, will be deactivated. The upcoming demise of this service has been announced previously.