Monday, April 28, 2014

Gratia Database Outage - Update - Monday, April 28, 2014

The Gratia Database is currently experiencing an outage. On Friday, April 25, the GOC reported instability in some of the services that rely on Gratia for information. Over the course of the weekend, this instability degraded to outage status. Fermilab is currently working to tune the Gratia database. Maintenance is ongoing and there is no estimated time of repair.

We will pass along updates as they become available to us.

During this time Gratia will continue to collect and store accounting records normally and local Gratia record caching will occur if the collector reaches it limits, however no updates will occur to the above referenced sites until it is fully restored.

Friday, April 25, 2014

GratiaWeb Instability

GratiaWeb is currently experiencing an instability. Additionally, OSG Display is experiencing delays with updates. We are currently investigating the cause of the instability. More information can be found here https://ticket.grid.iu.edu/20881. We will provide updates as they become available.

Please contact the GOC with any questions.

Gratia Functionality Restored

Gratia DB resumed normal operations at 09:15 GMT this morning. Gratiaweb and OSG Display have also resumed normal operations and Gratia-APEL will follow shortly.

Please contact the GOC with any questions.

Thursday, April 24, 2014

Gratia Maintenance Ongoing

Fermilab is still working on the gratia DB failures reported earlier. Maintenance is ongoing at this point and no estimated time of repair is available. We will pass on updates as they are avaliable. As a reminder, gratiaweb.opensciencegrid.org and display.opensciencegrid.org will be affected. During this time Gratia will continue to collect and store accounting records normally and local Gratia record caching will occur if the collector reaches it limits, however no updates will occur to the above referenced sites until it is fully restored.

GOC Service Update - Tuesday, April 29th at 13:00 UTC

The GOC announces a special maintenance period starting Tuesday, April 29th, 2014 at 13:00 UTC. The GOC reserves 8 hours in the unlikely event that unexpected problems are encountered.

Oasis
Upgrade oasis-replica to cvmfs-server v2.1. Only the GOC instance will be affected.

Tuesday, April 22, 2014

Gratia Emergency Maintenance - Follow Up Information

The Gratia accounting database investigation has completed and a restoration plan is underway. Due to the size of the database we expect it to remain unavailable for publishing data to gratiaweb.opensciencegrid.org and display.opensciencegrid.org until at least tomorrow afternoon. During this time Gratia will continue to collect and store accounting records normally and local Gratia record caching will occur if the collector reaches it limits, however no updates will occur to the above referenced sites until it is fully restored. We will inform the community when Gratia is back to normal operation.

Gratia Emergency Maintenance

Due to a hardware failure the Gratia accounting database has become corrupted. This is currently affecting the publishing of new information to gratiaweb.opensciencegrid.org and display.opensciencegrid.org. The Gratia collector is still functioning normally and all records are being collected and preserved. We are investigating the best way to restore the data from backup and process the new records held on the collector. Until we have completed this emergency maintenance no new data will be published to the sites referenced above.

Wednesday, April 16, 2014

GOC Service Update - Tuesday, April 22nd, 2014 at 13:00 UTC

The GOC will upgrade the following services beginning Tuesday, April 22nd, 2014 at 13:00 UTC. The GOC reserves 8 hours in the unlikely event that unexpected problems are encountered. We encourage users to test affected services before the production release.

GratiaWeb v1.2-28

Rebuilding with RHEL6 image (new version may require python2.6)
Moved all external ‘URL’s into a single file location for easier update and tracking.
Reworked WLCG Reporting (Overview) to contain all summary pledge and capacity information by Federation Name and Resource Group
Reworked WLCG Reporting (Detailed) to contain detailed resources supplied information by Federation Name and Resource Group
New WLCG report pages
Bug fix to ATLAS data transfer graphing
Improvement to wlcg reports error handling
Completion of move for all GratiaWeb URLs to a config file
graphtool / Reworked RPM packaging
graphtool / Updated database connection error handling

GOC-TX 1.36

Added fp114_fermide TX ID for test purpose.

GOC Ticket 1.74
(Didn’t get released during the last release)

Made Gratia software ticket to go to GOC support staff instead of SNOW-GratiaDev (https://ticket.grid.iu.edu/20235)
Oasis

Upgrade oasis-replica to cvmfs-server v2.1. Only the GOC instance will be affected.

All Services

There will be OS updates; reboots will be required. Downtime should be minimal, and the usual high-availability mechanisms will be used to reduce service downtime even further and eliminate it in most cases. However, services may experience degraded performance, and the services without HA mechanisms (OIM and Twiki) will still experience brief downtimes.

Tuesday, April 15, 2014

Notification of Gratia-reporters service to be decommissioned

Greetings,

On Thursday, April 17th, the gratia-reporters service will be
decommissioned. We recommend anyone still using this service to switch
to the GratiaWeb interface at http://gratiaweb.opensciencegrid.org/. You
should be able to search all gratia data there, however if you need to
do a local search there are local gratia collector instances available at
http://gratia-fermi-osg.fnal.gov:8100/gratia
http://gratia-fermi-transfer.fnal.gov:8100/gratia
http://gratia-fermi-itb.gov:8100/gratia

thanks,
-Kevin

Tuesday, April 8, 2014

Emergency Maintenance

Tomorrow, April 9th, starting at 9:00 (eastern) the GOC will perform emergency maintenance on some services. The usual
high-availability mechanisms will be used to eliminate visible downtime for all services except Oasis where a brief
outage should be expected.

Announcing OSG Software versions 3.1.32 and 3.2.8

We are pleased to announce OSG Software versions 3.1.32 and 3.2.8.

These releases contain updated Certificate Authority (CA) bundles.
It is imperative that these new CA packages are installed before
June 1st. At that time, DigiCert will begin issuing OSG certificates
that depend on the new CA bundles.

OSG 3.1.32 and 3.2.8 contain:
* Updated CA certificates (IGTF 1.56)
- New SHA-2 signed DigiCert CA certificates
- DOEGrids CA certificates removed
- Old format CA certificate bundles discontinued
* Update to CVMFS 2.1.17 (many enhancements and bug fixes)
* VO Package v52
- New Sub VOs: Lariat, Gendetrd, Lar1, and Okra
- New VOMS servers at CERN
* Bug fixes for VOMS admin
* Updated gratia probes (sge, lsf, htcondor)
* Update to RSV 3.7.15 (bug fixes)
* Update to MyProxy 5.9 (bug fixes)
* Many other minor bug fixes

OSG 3.2.8 also contains:
* Bug fix for crash in gridftp-hdfs

Release notes and pointers to more documentation can be found at:

https://www.opensciencegrid.org/bin/view/Documentation/Release3/Release328
https://www.opensciencegrid.org/bin/view/Documentation/Release3/Release3132

Need help? Let us know:

https://www.opensciencegrid.org/bin/view/Documentation/Release3/HelpProcedure

We welcome feedback on this release!

Monday, April 7, 2014

2014 OSG User School - application deadline extended

Applications for the 2014 OSG User School are due soon. To make it easier
to apply, we extended the application deadline as late as possible: Sunday,
April 13th. But there is no reason to wait ... apply today for acceptance
to the School and for financial support!


ANNOUNCING THE 2014 OPEN SCIENCE GRID USER SCHOOL!

If you could access thousands, maybe millions of hours of computing, how
would it transform your research? What discoveries would you make?

We are looking for qualified students to attend the 2014 Open Science Grid
(OSG) User School, where they will learn how to use high throughput
computing (HTC) to harness vast amounts of computing power for research.

Using lectures, discussions, roleplays, and lots of hands-on work with OSG
experts in high throughput computing, students will learn how HTC systems
work, how to run and manage many jobs and huge datasets to implement a full
scientific computing workflow, and where to turn for help and more info.

Worried about costs? Successful applicants will get financial support to
attend the OSG School (July 7-10) at the beautiful University of Wisconsin
in Madison.

Ideal candidates are science, technology, engineering, and mathematics
(STEM) graduate students whose research demands large-scale computing.
Also, we will consider applications from faculty, staff, and advanced
undergraduates, so make a good case for yourself!

IMPORTANT DATES

Application Period: March 10 - April 13 (EXTENDED!)
OSG User School: July 7-10

MORE INFORMATION AND APPLICATIONS

Web: http://www.opensciencegrid.org/UserSchool
Email: osg-school-2014@opensciencegrid.org
Facebook: https://www.facebook.com/OSGUserSchool
Twitter: https://twitter.com/OSGUserSchool

Please forward this announcement to help us reach potential students. And
consider posting our flyer where appropriate:

https://twiki.opensciencegrid.org/twiki/pub/Education/OSGUserSchool2014/2014-osg-user-school-flyer.pdf