Friday, December 23, 2016
OSG TWiki service restored
The problem causing instability on the TWiki has been addressed and the service has been restored to normal operation. Please contact us if you encounter any difficulties or unusual behavior. We apologize for any inconvenience and wish you a happy holiday.
Instability on OSG Twiki
Service instability on twiki.opensciencegrid.org was observed starting around 09:30 EST. This is under investigation, expect some intermittency until service is fully restored. We apologize for any inconvenience.
Tuesday, December 20, 2016
Holiday Greetings from OSG Operations
Dear OSG Collaborators,
OSG Operations would like to wish you a happy holiday season and a joyous new year!
We'd like to thank the OSG resource providers and users worldwide that have helped us provide more than 1.3 Billion hours of compute time this calendar year (see http://display.grid.iu.edu/ for a full summary). OSG Operations will continue to provide round the clock holiday coverage in case of operational emergencies over the holiday season. You can continue to open tickets at https://ticket.grid.iu.edu/, send us mail at goc@openscinecegrid.org, or call us at +1 (317)-278-9699. However, we will be operating on holiday procedures beginning on Monday, December 26th, and resuming regular staffing on Tuesday, January 3rd. During this time non-emergency issues will be handled on a best-effort basis.
Operations Weekly Calls will resume on Jan 9.
Thanks to everyone for another successful year! We look forward to working with everyone in 2017!
OSG Operations would like to wish you a happy holiday season and a joyous new year!
We'd like to thank the OSG resource providers and users worldwide that have helped us provide more than 1.3 Billion hours of compute time this calendar year (see http://display.grid.iu.edu/ for a full summary). OSG Operations will continue to provide round the clock holiday coverage in case of operational emergencies over the holiday season. You can continue to open tickets at https://ticket.grid.iu.edu/, send us mail at goc@openscinecegrid.org, or call us at +1 (317)-278-9699. However, we will be operating on holiday procedures beginning on Monday, December 26th, and resuming regular staffing on Tuesday, January 3rd. During this time non-emergency issues will be handled on a best-effort basis.
Operations Weekly Calls will resume on Jan 9.
Thanks to everyone for another successful year! We look forward to working with everyone in 2017!
Monday, December 19, 2016
Planned Retirement of OSG BDII
OSG Operations and Technology are planning the retirement of the BDII
information service located at is.grid.iu.edu on March
31st, 2017. We have
been working with WLCG, ATLAS and CMS to remove
dependencies or replace
the functionality within our HTCondor Collector service.
This work is
still ongoing. This message is to alert you to the
upcoming deprecation
date and to get feedback on any other existing
dependencies that might
exist to the BDII.
If you are dependent in any way on the OSG BDII or
information the OSG
BDII supplies to the WLCG or EGI BDIIs please contact us
at goc@opensciencegrid.org
Tuesday, December 13, 2016
Announcing OSG Software version 3.3.19
We are pleased to announce OSG Software version 3.3.19.
Changes to OSG 3.3.19 include:
- Update HTCondor-CE to provide data needed by the ATLAS AGIS system
- Provide a way for Gratia to avoid reporting local, non-OSG jobs*
- Provide better hold messages when the Job Router does not route a job
- Update several packages to better integrate with systemd on EL7
- Make osg-configure more robust when writing Gratia probe configuration files
- Update to frontier-squid 3 in the upcoming repository
*NOTE: Earlier versions of the Gratia probes required modifications to the
HTCondor configuration. See the release notes to revert those changes.
Release notes and pointers to more documentation can be found at:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/Release3319
Need help? Let us know:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/HelpProcedure
We welcome feedback on this release!
Changes to OSG 3.3.19 include:
- Update HTCondor-CE to provide data needed by the ATLAS AGIS system
- Provide a way for Gratia to avoid reporting local, non-OSG jobs*
- Provide better hold messages when the Job Router does not route a job
- Update several packages to better integrate with systemd on EL7
- Make osg-configure more robust when writing Gratia probe configuration files
- Update to frontier-squid 3 in the upcoming repository
*NOTE: Earlier versions of the Gratia probes required modifications to the
HTCondor configuration. See the release notes to revert those changes.
Release notes and pointers to more documentation can be found at:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/Release3319
Need help? Let us know:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/HelpProcedure
We welcome feedback on this release!
OSG Connect downtime (Dec 19 - Dec 26)
To prepare for the upcoming year, we will be taking a one week downtime of OSG Connect starting Monday, December 19th through Monday, December 26th.
We aim to accomplish two major service upgrades:
- Update of Stash to the latest long-term support release of Ceph (0.94.x -> 10.2.x)
- Migration of all home directories to a newer, faster array (40TB)
In addition to the service upgrades above, we will use the downtime to test and benchmark all Connect services and update system software. During this time, all Connect services will be down and users will not be able to log in.
Most importantly, please backup any critical data on Stash. We recommend using Globus to manage this, see help desk documentation at bit.ly/globus-xfer. Once the downtime has been completed or needs to be extended, we will let you know. As always, please contact us if you have any questions or concerns.
Sincerely
The OSG User Support Team
https://support.opensciencegrid.org/support/home
We aim to accomplish two major service upgrades:
- Update of Stash to the latest long-term support release of Ceph (0.94.x -> 10.2.x)
- Migration of all home directories to a newer, faster array (40TB)
In addition to the service upgrades above, we will use the downtime to test and benchmark all Connect services and update system software. During this time, all Connect services will be down and users will not be able to log in.
Most importantly, please backup any critical data on Stash. We recommend using Globus to manage this, see help desk documentation at bit.ly/globus-xfer. Once the downtime has been completed or needs to be extended, we will let you know. As always, please contact us if you have any questions or concerns.
Sincerely
The OSG User Support Team
https://support.opensciencegrid.org/support/home
Wednesday, December 7, 2016
GOC Service Update - Tuesday, December 13th at 14:00 UTC
The GOC will upgrade the following services beginning Tuesday, December 13 at 14:00 UTC. The GOC reserves 8 hours in the unlikely event unexpected problems are encountered.
Collector
* Update HTCondor-ceview to latest version
Software.grid
* Create alias to repo.grid in preparation to retire software.grid
OASIS
* Update cvmfs packages on oasis and oasis-replica to the latest version
* Update frontier-squid on oasis-replica to the latest version
* Add the BNL stashcache server to the default list of data servers for osgstorage.org repositories
VOMS
* Rebuild VOMS for mis and osgedu VOs at CentOS7
Twiki
* Reinstalled on CentOS6
Collector
* Update HTCondor-ceview to latest version
Software.grid
* Create alias to repo.grid in preparation to retire software.grid
OASIS
* Update cvmfs packages on oasis and oasis-replica to the latest version
* Update frontier-squid on oasis-replica to the latest version
* Add the BNL stashcache server to the default list of data servers for osgstorage.org repositories
VOMS
* Rebuild VOMS for mis and osgedu VOs at CentOS7
Twiki
* Reinstalled on CentOS6
GOC Service Update - Tuesday, December 13th at 14:00 UTC
The GOC will upgrade the following services beginning Tuesday, December 13 at 14:00 UTC. The GOC reserves 8 hours in the unlikely event unexpected problems are encountered.
Collector
* Update HTCondor-ceview to latest version
Software.grid
* Create alias to repo.grid in preparation to retire software.grid
OASIS
* Update cvmfs packages on oasis and oasis-replica to the latest version
* Update frontier-squid on oasis-replica to the latest version
* Add the BNL stashcache server to the default list of data servers for osgstorage.org repositories
VOMS
* Rebuild VOMS for mis and osgedu VOs at CentOS7
Twiki
* Reinstalled on CentOS6
Collector
* Update HTCondor-ceview to latest version
Software.grid
* Create alias to repo.grid in preparation to retire software.grid
OASIS
* Update cvmfs packages on oasis and oasis-replica to the latest version
* Update frontier-squid on oasis-replica to the latest version
* Add the BNL stashcache server to the default list of data servers for osgstorage.org repositories
VOMS
* Rebuild VOMS for mis and osgedu VOs at CentOS7
Twiki
* Reinstalled on CentOS6
Tuesday, November 29, 2016
UPDATE: Operations Emergency Maintenance - November 29, 2016
As was previously noted, there was an emergency maintenance period today starting at 8am EST to repair a damaged filesystem. The affected filesystem, /net/nas01, has been repaired as of 11:45 EST and all systems are operating as expected. We regret any inconvenience that this may have caused. Please feel free to contact us with any questions.
Monday, November 28, 2016
Operations Emergency Maintenance - November 29, 2016
There will be an emergency maintenance period tomorrow, November 29, starting at 8am EST to repair a damaged filesystem. The affected filesystem is /net/nas01, commonly used as scratch space. This will affect some internal service monitoring functionality, but should not affect other GOC services. We regret any inconvenience and will inform you when maintenance is completed.
Monday, November 21, 2016
Thanksgiving Holiday
On November 24th and 25th, the OSG Operations Center will be operating on a holiday schedule. Staff will be available to respond to emergencies but routine operations will resume at the start of business Monday, November 28.
OSG Operations wishes its users and OSG staff a happy Thanksgiving Holiday.
OSG Operations wishes its users and OSG staff a happy Thanksgiving Holiday.
Tuesday, November 15, 2016
OSG receives HPCwire's 'Top Supercomputing Achievement' awards
Multi-partner awards cite OSG's role in gravitational wave detection
The Open Science Grid (OSG) is a recipient of two HPCwire 'Top Supercomputing Achievement' awards for 2016, recognizing the use of high performance computing to verify Einstein's theory of gravitational waves.
Funded by the U.S. Department of Energy (DOE) and National Science Foundation (NSF), OSG is a multi-disciplinary research partnership specializing in high throughput computational services.
The HPCwire awards were presented at the 2016 International Conference for High Performance Computing, Networking, Storage and Analysis (SC16), in Salt Lake City, Utah. OSG won in both the online publication's annual Readers' Choice and Editors' Choice categories.
The awards also recognize the San Diego Supercomputer Center (SDSC) at the University of California San Diego, the NSF's Extreme Science and Engineering Discovery Environment (XSEDE), the Holland Computing Center at University of Nebraska-Lincoln (UNL), the National Center for Supercomputing Applications (NCSA), and the Texas Advanced Computing Center (TACC) at The University of Texas at Austin for their participation in verifying the existence of gravitational waves.
OSG provided access to numerous high performance computing systems for LIGO, seamlessly integrating XSEDE supercomputers like SDSC's Comet and TACC's Stampede with HPC clusters in academia and DOE national labs. LIGO used this integrated infrastructure over the course of several months to verify the statistical significance of the observed gravitational wave, thus leading to its unambiguous detection.
In the process, roughly five terabytes of LIGO data was processed many thousands of times, leading to many petabytes of exported data. The storage infrastructure at Holland Computing Center hosted the LIGO data for processing via OSG. The data was pulled as needed by the Pegasus workflow.
Frank Würthwein, who is the current executive director of the OSG, worked in tandem with XSEDE researchers to make these resources available to LIGO scientists. Würthwein is also SDSC's head of High Throughput Computing.
"There were a lot of institutions and researchers involved in this landmark discovery, as well as the actual verification process," said Würthwein, also a physicist with UC San Diego. "While high performance computing resources from all over have been used by LIGO for years, this award focused on the use of distributed high throughput computing across a wide range of HPC resources for the actual verification of this amazing discovery."
"From thought leaders to end users, the HPCwire readership reaches and engages every corner of the high performance computing community," said Tom Tabor, CEO of Tabor Communications, publisher of HPCwire. "Receiving their recognition signifies community support across the entire HPC space as well as the breadth of industries it serves. We are proud to recognize these efforts and make the voices of our readers heard, and our congratulations go out to all the winners."
A multi-partner collaboration
In February 2016, the NSF made a pivotal announcement: For the first time, scientists detected gravitational waves in the universe as hypothesized by Albert Einstein about 100 years ago. On September 14, 2015 scientists at the NSF-funded Laser Interferometer Gravitational-Wave Observatory (LIGO) detected gravitational waves using both LIGO detectors. The waves reached Earth from the southern hemisphere, passed through the Earth, and emerged at the Earth's surface, first at the LIGO detector near Livingston, Louisiana, and then, seven milliseconds later and 1,890 miles away at the second LIGO detector in Hanford, Washington.
More details on the LIGO discovery can be found at:
https://www.opensciencegrid.org/osg-helps-ligo-scientists-confirm-einsteins-last-unproven-theory/
http://www.sdsc.edu/News%20Items/PR20160225_ligo.html
https://www.xsede.org/xsede-resources-help-confirm-ligo-discovery.
The annual HPCwire Readers' and Editors' Choice Awards are determined through a nomination and voting process with the global HPCwire community, as well as selections from the HPCwire editors. The awards are an annual feature of the publication and constitute prestigious recognition from the HPC community. These awards are revealed each year to kick off the annual Supercomputing Conference, which showcases high-performance computing, networking, storage, and data analysis. More information on these awards can be found at the HPCwire website or on Twitter through the #HPCwireAwards hashtag.
About OSG
The Open Science Grid Consortium is a community-driven organization that spans academia and Department of Energy national laboratories to advance the state of the art of distributed high throughput computing. In addition to community contributions, the OSG project receives funding from DOE and NSF to operate a fabric of services, including a production infrastructure, an integrated software stack, and a variety of intellectual support services for educators, scientists, and IT professionals.
About HPCwire
HPCwire is an online news and information resource covering the fastest computers in the world and the people who run them. Started in 1986, HPCwire has enjoyed a legacy of world-class editorial and journalism, making it the news source of choice for science, technology, and business professionals interested in high-performance and data-intensive computing. Visit HPCwire at www.hpcwire.com.
Media Contacts:
Kyle Gross, OSG communications lead, kagross@iu.edu
Chelsea Lang, corporate marketing manager, Tabor Communications, 919 749-1895
Related Links:
The Open Science Grid: https://www.opensciencegrid.org
National Science Foundation: https://www.nsf.gov/
Department of Energy, Office of Science: http://science.energy.gov
Pegasus: https://pegasus.isi.edu
LIGO: http://ligo.org/
The Open Science Grid (OSG) is a recipient of two HPCwire 'Top Supercomputing Achievement' awards for 2016, recognizing the use of high performance computing to verify Einstein's theory of gravitational waves.
Funded by the U.S. Department of Energy (DOE) and National Science Foundation (NSF), OSG is a multi-disciplinary research partnership specializing in high throughput computational services.
The HPCwire awards were presented at the 2016 International Conference for High Performance Computing, Networking, Storage and Analysis (SC16), in Salt Lake City, Utah. OSG won in both the online publication's annual Readers' Choice and Editors' Choice categories.
The awards also recognize the San Diego Supercomputer Center (SDSC) at the University of California San Diego, the NSF's Extreme Science and Engineering Discovery Environment (XSEDE), the Holland Computing Center at University of Nebraska-Lincoln (UNL), the National Center for Supercomputing Applications (NCSA), and the Texas Advanced Computing Center (TACC) at The University of Texas at Austin for their participation in verifying the existence of gravitational waves.
OSG provided access to numerous high performance computing systems for LIGO, seamlessly integrating XSEDE supercomputers like SDSC's Comet and TACC's Stampede with HPC clusters in academia and DOE national labs. LIGO used this integrated infrastructure over the course of several months to verify the statistical significance of the observed gravitational wave, thus leading to its unambiguous detection.
In the process, roughly five terabytes of LIGO data was processed many thousands of times, leading to many petabytes of exported data. The storage infrastructure at Holland Computing Center hosted the LIGO data for processing via OSG. The data was pulled as needed by the Pegasus workflow.
Frank Würthwein, who is the current executive director of the OSG, worked in tandem with XSEDE researchers to make these resources available to LIGO scientists. Würthwein is also SDSC's head of High Throughput Computing.
"There were a lot of institutions and researchers involved in this landmark discovery, as well as the actual verification process," said Würthwein, also a physicist with UC San Diego. "While high performance computing resources from all over have been used by LIGO for years, this award focused on the use of distributed high throughput computing across a wide range of HPC resources for the actual verification of this amazing discovery."
"From thought leaders to end users, the HPCwire readership reaches and engages every corner of the high performance computing community," said Tom Tabor, CEO of Tabor Communications, publisher of HPCwire. "Receiving their recognition signifies community support across the entire HPC space as well as the breadth of industries it serves. We are proud to recognize these efforts and make the voices of our readers heard, and our congratulations go out to all the winners."
A multi-partner collaboration
In February 2016, the NSF made a pivotal announcement: For the first time, scientists detected gravitational waves in the universe as hypothesized by Albert Einstein about 100 years ago. On September 14, 2015 scientists at the NSF-funded Laser Interferometer Gravitational-Wave Observatory (LIGO) detected gravitational waves using both LIGO detectors. The waves reached Earth from the southern hemisphere, passed through the Earth, and emerged at the Earth's surface, first at the LIGO detector near Livingston, Louisiana, and then, seven milliseconds later and 1,890 miles away at the second LIGO detector in Hanford, Washington.
More details on the LIGO discovery can be found at:
https://www.opensciencegrid.org/osg-helps-ligo-scientists-confirm-einsteins-last-unproven-theory/
http://www.sdsc.edu/News%20Items/PR20160225_ligo.html
https://www.xsede.org/xsede-resources-help-confirm-ligo-discovery.
The annual HPCwire Readers' and Editors' Choice Awards are determined through a nomination and voting process with the global HPCwire community, as well as selections from the HPCwire editors. The awards are an annual feature of the publication and constitute prestigious recognition from the HPC community. These awards are revealed each year to kick off the annual Supercomputing Conference, which showcases high-performance computing, networking, storage, and data analysis. More information on these awards can be found at the HPCwire website or on Twitter through the #HPCwireAwards hashtag.
About OSG
The Open Science Grid Consortium is a community-driven organization that spans academia and Department of Energy national laboratories to advance the state of the art of distributed high throughput computing. In addition to community contributions, the OSG project receives funding from DOE and NSF to operate a fabric of services, including a production infrastructure, an integrated software stack, and a variety of intellectual support services for educators, scientists, and IT professionals.
About HPCwire
HPCwire is an online news and information resource covering the fastest computers in the world and the people who run them. Started in 1986, HPCwire has enjoyed a legacy of world-class editorial and journalism, making it the news source of choice for science, technology, and business professionals interested in high-performance and data-intensive computing. Visit HPCwire at www.hpcwire.com.
Media Contacts:
Kyle Gross, OSG communications lead, kagross@iu.edu
Chelsea Lang, corporate marketing manager, Tabor Communications, 919 749-1895
Related Links:
The Open Science Grid: https://www.opensciencegrid.org
National Science Foundation: https://www.nsf.gov/
Department of Energy, Office of Science: http://science.energy.gov
Pegasus: https://pegasus.isi.edu
LIGO: http://ligo.org/
Tuesday, November 8, 2016
Announcing OSG Software version 3.3.18
We are pleased to announce OSG Software version 3.3.18
Changes to OSG 3.3.18 include:
* GlideinWMS 3.2.16: can specify BOSCO user, start glidein manually, bug fixes
* SSLv3 is now disabled on several Globus tools
* Fix to prevent Globus GridFTP server process hangs
* Fixed edg-mkgridmap on Enterprise Linux 7
* PKI tools now generate CSRs using SHA2
* Fixed blahp qstat call to support torque-4.2.9
* Fixed crash in blahp when using glexec and limited proxies
* Augment Gratia PBS probe: process "exec_host", when "ALLPROCS" flag present
* GridFTP server script returns correct value when the service isn't running
* HTCondor-CE 2.0.11: Minor fixes
Release notes and pointers to more documentation can be found at:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/Release3318
Need help? Let us know:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/HelpProcedure
We welcome feedback on this release!
Changes to OSG 3.3.18 include:
* GlideinWMS 3.2.16: can specify BOSCO user, start glidein manually, bug fixes
* SSLv3 is now disabled on several Globus tools
* Fix to prevent Globus GridFTP server process hangs
* Fixed edg-mkgridmap on Enterprise Linux 7
* PKI tools now generate CSRs using SHA2
* Fixed blahp qstat call to support torque-4.2.9
* Fixed crash in blahp when using glexec and limited proxies
* Augment Gratia PBS probe: process "exec_host", when "ALLPROCS" flag present
* GridFTP server script returns correct value when the service isn't running
* HTCondor-CE 2.0.11: Minor fixes
Release notes and pointers to more documentation can be found at:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/Release3318
Need help? Let us know:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/HelpProcedure
We welcome feedback on this release!
Tuesday, November 1, 2016
GOC Service Update - Tuesday, November 8th at 14:00 UTC
The GOC will upgrade the following services beginning Tuesday, November 8th at 14:00 UTC. The GOC reserves 8 hours in the unlikely event unexpected problems are encountered.
Ticket
Configuration Changes tickets exchanged with BNL will be linked on the ticket view page
OIM
JIRA OIM-140: Retry CILogon host certs when we receive a "retry" error
VOMS
Remove obsolete voms for the CSIU, osg, osgcrossce and UC3 VOs
Ticket
Configuration Changes tickets exchanged with BNL will be linked on the ticket view page
OIM
JIRA OIM-140: Retry CILogon host certs when we receive a "retry" error
VOMS
Remove obsolete voms for the CSIU, osg, osgcrossce and UC3 VOs
Monday, October 24, 2016
GOC Service Update - Tuesday, October 25th at 13:00 UTC
The GOC will upgrade the following services beginning Tuesday, October 25th at 13:00 UTC.
The GOC reserves 8 hours in the unlikely event unexpected problems are encountered.
Ticket Exchange
* Configuration Changes
* Adding new exchange (XSEDE)
OIM
* Add “Created” timestamp field in OIM downtime table to preserve original log creation timestamp
Oasis
* Update the configuration for ligo.osgstorage.org to work better on Debian hosts.
MyOSG
* “Created” downtime timestamp to be available through reporting
All Services
* Operating system updates; reboots will be required. The usual HA mechanisms will be used, but some services will experience brief outages.
Thursday, October 20, 2016
Announcing OSG CA Certificate Package Update
We are pleased to announce our first data only release for the OSG Software Stack. Data releases will not contain any software changes. A typical data release would only have CA certificate and/or VO package changes.
This release contains updated CA Certificates based on IGTF 1.78
* Removed superseded INFN-CA-2006 CA (IT)
* Updated Debian packaging to support APT security improvements
* Updated namespaces and signing_policy files for CILogon Basic CA to permit DNs without "/C=US" (US)
* Added G2 series (sha-2) QuoVadis Root 2 and Grid ICA G2 (BM)
* Removed discontinued UniandesCA (CO)
Release notes and pointers to more documentation can be found at:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/Release33172
Need help? Let us know:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/HelpProcedure
We welcome feedback on this release!
This release contains updated CA Certificates based on IGTF 1.78
* Removed superseded INFN-CA-2006 CA (IT)
* Updated Debian packaging to support APT security improvements
* Updated namespaces and signing_policy files for CILogon Basic CA to permit DNs without "/C=US" (US)
* Added G2 series (sha-2) QuoVadis Root 2 and Grid ICA G2 (BM)
* Removed discontinued UniandesCA (CO)
Release notes and pointers to more documentation can be found at:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/Release33172
Need help? Let us know:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/HelpProcedure
We welcome feedback on this release!
Wednesday, October 19, 2016
End of OSG GRAM Support - 8 November 2016
As part of the OSG transition from GRAM to HTCondor-CE technology, OSG Operations and Software teams will stop supporting GRAM CE installations on November 8, 2016. A site that requests support for GRAM or GRAM-based software will be asked to upgrade to HTCondor-CE instead. However, GRAM packages will remain available for sites that accept full responsibility for the operation of GRAM.
Other milestones that will likely occur in early 2017:
* Ending support for GRAM CEs at the OSG pilot factories, which will leave many VOs unable to run at GRAM sites
* Dropping GRAM software from the OSG software stack (starting in OSG 3.4.0)
* Removing GRAM-based code from OSG tools like osg-configure
BACKGROUND
An OSG CE is the entry point for the OSG to your local resources. At the heart of the CE is the job gateway software, which accepts incoming jobs, authorizes them, and delegates them to your batch system for execution. These days, OSG jobs come from the OSG factories and are really pilot jobs, which in turn run actual end-user jobs. Since OSG started, the Globus GRAM gatekeeper has managed grid jobs, but starting in May 2014, the OSG added another job gateway option - HTCondor-CE - that is built on core HTCondor technology. For many reasons, OSG has decided to migrate all CEs to HTCondor-CE and is today in the midst of this change.
Since August 2014, new CE installs included both HTCondor-CE and GRAM, and since December 2014, HTCondor-CE is the default job gateway software for a new site. Today, HTCondor-CE is on 60 CEs and is the sole job gateway software on many of them; the list of sites that have already migrated includes some of the largest OSG sites, as well as medium and small ones.
This is a significant technical change for OSG, but the good news is that it is well underway and has been going very smoothly for most sites. If you are still running GRAM, please consider migrating to HTCondor-CE as soon as possible (you can run both for a while, if you like). Then, once your HTCondor-CE is running well, decommission the GRAM software.
Other milestones that will likely occur in early 2017:
* Ending support for GRAM CEs at the OSG pilot factories, which will leave many VOs unable to run at GRAM sites
* Dropping GRAM software from the OSG software stack (starting in OSG 3.4.0)
* Removing GRAM-based code from OSG tools like osg-configure
BACKGROUND
An OSG CE is the entry point for the OSG to your local resources. At the heart of the CE is the job gateway software, which accepts incoming jobs, authorizes them, and delegates them to your batch system for execution. These days, OSG jobs come from the OSG factories and are really pilot jobs, which in turn run actual end-user jobs. Since OSG started, the Globus GRAM gatekeeper has managed grid jobs, but starting in May 2014, the OSG added another job gateway option - HTCondor-CE - that is built on core HTCondor technology. For many reasons, OSG has decided to migrate all CEs to HTCondor-CE and is today in the midst of this change.
Since August 2014, new CE installs included both HTCondor-CE and GRAM, and since December 2014, HTCondor-CE is the default job gateway software for a new site. Today, HTCondor-CE is on 60 CEs and is the sole job gateway software on many of them; the list of sites that have already migrated includes some of the largest OSG sites, as well as medium and small ones.
This is a significant technical change for OSG, but the good news is that it is well underway and has been going very smoothly for most sites. If you are still running GRAM, please consider migrating to HTCondor-CE as soon as possible (you can run both for a while, if you like). Then, once your HTCondor-CE is running well, decommission the GRAM software.
Planned Retirement of OSG BDII
OSG Collaborators,
OSG Operations and Technology are planning the retirement of the BDII
information service located at is.grid.iu.edu on March 31st, 2017. We have
been working with WLCG, ATLAS and CMS to remove dependencies or replace
the functionality within our HTCondor Collector service. This work is
still ongoing. This message is to alert you to the upcoming deprecation
date and to get feedback on any other existing dependencies that might
exist to the BDII.
If you are dependent in any way on the OSG BDII or information the OSG
BDII supplies to the WLCG or EGI BDIIs please contact us at
Thursday, October 13, 2016
Announcing OSG Software version 3.3.17
We are pleased to announce OSG Software version 3.3.17.
Changes to OSG 3.3.17 include:
* HTCondor 8.4.9: Job Router prompts schedd reschedule, other bug fixes
* HTCondor-CE 2.0.10: handles unbounded accounting directory, other bug fixes
* Update gratia probe to work with more recent versions of Slurm
* Frontier-squid 2.7.STABLE9-27: fix unbounded growth of swap.state
* CVMFS 2.3.2: secure access to data in osgstorage.org repositories supported
* XRootD 4.4.0
* Tarballs contain RPM package version list
* Add fallback default in gratia probe for HTCondor-CE history folder
* Several configuration updates to better mesh with EL7 and systemd
* HTCondor 8.5.7 in upcoming: schedd can perform job ClassAd transformations
* VO Package v69: Added: miniclean VO; Removed: LNBE, CDF INFN
Release notes and pointers to more documentation can be found at:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/Release3317
Need help? Let us know:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/HelpProcedure
We welcome feedback on this release!
Changes to OSG 3.3.17 include:
* HTCondor 8.4.9: Job Router prompts schedd reschedule, other bug fixes
* HTCondor-CE 2.0.10: handles unbounded accounting directory, other bug fixes
* Update gratia probe to work with more recent versions of Slurm
* Frontier-squid 2.7.STABLE9-27: fix unbounded growth of swap.state
* CVMFS 2.3.2: secure access to data in osgstorage.org repositories supported
* XRootD 4.4.0
* Tarballs contain RPM package version list
* Add fallback default in gratia probe for HTCondor-CE history folder
* Several configuration updates to better mesh with EL7 and systemd
* HTCondor 8.5.7 in upcoming: schedd can perform job ClassAd transformations
* VO Package v69: Added: miniclean VO; Removed: LNBE, CDF INFN
Release notes and pointers to more documentation can be found at:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/Release3317
Need help? Let us know:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/HelpProcedure
We welcome feedback on this release!
Tuesday, October 4, 2016
GOC Service Update - Tuesday, October 11th at 13:00 UTC
The GOC will upgrade the following services beginning Tuesday, October 11th at 13:00 UTC.
The GOC reserves 8 hours in the unlikely event unexpected problems are encountered.
OIM
- reverting previous temporary fix for CILogon DNS issues, as issue has been fixed at DNS level
The GOC reserves 8 hours in the unlikely event unexpected problems are encountered.
OIM
- reverting previous temporary fix for CILogon DNS issues, as issue has been fixed at DNS level
Tuesday, September 20, 2016
GOC Service Update - Tuesday, September 27th at 13:00 UTC
The GOC will upgrade the following services beginning Tuesday, September 13th at 13:00 UTC. The GOC reserves 8 hours in the unlikely event unexpected problems are encountered.
OIM
* Changes to wording for instructions of command line host cert issuance
* Changes to DNS query for host cert issuance via CILogon. We will try to resolve the DNS address more than once on failure.
Perfsonar
* Configuration changes to extend 24-hour recovery limitations
Ticket Exchange
* Configuration changes
* GGUS synchronization format update
* FNAL synchronization format update
OSG Website
* Routine Wordpress version and plugin updates
All Services
* Operating system updates; reboots will be required. The usual HA mechanisms will be used, but some services will experience brief outages. Additionally, the primary and backup LDAP and DNS servers used internally will be exchanged to allow maintenance on the original primary server
OIM
* Changes to wording for instructions of command line host cert issuance
* Changes to DNS query for host cert issuance via CILogon. We will try to resolve the DNS address more than once on failure.
Perfsonar
* Configuration changes to extend 24-hour recovery limitations
Ticket Exchange
* Configuration changes
* GGUS synchronization format update
* FNAL synchronization format update
OSG Website
* Routine Wordpress version and plugin updates
All Services
* Operating system updates; reboots will be required. The usual HA mechanisms will be used, but some services will experience brief outages. Additionally, the primary and backup LDAP and DNS servers used internally will be exchanged to allow maintenance on the original primary server
Tuesday, September 13, 2016
Announcing OSG Software version 3.3.16
We are pleased to announce OSG Software version 3.3.16.
Changes to OSG 3.3.16 include:
* Updated most Globus Packages to latest available from EPEL
* Note: Now Globus Toolkit strictly checks host names against certificates
* BLAHP 1.18.25: Additional features supported for SGE, PBS Pro, and Slurm
* Update to GlideinWMS 3.2.15
* Fixed major scalability problem in GUMS on EL7
* HTCondor-CE 2.0.8: Support for Terana eScience, minor bug fixes
* The MyProxy server now produces RFC compliant proxies
* Fixed load-balancing in Globus GridFTP when using IPv6 addresses
* Added the HTCondor CREAM GAHP for EL7 platforms
* Completed porting components of OSG Software Stack to EL7
* Added RSV GlideinWMS Tester for VO Front-ends to test site support
* Updated to lcas-lcmaps-gt4-interface to version 0.3.1
* VO Package v68: Added project8 VO
Release notes and pointers to more documentation can be found at:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/Release3316
Need help? Let us know:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/HelpProcedure
We welcome feedback on this release!
Changes to OSG 3.3.16 include:
* Updated most Globus Packages to latest available from EPEL
* Note: Now Globus Toolkit strictly checks host names against certificates
* BLAHP 1.18.25: Additional features supported for SGE, PBS Pro, and Slurm
* Update to GlideinWMS 3.2.15
* Fixed major scalability problem in GUMS on EL7
* HTCondor-CE 2.0.8: Support for Terana eScience, minor bug fixes
* The MyProxy server now produces RFC compliant proxies
* Fixed load-balancing in Globus GridFTP when using IPv6 addresses
* Added the HTCondor CREAM GAHP for EL7 platforms
* Completed porting components of OSG Software Stack to EL7
* Added RSV GlideinWMS Tester for VO Front-ends to test site support
* Updated to lcas-lcmaps-gt4-interface to version 0.3.1
* VO Package v68: Added project8 VO
Release notes and pointers to more documentation can be found at:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/Release3316
Need help? Let us know:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/HelpProcedure
We welcome feedback on this release!
Scheduled FermiLab Power Outage
This weekend, Fermilab will have a scheduled power outage in the Feynman Computer Center to repair an automatic power transfer switch. The transfer switch ensures that the lab’s computing services have redundant power. This scheduled outage will cause many services at the laboratory to be unavailable.
The outage date is Saturday, Sept. 17, the same day as a scheduled Wilson Hall cooling outage, and is expected to last more than 8 hours. Services may be affected starting Friday, September 16 at 4:00 PM and are estimated to be restored by Saturday at 6:00 PM, though the outage could last longer. (All times US central) At this time we expect Fermilab email, listserv and analog telephones to be operational.
The Open Science Grid services expected to be affected include the OSG VOMS, Gratia, Indico and Docdb. One (of three) oasis replica will be out of service and replaced temporarily by another elsewhere. Details provided by FNAL can be found here: https://fermi.service-now.com/kb_view_customer.do?sysparm_article=KB0012205
Updates will be provided via Twitter throughout the outage, so follow the Service Desk at https://twitter.com/FNALServiceDesk to stay informed.
The outage date is Saturday, Sept. 17, the same day as a scheduled Wilson Hall cooling outage, and is expected to last more than 8 hours. Services may be affected starting Friday, September 16 at 4:00 PM and are estimated to be restored by Saturday at 6:00 PM, though the outage could last longer. (All times US central) At this time we expect Fermilab email, listserv and analog telephones to be operational.
The Open Science Grid services expected to be affected include the OSG VOMS, Gratia, Indico and Docdb. One (of three) oasis replica will be out of service and replaced temporarily by another elsewhere. Details provided by FNAL can be found here: https://fermi.service-now.com/kb_view_customer.do?sysparm_article=KB0012205
Updates will be provided via Twitter throughout the outage, so follow the Service Desk at https://twitter.com/FNALServiceDesk to stay informed.
Wednesday, September 7, 2016
Emergency Maintenance - psds0.grid.iu.edu - Wednesday, September 7th from 13:00-14:00 EDT
psds0.grid.iu.edu will be unavailable from 13:00-14:00 EDT while GOC Engineers perform maintenance to increase the amount of memory resources available to resolve
an issue with data collection. The GOC regrets any inconvenience this may cause.
an issue with data collection. The GOC regrets any inconvenience this may cause.
Tuesday, September 6, 2016
GOC Service Update - Tuesday, September 13th at 13:00 UTC
The GOC will upgrade the following services beginning Tuesday, September 13th at 13:00 UTC. The GOC reserves 8 hours in the unlikely event unexpected problems are encountered.
Ticket, Ticket Exchange, OIM
- Ticket update
- OIM Revocation Digicert user certs and host certs
- OIM message for CMS and Atlas users
- OIM change default certificate signer from DigiCert to CILogon
Ticket, Ticket Exchange, OIM
- Ticket update
- OIM Revocation Digicert user certs and host certs
- OIM message for CMS and Atlas users
- OIM change default certificate signer from DigiCert to CILogon
Monday, August 22, 2016
TWiki Outage Update
We have completed the previously announced restoration of the TWiki to its state as of Monday 15/Aug. The system is behaving normally at this time but we request you contact us if you encounter any unusual behavior.
We are also in the process of recovering changes to content made between Monday 15/Aug and Friday 19/Aug. If you have any content you need restored and would like prioritized, please let us know.
The GOC regrets any inconvenience and is taking steps to insure this will not reoccur.
We are also in the process of recovering changes to content made between Monday 15/Aug and Friday 19/Aug. If you have any content you need restored and would like prioritized, please let us know.
The GOC regrets any inconvenience and is taking steps to insure this will not reoccur.
Friday, August 19, 2016
TWiki Outage
We are currently encountering difficulties with TWiki and are restoring the
service from backup to its state as of Monday 15/Aug. We will attempt to recover
changes made after that time and will appraise you as to the results of our efforts.
The GOC regrets any inconvenience and will inform you as soon as resolution and
further information is available.
service from backup to its state as of Monday 15/Aug. We will attempt to recover
changes made after that time and will appraise you as to the results of our efforts.
The GOC regrets any inconvenience and will inform you as soon as resolution and
further information is available.
Tuesday, August 16, 2016
GOC Service Update - Tuesday, August 23rd
The GOC will upgrade the following services beginning Tuesday, August 23rd at 13:00 UTC. The GOC reserves 8 hours in the unlikely event unexpected problems are encountered.
PerfSonar
Updates to esmond, rsv-perfsonar
Collector, Redirector, Ticket, Ticket exchange, OIM
Rebuild from new content management system
OIM, change default certificate signer from DigiCert to CILogon
Glidein
Upgrade GlideinWMS to 3.2.14.1 VOs planning to run GlideinWMS 3.2.15 on their frontends will require all factories to run >= 3.2.14.1
WWW
Updates to wordpress
All Services
Operating system updates; reboots will be required. The usual HA mechanisms will be used, but some services will experience brief outages.
PerfSonar
Updates to esmond, rsv-perfsonar
Collector, Redirector, Ticket, Ticket exchange, OIM
Rebuild from new content management system
OIM, change default certificate signer from DigiCert to CILogon
Glidein
Upgrade GlideinWMS to 3.2.14.1 VOs planning to run GlideinWMS 3.2.15 on their frontends will require all factories to run >= 3.2.14.1
WWW
Updates to wordpress
All Services
Operating system updates; reboots will be required. The usual HA mechanisms will be used, but some services will experience brief outages.
Tuesday, August 9, 2016
Announcing OSG Software versions 3.3.15 and 3.2.41
We are pleased to announce OSG Software versions 3.3.15 and 3.2.41.
This is the last release in the 3.2 series.
Both 3.3.15 and 3.2.41 include:
* CA certificates based on IGTF 1.76
* VO Package v67 - correction for ILC
Changes to OSG 3.3.15 include:
* SLURM scalability enhancements in the BLAHP
* Fixed a bug in the BLAHP where HTCondor could not remove a SLURM job
* Enable XRootD-HDFS to use native HDFS libraries if available
* Add an extension to the GridFTP server to report space usage on the server
* Fix GUMS to properly display long Pool Account lists
* The RSV service will start even though its state file is corrupt
* Update GSI-OpenSSH from 5.7-4.3 to 7.1p2f
* voms-proxy-init generates RFC compliant proxies by default
* Configure voms-server for systemd startup in EL7
* Add voms-admin-client for EL7
* HTCondor 8.5.6 in the upcoming repository
Release notes and pointers to more documentation can be found at:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/Release3315
https://www.opensciencegrid.org/bin/view/Documentation/Release3/Release3241
Need help? Let us know:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/HelpProcedure
We welcome feedback on this release!
This is the last release in the 3.2 series.
Both 3.3.15 and 3.2.41 include:
* CA certificates based on IGTF 1.76
* VO Package v67 - correction for ILC
Changes to OSG 3.3.15 include:
* SLURM scalability enhancements in the BLAHP
* Fixed a bug in the BLAHP where HTCondor could not remove a SLURM job
* Enable XRootD-HDFS to use native HDFS libraries if available
* Add an extension to the GridFTP server to report space usage on the server
* Fix GUMS to properly display long Pool Account lists
* The RSV service will start even though its state file is corrupt
* Update GSI-OpenSSH from 5.7-4.3 to 7.1p2f
* voms-proxy-init generates RFC compliant proxies by default
* Configure voms-server for systemd startup in EL7
* Add voms-admin-client for EL7
* HTCondor 8.5.6 in the upcoming repository
Release notes and pointers to more documentation can be found at:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/Release3315
https://www.opensciencegrid.org/bin/view/Documentation/Release3/Release3241
Need help? Let us know:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/HelpProcedure
We welcome feedback on this release!
Tuesday, August 2, 2016
GOC Service Update Tuesday, August 9th at 13:00 UTC
The GOC will upgrade the following services beginning Tuesday, August 9th at 13:00 UTC. The GOC reserves 8 hours in the unlikely event unexpected problems are encountered.
www
Modifications to configuration for bosco
Will not request a certificate
Redirect to https
OASIS
Add configuration for /cvmfs/cernvm-devel.cern.ch to the config-osg.opensciencegrid.org repository
www
Modifications to configuration for bosco
Will not request a certificate
Redirect to https
OASIS
Add configuration for /cvmfs/cernvm-devel.cern.ch to the config-osg.opensciencegrid.org repository
Tuesday, July 19, 2016
GOC Service Update Tuesday, July 26th at 13:00 UTC
The GOC will upgrade the following services beginning Tuesday, July 26th at 13:00 UTC. The GOC reserves 8 hours in the unlikely event unexpected problems are encountered.
Condor Collector
Adding the service to new content management system.
Oasis-login
Adding the service to new content management system.
OIM
No longer suppress emails during the host cert request process when the submitter is a GA
StashCache Redirector
Adding the service to new content management system.
All Services
Operating system updates; reboots will be required.
The usual HA mechanisms will be used, but some services will experience brief outages.
Condor Collector
Adding the service to new content management system.
Oasis-login
Adding the service to new content management system.
OIM
No longer suppress emails during the host cert request process when the submitter is a GA
StashCache Redirector
Adding the service to new content management system.
All Services
Operating system updates; reboots will be required.
The usual HA mechanisms will be used, but some services will experience brief outages.
Friday, July 15, 2016
Planned Retirement of OSG BDII Information Service (is.grid.iu.edu) - March 31st, 2017
OSG Collaborators,
OSG Operations and Technology are planning the retirement of the BDII information service located at is.grid.iu.edu on March 31st, 2017. We have been working with WLCG, ATLAS and CMS to remove dependencies or replace the functionality within our HTCondor Collector service. This work is still ongoing. This message is to alert you to the upcoming deprecation date and to get feedback on any other existing dependencies that might exist to the BDII.
If you are dependent in any way on the OSG BDII or information the OSG BDII supplies to the WLCG or EGI BDIIs please contact us at goc@opensciencegrid.org.
OSG Operations and Technology are planning the retirement of the BDII information service located at is.grid.iu.edu on March 31st, 2017. We have been working with WLCG, ATLAS and CMS to remove dependencies or replace the functionality within our HTCondor Collector service. This work is still ongoing. This message is to alert you to the upcoming deprecation date and to get feedback on any other existing dependencies that might exist to the BDII.
If you are dependent in any way on the OSG BDII or information the OSG BDII supplies to the WLCG or EGI BDIIs please contact us at goc@opensciencegrid.org.
Wednesday, July 13, 2016
Requested Update of CVMFS Client Version for NoVA experiment
FabrIc for Frontier Experiments (FIFE) and the NoVA experiment are dependent on CVMFS Client version 2.2.3 or later.
OSG Operations requests all sites that would like to support FIFE and NoVA workflows update to the latest version of the CVMFS Client.
Details on the current release and update instructions can be found at
https://twiki.opensciencegrid.org/bin/view/Documentation/Release3/Release3314.
Thank you for your effort in keeping your OSG installation up to date.
OSG Operations requests all sites that would like to support FIFE and NoVA workflows update to the latest version of the CVMFS Client.
Details on the current release and update instructions can be found at
https://twiki.opensciencegrid.org/bin/view/Documentation/Release3/Release3314.
Thank you for your effort in keeping your OSG installation up to date.
Tuesday, July 12, 2016
Announcing OSG Software versions 3.3.14 and 3.2.40
We are pleased to announce OSG Software versions 3.3.14 and 3.2.40.
Both 3.3.14 and 3.2.40 include:
* CA certificates based on IGTF 1.75
Changes to OSG 3.3.13 include:
* HTCondor 8.4.8: bug fixes for bosco, schedd crash, memory leak using python
* GlideinWMS 3.2.14.1: Improved efficiency, works with any HTCondor version
* HTCondor-CE 2.0.7: Add htcondor-ce-bosco sub-package
* BLAHP 1.18.21: SLURM improvements
* gridFTP 7.30-1.2: adler32 checksum support, fix deadlock
* osg-configure 1.4.1: improved configuration of bosco, GUMS
* xrootd-voms-plugin 0.4.0: added support for 'all' group selection
* osg-system-profiler 1.4.0: detect unconfigured trustmanager
* gridFTP-HDFS 0.5.4: fixed ability to list/remove empty directories
* cvmfs-config-osg 1.2.5: use new CVMFS fall-back policies
* bigtop-utils: Fix default JAVA_HOME to prevent crash in hdfs utils
* osg-voms 3.3-3: Remove voms-admin (EL7 only)
Release notes and pointers to more documentation can be found at:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/Release3314
https://www.opensciencegrid.org/bin/view/Documentation/Release3/Release3240
Need help? Let us know:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/HelpProcedure
We welcome feedback on this release!
Both 3.3.14 and 3.2.40 include:
* CA certificates based on IGTF 1.75
Changes to OSG 3.3.13 include:
* HTCondor 8.4.8: bug fixes for bosco, schedd crash, memory leak using python
* GlideinWMS 3.2.14.1: Improved efficiency, works with any HTCondor version
* HTCondor-CE 2.0.7: Add htcondor-ce-bosco sub-package
* BLAHP 1.18.21: SLURM improvements
* gridFTP 7.30-1.2: adler32 checksum support, fix deadlock
* osg-configure 1.4.1: improved configuration of bosco, GUMS
* xrootd-voms-plugin 0.4.0: added support for 'all' group selection
* osg-system-profiler 1.4.0: detect unconfigured trustmanager
* gridFTP-HDFS 0.5.4: fixed ability to list/remove empty directories
* cvmfs-config-osg 1.2.5: use new CVMFS fall-back policies
* bigtop-utils: Fix default JAVA_HOME to prevent crash in hdfs utils
* osg-voms 3.3-3: Remove voms-admin (EL7 only)
Release notes and pointers to more documentation can be found at:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/Release3314
https://www.opensciencegrid.org/bin/view/Documentation/Release3/Release3240
Need help? Let us know:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/HelpProcedure
We welcome feedback on this release!
Monday, July 11, 2016
Registration open for free Workflows Workshop
Registration is now open for a free Workflows Workshop to be held August 9-10 at multiple institutions across the country. Sponsored by the Blue Waters sustained-petascale computing project, this workshop will provide an overview of workflows and how they can enhance research productivity.
A general session on the value of workflows will be followed by presentations and hands-on sessions with six different workflows. The objective is to assist the community in understanding the capabilities of these various workflows and to get people started with their usage. These include:
- General overview of workflows; Why use them?, presented by Scott Callaghan, University of Southern California
- Copernicus, presented by Peter Kasson, University of Virgini
- Galaxy, presented by Dave Clements, Johns Hopkins University
- Makeflow/WorkQueue, presented by Nicholas Hazekamp, University of Notre Dame
- Pegasus, presented by Karan Vahi and Mats Rynge, Information Sciences Institute
- RADICAL Cybertools, presented by Shantenu Jha, Rutgers University
- Swift, presented by Mike Wilde, Argonne National Laboratory
The presentations will be followed by a question and answer period to address questions from the community. Additional information on the workshop is available at https://sites.google.com/a/illinois.edu/workflows-workshop/home
The sites hosting this workshop include:
Georgia State University
NCSA University of Illinois at Urbana-Champaign
Michigan State University
Oklahoma State University
Purdue University
Stanford University
Texas Tech University
UCLA
University of Kentucky
University of Houston
University of Utah
University of Wyoming
You may register for this workshop through the XSEDE User Portal at: https://portal.xsede.org/course-calendar by August 2, 2016. There is a registration button for each site, be sure you select the site where you will be attending.
The Blue Waters sustained-petascale computing project is supported by the National Science Foundation (awards OCI-0725070 and ACI-1238993) and the state of Illinois. Blue Waters is a joint effort of the University of Illinois at Urbana-Champaign and its National Center for Supercomputing Applications.
A general session on the value of workflows will be followed by presentations and hands-on sessions with six different workflows. The objective is to assist the community in understanding the capabilities of these various workflows and to get people started with their usage. These include:
- General overview of workflows; Why use them?, presented by Scott Callaghan, University of Southern California
- Copernicus, presented by Peter Kasson, University of Virgini
- Galaxy, presented by Dave Clements, Johns Hopkins University
- Makeflow/WorkQueue, presented by Nicholas Hazekamp, University of Notre Dame
- Pegasus, presented by Karan Vahi and Mats Rynge, Information Sciences Institute
- RADICAL Cybertools, presented by Shantenu Jha, Rutgers University
- Swift, presented by Mike Wilde, Argonne National Laboratory
The presentations will be followed by a question and answer period to address questions from the community. Additional information on the workshop is available at https://sites.google.com/a/illinois.edu/workflows-workshop/home
The sites hosting this workshop include:
Georgia State University
NCSA University of Illinois at Urbana-Champaign
Michigan State University
Oklahoma State University
Purdue University
Stanford University
Texas Tech University
UCLA
University of Kentucky
University of Houston
University of Utah
University of Wyoming
You may register for this workshop through the XSEDE User Portal at: https://portal.xsede.org/course-calendar by August 2, 2016. There is a registration button for each site, be sure you select the site where you will be attending.
The Blue Waters sustained-petascale computing project is supported by the National Science Foundation (awards OCI-0725070 and ACI-1238993) and the state of Illinois. Blue Waters is a joint effort of the University of Illinois at Urbana-Champaign and its National Center for Supercomputing Applications.
Tuesday, July 5, 2016
GOC Service Update Tuesday, July 12th at 13:00 UTC
The GOC will upgrade the following services beginning Tuesday, July 12th at 13:00 UTC. The GOC reserves 8 hours in the unlikely event unexpected problems are encountered.
GratiaWeb
Adding PNG image generation capability (deferred from previous release)
Oasis
Reinstalling oasis-login with new management system
GratiaWeb
Adding PNG image generation capability (deferred from previous release)
Oasis
Reinstalling oasis-login with new management system
Friday, July 1, 2016
OSG Security Position Posting at Indiana University
As of OSG year 5, the Indiana University Center for Applied Cybersecurity
Research (CACR) is joining the OSG security team. As part of that effort,
CACR has an open position (closing July 7th) suitable for an
entry-to-medium level analyst.
We would encourage anyone in the community with interest in this position
to apply. If you know someone who might be interested in the position to
please point them to the link listed below:
https://iujobs.peopleadmin.com/postings/25274
Research (CACR) is joining the OSG security team. As part of that effort,
CACR has an open position (closing July 7th) suitable for an
entry-to-medium level analyst.
We would encourage anyone in the community with interest in this position
to apply. If you know someone who might be interested in the position to
please point them to the link listed below:
https://iujobs.peopleadmin.com/postings/25274
Friday, June 24, 2016
OSG website available
Colleagues,
The OSG Website (https://www.opensciencegrid.org/) was unavailable due to maintenance. No other operations services were affected. System engineers restored service at approximately 10:34am Eastern. We apologize for any inconvenience this caused.
The OSG Website (https://www.opensciencegrid.org/) was unavailable due to maintenance. No other operations services were affected. System engineers restored service at approximately 10:34am Eastern. We apologize for any inconvenience this caused.
OSG website currently unavailable
The OSG Website (https://www.opensciencegrid.org/) is currently unavailable due to maintenance. No other operations services are affected at this time. System engineers are currently working to restore service. We will let you know when things have returned to normal. Sorry for any inconvenience.
Tuesday, June 21, 2016
GOC Service Update Tuesday, June 28th at 13:00 UTC
The GOC will upgrade the following services beginning Tuesday, June 28th at 13:00 UTC. The GOC reserves 8 hours in the unlikely event unexpected problems are encountered.
GratiaWeb
Adding PNG image generation capability
OIM
Adding error handling for CILogon errors during cert request
Repo
Updating mash configuration files
WWW
Adding rate limiter to minimize DoS possibilities
Disabling a vulnerable service component
All Services
Operating system updates; reboots will be required.
The usual HA mechanisms will be used, but some services will experience brief outages.
GratiaWeb
Adding PNG image generation capability
OIM
Adding error handling for CILogon errors during cert request
Repo
Updating mash configuration files
WWW
Adding rate limiter to minimize DoS possibilities
Disabling a vulnerable service component
All Services
Operating system updates; reboots will be required.
The usual HA mechanisms will be used, but some services will experience brief outages.
Tuesday, June 14, 2016
Announcing OSG Software versions 3.3.13 and 3.2.39
We are pleased to announce OSG Software versions 3.3.13 and 3.2.39.
NOTE: There is a known incompatibility between GlideinWMS and HTCondor
8.4.7 and 8.5.5. Please read the release notes for further details.
Both 3.3.13 and 3.2.39 include:
* CA certificates based on IGTF 1.74
Changes to OSG 3.3.13 include:
* GridFTP-HDFS: improve buffering and avoid deadlocks
* CVMFS 2.2.3: bug fix for chunked files
* jGlobus: patched to reduce resource consumption, no need to restart BeStMan
* HTCondor 8.4.7: bug fixes
* BLAHP 1.18.20: dynamic environment variables, multi-core HTCondor jobs
* RSV: support for CREAM and NorduGrid
* GUMS: Now available on EL7 platforms!
* BeStMan: minor improvements (see notes) and a new OSG code base
* HTCondor 8.5.5 in the Upcoming repository
* CVMFS 2.3.0 in the Upcoming repository
Release notes and pointers to more documentation can be found at:
Documentation Release3: Release3313
Documentation Release3: Release3239
Need help? Let us know:
Documentation Release3: HelpProcedure
We welcome feedback on this release!
NOTE: There is a known incompatibility between GlideinWMS and HTCondor
8.4.7 and 8.5.5. Please read the release notes for further details.
Both 3.3.13 and 3.2.39 include:
* CA certificates based on IGTF 1.74
Changes to OSG 3.3.13 include:
* GridFTP-HDFS: improve buffering and avoid deadlocks
* CVMFS 2.2.3: bug fix for chunked files
* jGlobus: patched to reduce resource consumption, no need to restart BeStMan
* HTCondor 8.4.7: bug fixes
* BLAHP 1.18.20: dynamic environment variables, multi-core HTCondor jobs
* RSV: support for CREAM and NorduGrid
* GUMS: Now available on EL7 platforms!
* BeStMan: minor improvements (see notes) and a new OSG code base
* HTCondor 8.5.5 in the Upcoming repository
* CVMFS 2.3.0 in the Upcoming repository
Release notes and pointers to more documentation can be found at:
Documentation Release3: Release3313
Documentation Release3: Release3239
Need help? Let us know:
Documentation Release3: HelpProcedure
We welcome feedback on this release!
Tuesday, June 7, 2016
GOC Service Update Tuesday, June 14th at 13:00 UTC
The GOC will upgrade the following services beginning Tuesday, June 14th at 13:00 UTC. The GOC reserves 8 hours in the unlikely event unexpected problems are encountered.
Oasis
* Upgrade oasis and oasis-replica to cvmfs-2.2.2 and oasis-goc-2.1.9
* Add tool to copy config-osg repository from oasis-itb to oasis
* Update config-osg configuration of osgstorage.org repositories
OIM
* Adding the CN text describing the host name in all emails related to the certificate process
Oasis
* Upgrade oasis and oasis-replica to cvmfs-2.2.2 and oasis-goc-2.1.9
* Add tool to copy config-osg repository from oasis-itb to oasis
* Update config-osg configuration of osgstorage.org repositories
OIM
* Adding the CN text describing the host name in all emails related to the certificate process
Tuesday, May 17, 2016
GOC Service Update Tuesday, May 24th at 13:00 UTC
The GOC will upgrade the following services beginning Tuesday, May 24th at 13:00 UTC. The GOC reserves 8 hours in the unlikely event unexpected problems are encountered.
Condor Collector
Install HTCondor-CE-View, a web based monitoring tool that displays attributes of HTCondor-CEs
Oasis
Conduct a security drill, temporarily blank two repositories
OIM
Color coding Search by Signer results for user certs (OSGPKI-400)
TICKET
Preserving formatting in blog posts (TICKET-124)
WWW
Prevent Apache from requiring a client certificate (HOME-5, 29653) Configure Apache to redirect http to https (same tickets)
All services
Operating system updates; reboots will be required. The usual HA mechanisms will be used, but some services will experience brief outages.
Condor Collector
Install HTCondor-CE-View, a web based monitoring tool that displays attributes of HTCondor-CEs
Oasis
Conduct a security drill, temporarily blank two repositories
OIM
Color coding Search by Signer results for user certs (OSGPKI-400)
TICKET
Preserving formatting in blog posts (TICKET-124)
WWW
Prevent Apache from requiring a client certificate (HOME-5, 29653) Configure Apache to redirect http to https (same tickets)
All services
Operating system updates; reboots will be required. The usual HA mechanisms will be used, but some services will experience brief outages.
Tuesday, May 10, 2016
Announcing OSG Software versions 3.3.12 and 3.2.38
We are pleased to announce OSG Software versions 3.3.12 and 3.2.38.
Both 3.3.12 and 3.2.38 include:
* VO Package v66 - final wave of OSG CA transitions
Changes to OSG 3.3.12 include:
* HTCondor-CE-BOSCO: Setup a CE with only SSH access to the cluster
* HTCondor 8.4.6: Fixes serious regression in HTCondor 8.4.5
* CVMFS 2.2.2: See Release Notes for important upgrade information
* OSG CE packages will no longer install GRAM components
* BeStMan on EL7 platforms
* HTCondor-CE 2.0.5: HTCondor-CE-CEView bug fixes, support HTCondor-CE-BOSCO
* BLAHP 1.18.19: Fix memory request for PBS, updated SLURM support
* Pegasus 4.6.1: Updated from version 4.3.1
* osg-pki-tools 1.2.17: Fix timeout handling, improved error messages
* osg-configure 1.4.0: configuration of HTCondor-CE-BOSCO plus other changes
* HTCondor 8.5.4 in the Upcoming repository
Release notes and pointers to more documentation can be found at:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/Release3238
https://www.opensciencegrid.org/bin/view/Documentation/Release3/Release3312
Need help? Let us know:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/HelpProcedure
We welcome feedback on this release!
Both 3.3.12 and 3.2.38 include:
* VO Package v66 - final wave of OSG CA transitions
Changes to OSG 3.3.12 include:
* HTCondor-CE-BOSCO: Setup a CE with only SSH access to the cluster
* HTCondor 8.4.6: Fixes serious regression in HTCondor 8.4.5
* CVMFS 2.2.2: See Release Notes for important upgrade information
* OSG CE packages will no longer install GRAM components
* BeStMan on EL7 platforms
* HTCondor-CE 2.0.5: HTCondor-CE-CEView bug fixes, support HTCondor-CE-BOSCO
* BLAHP 1.18.19: Fix memory request for PBS, updated SLURM support
* Pegasus 4.6.1: Updated from version 4.3.1
* osg-pki-tools 1.2.17: Fix timeout handling, improved error messages
* osg-configure 1.4.0: configuration of HTCondor-CE-BOSCO plus other changes
* HTCondor 8.5.4 in the Upcoming repository
Release notes and pointers to more documentation can be found at:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/Release3238
https://www.opensciencegrid.org/bin/view/Documentation/Release3/Release3312
Need help? Let us know:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/HelpProcedure
We welcome feedback on this release!
Tuesday, May 3, 2016
The GOC will upgrade the following services beginning Tuesday, May 10th at 13:00 UTC.
The GOC will upgrade the following services beginning Tuesday, May 10th at 13:00 UTC. The GOC reserves 8 hours in the unlikely event unexpected problems are encountered.
MyOSG
MyOSG
- Adding CILogon status to the status board OIM
- Support for email signing will be added to CILogon OSG CA user certificates.
- OSG users will be able to utilize their OSG personal certificate to receive encrypted email and prove email came from them.”
- Fixing revocation for Host certificates - Adding “Search by Signer” feature for host certs
Upcoming gratia outage
Gratia will undergo emergency maintenance this afternoon to address
disk space issues. Exact timing of the outage is unknown but is being
determined at this time.
Unstable and/or unavailable behavior should be expected but no loss of data is anticipated. Some GOC services (GratiaWeb and display.opensciencegrid.org, for example) will also be affected.
The OSG Operations apologizes for any inconvenience you may encounter.
Unstable and/or unavailable behavior should be expected but no loss of data is anticipated. Some GOC services (GratiaWeb and display.opensciencegrid.org, for example) will also be affected.
The OSG Operations apologizes for any inconvenience you may encounter.
Tuesday, April 19, 2016
GOC Service Update - Tuesday, April 26, 2016 at 13:00 UTC
The GOC will upgrade the following services beginning Tuesday, April 26 at 13:00 UTC. The GOC reserves 8 hours in the unlikely event unexpected problems are encountered.
GratiaWeb
Fix for project accounting plots
inclusion of HTML site for Google Charts
inclusion of WLCG reporting link
OIM
Increase the number of Security and Administrative contacts for a resource - release 3.52
All services
Operating system updates; reboots will be required. The usual HA mechanisms will be used, but some services will experience brief outages.
GratiaWeb
Fix for project accounting plots
inclusion of HTML site for Google Charts
inclusion of WLCG reporting link
OIM
Increase the number of Security and Administrative contacts for a resource - release 3.52
All services
Operating system updates; reboots will be required. The usual HA mechanisms will be used, but some services will experience brief outages.
Friday, April 15, 2016
HTCondor 8.4.5 causes problems with partitionable slots
HTCondor 8.4.5 (released in OSG 3.3.11 on Tuesday, April 12) contains a bug that may affect some sites significantly. If your site uses partitionable slots on execute nodes, OSG recommends that you avoid HTCondor 8.4.5, either by skipping the update or reverting to a previous version (e.g., HTCondor 8.4.4).
(Tech details: Jobs that land on partitionable slots will fail to start about 10% of the time. The failed job correctly returns to the queue in Idle state, and HTCondor will continue trying to match and run the job which should complete eventually. But the recycling process adds inefficiencies into the overall system.)
The HTCondor development and OSG Software teams discovered the bug after the OSG release, and plan to release a patched version soon.
(Tech details: Jobs that land on partitionable slots will fail to start about 10% of the time. The failed job correctly returns to the queue in Idle state, and HTCondor will continue trying to match and run the job which should complete eventually. But the recycling process adds inefficiencies into the overall system.)
The HTCondor development and OSG Software teams discovered the bug after the OSG release, and plan to release a patched version soon.
Thursday, April 14, 2016
OSG GRAM Support Ending November 2016
As part of the OSG transition from GRAM to HTCondor-CE technology, OSG Operations and Software teams will stop supporting GRAM CE installations in early November. A site that requests support for GRAM or GRAM-based software will be asked to upgrade to HTCondor-CE instead. However, GRAM packages will remain available for sites that accept full responsibility for operation of GRAM.
Toward that goal, OSG CE convenience meta-packages (e.g., osg-ce-condor) will drop GRAM components starting with the April 12 release of OSG 3.3. The change will affect only new installations that use the osg-ce-* packages — OSG will still ship GRAM software, will document how to install GRAM, and will support GRAM in tools like osg-configure.
Other milestones that will likely occur in early 2017:
* Ending support for GRAM CEs at the OSG pilot factories, which will leave many VOs unable to run at GRAM sites
* Dropping GRAM software from the OSG software stack (starting in OSG 3.4.0)
* Removing GRAM-based code from OSG tools like osg-configure
BACKGROUND
An OSG CE is the entry point for the OSG to your local resources. At the heart of the CE is the job gateway software, which accepts incoming jobs, authorizes them, and delegates them to your batch system for execution. These days, OSG jobs come from the OSG factories and are really pilot jobs, which in turn run actual end-user jobs. Since OSG started, the Globus GRAM gatekeeper has managed grid jobs, but starting in May 2014, the OSG added another job gateway option — HTCondor-CE — that is built on core HTCondor technology. For many reasons, OSG has decided to migrate all CEs to HTCondor-CE and is today in the midst of this change.
Since August 2014, new CE installs included both HTCondor-CE and GRAM, and since December 2014, HTCondor-CE is the default job gateway software for a new site. Today, HTCondor-CE is on 60 CEs and is the sole job gateway software on many of them; the list of sites that have already migrated includes some of the largest OSG sites, as well as medium and small ones.
This is a significant technical change for OSG, but the good news is that it is well underway and has been going very smoothly for most sites. If you are still running GRAM, please consider migrating to HTCondor-CE as soon as possible (you can run both for a while, if you like). Then, once your HTCondor-CE is running well, decommission the GRAM software.
Tuesday, April 12, 2016
Reminder about the OSG User School 2016
REMINDER:
The deadline for applications is this Friday, April 15!!!
Please help spread the word, especially through personal connections to people who might benefit from the School. Announcing the Open Science Grid User School 2016! If you could access hundreds, thousands, or even more computers for your scholarly work, what could you do? How could it transform your work? What discoveries might you make? We are seeking applicants for the Open Science Grid (OSG) User School 2016, which takes place 25 - 29 July at the beautiful University of Wisconsin in Madison.
Participants will learn to use high throughput computing (HTC) to harness vast amounts of computing power for research, applicable to nearly any field of study (e.g., physics, chemistry, engineering, life sciences, earth sciences, agricultural and animal sciences, economics, social sciences, medicine, and more). Using lectures, discussions, roleplays, and lots of hands-on work with OSG experts in HTC, participants will learn how HTC systems work, how to run and manage many jobs and huge datasets to implement a full scientific computing workflow, and where to turn for help and more info.
Worried about costs? Successful applicants will receive financial support to attend the OSG School, covering all basic travel, hotel, and food costs. This is a great deal! Ideal candidates are graduate students whose research involves or could involve large-scale computing - work that cannot be done on one laptop or a handful of computers. And every year, we accept some post-doctoral students, faculty, staff, and advanced undergraduates, so make a good case for yourself!
IMPORTANT DATES
Application Period (OPEN NOW): 14 March - 15 April 2016 OSG
User School: 25 - 29 July 2016
MORE INFORMATION AND APPLICATIONS
Web: http://www.opensciencegrid.org/UserSchool
Email: user-school@opensciencegrid.org
Facebook: https://www.facebook.com/OSGUserSchool
Twitter: https://twitter.com/OSGUserSchool
The deadline for applications is this Friday, April 15!!!
Please help spread the word, especially through personal connections to people who might benefit from the School. Announcing the Open Science Grid User School 2016! If you could access hundreds, thousands, or even more computers for your scholarly work, what could you do? How could it transform your work? What discoveries might you make? We are seeking applicants for the Open Science Grid (OSG) User School 2016, which takes place 25 - 29 July at the beautiful University of Wisconsin in Madison.
Participants will learn to use high throughput computing (HTC) to harness vast amounts of computing power for research, applicable to nearly any field of study (e.g., physics, chemistry, engineering, life sciences, earth sciences, agricultural and animal sciences, economics, social sciences, medicine, and more). Using lectures, discussions, roleplays, and lots of hands-on work with OSG experts in HTC, participants will learn how HTC systems work, how to run and manage many jobs and huge datasets to implement a full scientific computing workflow, and where to turn for help and more info.
Worried about costs? Successful applicants will receive financial support to attend the OSG School, covering all basic travel, hotel, and food costs. This is a great deal! Ideal candidates are graduate students whose research involves or could involve large-scale computing - work that cannot be done on one laptop or a handful of computers. And every year, we accept some post-doctoral students, faculty, staff, and advanced undergraduates, so make a good case for yourself!
IMPORTANT DATES
Application Period (OPEN NOW): 14 March - 15 April 2016 OSG
User School: 25 - 29 July 2016
MORE INFORMATION AND APPLICATIONS
Web: http://www.opensciencegrid.org/UserSchool
Email: user-school@opensciencegrid.org
Facebook: https://www.facebook.com/OSGUserSchool
Twitter: https://twitter.com/OSGUserSchool
Announcing OSG Software versions 3.3.11 and 3.2.37
We are pleased to announce OSG Software versions 3.3.11 and 3.2.37.
Both 3.3.11 and 3.2.37 include:
* VO Package v65 - more OSG CA transitions
* CA certificates based on IGTF 1.73
Changes to OSG 3.3.11 include:
* XRootD 4.3.0: Several important fixes for bugs affecting CMS
* HDFS 2.0.0+1612: Support ACLs, Support the EL7 platform
* Update to GlideinWMS 3.2.13
* Add gfal functionality to xrootd-dsi
* HTCondor CE 2.0.4: Accept full subject DNs in extattr_table.txt
* BLAHP 1.18.18: Changes in the BLAHP to support PBS Pro
* osg-pki-tools 1.2.15: Better error messages and checking of arguments
* HTCondor 8.4.5: Various bug fixes
* Pull in the required log4j package when installing the emi-trustmanager
* HTCondor 8.5.3 in the Upcoming repository
* Support for an OSG CVMFS configuration repository in the Upcoming repository
Release notes and pointers to more documentation can be found at:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/Release3237
https://www.opensciencegrid.org/bin/view/Documentation/Release3/Release3311
Need help? Let us know:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/HelpProcedure
We welcome feedback on this release!
Both 3.3.11 and 3.2.37 include:
* VO Package v65 - more OSG CA transitions
* CA certificates based on IGTF 1.73
Changes to OSG 3.3.11 include:
* XRootD 4.3.0: Several important fixes for bugs affecting CMS
* HDFS 2.0.0+1612: Support ACLs, Support the EL7 platform
* Update to GlideinWMS 3.2.13
* Add gfal functionality to xrootd-dsi
* HTCondor CE 2.0.4: Accept full subject DNs in extattr_table.txt
* BLAHP 1.18.18: Changes in the BLAHP to support PBS Pro
* osg-pki-tools 1.2.15: Better error messages and checking of arguments
* HTCondor 8.4.5: Various bug fixes
* Pull in the required log4j package when installing the emi-trustmanager
* HTCondor 8.5.3 in the Upcoming repository
* Support for an OSG CVMFS configuration repository in the Upcoming repository
Release notes and pointers to more documentation can be found at:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/Release3237
https://www.opensciencegrid.org/bin/view/Documentation/Release3/Release3311
Need help? Let us know:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/HelpProcedure
We welcome feedback on this release!
Tuesday, April 5, 2016
GOC Service Update - Tuesday, April 12, 2016 at 14:00 UTC
The GOC will upgrade the following services beginning Tuesday, April 12 at 14:00 UTC. The GOC reserves 8 hours in the unlikely event unexpected problems are encountered.
OIM
OIM
- Bugfix for previous release to allow alternate names for host certs in different domains https://jira.opensciencegrid.org/browse/OIM-135, code release 3.51
- Adding signer search for user certs
- Change to remove renew/request option for ATLAS
Thursday, March 31, 2016
ANNOUNCING THE OPEN SCIENCE GRID USER SCHOOL 2016!
If you could access hundreds, thousands, or even more computers for your
scholarly work, what could you do? How could it transform your work? What
discoveries might you make?
We are seeking applicants for the Open Science Grid (OSG) User School 2016,
which takes place 25-29 July at the beautiful University of Wisconsin in
Madison. Participants will learn to use high throughput computing (HTC) to
harness vast amounts of computing power for research, applicable to nearly
any field of study (e.g., physics, chemistry, engineering, life sciences,
earth sciences, agricultural and animal sciences, economics, social
sciences, medicine, and more).
Using lectures, discussions, roleplays, and lots of hands-on work with OSG
experts in HTC, participants will learn how HTC systems work, how to run
and manage many jobs and huge datasets to implement a full scientific
computing workflow, and where to turn for help and more info.
Worried about costs? Successful applicants will receive financial support
to attend the OSG School, covering all basic travel, hotel, and food costs.
This is a great deal!
Ideal candidates are graduate students whose research involves or could
involve large-scale computing - work that cannot be done on one laptop or a
handful of computers. And every year, we accept some post-doctoral
students, faculty, staff, and advanced undergraduates, so make a good case
for yourself!
IMPORTANT DATES
Application Period (OPEN NOW): 14 March - 15 April 2016
OSG User School: 25-29 July 2016
MORE INFORMATION AND APPLICATIONS
Web: http://www.opensciencegrid.org/UserSchool
Email: user-school@opensciencegrid.org
Facebook: https://www.facebook.com/OSGUserSchool
Twitter: https://twitter.com/OSGUserSchool
Please forward this announcement to help us reach potential participants.
And consider posting our flyer where appropriate:
https://twiki.opensciencegrid.org/twiki/pub/Education/OSGUserSchool2016/osg-user-school-2016-flyer.pdf
scholarly work, what could you do? How could it transform your work? What
discoveries might you make?
We are seeking applicants for the Open Science Grid (OSG) User School 2016,
which takes place 25-29 July at the beautiful University of Wisconsin in
Madison. Participants will learn to use high throughput computing (HTC) to
harness vast amounts of computing power for research, applicable to nearly
any field of study (e.g., physics, chemistry, engineering, life sciences,
earth sciences, agricultural and animal sciences, economics, social
sciences, medicine, and more).
Using lectures, discussions, roleplays, and lots of hands-on work with OSG
experts in HTC, participants will learn how HTC systems work, how to run
and manage many jobs and huge datasets to implement a full scientific
computing workflow, and where to turn for help and more info.
Worried about costs? Successful applicants will receive financial support
to attend the OSG School, covering all basic travel, hotel, and food costs.
This is a great deal!
Ideal candidates are graduate students whose research involves or could
involve large-scale computing - work that cannot be done on one laptop or a
handful of computers. And every year, we accept some post-doctoral
students, faculty, staff, and advanced undergraduates, so make a good case
for yourself!
IMPORTANT DATES
Application Period (OPEN NOW): 14 March - 15 April 2016
OSG User School: 25-29 July 2016
MORE INFORMATION AND APPLICATIONS
Web: http://www.opensciencegrid.org/UserSchool
Email: user-school@opensciencegrid.org
Facebook: https://www.facebook.com/OSGUserSchool
Twitter: https://twitter.com/OSGUserSchool
Please forward this announcement to help us reach potential participants.
And consider posting our flyer where appropriate:
https://twiki.opensciencegrid.org/twiki/pub/Education/OSGUserSchool2016/osg-user-school-2016-flyer.pdf
Thursday, March 17, 2016
GOC Service Update - Tuesday, March 22 at 14:00 UTC
The GOC will upgrade the following services beginning Tuesday, March 22 at 14:00 UTC. The GOC reserves 8 hours in the unlikely event unexpected problems are encountered.
Oasis
move staging of files from /net/nas01 to disks attached to oasis hypervisor.
upgrade server software to cvmfs-server-2.2.1
add cvmfs configuration repository
replace all repositories data with a copy rebuilt from scratch to save disk space
other small changes, details at https://jira.opensciencegrid.org/browse/OO-125
OIM
Remove rerequest tab when VO has switched signers since the certificate expired https://jira.opensciencegrid.org/browse/OSGPKI-399
Allow alternate names for host certs in different domains https://jira.opensciencegrid.org/browse/OIM-135
Oasis
move staging of files from /net/nas01 to disks attached to oasis hypervisor.
upgrade server software to cvmfs-server-2.2.1
add cvmfs configuration repository
replace all repositories data with a copy rebuilt from scratch to save disk space
other small changes, details at https://jira.opensciencegrid.org/browse/OO-125
OIM
Remove rerequest tab when VO has switched signers since the certificate expired https://jira.opensciencegrid.org/browse/OSGPKI-399
Allow alternate names for host certs in different domains https://jira.opensciencegrid.org/browse/OIM-135
Thursday, March 10, 2016
FINAL REMINDER: Dont forget to register! - All Hands Meeting 2016
Don't miss your opportunity to register for the All Hands Meeting of the Open Science Grid hosted by the Cyberinfrastructure Technology Integration group (CITI - http://citi.clemson.edu/) of Clemson University, March 14-18, 2016.
Register now at http://citi.clemson.edu/osg-ahm-2016/registration/
Topics to be discussed will include:
* How high-throughput computing accelerates research, and how OSG can help users scale up.
* Usability challenges and solutions for distributed high-throughput computing applications.
* Connecting virtual organizations, campus researchers and XSEDE users to the OSG: command line, science gateways, and workflow frameworks.
* Training and education: partnerships with ACI-REF and Software Carpentry.
* Serving more of the "long tail" of science with high-throughput parallel computing: incorporating multi-core, GPU and virtual cluster resources into science workflows using shared and allocated distributed infrastructure.
* Broadening participation and use of OSG from various disciplines, strategy for expanding use, potential areas of impact.
* Advanced network analytics services for national Science DMZ infrastructure.
* Cyberinfrastructure partnerships: university research computing HPC centers, XSEDE XD providers, DOE laboratories, NSF Large Facility computing organizations and commercial cloud providers. Technologies, strategies, ideas, and discussions on how OSG can foster partnerships across the widest possible range of CI.
The program this year has been expanded to include hands-on workshops for advanced cyberinfrastructure providers and scientific user communities accessing the shared resources connected by the OSG. As has been the custom, the 2016 OSG AHM will be co-located with the U.S. Large Hadron Collider (LHC at CERN) computing facility meetings.
Logistical information, registration and agenda are available at http://citi.clemson.edu/osg-ahm-2016/index.html
Register now at http://citi.clemson.edu/osg-ahm-2016/registration/
Topics to be discussed will include:
* How high-throughput computing accelerates research, and how OSG can help users scale up.
* Usability challenges and solutions for distributed high-throughput computing applications.
* Connecting virtual organizations, campus researchers and XSEDE users to the OSG: command line, science gateways, and workflow frameworks.
* Training and education: partnerships with ACI-REF and Software Carpentry.
* Serving more of the "long tail" of science with high-throughput parallel computing: incorporating multi-core, GPU and virtual cluster resources into science workflows using shared and allocated distributed infrastructure.
* Broadening participation and use of OSG from various disciplines, strategy for expanding use, potential areas of impact.
* Advanced network analytics services for national Science DMZ infrastructure.
* Cyberinfrastructure partnerships: university research computing HPC centers, XSEDE XD providers, DOE laboratories, NSF Large Facility computing organizations and commercial cloud providers. Technologies, strategies, ideas, and discussions on how OSG can foster partnerships across the widest possible range of CI.
The program this year has been expanded to include hands-on workshops for advanced cyberinfrastructure providers and scientific user communities accessing the shared resources connected by the OSG. As has been the custom, the 2016 OSG AHM will be co-located with the U.S. Large Hadron Collider (LHC at CERN) computing facility meetings.
Logistical information, registration and agenda are available at http://citi.clemson.edu/osg-ahm-2016/index.html
Tuesday, March 8, 2016
Announcing OSG Software versions 3.3.10 and 3.2.36
We are pleased to announce OSG Software versions 3.3.10 and 3.2.36. The focus is now on the 3.3 release series. The 3.2 series will receive only security updates and critical bug fixes. Both 3.3.10 and 3.2.36 include: * VO Package v64 - more OSG CA transitions * CA certificates based on IGTF 1.72 * osg-ca-certs-updater no longer requires certificate compatibility packages * GFAL environment variables are now set in the client tarballs * Gratia probes: Improve robustness of handling batch system records Changes to OSG 3.3.10 include: * BLAHP 1.18.17: Additional support for LSF, SGE, and BOSCO * osg-configure: Propagate SGE parameters into the BLAHP * HTCondor 8.4.4: Various bug fixes * GUMS 1.5.2: Bug fix for LDAP-based user groups * StashCache: Updated configuration templates * AutoPyFactory 2.4.6: a provisioning factory layered on top of HTCondor * Support for XRootD and GridFTP HDFS plugins on EL7 platforms * HTCondor CE 2.0.2: HTCondor CE view installable with HTCondor CE collector * HTCondor 8.5.2 in the Upcoming repository Also, OSG 3.2.36 includes: * Updated emi-trustmanager to bring it in line with the OSG 3.3 series Release notes and pointers to more documentation can be found at: https://www.opensciencegrid.org/bin/view/Documentation/Release3/Release3236 https://www.opensciencegrid.org/bin/view/Documentation/Release3/Release3310
Monday, March 7, 2016
3rd Advancing Research Computing on Campuses (ARCC): Best Practices
3rd Advancing Research Computing on Campuses (ARCC): Best Practices Workshop
University of Illinois, Champaign IL
Tutorials: March 22, 2016
Workshop: March 23-24 2016
Contact: arcc-info@ncsa.illinois.edu
The 3rd Advancing Research Computing on Campuses (ARCC): Best Practices Workshop, will take place March 22-24, 2016 in Champaign, Illinois. March 22 will be a tutorial day and the plenary sessions will take place March 23-24. This workshop is presented in partnership with Advanced Cyberinfrastructure Research and Education Facilitators (ACI-REF) and the National Center for Supercomputing Applications (NCSA). Professionals involved in operating and supporting campus shared research computing infrastructure are invited to join in this workshop event and share experiences and expertise. New this year will be tutorial sessions that will take place on March 22, 2016.
The tutorial sessions will include the following:
- Morning Track 1: Facilitation: Best Practices on the Front-lines with Researchers
- Morning Track 2: Cyberinfrastructure from Scratch
- Afternoon Track 1: High Throughput Computing: Value Added for Campus Research
- Afternoon Track 2: A New Career Path: The Cyberinfrastructure Professional
The workshop agenda topics will include:
- Efforts/organizations that can be leveraged by campus providers
- Collaboration with researcher
- Challenges of managing condo-style resources
- Business models/Policies
The keynote speaker for the workshop will be Rajiv Ramnath, from the National Science Foundation who will present on “NSF Software Programs and the Relationship with Campus Cyberinfrastructure”
Questions: arcc-info@ncsa.illinois.edu
To register or further details, please visit the ARCC website at http://www.ncsa.illinois.edu/Conferences/ARCC/index.html
Thursday, March 3, 2016
Dont forget to register! - All Hands Meeting 2016
Dont miss your opportunity to register for the All Hands Meeting of the Open Science Grid hosted by the Cyberinfrastructure Technology Integration group (CITI - http://citi.clemson.edu/) of Clemson University, March 14-18, 2016.
Register now at http://citi.clemson.edu/osg-ahm-2016/registration/
Topics to be discussed will include:
* How high-throughput computing accelerates research, and how OSG can help users scale up.
* Usability challenges and solutions for distributed high-throughput computing applications.
* Connecting virtual organizations, campus researchers and XSEDE users to the OSG: command line, science gateways, and workflow frameworks.
* Training and education: partnerships with ACI-REF and Software Carpentry.
* Serving more of the "long tail" of science with high-throughput parallel computing: incorporating multi-core, GPU and virtual cluster resources into science workflows using shared and allocated distributed infrastructure.
* Broadening participation and use of OSG from various disciplines, strategy for expanding use, potential areas of impact.
* Advanced network analytics services for national Science DMZ infrastructure.
* Cyberinfrastructure partnerships: university research computing HPC centers, XSEDE XD providers, DOE laboratories, NSF Large Facility computing organizations and commercial cloud providers. Technologies, strategies, ideas, and discussions on how OSG can foster partnerships across the widest possible range of CI.
The program this year has been expanded to include hands-on workshops for advanced cyberinfrastructure providers and scientific user communities accessing the shared resources connected by the OSG. As has been the custom, the 2016 OSG AHM will be co-located with the U.S. Large Hadron Collider (LHC at CERN) computing facility meetings.
Logistical information, registration and agenda are available at http://citi.clemson.edu/osg-ahm-2016/index.html
OSG Grid Operations Center (GOC)
Email/Phone: goc@opensciencegrid.org, 317-278-9699
GOC Homepage: http://www.opensciencegrid.org/ops
RSS Feed: http://osggoc.blogspot.com
Wednesday, March 2, 2016
RESOLVED: RSV Collector Service Unavailable
Yesterday, the RSV collector at the GOC encountered unstable operation and was non-functional. The problems are resolved and the RSV Collector is now functioning as expected. We apologize for any inconvenience this caused.
Tuesday, March 1, 2016
GOC Service Update - Tuesday, March 8, 2016
The GOC will upgrade the following services beginning Tuesday, March 8 at 14:00 UTC. The GOC reserves 8 hours in the unlikely event unexpected problems are encountered.
OIM
Repair certificate revocation problem for VOs who have switched signers, https://jira.opensciencegrid.org/browse/OIM-133
OIM
Repair certificate revocation problem for VOs who have switched signers, https://jira.opensciencegrid.org/browse/OIM-133
Monday, February 29, 2016
RSV Collector Service Impacted Over The Weekend
Over the weekend, the central RSV Collector at the GOC experienced a drive failure which impacted the service. The central RSV collector is currently back in service. The affected RAID array is running in "degraded" mode and will be restored to full functionality when a replacement drive arrives.
Thursday, February 25, 2016
Dont forget to register! - All Hands Meeting 2016
Dont miss your opportunity to register for the All Hands Meeting of the Open Science Grid hosted by the Cyberinfrastructure Technology Integration group (CITI - http://citi.clemson.edu/) of Clemson University, March 14-18, 2016.
Register now at http://citi.clemson.edu/osg-ahm-2016/registration/
Topics to be discussed will include:
* How high-throughput computing accelerates research, and how OSG can help users scale up.
* Usability challenges and solutions for distributed high-throughput computing applications.
* Connecting virtual organizations, campus researchers and XSEDE users to the OSG: command line, science gateways, and workflow frameworks.
* Training and education: partnerships with ACI-REF and Software Carpentry.
* Serving more of the "long tail" of science with high-throughput parallel computing: incorporating multi-core, GPU and virtual cluster resources into science workflows using shared and allocated distributed infrastructure.
* Broadening participation and use of OSG from various disciplines, strategy for expanding use, potential areas of impact.
* Advanced network analytics services for national Science DMZ infrastructure.
* Cyberinfrastructure partnerships: university research computing HPC centers, XSEDE XD providers, DOE laboratories, NSF Large Facility computing organizations and commercial cloud providers. Technologies, strategies, ideas, and discussions on how OSG can foster partnerships across the widest possible range of CI.
The program this year has been expanded to include hands-on workshops for advanced cyberinfrastructure providers and scientific user communities accessing the shared resources connected by the OSG. As has been the custom, the 2016 OSG AHM will be co-located with the U.S. Large Hadron Collider (LHC at CERN) computing facility meetings.
Logistical information, registration and agenda are available at http://citi.clemson.edu/osg-ahm-2016/index.html
Register now at http://citi.clemson.edu/osg-ahm-2016/registration/
Topics to be discussed will include:
* How high-throughput computing accelerates research, and how OSG can help users scale up.
* Usability challenges and solutions for distributed high-throughput computing applications.
* Connecting virtual organizations, campus researchers and XSEDE users to the OSG: command line, science gateways, and workflow frameworks.
* Training and education: partnerships with ACI-REF and Software Carpentry.
* Serving more of the "long tail" of science with high-throughput parallel computing: incorporating multi-core, GPU and virtual cluster resources into science workflows using shared and allocated distributed infrastructure.
* Broadening participation and use of OSG from various disciplines, strategy for expanding use, potential areas of impact.
* Advanced network analytics services for national Science DMZ infrastructure.
* Cyberinfrastructure partnerships: university research computing HPC centers, XSEDE XD providers, DOE laboratories, NSF Large Facility computing organizations and commercial cloud providers. Technologies, strategies, ideas, and discussions on how OSG can foster partnerships across the widest possible range of CI.
The program this year has been expanded to include hands-on workshops for advanced cyberinfrastructure providers and scientific user communities accessing the shared resources connected by the OSG. As has been the custom, the 2016 OSG AHM will be co-located with the U.S. Large Hadron Collider (LHC at CERN) computing facility meetings.
Logistical information, registration and agenda are available at http://citi.clemson.edu/osg-ahm-2016/index.html
Tuesday, February 16, 2016
GOC Service Update - Tuesday, Feb 23, 2016 at 14:00 UTC
The GOC will upgrade the following services beginning Tuesday, February 23 at 14:00 UTC. The GOC reserves 8 hours in the unlikely event unexpected problems are encountered.
OASIS
Increase size of shared volume from 8 TB to 9 TB
OIM
Repair certificate revocation problem for VOs who have switched signers, https://jira.opensciencegrid.org/browse/OIM-133
perfSONAR
Upgrade check_mk, https://ticket.grid.iu.edu/28470 Bring psds to OSG 3.3, https://ticket.opensciencegrid.org/28461
Gratia Web
Fix 12 month per VO plots, https://ticket.opensciencegrid.org/28342
MyOSG 2.38
Exposing APEL Normalization Factor field under Resource Group summary (MYOSG-88)
All services
Operating system updates; reboots will be required. The usual HA mechanisms will be used, but some services will experience brief outages.
OASIS
Increase size of shared volume from 8 TB to 9 TB
OIM
Repair certificate revocation problem for VOs who have switched signers, https://jira.opensciencegrid.org/browse/OIM-133
perfSONAR
Upgrade check_mk, https://ticket.grid.iu.edu/28470 Bring psds to OSG 3.3, https://ticket.opensciencegrid.org/28461
Gratia Web
Fix 12 month per VO plots, https://ticket.opensciencegrid.org/28342
MyOSG 2.38
Exposing APEL Normalization Factor field under Resource Group summary (MYOSG-88)
All services
Operating system updates; reboots will be required. The usual HA mechanisms will be used, but some services will experience brief outages.
Tuesday, February 9, 2016
Announcing OSG Software versions 3.2.35 and 3.3.9
We are pleased to announce OSG Software versions 3.2.35 and 3.3.9.
After this release, the 3.2 series will only receive security updates
and critical bug fixes.
Changes to OSG 3.2.35 and 3.3.9 include:
* Update to GSI-OpenSSH 5.7 to address CVE-2016-0777
* Update to GlideinWMS 3.2.12.1
* XRootD HDFS plugin support for non-world-readable files
* Fix a memory leak in lcmaps-plugins-scas-client
* CA certificates based on IGTF 1.71
* VO Package v63 - more CILogon transitions
* VOMS admin updated to disable an unnecessary CA check
* Better warnings and message handling in RSV-perfsonar
* Support for HDFS on EL7 platforms
In addition, OSG 3.3.9 includes:
* CVMFS over StashCache, including CVMFS 2.2.0, in the Upcoming repository
Also, OSG 3.2.35 includes:
* Much smaller tarballs
Note: The CHANGES file in the OSG CA certificate distribution labels
the 1.52 changes under a 1.51 heading. This problem is merely cosmetic.
Release notes and pointers to more documentation can be found at:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/Release3235
https://www.opensciencegrid.org/bin/view/Documentation/Release3/Release339
Need help? Let us know:
https://www.opensciencegrid.org/bin/view/Documentation/Release3/HelpProcedure
We welcome feedback on this release!
Tuesday, February 2, 2016
GOC Service Update - Tuesday, Feb 9, 2016 at 14:00 UTC
The GOC will upgrade the following services beginning
Tuesday, Feb 9, 2016 at 14:00 UTC. The GOC reserves 8 hours in the unlikely event unexpected
problems are encountered.
OASIS
* increase size of shared volume from 8 TB to 9 TB
* modify /etc/hosts to see the fileserver and shared
volume
OIM
* remove wildcarding from id searches that causes too
many matches - https://jira.opensciencegrid.org/browse/OIM-132
* upgrade captcha code - https://jira.opensciencegrid.org/browse/OIM-130
Ticket
* add timestamp to file attachments - https://jira.opensciencegrid.org/browse/TICKET-113
perfSONAR
* install monitoring components
Subscribe to:
Posts (Atom)
Posts
Copyright 2009 Indiana University - Developed for Open Science Grid