DOE Grids service transition

Dear colleagues,

As we have been communicating with you over the past several months, ESnet will be fully transitioning support for the DOE Grids certificate service to a new service operated by the Open Science Grid (OSG).

We have reached a critical juncture in the transition where it is necessary for you to begin solidifying your organization’s future certificate service plans to ensure no interruption of service. We encourage you to read the following information carefully.

In mid-March 2013 the DOE Grids PKI will cease issuing new certificates. The exact timing will coincide with the planned Large Hadron Collider (LHC) shutdown. The exact date of the shutdown is still being determined and we will share that specific date as soon as it is set. After DOE Grids concludes offering certificate services, all users will either need to use the OSG certificate service (the “OSG PKI”), or some other provider, to obtain or renew certificates. All certificates issued by DOE Grids prior to its cessation of service in mid-March 2013 will continue to function for 12 months after the date of issue.

To help ensure a seamless transition of service, beginning October 1, 2012, the OSG PKI will begin issuing production-ready certificates. This will allow you time to transition to the new service and resolve any issues that may arise before the DOE Grids service ends. The new OSG PKI will have a new look and feel, but effort has been made to keep the workflow and processes similar to the DOE Grids PKI. There is friendly testing already underway. If you would like to participate in this current trial, please contact Von Welch at vwelch@indiana.edu.

Your organization should begin planning for the effort involved in the transition. Specifically, please note the following action items that will need to be taken:

* Registration Authorities (RAs) and Grid Admins will need to register with the OSG PKI. This will involve making a formal request for the service and accepting the OSG/DigiCert agreement. A draft process can be found here: https://twiki.grid.iu.edu/bin/view/Operations/OSGPKITrustedAgent

* In the new OSG PKI, the role of the Grid Admin will change. Virtual Organization (VO) RAs will approve users in their VO and Grid Admins will approve hosts in a given domain (e.g., iu.edu).

* In the new service, users identities (distinguished names) will change. VOs will need to prepare for the extra effort for handling this change. For example, VOs will need to be prepared to re-register users in the Virtual Organization Management System (VOMS) or other access control mechanisms, if applicable.

* The DigiCert CA used by the new OSG PKI is in the IGTF CA distribution starting with the January 2012 v1.44 distribution. VOs and Sites should ensure they update as soon as possible. Please see: https://dist.eugridpma.info/distribution/igtf/current/accredited/

* Beginning in October 2012, training and other resources will be available to help organizations prepare for the service transition. We will be sending an update to this mail list when training is scheduled or otherwise available. Please contact Von (vwelch@indiana.edu) if you have interest in participating in this training so we can plan appropriate venues and delivery mechanisms. For more information, visit: https://opensciencegrid.org/bin/view/Security/DigiCertTrainingPlan

Von is also currently hosting weekly calls on Tuesdays at 3pm ET focused on details of the transition. All are invited to join these calls to discuss your needs and to learn the progress of the roll-out. Call-in details can be found below and on the OSG PKI Planning Website, which provides up to date information on all aspects of the transition.

Ruth Pordes (ruth@fnal.gov) and Von (vwelch@indiana.edu) together with our ESnet staff are available at anytime for individual conversations about the service to answer any questions you may have. We will be contacting each of you in the coming weeks to schedule a one-on-one conversation to answer questions and ensure your transition planning is underway. We look forward to speaking with you soon.

Regards,
Von Welch
OSG PKI Transition Project Lead

For the latest information:
https://twiki.grid.iu.edu/bin/view/Security/OSGCATransition2012

Conference call details:
Weekly calls Tuesday at 3pm ET
Phone Number: (800) 940-6112 or (812) 856-3600
Participant PIN: 001174#

GOC Service Update - Tuesday, September 11th at 13:00 UTC

The GOC will upgrade the following services beginning Tuesday, September 11th, 2012 at 13:00 UTC. The GOC reserves 8 hours (13:00 - 21:00 UTC) in the unlikely event that unexpected problems are encountered. We encourage users to test affected services before the production release.


MyOSG 2.1

ITB version is now available for testing at https://myosg-itb.grid.iu.edu
CPU count will be increased to 2 on myosg1/2
remove osg-xsede from the status board
Minor cosmetic updates on VO summary / resource list
Updated stylesheet for various error pages
(patched) Fixed the IE base url issue
(patched) Fixed broken status map url issue


Reports / Installed Capacity Report 1.0-12

Adding sites supported by USCMS_Tier2 Support Center to the report.
Moving report script from OIM to reports.grid.iu.edu
Add deprecation message to RSV reports.


RSV-Client

Rebuild VM instance using the latest install script which uses the RSV v2 RPMs.


OIM 3.6

ITB version can be tested via https://oim-itb.grid.iu.edu
Added display for log based on composite key (not through topology class yet) [OIM-23]
PKI / Added domain name validator for GridAdmin [OSGPKI-89]
PKI / Added search capability for user/host certificates [OSGPKI-105]
Added check for session destruction to report private key deletion [OSGPKI-80]
Improved the URL handling for login/out.
PKI / Differentiated the behavior of user certificate approval process between new approval and renew approval.
PKI / Reset CSR when user certificate renewal is requested.
PKI / Added serial number to log comment
PKI / Updated label from Serial ID to Serial Number
Updated the Google Map API from v2 to v3.
PKI / Added listing for "certificate I approve" for user/host certificates.
Updated returned content type to application/json for all REST APIs.
Removed error message in case of missing session for message (could happen on error page)
PKI / Removed the 500 error code reporting of the OIM Rest API error return.
PKI / Improved the way secure / guest URLs were composed.
Made IP address to be displayed at all time
PKI / Updated cert-retrieve-new to osg-cert-retrieve based on recent name change.
PKI / Added VO name in the title of GOC ticket generated for user certificate request
PKI / Added more checks for various error condition during host cert request.
PKI / Made read-only GridAdmin page for non PKI staff.
PKI / Added GridAdmin & RA enrollment request buttons.
PKI / Added capability to assign VO managers
PKI / Enabled user cert revoke capability
Implemented logout action to invalidate current session.
(patched) removed anti-session spoofing mechanism.
Other minor bug fixes


All Services

We will be updating all RHEL 5 and RHEL 6 hosts to the latest Red Hat packages, and we will be updating the firmware on physical hosts where updates are available. This will require reboots.
DOEGrids host certificates on ITB services will be changed to DOEGrids host certificates with SHA-2 signatures.

OSG Software version 3.1.8 has been released

Hello,

We're pleased to announce OSG Software version 3.1.8. This is the new OSG Software distributed via RPMs for Scientific Linux 5 and 6, CentOS 5 and 6, and Red Hat Enterprise Linux 5 and 6. The changes in this release include changes that affect the client, and the compute element:

Condor 7.8.2 is a recommended security update.
A new version of the Gratia probe that fixes various bugs
A new version of osg-cleanup that fixes a bug that prevented it from cleaning up all appropriate files.
A new version of lcmaps that makes it work better with Condor. Please note that you will get an updated lcmaps configuration file that you need to merge with your existing configuration. (/etc/lcmaps.db)

Release notes and pointers to documentation can be found at: https://twiki.grid.iu.edu/bin/view/Documentation/Release3/Release318

Need help? Let us know: https://twiki.grid.iu.edu/bin/view/Documentation/Release3/HelpProcedure

GOC Service Update - Tuesday, August 14th at 13:00 UTC

The GOC will upgrade the following services beginning Tuesday, August 14th, 2012 at 13:00 UTC. The GOC reserves 8 hours (13:00 - 21:00 UTC) in the unlikely event that unexpected problems are encountered. We encourage users to test affected services before the production release.


OIM 3.4
ITB version can be tested via https://oim-itb.grid.iu.edu
Limited availability for OSG PKI user interface


MyOSG 2.0
ITB version is now available for testing at https://myosg-itb.grid.iu.edu
Updated look & feel.
Compacted various pages to fit more information.


GOC-TX 1.26.2
ITB version can be tested via https://ticket-itb.grid.iu.edu
Updated FNAL/ServiceNow SOAP client to the latest version which is to be released on 8/10 at FNAL.


GOC Ticket 1.55
ITB version is now available for testing at https://ticket-itb.grid.iu.edu
Added campus research club request form (prototype)


Blogs
ITB version can be tested via http://blogs-itb.grid.iu.edu
Added Erik Erlandson Blog


Installed Capacity Report 1.0-10
Adding sites supported by USCMS_Tier2 Support Center to the report.
Moving report script from OIM to reports.grid.iu.edu


VM Maintenance

We will be reorganizing the files on the VM hosts to improve performance -- most have already been done; this is the last group of changes. On Tuesday, August 7, only ITB VMs will be affected (and then only data-itb, display-itb, gratiaweb-itb, myosg-itb, rsv-itb, ticket-itb, and tx-itb). All other ITB VMs have already been reorganized. On Tuesday, August 14, the last production VMs will be reorganized. The only public-facing services experiencing downtime will be display.grid.iu.edu, tx.grid.iu.edu, and twiki.grid.iu.edu. Ticket.grid.iu.edu may experience degradation of service due to load but will remain online. Production downtime should not exceed 20 minutes.

OSG Software version 3.1.7 has been released

Hello,

We're pleased to announce OSG Software version 3.1.7. This is the new OSG Software distributed via RPMs for Scientific Linux 5 and 6, CentOS 5 and 6, and Red Hat Enterprise Linux 5 and 6. The changes in this release include changes that primarily affect the OSG CE, the Bestman SE, and the HDFS SE. Changes include:

Several bug fixes, especially:
* Fixed missing dependencies for Bestman2
* Fixed lcmaps integration for xrootd
* Fixed GridFTP HDFS integration so files are properly deleted when requested.
Updated software, especially:
* New version of glideinwms, 2.6.0
* Condor job manager now respects max_wall_time RSL parameter

Release notes and pointers to documentation can be found at: https://twiki.grid.iu.edu/bin/view/Documentation/Release3/Release317

Need help? Let us know: https://twiki.grid.iu.edu/bin/view/Documentation/Release3/HelpProcedure

We look forward to your feedback on this new release.

GOC Service Update - Tuesday, July 24th at 13:00 UTC

The GOC will upgrade the following services beginning Tuesday, June 24th, 2012 at 13:00 UTC. The GOC reserves 8 hours (13:00 - 21:00 UTC) in the unlikely event that unexpected problems are encountered. We encourage users to test affected services before the production release.

GOC Ticket Synchronizer (GOC-TX) 1.26.1

ITB version can be tested via https://ticket-itb.grid.iu.edu
Updated default value for FNAL/SNOW Reported Source to “Direct Input”.
Fixed FNAL/SNOW mapping for pending status (was unknown previously)
Updated default assignment group for SNOW to be “Service Desk”.
Replaced GGUS/concerned VO to GOC/SC conversion with GGUS/notified site to GOC/SC [GOCTX-21]


MyOSG 1.53

ITB version is now available for testing at https://myosg-itb.grid.iu.edu
Added campus grid site marks on RSV Status Map (to be exposed via OSG Display)


OSG Display 1.0.10

ITB version can be tested via http://display-itb.grid.iu.edu/
Requesting MyOSG to render Campus Grid site icons on Status Map tab


Blogs

ITB version can be tested via http://blogs-itb.grid.iu.edu
Added OSG Security Blog


OIM 3.3

ITB version can be tested via https://oim-itb.grid.iu.edu
Added capability to display all DNs associated with the user contact, and show which cert was used to login.
Added email display for contacts in contact editor.
Various updates related to OSG PKI certificate interface.


TWiki

ITB version can be tested via https://twiki.grid.iu.edu
Fixed the Mime type issue for MS Office documents


All Services

We will be updating all RHEL 5 and RHEL 6 hosts to the latest Red Hat packages, and we will be updating the firmware on physical hosts where updates are available. This will require reboots.

The Large Synoptic Survey Telescope (LSST) Needs Your Opportunistic Cycles

OSG Resource Providers,

The LSST VO (http://www.lsst.org/lsst/) is looking for cycles. LSST is currently running on OSG resources using the North West Indiana Campus Grid (NWICG) VO as an umbrella VO for their activities. Eventually, we would like for all of the LSST work to be accounted to the LSST VO. Please take a look to see if your resource is enabled to allow LSST VO jobs, and if it is not please consider enabling it. Below is a description of the application. LSST jobs run already on the OSG and the LSST simulation group can work closely with the OSG support group and the resource providers in case of need. If you'd like to help please enable the LSST VO: we are happy to provide assistance if help is needed.

Thank you in advance for your participation,
Rob Quick
OSG Operations Coordinator

Gabriele Garzoglio
OSG Support team

John Peterson
LSST Photon Simulator Coordinator


About LSST simulations
-----------
The Large Synoptic Survey Telescope (LSST) will image a large area of the sky with each exposure. This will help it in accomplishing its science missions:
1.Probing dark energy and dark matter
2.Taking an inventory of the solar system
3.Exploring the transient optical sky
4.Mapping the Milky Way

One of the main features of the telescope is a 3.2 Gigapixel camera, which is anticipated to produce about 15TB of uncompressed image data a night. Another is its large mirror, which allows the telescope to quickly detect faint objects in a large area of the sky.

The OSG user support team and the LSST simulation group have collaborated in 2010 to port the LSST image simulation application to the OSG. Today, LSST simulation jobs "overspill" to OSG from Purdue whenever additional cycles are needed. Simulated images are used to refine and validate the data analysis software used to accomplish LSST scientific missions. For one exposure the software simulates the path of 1011 photons from their sources, through the atmosphere, the telescope optics, and to the CCDs.


LSST Simulation Execution on the OSG
--------------------------

The client supports following important features:

* integration with glideinWMS
* automatic recovery of jobs through glideinWMS
* full support for eviction of jobs, automatic suspend and restart of the job
* ~30 MB data stage in / stage out per job without a need to prestage data at sites
* 1-5 hours per job
* the "unit" of computation is a "visit" of the sky i.e. two 15 sec exposures of all 189 camera CCDs i.e. 378 jobs. This takes ~1000 CPU hours.


Benefits of running LSST Simulation on your Site
--------------------------------------

LSST simulation runs approximately monthly "bursts" of job "overspilling" from Purdue to OSG and other resources. These jobs will utilize idles cycles at your site that might otherwise not be utilized. At the same time, through the integration with GlideinWMS, the jobs will not flood any one site.