System Status

All Systems ActiveAll Systems Active: No current issues

Emergency Patching at 11PM Tonight (Feb 17, 2016)

Wednesday, February 17, 2016 - 3:42pm

There is a new glibc vulnerability (CVE-2015-7547) that affects most Linux systems. It may allow remote code execution on the servers, so I will patch and reboot all CARLI servers starting at 11PM tonight (Wednesday, Feb 17).

For most servers this will be a quick reboot. For Voyager, it will probably be down for about 20 minutes while the Oracle database server is rebooted.

Brandon Gant
CARLI

Voyager Outage (Dec 4th, 2015)

Friday, December 4, 2015 - 9:04am

We have a batch job that has been failing this week and we have a ticket open with Ex Libris. We did not realize that the failed jobs were generating 4GB core dumps. At 2:10AM this morning the core dumps pushed the disk volume to 100% capacity. At approximately 8:50AM we identified and removed the core dump files to restore service.

This failure did not generate any automated alerts, so we will need to revise our monitoring.

Brandon Gant
CARLI

All CARLI Services scheduled downtime: Friday, October 23, 2015, 3am-5am

Wednesday, October 21, 2015 - 10:19am

UIUC networking staff will perform maintenance this week affecting CARLI services. Please notify your library staff working Friday morning.

  • Friday 10/23/2015, between 3AM and 5AM, all CARLI services including Voyager, I-Share, VuFind, SFX, CONTENTdm, and the CARLI website will be unavailable due to UIUC firewall maintenance.
    • This outage is expected to last approximately 15 minutes, but it could occur at any time during the scheduled hours.
    • UIUC patrons and staff may be unaffected.

Since this outage will affect the availability of the CARLI website, you will be unable to review the System Status updates on the CARLI homepage until the outages are resolved.

If the outages extend beyond the scheduled times, you can contact CARLI by telephone at 866-904-5843 or by email at and we will pass the outage information along to the UIUC networking staff.
 
Please send any questions about these upcoming outages to .

CARLI website scheduled downtime: Wednesday, October 21, 2015, 10pm-midnight

Wednesday, October 21, 2015 - 10:13am

UIUC networking staff will perform maintenance this week affecting CARLI services. Please notify your library staff working Wednesday night and Friday morning.

  • Tonight, Wednesday 10/21/2015, between 10pm and midnight, the CARLI website will be unavailable due to UIUC network maintenance.
     
  • Friday 10/23/2015, between 3AM and 5AM, all CARLI services including Voyager, I-Share, VuFind, SFX, CONTENTdm, and the CARLI website will be unavailable due to UIUC firewall maintenance.
    • This outage is expected to last approximately 15 minutes, but it could occur at any time during the scheduled hours.
    • UIUC patrons and staff may be unaffected.

Since both outages will affect the availability of the CARLI website, you will be unable to review the System Status updates on the CARLI homepage until the outages are resolved.

If the outages extend beyond the scheduled times, you can contact CARLI by telephone at 866-904-5843 or by email at and we will pass the outage information along to the UIUC networking staff.
 
Please send any questions about these upcoming outages to .  

 

Network Issues Friday (10/02) and Saturday (10/03)

Monday, October 5, 2015 - 4:41pm

The CARLI servers are in a Data Center on the University of Illinois at Urbana-Champaign (UIUC) campus. The UIUC campus firewalls automatically download the latest intrusion detection signatures and one of those signatures from the vendor was bad. At 9AM Friday the firewalls crashed and automatically restarted. A firewall restart drops all connections to our Voyager server (it probably has no effect on web traffic). At 3PM Friday, the firewalls crashed and restarted again, but now they were in a bad state that also slowed down network performance.

The poor network performance continued through Saturday evening until UIUC network staff and the firewall vendor identified the issue and removed the bad signature. Everything appears to be working normally again at this time.

Brandon Gant
CARLI

CONTENTdm Down for Software Upgrade

Tuesday, June 23, 2015 - 3:43pm

CARLI's instance of CONTENTdm is down for a software upgrade from June 23 – 28, 2015. CARLI Digital Collections is still available for searching and browsing, though collections are static as of 5:00pm, June 23, 2015.

Note to library staff: Access to the CONTENTdm Project Clients and CONTENTdm Web Administration will not be available during the upgrade. Collections cannot be created or published, and no changes can be made to collections or collection home pages.

We expect CONTENTdm and the improved CARLI Digital Collections website to be back online no later than Monday, June 29.

Please refer to the CONTENTdm Upgrade webpage for more information including what changes libraries will need to make after he upgrade.

Send any questions to the CARLI Office at

Resolved: Oracle Issues Affecting I-Share 6/17/2015

Wednesday, June 17, 2015 - 9:10am

This issue has been resolved. If you start seeing problems with holdings displays in I-Share please email support@carli.illinois.edu

Oracle Issues Affecting I-Share 6/17/2015

Wednesday, June 17, 2015 - 8:38am

This morning we are seeing Oracle issues affecting the display of holdings in I-Share. Searching is not affected. We are actively investigating this issue.

I-Share Voyager Down for Software Upgrade

Friday, June 12, 2015 - 5:18pm

I-Share Voyager and its VuFind catalog interface are down this weekend (June 12-15) while CARLI staff upgrade its software to Voyager version 9.1.1 .

Note to library staff: Voyager batch jobs will be suspended until Monday night, June 15, so there will be no new circulation notices and no UB request promotion (promoted call slips) until Tuesday morning, June 16. Any bibliographic record you saved in cataloging on Friday, June 12, or updated in WorldCat using a bulk import production mode (e.g., "Express mode"), will not appear in your local VuFind catalog until Tuesday June 16, or in the I-Share union catalog until Wednesday, June 17.

We expect all systems to be back online at 7am, Monday June 15.

Please refer to the I-Share upgrade page for more information including a list of post-upgrade tasks I-Share library staff will want to review.

As always, send any questions or problem reports to us at support@carli.illinois.edu

 

VuFind Outage, Saturday, Jan 31

Monday, February 2, 2015 - 10:28am

On Saturday, January 31st, at 11:53 AM we received reports of VuFind not responding. CARLI System Services staff investigated the problem and determined that it was due to one of VuFind's services exhausting its memory resources. More memory was added and VuFind returned to normal operation by 3:30 PM.

Todd Pavlik
CARLI

Pages