Wufoo Status: Archive

Planned Maintenance

Update, 6:11pm: And we are back! Thanks for your patience during this downtime. Friday, April 19th at 6:00pm PDT, Wufoo will have a short downtime. Our goal is to keep the downtime to approximately 30 minutes or less. All public forms and reports will see error messages as shown in this screenshot. Friday, April 19th at 6:00pm PDT [ View All Timezones ] If you’re up or interested, you...

Apr 18th

Planned Maintenance

Update, 8:22pm: And we are back! Thanks for your patience during this downtime. Update, 8:01pm: We’ve begun our short maintenance. We will let you know when we’re back. Update, 7:22pm: We’ve run into a small issue and had to delay the scheduled downtime until 8:00pm PDT. Sorry for the delay. The Wufoo Team Friday, April 5th at 7:00pm PDT, Wufoo will have a short downtime....

Apr 4th

Issue with Notifications (FIXED)

Update: Thank you everyone for your continued patience with this. The issue looks to now be resolved. All emails are now working correctly again. Thanks again for your support! The Wufoo Team We’re currently experiencing an issue and our engineers are looking into it. Thank you for your patience with this. We’ll update you as soon as we get more information. The Wufoo Team

Feb 18th

Planned Maintenance

Tuesday, February 12th at 5:00pm PST, Wufoo will have a short downtime. Our goal is to keep the downtime to 5 minutes or less, but you should be prepared for a downtime of up to 30 minutes. All public forms and reports will see error messages as shown in this screenshot. Tuesday, February 12th at 5:00pm PST [ View All Timezones ] If you’re up or interested, you can follow our progress during the...

Feb 11th

Planned Maintenance

Friday, November 2nd at 6:30pm PST, we will be performing updates on our database servers. For the past few weeks, we have had problems that have caused a subset of users to experience slow downs or site unavailability. We have identified the core problem, and will be implementing the appropriate fixes. During the downtime, we are expecting all services to be unavailable for 30 minutes. All public...

Nov 1st

Unexpected Downtime

Summary: Wufoo was unavailable for a subset of users for portions of Sunday. Form submissions and third-party integrations were slow or not working for a subset of users for portions of Sunday night and Monday morning. Hi everyone, we have had a rough two days on the server front. We apologize for the issues that you’ve been seeing, and we want to take a minute and recap what has happened. ...

Oct 22nd

Downtime Over

Forms and account access should now be up for all users (as of 6:00pm PDT). We now return to your regularly scheduled form-building. Thanks for bearing with us during the downtime.

Oct 22nd

Current Downtime for Some Users

We’re in the process of restarting a database server because of a technical issue that popped-up earlier today. This is taking a little time, but effected forms and accounts should be accessible again in the very near future. Stay tuned for updates.

Oct 21st

Planned Maintenance

Sunday, September 16th at 8:00am PST, we will be performing emergency updates on our database servers. Users have been experiencing slow downs and poor performance throughout the week, so we’re going to address this on short notice. During that time, we are expecting all services to be unavailable for 30 minutes. All public forms and reports will see error messages as shown in...

Sep 15th

Unexpected Downtime

Summary: Wufoo was unavailable tonight from 11:57pm PST to 12:59am PST. Leading up to the outage, Wufoo received a surge of traffic. This increase lead to delays on the database server, which then made the web servers unresponsive. Unfortunately, the traffic was not genuine, and was the result of an attack on Wufoo. Upon realizing the nature of the attack, we applied code changes to improve...

May 9th

Downtime to Expand Storage

Thursday, December 15th at 9:30pm PST, we will be expanding our storage capabilities. This will require the entire site to be taken offline for an unknown amount of time. In our best case scenario, we expect the site to be down for 30 minutes. The worst case scenario would put us at a 90 minute downtime. During the downtime, we will bring up existing forms as soon as possible, but possibly...

Dec 14th

Quarterly Patching

Wednesday, November 9th at 10:00pm PST, we will be performing routine updates on all of our servers. For two specific servers, we require the site to be offline in order to reliably patch them. Patching for those two specific servers will begin at 10:00pm PST. During that time, we are expecting all services to be unavailable for 10-20 minutes. All public forms and reports will see error...

Nov 9th

1 note

Quarterly Patching

Thursday, August 4th at 10:00pm PST, we will be performing routine updates on all of our servers. For two specific servers, we require the site to be offline in order to reliably patch them. Patching for those two specific servers will begin at 10:00pm PST. During that time, we are expecting all services to be unavailable for 10-20 minutes. All public forms and reports will see error messages...

Aug 3rd

1 note

New Domains and Languages

We’re beginning to roll out new domains and languages. From an end user perspective, you may see yourself redirected from http://wufoo.com to http://wufoo.eu, or some other Wufoo domain depending on your language, and where you are located in the world. The way redirection currently works is that when you log in, you are directed to the proper domain. This only affects private pages in the...

Aug 2nd

1 note

Revisiting June 20th Maintenance

Our previous maintenance scheduled for June 20th never occurred because more preparation was required. We have rescheduled the downtime for this Thursday, June 30th. We are expecting the service to be unavailable for 15 to 30 minutes. During the downtime, public forms and reports will see error messages as shown in this screenshot. Please note that this event is more complicated than our standard...

Jun 29th

Potential Downtime

UPDATE - This downtime never occurred. Will be rescheduled at a later date. Today, June 20th at 9:40pm PST, we will be upgrading our file servers to allow for more storage space. This change will allow the site to handle more users, forms and file uploads. We will try to do this without any downtime, but there is a chance we have to put the site into maintenance mode for stability. If that is...

Jun 20th

Downtime for New Servers

Today, May 24th at 9:15pm PST, we will fully cutting over to our new servers that were added to alleviate some of our recent problems. This change will allow the site to handle more users, and it will also minimize the risk of one server affecting the performance of others. Downtime will begin between 9:15pm PST and 9:30pm PST. During that time, we are expecting all services to be unavailable...

May 24th

3 notes

Quarterly Patching

Today, May 19th at 9:15pm PST, we will be performing routine updates on all of our servers. For two specific servers, we require the site to be offline in order to reliably patch them. Patching for those two specific servers will begin between 9:15pm PST and 9:30pm PST. During that time, we are expecting all services to be unavailable for 5-15 minutes. All public forms and reports will see...

May 19th

5 notes

Status Update

As many of our users have noticed, the site has been having trouble staying healthy lately. Before getting to the details, please note that while we have moved away from critical into a much more stable zone, we are not fully in the clear just yet. We may have to bring the site down for patching 1 or 2 more times this week. If maintenance is required, we will try to give as much advance notice as...

May 9th

2 notes

Server Tuning

We’ll be performing some checks this evening, May 6th, to try to improve performance, and isolate some issues that we have noticed on a recurring basis. This work will begin at approximately 9pm PST / midnight EST, and we will be performing tests for a couple of hours. We aren’t certain, but this may result in periodic downtime. We don’t anticipate a worst case scenario of more...

May 6th

Quarterly Patching

Today, February 10th at 9:00pm PST, we will be performing routine updates on all of our servers. For two specific servers, we require the site to be offline in order to reliably patch them. Patching for those two specific servers will begin between 9:00pm PST and 9:30pm PST. During that time, we are expecting all services to be unavailable for 5-15 minutes. All public forms and reports will see...

Feb 10th

1 note

Update on Outages

As you know, we had a brief outage this past Saturday, and then a shorter occurrence of the same problem on Monday afternoon. For the issue on Monday, we could not update this status blog because Tumblr was experiencing issues as well. Between the two days, we had approximately 45 minutes of downtime, and we apologize for that. We do our best to keep Wufoo running as smoothly as possible, so...

Dec 8th

2 notes

Unexpected Downtime

Today, around 1:45pm EST, we started noticing issues with our web servers. It started with one, and eventually spread to all. We’re currently working on the issue, and we will update you as we know more. *UPDATE* As of 2:15, the site became responsive again. An unexpected process consumed all of the memory available in the web servers, which caused them to become unresponsive. We...

Dec 4th

Quarterly Patching

This Monday, November 1st at 9:30pm PST, we will be performing routine updates on all of our servers. For two specific servers, we require the site to be offline in order to reliably patch them. Patching for those two specific servers will begin between 9:30pm PST and 10:00pm PST. During that time, we are expecting all services to be unavailable for 15 minutes. All public forms and reports will...

Nov 1st

1 note

Scheduled Maintenance

This Tuesday, September 14th at 9:30pm PST, we will be performing scheduled maintenance. Small performance tweaks and general routine monitoring will be going on. We are expecting all services to be unavailable for 15 minutes. All public forms and reports will see error messages as shown in this screenshot. Tuesday, September 14th at 9:30pm PST [ View All Timezones ] If you’re up or...

Sep 14th

3 notes

3rd Party Integrations

Update: We have identified and fixed the problem. The backlog is starting to clear out, and we imagine it will be caught up within an hour. We’ll continue to monitor. We are beginning to notice severe delays in the processing of our third party integrations. Users with a form configured to send data to a third party are affected. The data still comes immediately through to the Wufoo...

Aug 12th

Scheduled Maintenance

This Tuesday, June 22nd at 9pm PST, we will be performing scheduled maintenance in preparation for the addition of more servers. To ensure a safe expansion, we are expecting all services to be unavailable for 15 minutes. All public forms and reports will see error messages as shown in this screenshot. Tuesday, June 22nd at 9pm PST [ View All Timezones ] If you’re up or interested, you can...

Jun 22nd

Scheduled Patching

This Thursday, April 29th at 9pm PST, we will be performing scheduled maintenance to apply routine patches to core servers. During tonights maintenance we are expecting all services to be unavailable for 15 minutes. All public forms and reports will see error messages as shown in this screenshot. Thursday, April 29th at 9pm PST [ View All Timezones ] If you’re up or interested, you can follow...

Apr 29th

Scheduled Maintenance

This Tuesday, April 6th at 9pm PST, we will be performing scheduled maintenance to address some of the connectivity issues we found yesterday. The connectivity issues prevented random connections from accessing the web site for about 5 minutes. During tonights maintenance we are expecting all services to be unavailable for 15 minutes. All public forms and reports will see error messages as shown...

Apr 6th

Server Upgrade Issue

We did experience server issue starting around 3:30am EST. One of our recently upgraded database servers wasn’t handling I/O as intended, which forced us to reboot it. We were able to gather a few data points, so hopefully we can make a proper conclusion about the events leading up to the problem. We’re sorry for the outage.

Mar 18th

Web Server Issue

Around 1pm EST, for about 25 minutes, one of our web servers was dropping requests to public forms. Only requests sent to this web server were affected. Thanks to everyone who reported this, and for helping us narrow it down. We’re sorry for trouble.

Mar 1st

Scheduled Maintenance

This Thursday, February 25th at 10pm PST, we will be performing scheduled maintenance to address some of the server issues we found last week. During this outage we are expecting all services to be unavailable for 20 minutes. All public forms and reports will see error messages as shown in this screenshot. Thursday, February 25th at 10pm PST [ View All Timezones ] If you’re up or interested,...

Feb 24th

Database / Page Load Issue

Thursday, February 18th. Earlier this morning, for about 5 minutes, some pages were not fully loading. The problem took care of itself, and we investigated the root cause. After noticing the master lookup database was trying to read from disk occasionally, we decided the best course would be a reboot so that we could watch the memory usage upon restart. Our planned reboot was for midnight. ...

Feb 18th

Moving to Rackspace

This Saturday, October 10th, at 11:00 PM EST, we will be moving Wufoo to a new home with Rackspace. During this move we are expecting all services to be unavailable for 30 minutes. October 10th, 2009 at 11:00 PM EST [ View All Timezones ] If you’re up or interested, you can follow our progress during the downtime on our Tumblr Status Blog and/or Twitter feed. We also have more details about the...

Oct 8th

1 note

Server Reboot

We are experiencing a problem with replication on one of our main database servers, so we have to take the site down for a short period of time. We’re estimating about 15 minutes. Downtime started at: 2:20pm EST View all timezones *Update - Back up at 2:28 EST. Thanks for your understanding.

Oct 5th

Server Maintenance : Thursday, July 23, 2009 at...

This Thursday at 9:00 PM PDT, we’ll be doing some software and security upgrades to the web and file servers in addition to running a failover test on the database server. During the upgrades and tests, Wufoo’s services will be unavailable for at least 30 minutes up to an hour. The standard maintenance message will show on your forms and reports during this period. July 23rd, 2009 at 9:00...

Jul 22nd

Explanation of this Morning's Downtime

Apologies for the delayed announcement on this recent downtime. We’re still gathering information from last night and because it was a long night for everyone involved we tried and let people catch some sleep once everything seem up and secure again. Here’s the information that we know so far: As most of you know, we had a planned maintenance period last Thursday to restart some of our servers to...

Jul 5th

Touch Ups from Server Move

Tonight, Thursday July 2nd, 2009, at midnight eastern, we would like to clean up a few loose ends from our server move. We believe all fixes can be accomplished with a simple reboot. And unfortunately, our replication servers are affected as well, so we can’t do a rolling downtime. Plan for approximately 10 minutes of downtime. The standard error messages will show...

Jul 2nd

Server Move

As planned, our server move has begun. We’ll be working on it over the next 4 hours, and we’ll update this post if there are any advances/delays in the time frame. UPDATE Three and a half hours in, all services have been restored. Everything went as planned, and all accounts should be functioning as expected. Thanks for your patience as we went through this move.

Jun 27th

Power Problems

We just received word that there are power problems at the data center. All customers at the center are affected. Will know more soon. *Edit on 5/19 3:30 EDT* It appears the unthinkable is happening again. As we were evaluating the downtime yesterday, power went out again. The root of the problem bypasses our redundant circuits, and we are completely dependent on a third party. *Edit on...

May 18th

New Database Server - 5/15/2009

At midnight we made the switch to a new primary database server (our master lookup database). We tried to do this with no downtime, but it didn’t work as expected. Wufoo was unavailable for 37 minutes during this cutover.

May 16th

Outage Earlier Today

Monday, May 11th, 2009, we were down between 5:00pm and 6:15pm EST. The trouble began when one of our DB servers had a process list build up that began affecting the web servers. For about 10 minutes, starting at 5:10pm, the DB server recovered and the site was responsive. We thought the problem was fixed. Shortly after, we found out the server could not fully recover, so we began the reboot and...

May 11th

1 note

Unexpected Reboots

Tonight, April 29th 2009 at 12:30, we had to reboot one of our primary servers. The service was unavailable for a few minutes while this took place. We apologize for any inconvenience caused.

Apr 29th

1 note

Server Reboots

We will be doing a round of reboots and general maintenance to the servers tonight, 4/16/09, at 7pm PDT. There will be a brief window when Wufoo will be unavailable. Plan for roughly 15 to 30 minutes of unavailability for your forms. Normally, this sort of maintenance can be done with no downtime as we have redundant servers and can bring down one at a time. But with the settings we have to change...

Apr 16th

Free Plan Outages - 4/2/09

We will be doing maintenance tonight that will affect new free plans created within the past 3 weeks. There are also some very old accounts that will be affected by this as well. We are streamlining everything into our new server configuration. These recent accounts are affected because they were temporarily created in a holding server. No paid accounts, and a majority of free accounts will not be...

Apr 3rd

Downtime Tonight

This post is to inform you that we will be performing maintenance on Wufoo tonight. The scheduled outage is: Saturday, March 27th, 2009 12:30am EST - 1:30am EST While the scheduled time slot is one hour, we do want to alert you to potential uncertainties. The maintenance could take as little as 10 minutes, or it could span longer than the hour scheduled. If it does take longer, we will...

Mar 27th

Unexpected Maintenance

Edit - 1:18pm EST - All servers are back up. Any accounts on the affected servers will be slow as the cache rebuilds, but all functionality is there. The speed will get progressively faster over the next 10 minutes. It is 12:55pm EST, and we are currently performing maintenance on a couple of our servers. Not all accounts are affected, but about 60% of forms will have problems loading. The...

Mar 17th

Downtime This Weekend

This post is to inform you that we will be performing maintenance on Wufoo this coming Saturday. The scheduled outage is: Saturday, March 1st, 2008 7:00pm EST - 8:00pm EST While the scheduled time slot is one hour, we do want to alert you to the potential of delays. Wufoo is unique in the way that each account is isolated for security, which can cause unexpected behavior when performing...

Feb 26th