September 2015

[Scheduled] Datacenter network maintenance
The scheduled maintenance has been completed.
Sep 22, 01:30-07:30 EDT

August 2015

[Scheduled] Server software updates
The scheduled maintenance has been completed.
Aug 22, 09:45-09:48 EDT

July 2015

[Scheduled] Server software updates
The scheduled maintenance has been completed.
Jul 25, 09:45-09:56 EDT
[Scheduled] Server software updates
The scheduled maintenance has been completed.
Jul 11, 09:30-09:35 EDT
Developer website not reachable
This incident has been resolved.
Jul 6, 17:37-18:39 EDT

June 2015

[Scheduled] Server software updates
The scheduled maintenance has been completed.
Jun 20, 02:30-02:32 EDT
500 Errors
This incident has been resolved.
Jun 3, 12:52-14:34 EDT

May 2015

500 errors
This incident has been resolved.
May 26, 16:16-17:09 EDT
500 errors
Freckle is all back in action. The root cause of the errors was a temporary traffic spike on our API. Our emergency systems we have in place to recover from an overload like this worked as expected and Freckle came back after a downtime of about 4 minutes. Obviously this still sucks, and we're taking steps to prevent the same thing from happening again in the future so that temporary traffic spikes will no longer cause Freckle to become unavailable.
May 25, 16:34-17:05 EDT
[Scheduled] Urgent software update
The scheduled maintenance has been completed.
May 14, 08:45-09:05 EDT
[Scheduled] Server software updates
The scheduled maintenance has been completed.
May 9, 08:30-08:33 EDT

April 2015

[Scheduled] Server software updates
The scheduled maintenance has been completed.
Apr 18, 09:00-09:12 EDT

March 2015

Freckle is down
This incident has been resolved.
Mar 5, 06:44-08:07 EDT
[Scheduled] Datacenter infrastructure maintenance
The scheduled maintenance has been completed.
Mar 5, 01:00-05:00 EDT
Timers not in sync
This incident has been resolved.
Mar 3, 11:25-12:07 EDT
[Scheduled] Software updates
The scheduled maintenance has been completed.
Mar 1, 09:30-09:35 EDT

February 2015

Freckle is down
This incident has been resolved.
Feb 18, 09:20-12:30 EDT
Support chat is currently down
Support chat is available again!
Feb 17, 10:48-12:21 EDT
[Scheduled] Software updates
The scheduled maintenance has been completed.
Feb 14, 09:00-09:03 EDT
[Scheduled] Datacenter network maintenance
The scheduled maintenance has been completed.
Feb 11, 02:00-08:00 EDT
[Scheduled] Datacenter network maintenance
The scheduled maintenance has been completed.
Feb 10, 02:00-08:00 EDT
Failing PDF downloads
This incident has been resolved.
Feb 3, 18:23-18:42 EDT

January 2015

[Scheduled] Security patches and updates
The scheduled maintenance has been completed.
Jan 17, 08:45-09:03 EDT
Background tasks not processing
This incident has been resolved.
Jan 10, 09:50-14:20 EDT
Mail sending and integrations syncing partially unavailable
All queues are processing normally. There's some backlog on syncing with Basecamp, but we're catching up. If that is the case for your account it should be resolved within the next hour or so.
Jan 10, 07:39-08:55 EDT

December 2014

Background processing slowdown
This incident has been resolved.
Dec 18, 07:37-09:05 EDT
Freckle is down due to a deployment issue
All systems go! :)
Dec 15, 15:25-15:29 EDT
[Scheduled] Software updates
The scheduled maintenance has been completed.
Dec 13, 09:15-09:23 EDT
[Scheduled] Software updates
The scheduled maintenance has been completed.
Dec 6, 09:30-09:34 EDT
500 errors
We've tuned some background processing to lighten the load and haven't seen 500 errors in several hours. We're marking this issue as solved, but will continue to keep an eye on it.
Dec 1, 14:13-18:27 EDT
Loading time delays and 500 errors
We have increased capacity and tweaked a few things here and there for smoother loading.
Dec 1, 13:26-13:59 EDT

November 2014

CDN issues preventing Freckle from being displayed correctly
Issues with Amazon's CloudFlare DNS not resolving have been fixed by Amazon. Happy Thanksgiving!
Nov 26, 19:41-21:37 EDT
Background processing unresponsive
All background processing has been restored to normal.
Nov 8, 08:07-08:14 EDT
[Scheduled] Reboot of Freckle servers
The scheduled maintenance has been completed.
Nov 8, 07:51 EDT

October 2014

Background processing queues not working
This incident has been resolved.
Oct 11, 09:50-12:56 EDT
[Scheduled] Software updates
All is good! Have a nice weekend! :)
Oct 11, 08:30-08:32 EDT

September 2014

[Scheduled] Hosting updates
The scheduled maintenance has been completed.
Sep 28, 23:07 - Sep 29, 07:00 EDT
[Scheduled] Security patches and updates
The scheduled maintenance has been completed.
Sep 4, 08:00-08:02 EDT

August 2014

[Scheduled] Server software updates
The scheduled maintenance has been completed.
Aug 9, 08:30-08:42 EDT
Intermittent 500 errors
This incident has been resolved.
Aug 4, 11:49-14:53 EDT

July 2014

[Scheduled] Software updates
The scheduled maintenance has been completed.
Jul 21, 21:58 EDT
500 errors
This incident has been resolved.
Jul 21, 10:46-13:45 EDT
Billing partially unavailable
Everything is back to normal. We've reached out to customers who couldn't update their credit card information in the time the payment provider wasn't available.
Jul 8, 08:42-09:12 EDT
[Scheduled] Software updates
The scheduled maintenance has been completed.
Jul 6, 09:00-09:12 EDT

June 2014

[Scheduled] Server software updates
The scheduled maintenance has been completed.
Jun 22, 10:04-10:16 EDT
[Scheduled] Security patches and updates
All is good!
Jun 5, 11:00-11:46 EDT

May 2014

[Scheduled] Software updates
Software updates are complete! Have a nice weekend! :)
May 11, 10:00-10:07 EDT

April 2014

Freckle Blog is intermittently down
Landing page and blog are now available again.
Apr 24, 06:13-07:22 EDT
[Scheduled] Software updates
We've finished our maintenance. All systems go! :)
Apr 8, 08:00-08:09 EDT

March 2014

[Scheduled] Hosting updates and DNS change
Freckle is back! New DNS means that it might take a while for it to be available from your location again. Enjoy your weekend. :)
Mar 23, 16:00-17:23 EDT
Norton DNS has issues resolving Freckle's DNS entries
We've not seen any more issues with Norton DNS in several hours.
Mar 20, 11:09-12:47 EDT

February 2014

Slow loading and timeouts
Freckle is back! There was a queued process using up too much memory causing a server load issue. We're working on a fix to prevent this from happening again in the future.
Feb 21, 17:18-17:28 EDT
Freckle outage
Database performance is back to normal. We will continue to monitor for slow queries.
Feb 16, 04:03-04:12 EDT
Intermittent outage
All is back to normal. The root cause was an issue with an import that was processed in the background—we're now debugging exactly why this import stalled and used up resources and working on a fix to prevent this in the future.
Feb 14, 14:46-14:59 EDT

January 2014

No incidents reported for this month.

December 2013

Freckle is down
We're all back and Freckle works normally. We're still researching the root cause of this problem.
Dec 27, 17:31-17:39 EDT

November 2013

Partial outage of Freckle
Performance is back to normal. This problem was caused by an issue with Freckle API, which we have identified and will roll out a fix very soon. We'll keep on monitoring'.
Nov 25, 16:57-17:13 EDT
DNS resolving problem
This incident has been resolved.
Nov 15, 08:57-09:12 EDT
Freckle is currently down
All systems are go—the unicorns are prancing once more! :)
Nov 7, 12:00-12:04 EDT

October 2013

[Scheduled] Landing page and blog server migration
We've moved hosting providers for our landing page and blog, which means the scheduled maintenance will no longer affect the Freckle landing page and blog. Have a nice weekend! :)
Oct 25, 21:05 EDT

September 2013

Homepage and blog down
This incident has been resolved.
Sep 19, 00:36-00:49 EDT
Unexpected downtime due to deploying problem
Freckle is back, all systems go!
Sep 8, 10:23-10:28 EDT

August 2013

High server load
All systems go. We had to kill and restart a spurious Ruby process. Bad Ruby!
Aug 6, 17:31-19:50 EDT
Report errors
This incident has been resolved.
Aug 1, 10:05-11:54 EDT

July 2013

Freckle is currently down
Freckle is back and up and running!
Jul 5, 09:40-09:43 EDT
[Scheduled] Scheduled data migration
Database updates are complete and Freckle is available again! Happy 4th of July!
Jul 4, 09:00-09:08 EDT

June 2013

[Scheduled] Infrastructure updates
The scheduled maintenance has been completed.
Jun 20, 01:00-06:05 EDT
Short downtime due to DB update
...and we're back. :(
Jun 18, 14:24-15:03 EDT
Github integration logging all commits
The Github integration now works as expected and will no longer log commits without time information. We also removed empty entries that where created because of this bug.
Jun 18, 09:03-09:46 EDT
Slow loading and timeouts
We've identified an issue with a hanging PhantomJS process that we use for generating graphs for the weekly report emails causing timeouts across the board with Freckle being unavailable for short periods of time. We've restarted the process and performance is back to normal now.
Jun 17, 10:03 EDT

May 2013

No incidents reported for this month.

April 2013

No incidents reported for this month.

March 2013

Timers not working correctly
The Timer will no longer display bogus "disconnected" alerts. Please be sure to reload your timer window or simply close and open it again.
Mar 27, 11:15-11:26 EDT
Sign in page does not work correctly
Due to a routing misconfiguration the sign in page tried to reload itself—this problem is now fixed and all systems are go!
Mar 26, 14:41-14:50 EDT
Network outage
Freckle was down for about 12 minutes due to a network outage at our hoster. All systems are go now.
Mar 21, 19:55 EDT