In January, we experienced two incidents. One that resulted in degraded performance for GitHub Packages and GitHub Pages, and another that impacted git users.
January 30 21:48 UTC (lasting 35 minutes)
Our service monitors detected degraded performance for GitHub Packages and GitHub Pages. Most requests to the container registry were failing and some GitHub Pages builds were also impacted. We determined this was caused by a backend change and mitigated by reverting that change.
Due to the recency of this incident, we are still investigating the contributing factors and will provide a more detailed update in next month’s report.
January 30 18:35 UTC (lasting 7 hours)
We upgraded our production Git binary with a recent version from upstream. The updates included a change to use an internal implementation of
gzip when generating archives. This resulted in subtle changes to the contents of the “Download Source” links served by GitHub, leading to checksum mismatches. No content was changed.
After becoming aware of the impact to many communities, we rolled back the compression change to restore the previous behavior.
Similar to the above, we are still investigating the contributing factors of this incident, and will provide a more thorough update in next month’s report.
Please follow our status page for real-time updates on status changes. To learn more about what we’re working on, check out the GitHub Engineering Blog.