From fb37a1cb65126242d381567dcdda0f32803dc06e Mon Sep 17 00:00:00 2001 From: Adam Harvey Date: Tue, 2 Jun 2026 09:00:17 -0700 Subject: [PATCH] crates.io: remove outdated ops info We have this encoded in the crates.io ops guide, along with much more, so this is duplicative and out of date. I've also restructured the index page to look more like the docs.rs page, but I'm also open to the idea that we should just remove it altogether. --- src/SUMMARY.md | 2 - src/crates-io/crate-removal.md | 52 -------- src/crates-io/db-maintenance.md | 203 -------------------------------- src/crates-io/index.md | 13 +- 4 files changed, 12 insertions(+), 258 deletions(-) delete mode 100644 src/crates-io/crate-removal.md delete mode 100644 src/crates-io/db-maintenance.md diff --git a/src/SUMMARY.md b/src/SUMMARY.md index 0a146ecd2..705d25d4e 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -69,8 +69,6 @@ - [Working Areas](./compiler/working-areas.md) - [rustc-dev-guide](./rustc-dev-guide/index.md) - [crates.io](./crates-io/index.md) - - [Crate removal](./crates-io/crate-removal.md) - - [Database maintenance](./crates-io/db-maintenance.md) - [docs.rs](./docs-rs/index.md) - [Adding dependencies to the build environment](./docs-rs/add-dependencies.md) - [Self-hosting a docs.rs instance](./docs-rs/self-hosting.md) diff --git a/src/crates-io/crate-removal.md b/src/crates-io/crate-removal.md deleted file mode 100644 index 42e1c0877..000000000 --- a/src/crates-io/crate-removal.md +++ /dev/null @@ -1,52 +0,0 @@ -# Crate removal procedure - -If we get a DMCA takedown notice, here's what needs to happen: - -## Contact Legal - -Before removing the crates, get in touch with legal support, -and ask an opinion from them on the received request and -whether we have to comply with it. - -## Remove relevant version(s) and/or entire crates from crates.io - -* Remove it from the database: - - ```console - heroku run -a crates-io -- target/release/crates-admin delete-crate [crate-name] - ``` - - or - - ```console - heroku run -a crates-io -- target/release/crates-admin delete-version [crate-name] [version-number] - ``` - -* Remove the crate or version from the index. To remove an entire crate, remove - the entire crate file. For a version, remove the line corresponding to the - relevant version. - -* Remove the crate archive(s) and readme file(s) from S3. - -* Invalidate the CloudFront cache: - - ```console - aws cloudfront create-invalidation --distribution-id EJED5RT0WA7HA --paths '/*' - ``` - -## Remove entire crates from docs.rs - -The docs.rs application supports deleting all the documentation ever published -of a crate, by running a CLI command. The people who currently have permissions -to access the server and run it are: - -* docs.rs Team: - * [@pietroalbini](https://github.com/pietroalbini) - * [@jyn514](https://github.com/jyn514) -* Infrastructure Team: - * [@Mark-Simulacrum](https://github.com/Mark-Simulacrum) -* People with elevated 1password access - -You can find the documentation on how to run the command [here][docsrs-howto]. - -[docsrs-howto]: https://forge.rust-lang.org/infra/docs/docs-rs.html#removing-a-crate-from-the-website diff --git a/src/crates-io/db-maintenance.md b/src/crates-io/db-maintenance.md deleted file mode 100644 index 16f368708..000000000 --- a/src/crates-io/db-maintenance.md +++ /dev/null @@ -1,203 +0,0 @@ -# Database maintenance - -There are times when Heroku needs to perform a maintenance on our database -instances, for example to apply system updates or upgrade to a newer database -server. - -We must **not** let Heroku run maintenances during the maintenance window to -avoid disrupting production users (move the maintenance window if necessary). -This page contains the instructions on how to perform the maintenance with the -minimum amount of disruption. - -# Primary database - -Performing maintenance on the primary database requires us to temporarily put -the application in read-only mode. Heroku performs maintenances by creating a -hidden database follower and switching over to it, so we need to prevent writes -on the primary to let the follower catch up. - -Maintenance should take less than 5 minutes of read-only time, but we should -still announce it ahead of time on our status page. This is a sample message we -can use: - -> The crates.io team will perform a database maintenance on YYYY-MM-DD from -> hh:mm to hh:mm UTC. -> -> We expect this to take less than 5 minutes to complete. During maintenance, -> crates.io will only be available in read-only mode: downloading crates and -> visiting the website will still work, but logging in, publishing crates, -> yanking crates, or changing owners will not work. - -## Primary database checklist - -**1 hour before the maintenance** - -1. Go into the Heroku Scheduler and disable the job enqueueing the downloads - count updater. You can "disable" it by changing its schedule not to run - during the maintenance window. The job uses a lot of database resources, and - we should not run it during maintenance. - -**5 minutes before the maintenance** - -2. Scale the background worker to 0 instances: - - ```console - heroku ps:scale -a crates-io background_worker=0 - ``` - -**At the start of the maintenance** - -3. Update the status page with this message: - - > Scheduled maintenance on our database is starting. - > - > We expect this to take less than 5 minutes to complete. During maintenance, - > crates.io will only be available in read-only mode: downloading crates and - > visiting the website will still work, but logging in, publishing crates, - > yanking crates, or changing owners will not work. - -3. Configure the application to be in read-only mode without the follower: - - ```console - heroku config:set -a crates-io READ_ONLY_MODE=1 DB_OFFLINE=follower - ``` - - The follower is removed because while Heroku tries to prevent connections to - the primary database from failing during maintenance we observed that the - same does not apply to the follower database, and there could be brief - periods while the follower is not available. - -3. Wait for the application to be redeployed with the new configuration: - - ```console - heroku ps:wait -a crates-io - ``` - -3. Run the database maintenance: - - ```console - heroku pg:maintenance:run --force -a crates-io - ``` - -1. Wait for the maintenance to finish: - - ```console - heroku pg:wait -a crates-io - ``` - -3. Confirm all the databases are online: - - ```console - heroku pg:info -a crates-io - ``` - -3. Confirm the primary database fully recovered (should output `false`): - - ```console - echo "SELECT pg_is_in_recovery();" | heroku pg:psql -a crates-io DATABASE - ``` - -3. Switch off read-only mode: - - ```console - heroku config:unset -a crates-io READ_ONLY_MODE - ``` - - **WARNING:** the Heroku Dashboard's UI is misleading when removing an - environment variable. A red badge with a "-" (minus) in it means the - variable was *successfully removed*, it doesn't mean removing the variable - failed. Failures are indicated with a red badge with a "x" (cross) in it. - -3. Wait for the application to be redeployed with the new configuration: - - ```console - heroku ps:wait -a crates-io - ``` - -3. Update the status page and mark the maintenance as completed with this - message: - - > Scheduled maintenance finished successfully. - - The message is posted right now and not at the end because this is when - production users are not impacted by the maintenance anymore. - -3. Scale the background worker up again: - - ```console - heroku ps:scale -a crates-io background_worker=1 - ``` - -3. Confirm the follower database is available: - - ```console - echo "SELECT 1;" | heroku pg:psql -a crates-io READ_ONLY_REPLICA - ``` - -3. Enable connections to the follower: - - ```console - heroku config:unset -a crates-io DB_OFFLINE - ``` - -3. Re-enable the background job disabled during step 1. - -# Follower database - -Performing maintenance on the follower database doesn’t require any external -communication nor putting the application in read-only mode, as we can just -redirect all of the follower’s traffic to the primary database. It shouldn’t be -done during peak traffic periods though, as we’ll increase the primary database -load by doing this. - -## Follower database checklist - -**At the start of the maintenance** - -1. Configure the application to operate without the follower: - - ```console - heroku config:set -a crates-io DB_OFFLINE=follower - ``` - -1. Wait for the application to be redeployed with the new configuration: - - ```console - heroku ps:wait -a crates-io - ``` - -1. Start the database maintenance: - - ```console - heroku pg:maintenance:run --force -a crates-io READ_ONLY_REPLICA - ``` - -1. Wait for the maintenance to finish: - - ```console - heroku pg:wait -a crates-io READ_ONLY_REPLICA - ``` - -1. Confirm the follower database is ready: - - ```console - heroku pg:info -a crates-io - ``` - -1. Confirm the follower database is responding to queries: - - ```console - echo "SELECT 1;" | heroku pg:psql -a crates-io READ_ONLY_REPLICA - ``` - -1. Enable connections to the follower: - - ```console - heroku config:unset -a crates-io DB_OFFLINE - ``` - -1. Wait for the application to be redeployed with the new configuration. - - ```console - heroku ps:wait -a crates-io - ``` diff --git a/src/crates-io/index.md b/src/crates-io/index.md index ca189a0ce..441bc54bc 100644 --- a/src/crates-io/index.md +++ b/src/crates-io/index.md @@ -1,3 +1,14 @@ # crates.io -This section documents the processes of the crates.io team. +crates.io hosts crates for the Rust ecosystem. + +## External links + +- Source code: [`rust-lang/crates.io`][repo] +- Hosted at: https://crates.io/ +- Maintainers: [crates.io team][maintainers] +- [Ops guide][ops] (only available to team members) + +[maintainers]: https://rust-lang.org/governance/teams/dev-tools/#team-crates-io +[ops]: https://ops-guide.crates.io/ +[repo]: https://github.com/rust-lang/crates.io