Add AI tools policy #13726
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| # Guidelines for using AI tools | ||
|
|
||
| The person submitting an issue or PR is responsible for its content, | ||
| regardless of whether AI tools were used in its creation. Generative AI | ||
| tools can produce output quickly, but discretion, good judgment, and | ||
| critical thinking are the foundation of all good contributions. We value | ||
| good code, concise accurate documentation, and well scoped PRs without | ||
| unneeded code churn. | ||
|
|
||
| ## Considerations for success | ||
|
|
||
| Authors must review the work done by AI tooling in detail to ensure it | ||
| actually makes sense before proposing it as a PR or filing it as an issue. | ||
|
|
||
| We expect PR authors and those filing issues to be able to explain their | ||
| proposed changes in their own words. | ||
|
|
||
| Disclosure of the use of AI tools in the PR description is appreciated, | ||
| while not required. Be prepared to explain how the tool was used and what | ||
| changes it made. | ||
|
|
||
| Whether you are using AI tools or not, keep the following principles in | ||
| mind for the quality of your contribution: | ||
|
|
||
| - Consider whether the change is necessary | ||
| - Make minimal, focused changes | ||
| - Follow existing coding style and patterns | ||
| - Write tests that exercise the change | ||
| - Keep backwards compatibility with prior releases in mind. Existing | ||
| tests may be ensuring specific API behaviors are maintained. | ||
|
|
||
| Pay close attention to AI generated recommendations for testing changes. | ||
| Always review the output before opening a pull request or issue, | ||
| including proposed PR or issue titles and descriptions. | ||
|
|
||
| ## Acceptable uses | ||
|
|
||
| Some of the acceptable uses of generative AI include: | ||
|
|
||
| - Assistance with writing comments, especially in a non-native language | ||
| - Gaining understanding of existing code | ||
| - Supplementing contributor knowledge for code, tests, and documentation | ||
|
|
||
| ## Unacceptable uses | ||
|
|
||
| Maintainers may close issues and PRs that are not useful or productive, | ||
|
There was a problem hiding this comment. So, I think it would be worth clarifying a bit more what "productive" means here, since it feels a little vague and it's not exactly clear what you're looking for. (I'm fine if you keep using the term "productive" as long as it's defined, just, maybe a few examples would help.) Just guessing at what you mean, what might be worth mentioning:
Member
Author
There was a problem hiding this comment. Thanks, I see where you're coming from but I'd actually prefer to leave "productive" deliberately vague. To be honest I'm not sure all your examples land for me. Typo fixes for instance often seem fine. The kind of thing I'd see as unproductive is more like a PR where the author clearly doesn't understand what the LLM produced and is just acting as a transport medium between the model and the reviewer. But that's just one example, and I'd really rather leave it to maintainer judgment than enumerate it. My worry with putting cases in the policy is that they get read as an exhaustive list. A PR that doesn't fit any listed category becomes harder to push back on ("but the policy doesn't say this counts"), and we'd be locking ourselves into whatever taxonomy we write down today. The point of this section, as I read it, is to give maintainers cover to use judgment on cases that don't fit cleanly into rules. Happy to revisit if it turns out to be confusing in practice. |
||
| regardless of whether AI tools were used or not. | ||
|
|
||
| If a contributor repeatedly opens unproductive issues or PRs, they may be | ||
| blocked from contributing to the project because it is disruptive and | ||
| disrespectful of the maintainers time. | ||
|
|
||
| It is not acceptable to alter or bypass existing tests, or remove desired | ||
| functionality, in order to make a failing test pass. Such changes are not | ||
| a real fix. | ||
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So ultimately I don't expect crates.io to have a lot of PRs or issues so I think that ultimately this is just me giving advice that you're free to ignore. But at least from what I found, even projects which strongly support AI tools have been in favour of disclosing just so they know what they're dealing with, and I think that it's okay to have a hard rule with no punishments for forgetting it.
Like, I figure the rate of issues/PRs is so low that you have the time/resources to just individually talk to each person and figure out what they're working with anyway, but it can be helpful to ask for to see what people are using.
Just a few examples of other policies that feel relevant here:
View changes since the review
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This would also be my preference.
I litigated this at some length within the Foundation when we were drafting our internal AI policy, so my apologies to @Turbo87 for the repeat, but the short version is that I believe that we should require disclosure for two primary reasons:
I don't want this to be some onerous process. I'd be happy with an
Assisted-ByorCo-Authored-Bytrailer in either the commit message or the PR description, much like the Linux kernel policy. But I do think it's important that we require this.Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be interesting to hear from counsel on the first point. I would expect to hear a mixed set of pros and cons for each choice — widespread disclosure seems likely to carry some risks too for the Project. Regarding the second point, the relevant comment in the Python discussions was this one — in short, reviewers need to assume LLM use anyway.
Disclosure has more than minimal cost:
Mandating disclosure has additional costs. It becomes something we need to enforce fairly and consistently. As part of a moderation policy, we'd be talking about banning people from the Project over this. If it turns out the policy is unenforceable and that people choose not to disclose due to these costs, then we'll have created a kind of policy fiction. That's not any good for us.
Similar to Python, we can encourage disclosure without incurring the costs that accompany mandating it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So, it is worth reiterating on the legal point that generally the result of using LLMs is the removal of copyright protections, not the addition of extra burdens, which does not matter given the extremely permissive license crates.io uses. It does matter for proprietary code and copyleft code, but for open source, permissively licensed work, it seems fine.
Additionally, it seems incredibly likely that if any copyright questions came up, e.g. someone was upset that code verbatim was used without authorisation, there would be ample time to fix the issue (removing the offending code) rather than these being immediate repercussions.
Most of the projects that I've seen be concerned with copyright issues are copyleft and thus specifically rely on copyright to enforce licencing, which just isn't the case here. So, while I do think there are other points to discuss, I don't think the legal point is particularly compelling here, which also matches the existing advice given by Foundation counsel on the matter.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think I do most of the PR reviews on crates.io at the moment (obviously excluding my own PRs…), and knowing whether AI was involved wouldn't change much how I review. The "explain your changes" requirement is already what I actually care about. For first-time contributors with zero context I agree disclosure can help, but that's a pretty different situation from someone opening a dozens of PRs each month.
On the legal point: the PSF landed on this exact wording without requiring disclosure. I don't know what their counsel actually said about it, so I'm only guessing, but I'd be a little surprised if a project that size shipped an AI policy without running it past their lawyers at some point. If that's roughly right, it makes me want to understand what's specifically different about our situation before treating the legal angle as a strong reason to go further than they did.
And I think Travis is right that disclosure has real downsides in the current climate. Adding a trailer to every PR also adds quite a bit of friction when LLM tooling is opening them, and getting that tooling to reliably add trailers is a whole separate problem.
That said, here are a few things I'd be happy to put in the policy, if it helps:
Two other options to avoid the additional friction for regular contributors:
We can flip the framing: say explicitly that reviewers should assume AI may have been used, and make disclosure of non-use the optional opt-in. At this point the vast majority of recent crates.io PRs had LLM involvement and IMHO it's better to optimize for the 90% case.
One-time disclosure for regulars, recorded privately: contributors who use AI tooling regularly disclose it once (e.g. via a comment on a designated issue in the team's private ops guide repo) instead of repeating it on every PR. The record exists, so reviewers on the team have access and the legal/review-practicalities angle is covered, but the disclosure isn't sitting in a public list that anyone can scrape and turn into a target. For someone like me who opens a lot of PRs that drops the per-PR friction effectively to zero while still leaving the information available where it matters. First-time or external contributors would still get asked by a PR template or maybe by the reviewer.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I mean, ultimately, I do think that the small number of contributions is more than sufficient to prefer an ad hoc method of explaining/disclosing since, like I said, you probably have the bandwidth to just talk to everyone individually about their changes. I think that encouraging more transparency is good, but if you think the downsides outweigh the upsides here, that's fair.