-
Notifications
You must be signed in to change notification settings - Fork 233
Add an LLM policy for rust-lang/rust
#1040
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from 2 commits
772edeb
815da6e
17a35f4
61e5e2c
8ee5ed4
7cd8c17
2db7465
9b2b3c2
e3b1394
e3f2aec
864428f
593d538
75050a2
b6a8662
9a944f7
14956c3
8fe7281
791e46f
8520038
b14e8ca
69b6dc1
d682475
ea4e504
7eeecbb
cd9aecd
ee4f26c
7956574
b88855a
4305e14
ab6f8a4
d9d8238
83b9363
9efffad
24f236c
f85aac6
adfc5e2
014cf85
db8fdae
196cf63
6374d57
8a1ce25
742d9f4
235432b
9396306
33b1407
08a6b17
3740dc5
fb37c69
8444f54
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
|
oli-obk marked this conversation as resolved.
This comment was marked as a violation of GitHub Acceptable Use Policies
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I understand you're frustrated, but I don't think this kind of language helps anyone. About your point: you explicitly state that people are allowed to have their own opinion, but apparently they are not allowed, per you, to have an opinion on whether the discussion of ethics is relevant here. And for the record, I don't use LLMs and I dislike them for many reasons. But I also think that setting a policy on the basis of the ethical consequences of them is not what we should do. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The missing link to said RFC: rust-lang/rfcs#3959 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not the one who decided to omit ethics from the discussion here; I'm respecting the opinions of the author on that, and conceding that it's fair to get a simpler policy out sooner in this particular case. There are allowed to be concurrent policy decisions and this policy's opinion is that focusing entirely on pragmatic labelling of LLM contributions, alongside calling out certain problematic usages, is the best way to get a policy out sooner. Is that the best choice? No idea! But I respect the author and their wishes and have an explicit alternative that will almost certainly take longer to figure out. So, instead of bothering jyn about it, I'm inviting the discussion to happen there instead. |
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,116 @@ | ||||||
| ## Policy | ||||||
|
nikomatsakis marked this conversation as resolved.
Outdated
|
||||||
|
|
||||||
| For additional information about the policy itself, see [the appendix](#appendix). | ||||||
|
|
||||||
| ### Overview | ||||||
|
nikomatsakis marked this conversation as resolved.
|
||||||
|
|
||||||
| Using an LLM while working on `rust-lang/rust` is conditionally allowed. | ||||||
| However, we find it important to keep the following points in mind: | ||||||
|
|
||||||
| - Many people find LLM-generated code and writing deeply unpleasant to read or review. | ||||||
| - Many people find LLMs to be a significant aid to learning and discovery. | ||||||
|
|
||||||
| Therefore, the guidelines are roughly as follows: | ||||||
|
|
||||||
| > It's fine to use LLMs to answer questions, analyze, distill, refine, check, suggest, review. But not to **create**. | ||||||
|
jyn514 marked this conversation as resolved.
|
||||||
|
|
||||||
| > LLMs work best when used as a tool to write *better*, not *faster*. | ||||||
|
|
||||||
|
Comment on lines
+16
to
+17
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
Having this as a high-level summary is offering a judgement on LLMs that feels like it isn't necessary for the policy, and makes consensus more difficult to reach. For anti-LLM folks it's saying that they work best when used to write "better", which is a point in dispute. I would also expect (but don't want to put words in people's mouths) that for pro-LLM folks the point that they don't work well when used to work faster may be in dispute. I've tried to rephrase this in a fashion that, rather than expressing a general statement on when "LLMs work best", is instead expressing what is desired *for
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is adapted from a quote by @ubiratansoares. This edit changes the quote beyond recognition, and I would rather remove it than edit this much.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Then I think it would be best removed, on the basis that previous line covers similar territory and seems less controversial.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Tbh I don't actually understand what this quote is supposed to mean; if anything, I would phrase it the other way around (you can use LLMs to do [things you can already do] to get them done faster, but you shouldn't use them to do things you don't already know how to do yourself).
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Honestly, it was the
that I took back to my team and reworked our approach to AI-generated code. I think that statement itself has a lot of weight. |
||||||
| #### Legend | ||||||
|
|
||||||
| - ✅ Allowed | ||||||
| - ❌ Banned | ||||||
| - ⚠️ Allowed with caveats. Must disclose that an LLM was used. | ||||||
|
jyn514 marked this conversation as resolved.
|
||||||
| - ℹ️ Adds additional detail to the policy. These bullets are normative. | ||||||
|
|
||||||
| ### Rules | ||||||
|
|
||||||
| #### ✅ Allowed | ||||||
| The following are allowed. | ||||||
| - Asking an LLM questions about an existing codebase. | ||||||
| - Asking an LLM to summarize comments on an issue, PR, or RFC. | ||||||
|
jyn514 marked this conversation as resolved.
Outdated
|
||||||
| - ℹ️ This does not allow reposting the summary publicly. This only includes your own personal use. | ||||||
|
jyn514 marked this conversation as resolved.
Outdated
|
||||||
| - Asking an LLM to privately review your code or writing. | ||||||
| - ℹ️ This does not apply to public comments. See "review bots" under ⚠️ below. | ||||||
| - Writing dev-tools for your own personal use using an LLM, as long as you don't try to merge them into `rust-lang/rust`. | ||||||
| - Using an LLM to discover bugs, as long as you personally verify the bug, write it up yourself, and disclose that an LLM was used. | ||||||
| Please refer to [our guidelines for fuzzers](https://rustc-dev-guide.rust-lang.org/fuzzing.html#guidelines). | ||||||
| - ℹ️ This also includes reviewers who use LLMs to discover bugs in unmerged code. | ||||||
|
jyn514 marked this conversation as resolved.
Outdated
|
||||||
|
|
||||||
|
traviscross marked this conversation as resolved.
traviscross marked this conversation as resolved.
jyn514 marked this conversation as resolved.
|
||||||
| #### ❌ Banned | ||||||
| The following are banned. | ||||||
| - Comments from a personal user account that are originally authored by an LLM. | ||||||
|
jyn514 marked this conversation as resolved.
Outdated
|
||||||
| - ℹ️ This also applies to issue bodies and PR descriptions. | ||||||
|
traviscross marked this conversation as resolved.
|
||||||
| - ℹ️ See also "machine-translation" in ⚠️ below. | ||||||
| - Documentation that is originally authored by an LLM. | ||||||
|
jyn514 marked this conversation as resolved.
Outdated
|
||||||
| - ℹ️ This includes non-trivial source comments, such as doc-comments or multiple paragraphs of non-doc-comments. | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
Reordering this to make it clear first and foremost that "Documentation" includes any doc comments, moving "non-trivial source comments" second. This also drops the quantitative "multiple paragraphs"; some multi-paragraph comments may be trivial, and some one-sentence comments may not be.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If you are using an LLM to write a multi-paragraph comment that is trivial, IMO that should also be banned. If you have a load-bearing single-line comment, I think that falls under "code changes authored by an LLM", although I'm not sure how to say that concisely.
jyn514 marked this conversation as resolved.
Outdated
|
||||||
| - ℹ️ This includes compiler diagnostics. | ||||||
|
jyn514 marked this conversation as resolved.
|
||||||
| - Code changes that are originally authored by an LLM. | ||||||
|
jackh726 marked this conversation as resolved.
Outdated
jyn514 marked this conversation as resolved.
Outdated
|
||||||
| - This does not include "trivial" changes that do not meet the [threshold of originality](https://fsfe.org/news/2025/news-20250515-01.en.html), which fall under ⚠️ below. | ||||||
| We understand that while asking an LLM research questions it may, unprompted, suggest small changes where there really isn't another way to write it. | ||||||
| However, you must still type out the changes yourself; you cannot give the LLM write access to your source code. | ||||||
|
jyn514 marked this conversation as resolved.
Outdated
jackh726 marked this conversation as resolved.
Outdated
|
||||||
| - We do not accept PRs made up solely of trivial changes. | ||||||
|
jyn514 marked this conversation as resolved.
Outdated
jyn514 marked this conversation as resolved.
Outdated
|
||||||
| See [the compiler team's typo fix policy](https://rustc-dev-guide.rust-lang.org/contributing.html#writing-documentation:~:text=Please%20notice%20that%20we%20don%E2%80%99t%20accept%20typography%2Fspellcheck%20fixes%20to%20internal%20documentation). | ||||||
| - See also "learning from an LLM's solution" in ⚠️ below. | ||||||
| - Treating an LLM review as a sufficient condition to merge a change. | ||||||
|
jyn514 marked this conversation as resolved.
Outdated
|
||||||
| LLM reviews, if enabled by a team, **must** be advisory-only. | ||||||
|
jyn514 marked this conversation as resolved.
Outdated
|
||||||
| Teams can have a policy that code can be merged without review, and they can have a policy that code must be reviewed by at least one person, | ||||||
|
jackh726 marked this conversation as resolved.
Outdated
|
||||||
| but they may not have a policy that an LLM counts as a person. | ||||||
|
jyn514 marked this conversation as resolved.
Outdated
|
||||||
| - ℹ️ See "review bots" in ⚠️ below. | ||||||
| - ℹ️ An LLM review does not substitute for self-review. Authors are expected to review their own code before posting and after each change. | ||||||
|
|
||||||
| #### ⚠️ Allowed with caveats | ||||||
| The following are decided on a case-by-case basis. | ||||||
| Please avoid them where possible. | ||||||
| In general, existing contributors will be treated more leniently here than new contributors. | ||||||
|
jyn514 marked this conversation as resolved.
Outdated
|
||||||
| We may ask you for the original prompts or design documents that went into the LLM's output; | ||||||
| please have them on-hand, and be available yourself to answer questions about your process. | ||||||
|
jyn514 marked this conversation as resolved.
Outdated
|
||||||
|
|
||||||
| - Using an LLM to generate a solution to an issue, learning from its solution, and then rewriting it from scratch in your own style. | ||||||
|
jyn514 marked this conversation as resolved.
Outdated
jackh726 marked this conversation as resolved.
Outdated
|
||||||
| - Using machine-translation from your native language without posting your original message. | ||||||
|
jyn514 marked this conversation as resolved.
Outdated
|
||||||
| Doing so can introduce new miscommunications that weren't there originally, and prevents someone who speaks the language from providing a better translation. | ||||||
| - ℹ️ Posting both your original message and the translated version is always ok, but you must still disclose that machine-translation was used. | ||||||
| - ℹ️ This policy also applies to non-LLM machine translations such as Google Translate. | ||||||
|
jyn514 marked this conversation as resolved.
Outdated
jyn514 marked this conversation as resolved.
Outdated
|
||||||
| - Using an LLM as a "review bot" for PRs. | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Maybe I'm OOTL but I find this section situationally strange — where did the "review bot" come from? IME AI-powered review bots that directly participates in PR discussions (esp the "app" ones) are configured by repository owner, but AFAIK r-l/r (which this policy applies solely to) did not have any such bots. I highly doubt a contributor will bring in their own review bot in public. So practically this has to be either
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I wish it worked like that :( People can just trigger GitHub copilot, or I suppose any other review bot, and let it comment on a r-l/r PR. Some people don't even do it willingly, but GH does it automatically for them, as GH copilot has a tendency to re-enable itself even if you sometimes disable it. It is also not possible to opt-out of the PR author requesting a Copilot review, if I remember correctly. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I’ve seen this behavior elsewhere on GitHub, where contributors effectively use a personal account as a kind of "review bot" to comment on PRs without approval from maintainers.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Yeah currently disabling review is a personal/license-owner setting, it is not possible to configure from the repository PoV 😞 but I think this is something that we may bring up to GitHub. It may be possible to use content exclusion to blind Copilot, but I'm not sure if this hack is going to produce any overreaching effects (e.g. affecting private IDE usage too).
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I think this is exactly the point of pointing that out in our policy. Some people trigger a "[at]copilot review" in our repos without asking us for consent. This is rude behaviour and we don't want that. And, yes, as you point out opting out of this "trigger" is currently only a project-wide setting, not at a repository level so we are looking with GitHub if they could make this setting more fine-grained (here on Zulip a discussion with the Infra team) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. yeah, you're right; I deleted the comment There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Unsolicited review bots are becoming an increasing problem; for example: https://web.archive.org/web/20260426133344/https://github.com/rust-lang/rust-clippy/issues/16893#issuecomment-4321880160 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thank you for flagging xtqqczze - the same bot has commented in 6+ issues on the rust-clippy repo and in my case was giving unsolicited advice in a completely derailing direction (solving a specific case I obviously already worked around rather than the general case rust-lang/rust-clippy#16901 (comment))
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @xtqqczze both rust-lang/rust-clippy#16893 and rust-lang/rust-clippy#16901 are issues not PRs, and that There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I find this review bot policy to be a messy compromise that doens't seem to be serving anyone particularly well. Speaking from general experience, it is rather frustrating as a reviewer to be met with a review request that has then already been answered by an LLM. Even if a maintainer allows the bot, that doesn't necessarily mean that the reviewer consents to it. I think contributors and reviewers who wish to use LLMs for review are better served by running these tools locally, in compliance with the other policies. I propose removing the language around review bots. |
||||||
| - ℹ️ Review bots **must** have a separate GitHub account that marks them as an LLM. They **must not** post under a personal account. | ||||||
|
jyn514 marked this conversation as resolved.
Outdated
jyn514 marked this conversation as resolved.
Outdated
|
||||||
| - ℹ️ Review bots that post without being approved by a maintainer will be banned. | ||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm concerned this leaves room for reviewers to trigger a review bot without consent of the author of the PR, which could alienate the PR author. If I opened a PR and it got reviewed by an LLM bot, I would probably close the PR and never try contributing to the project again. I've seen this happen in another project. I think there should be an agreement between the reviewer and PR author before triggering a review bot.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. "approved by a maintainer" is the key point here, if an LLM review bot is "approved by a maintainer" it means such is a public decision and should be mentioned in CONTRIBUTING.md, and that's the agreement. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. An agreement among maintainers to impose LLM review bots on nonconsenting contributors would drive those contributors away. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If a reviewer really wants to use an LLM to review, they could run that LLM on their own, filter through the output to determine what is actually relevant and correct, and post in their own words about the identified problems. That doesn't require bothering a nonconsenting PR author with LLM output.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Rephrasing LLM output is already addressed in lines 67-68. The premise of this whole section is that somehow a bot (as a separate account, line 69) can be officially " If you think that a review bot account should not be allowed, even if approved by maintainers, this whole thread would be more relevant on the parent item (line 66; I've commented about this before). P.S. I don't think this policy implies any LLM review bot account will be allowed "right now" or "soon", I believe there must at least be an FCP. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Thinking about this further, this seems like an overall better process than having a review bot comment on a PR. There's no room for ambiguity about whether a PR author is responsible for responding to LLM output; only the reviewer who decides to use an LLM is in a position to interpret the LLM output because "Comments from a personal user account that are originally authored by an LLM" are explicitly forbidden. |
||||||
| - ℹ️ If a linter already exists for the language you're writing, we strongly suggest using that linter instead of or in addition to the LLM. | ||||||
|
jyn514 marked this conversation as resolved.
Outdated
|
||||||
| - ℹ️ Please keep in mind that it's easy for LLM reviews to have false positives or focus on trivialities. We suggest configuring it to the "least chatty" setting you can. | ||||||
|
jyn514 marked this conversation as resolved.
Outdated
|
||||||
| - ℹ️ LLM comments **must not** be blocking; reviewers must indicate which comments they want addressed. It's ok to require a *response* to each comment but the response can be "the bot's wrong here". | ||||||
|
jyn514 marked this conversation as resolved.
Outdated
|
||||||
| - In other words, reviewers must explicitly endorse an LLM comment before blocking a PR. They are responsible for their own analysis of the LLM's comment and cannot treat it as a CI failure. | ||||||
| - ℹ️ This does not apply to private use of an LLM for reviews; see ✅ above. | ||||||
|
|
||||||
| All of these **must** disclose that an LLM was used. | ||||||
|
|
||||||
| ## Appendix | ||||||
|
|
||||||
| ### No witch hunts | ||||||
|
jyn514 marked this conversation as resolved.
Outdated
|
||||||
| ["The optimal amount of fraud is not zero"](https://www.bitsaboutmoney.com/archive/optimal-amount-of-fraud/). | ||||||
| Do not try to be the police for whether someone has used an LLM. | ||||||
| If it's clear they've broken the rules, point them to this policy; if it's borderline, report it to the mods and move on. | ||||||
|
jyn514 marked this conversation as resolved.
Outdated
jyn514 marked this conversation as resolved.
Outdated
|
||||||
|
|
||||||
| Conversely, lying about whether you've used an LLM is an instant [code of conduct](https://rust-lang.org/policies/code-of-conduct/) violation. | ||||||
| If you are not sure where you fall in this policy, please talk to us. | ||||||
|
jyn514 marked this conversation as resolved.
Outdated
|
||||||
| Don't try to hide it. | ||||||
|
|
||||||
| ### Responsibility | ||||||
|
|
||||||
| All contributions are your responsibility; you cannot place any blame on an LLM. | ||||||
|
jyn514 marked this conversation as resolved.
Outdated
|
||||||
| - ℹ️ This includes when asking people to address review comments originally authored by an LLM. See "review bots" under ⚠️ above. | ||||||
|
jyn514 marked this conversation as resolved.
Outdated
|
||||||
|
|
||||||
| ### "originally authored" | ||||||
|
jyn514 marked this conversation as resolved.
Outdated
jyn514 marked this conversation as resolved.
Outdated
|
||||||
|
|
||||||
| This document uses the phrase "originally authored" to mean "text that was generated by an LLM (and then possibly edited by a human)". | ||||||
|
jyn514 marked this conversation as resolved.
Outdated
|
||||||
| No amount of editing can change authorship; authorship sets the initial style and it is very hard to change once it's set. | ||||||
|
jyn514 marked this conversation as resolved.
Outdated
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
Taking a different approach here, of narrowing the focus to the phrasing in this policy, rather than trying to get people to agree with the fully general statement. |
||||||
|
|
||||||
| For more background about analogous reasoning, see ["What Colour are your bits?"](https://ansuz.sooke.bc.ca/entry/23) | ||||||
|
|
||||||
| ### Non-exhaustive policy | ||||||
|
jyn514 marked this conversation as resolved.
|
||||||
|
|
||||||
| This policy does not aim to be exhaustive. | ||||||
| If you have a use of LLMs in mind that isn't on this list, judge it in the spirit of this overview: | ||||||
| - Usages that do not use LLMs for creation and do not show LLM output to another human are likely allowed ✅ | ||||||
| - Usages that use LLMs for creation or show LLM output to another human are likely banned ❌ | ||||||
|
jyn514 marked this conversation as resolved.
Outdated
jyn514 marked this conversation as resolved.
Outdated
jyn514 marked this conversation as resolved.
Outdated
|
||||||
|
|
||||||
| This policy is not set in stone. | ||||||
| We can evolve it as we gain more experience working with LLMs. | ||||||
|
jyn514 marked this conversation as resolved.
Outdated
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would feel better if we made this policy explicitly time-limited or tied to a process of gathering more information. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Niko, you're one of the loudest voices trying to dictate the direction we're going. I would argue that a majority of the pushback from sensible policies like this one have come from you; since you're effectively the project manager for the project, your voice carries further than a dozen people's, and it feels like you're genuinely oblivious to this. Plus, a lot of the arguments you've offered have been from the position that whatever you think is reasonable is canonically reasonable, which is perspective that resists all form of negotiation. We all agree that this policy is not going to be permanent, but a large portion of the project seems to be in agreement that this should be the policy we adopt until a project-wide policy is adopted. It's also worth noting, since it's been brought up multiple times, that we don't do policy by majority vote. This is even true for a policy like this one: if we did majority vote, we'd just ban all LLM usage, but we're not doing that because we're willing to compromise. Right now, it seems pretty unsubstantiated that a handful of voices have dictated this position. While it's true that a small number of people have been active in the policy channel, a majority of the project have pointed out their desire for a total ban on LLM usage. This, being noticeably more lenient on that, is a compromise from us. You should consider whether you're willing to compromise at all on your stance, and what compromise would mean for you. As I mentioned in one of the discussions, I do think it's a false equivalence that both sides need to concede something, but if you don't even know what it means to compromise, then negotiation is utterly impossible. I really am not convinced that you understand what a compromise of the pro-LLM position would be, based upon the utter confusion you've expressed when mentioning that some of the contributions you've done would not be acceptable under some of the proposed policies. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do not plan to actually engage in this conversation any further (I acknowledge my biases and when to step out), but I think it's worth pointing out to the at-least-5 people who gave a thumbs-down reaction to my comment that I personally have a rule when it comes to this. If I ever decide to mark my dissent on a comment with the thumbs-down emoji, I always reply explaining why unless everything I wish to say has already been said. Many times, the result is far more critical to the poster than a simple emoji, but I do this because I genuinely want people to understand why I feel a particular way, rather than just saying "I don't like this and will not explain why." We don't improve if we don't know what's wrong. My above comment, in my mind, is required to give Niko's a thumbs-down reaction, because otherwise I'm being insincere to him and everyone else reading. I do not say that I disagree with something without saying why; in that case, it's better to not say anything at all. Again, I acknowledge that my explanation can be deeply hurtful. Disagreement is a painful but necessary process. I also know that there are plenty of times where I have been excessively hurtful without providing the relevant constructive feedback, and think it's worth calling me out for that. I don't apply my standards to anyone else. Lots of people just don't have time to write up a full response. But I personally, in these cases, simply don't respond at all. So, consider whether your simple thumbs-down emoji constitutes genuinely useful feedback, or whether you're just being excessively hurtful instead. And, if you would like to express your dissent in private, I'm open to DMs on Zulip too; this is an open invitation to just say what you feel without a filter. It would be hypocritical of me to be so blunt with my opinions and not accept the same in kind.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'll reply here since this is 'the thread', but I want to say first that I don't agree with much of what you wrote @clarfonthey. I believe Niko is raising a concern in good faith, though I'd like to understand it better.
@nikomatsakis, can you elaborate on specifically what information you think we should be seeking, and what process you're imagining for iterating towards a better policy? Are you seeking commitment from folks who have engaged so far to keep engaging in discussion on Zulip? Something else? I'd be happy to chat offline (Zulip or more sync meeting if you'd prefer) if that makes more sense. I see @jyn514 left a comment below with some more data on project opinions, but it's not clear to me if that's the kind of data you're seeking, or something else. Could you elaborate on what you're looking for and what kinds of process/timeline you would find better than the copious discussion and iteration that has landed us on this (and some other) proposals? I personally think a policy like this one that is relatively restrictive, but scope-limited and leaving room for usage in other areas of the Project gives us a good balance for continued input on where the world is and leaving the door open for private usage for those comfortable with doing so. That combination seems guaranteed to ensure we're not going to stop discussing since everyone seems to want something different from this policy, even if we manage to get to consensus on landing this in the meantime.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I appreciate the vote of confidence, Mark. And @clarfonthey I appreciate that I have reputational clout in the project -- though I'd also note that it doesn't usually translate into me getting my way without fighting for it tooth and nail. =) In any case, I wouldn't be speaking up this much if I didn't feel it was important. To answer your questions Mark:
No, I think the Zulip discussions are not useful. I want to see a more structured process. I think it would look like this:
For example, @jyn514 has expressed openness to having a separate review queue for "LLM-authored content". How many others on the compiler team share that opinion? I have no idea. And of course @clarfonthey has expressed ethical concerns, and I don't really know how many people share that bright red line. And that's just existing maintainers, what about people who've opened PRs in the last year? How many of them work with LLMs at work or on a daily basis? What are there experiences like? Another thing I'm very curious to understand, something I think could be useful, is -- what are people afraid of or hopeful for as a result of this policy. That might inform the conversation. For example, for me, one of my big fears is that if we will be distancing ourselves from future contributors, many of whom will be coding with LLMs. When Rust started, we made a deliberate choice to use Github and not Bugzilla because, frankly, Github is where the people are. I would be interested to see if the perspectives around LLM usage vary between existing maintainers and future contributors or along other lines. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. According to https://rethinkpriorities.org/research-area/adoption-llms-tech-workers/ 91% of respondants have used LLMs for work with 29% using it daily. This data is a year old now, and I have many reasons to believe usage of tools like Claude Code has only increased since then, and dramatically at that. Of the four tech companies I have direct knowledge of, all four have gone from AI being used for coding by a minority of developers, to being used by almost every developer for coding in that time. In two of them using AI coding tools is practically mandatory. It's also quite clear to everyone that companies like Anthropic are struggling to keep up with the growth in usage. I personally have approached AI with extreme skepticism from the beginning, and I still consider its functionality to be dramatically oversold by the companies selling it, but it's extremely widely used already, is already a very effective tool when used correctly, and I think @nikomatsakis is absolutely correct in thinking this will distance contributors who would use it, which is now essentially all new programmers. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. At the company I work for, it's also the case that most people use AI quite a bit; however, multiple new employees have expressed that they don't want to use it much or at all because it could interfere with learning. They want to go from junior engineers to senior engineers, and the best way we know to do that is hands-on experience. So I agree that it's become an industry standard (and policies which do not reflect this may be unsustainable); however, it's not necessarily true that all new programmers will be AI users initially.
jackh726 marked this conversation as resolved.
Outdated
|
||||||
Uh oh!
There was an error while loading. Please reload this page.