As one of the maintainers of Apache Arrow, lately I’ve been noticing an increase in AI-generated pull requests. In this blog post, I discuss how AI tooling affects open source, the actions maintainers have been taking to address the less positive aspects, and emerging policies that open source projects are implementing around the topic of AI-generated pull requests.
Attitudes to AI tools
Earlier reactions to the use of AI tools in some open source projects have been quite negative, and there are lots of examples of open source projects explicitly banning the use of AI tools in generating contributions, for example, Gentoo Linux in 2024 banned all AI contributions.
I have some sympathy for this perspective, and there are certainly good and valid reasons around complications relating to licensing and AI-generated code that I believe open source developers need to be cautious of. The ASF have published some high-level guidelines on the use of generative tooling that capture the risk of replicating copyrighted materials.
That said though, I also believe that the evolution of these tools means that condemning them outright is throwing the metaphorical baby out with the bathwater. Entire features created with AI absolutely can be problematic from a licensing perspective, but much of the maintenance work required for existing codebases is less fraught. In my experience, fixes and changes generated based on the existing codebase and reasoning around it rarely end up replicating other codebases, and these tools are a net win for maintainers when used well.
As the sophistication of tools has increased, I found that at some point in 2025, the balance tipped for me from relatively poor results towards them becoming net useful. I also believe a lot of developers are in this place where many of us are actively using these tools but fewer are openly admitting it, due to the scepticism out there.
While there is some validity in the more cautious perspectives, it’s unrealistic to pretend that they aren’t being used pretty widely at this point.
The problem of AI-generated PRs in open source
There is a certain class of AI-generated pull request that is becoming problematic. There have a distinct pattern to them:
- a pull request is submitted by a contributor who has never previously contributed to the codebase
- code changes are often accompanied by verbose and unnecessary comments
- the scope of the changes is overly broad scope
- unit tests are either excessive or non-existent
- the code itself doesn’t match the style or conventions of the other code
- the contributor doesn’t ask any questions about anything they’re unsure about
When I see a pull request with these characteristics, I tend to assume that the contributor wanted to contribute to the codebase for whatever reason, decided to use AI to generate the code, but then didn’t engage much if at all with the output before submitting the PR.
To a certain extent I get it. Some of my first open source pull requests were absolutely done for the sake of something that I viewed as a real achievement and not a genuine investment in a codebase, because I didn’t really understand how open source worked at that point. I was fortunate to end up interacting with patient people who took their time to guide me through the process and help me learn, for which I’m grateful.
And I’ve made a fair share of embarrassing mistakes in my own use of AI tools. In my earlier explorations, I accidentally allowed Claude to push to an upstream branch instead of my own fork, and proposed poorly thought-through changes that I didn’t actually understand well enough to take responsibility for when other maintainers responded with thoughtful questions. The realisation of this potential for harm made me drastically alter my approach, but I had to make the mistake to learn the lesson.
All of that said though, most open source maintainers are pressed for time to work on the project. It’s great working with new contributors, both for the human connection and because it improves project health by increasing the number of people who are able to contribute, but there is very little value in a maintainer reviewing AI-generated code which the author has either abandoned or will just continue to use AI to respond to the provided suggestions.
Thoughtful use of AI tools requires multiple iterations and changes, and questioning of the tooling as to the reasoning behind changes, and taking ownership of the changes through the understanding gained during the process.
Taking it to the mailing list
After noticing the increase in AI generated PRs, I wanted to validate the preferred approach to handling these PRs with the wider Arrow community, and so I started a discussion on the Arrow developer mailing list about defining a set of community-agreed guidelines for AI-generated PRs.
It takes a chunk of emotional energy trying to respond to a PR with the empathy and welcoming nature that we want to show towards new contributors, while simultaneously not investing too much time and energy in a pull request which may well be abandoned. Conversely though, if the contributor really is interested in engaging, we don’t want to drive them away.
In reality, we can’t necessarily tell when people make thoughtful use of AI tooling, and so my own personal motivation for suggesting this guidance was to have an official source that we can link to during reviews of PRs which appear to be AI generated.
Given all the negativity I’ve seen online around the use of AI tooling, and also my own use of such tools, I was a little nervous to see what the conversation would look like when I opened the discussion on the Apache Arrow developer mailing list.
It was great to see that, as ever, the Apache Arrow developer community engaged in a thoughtful discussion in which folks suggested additions to my suggested phrasing of policy, and chimed in with their own experiences. This clearly isn’t a novel or new idea, and Andrew Lamb pointed me straight to some excellent guidelines which already existed in the DataFusion project. (For context, DataFusion is a query engine built on Arrow, used to be kept in the former Arrow monorepo, and is part of the Arrow community). I borrowed some ideas from the DataFusion guidelines as well as the contributions from other folks on the mailing list. The key recommendations we ended up with were:
- Only submit a PR if you are able to debug and own the changes yourself - review all generated code to understand every detail
- Match the style and conventions used in the rest of the codebase, including PR titles and descriptions
- Be upfront about AI usage and summarise what was AI-generated
- If there are parts you don’t fully understand, leave comments on your own PR explaining what steps you took to verify correctness
- Watch for AI’s tendency to generate overly verbose comments, unnecessary test cases, and incorrect fixes
- Break down large PRs into smaller ones to make review easier
Since we had this discussion, the PR that we merged was linked to by a couple of other projects and after following the links back I found even more open source projects which had already had engaged in similar discussions or recent policy updates, for example, Jupyter, Zulip, LLVM, and Python.
Other challenges
Pull requests aren’t the only challenge that open source is seeing in terms of AI tool usage; multiple projects have also recently experienced an uptick in security reports. Some of these are valid, but others appear to constitute things that would be considered bugs or edge cases. While deploying fixes for these may genuinely improve the codebase, some examples appear to include strategic phrasing to depict bugs as security incidents, and feel a little disingenuous.
This is becoming more prevalent, as shown by the curl project recently closing its bug bounty program, and similar issues experienced by Apache Log4j.
The irony of the spurious reporting of alleged potential Denial-of-Service attacks is delightfully captured by curl author Daniel Stenberg in the abstract of his upcoming FOSDEM talk:
“Sloppy humans causing Denial-of-Service attacks by overloading maintainers with quickly produced almost-real-looking rubbish”.
AI is having a significant effect on open source development, some of it positive, but it’s certainly not without real challenges as well.