2025-07-01
AI
Even the most advanced AI assistants can stumble on project-specific nuances—here’s how a simple yarn-to-npm migration went wrong.
When you’re working with one of the “best” AI coding assistants, you expect magic—flawless suggestions, zero second-guessing. But sometimes that very magic misfires on the simplest changes.
Last week, I embarked on a straightforward task: swap every `yarn` command in my GitHub Actions workflow for the equivalent `npm` form. My team standardizes on npm everywhere, so consistency felt like a no-brainer. I ran through the diff, eyeballed most replacements, and the only one I overlooked the transformation for `yarn eslint ./src`, AI had replaced it with—`npm run eslint ./src`.
- **Consistency** across all CI workflows.
- I saw at least three to four other workflows that were using npm except for this one. I didn't want to touch all of them at this time, so I thought I would just change this one.
Original command: `yarn eslint ./src`. AI’s first suggestion: `npm run eslint ./src`. At a glance, it looks right—`npm run <script>` invokes a script named `eslint` in your package.json.
On the next CI run, GitHub Actions blew up with an error: missing script 'eslint'. There was no 'eslint' entry in our scripts block. Instant face-palm: how did both me and the AI miss this?
I asked the AI to confirm that `npm run eslint ./src` truly mirrored the Yarn command. I wanted it to discover the error by itself. It checked and noted there was no 'eslint' script… but still couldn’t suggest the right fix (`npx eslint ./src`). Instead, it drifted to `npm run lint`, which runs `eslint .` at the project root—close, but not the same path-specific invocation that was in the yarn command.
- Pattern-matching over context: it saw a lot of commands like yarn <script> earlier in the file which it correctly replaced with npm run <script>, but it assumed an `eslint` npm script existed based on that pattern.
- The disappointment happened when even after realizing the script was missing, it couldn’t infer the correct `npx` command to run ESLint directly. Because the original command included the path ./src, the model should have inferred that ESLint needed to be invoked directly via npx.
- There are areas where AI can slip up easily. There are many vendors overselling AI currently in the market esp. via social media influencers. AI tools have gotten insanely powerful but they require an expert human to guide them.
- AI is a force-multiplier—but you’re still the pilot. If it’s operated by a skilled human, it will multiply that benefit, but if it is operated by a novice it will multiply the mistakes. Moreover, mistakes compound as the number of lines of code increase.
AI can turbo-charge your development workflow, but it isn’t a drop-in replacement for domain expertise. Treat it as your co-pilot, not your autopilot—and you’ll get the best of both worlds: speed and reliability.
If the work is for a side project or startup the considerations would be different. You might not have the time to do a deep dive into the AI model's suggestions, but you should still be aware of its limitations.
Have you had similar experiences where AI led you astray?