Editors Note
This was written at a very different time in my AI experience so take it with a grain of salt. Things changed fast between when this post was created and where AI (and its usage) is at now.
The LLM Matrix for Software Engineers
As AI tools become increasingly powerful, software engineers face a new challenge: when should we use AI assistance, and when should we write code ourselves? I’ve developed a simple framework that helps me make this decision quickly and consistently.
The Matrix
Understanding the Axes
X-Axis: Verification Complexity
The horizontal axis represents how easy it is to verify that your code works correctly.
The faster and more thoroughly you can verify the code, the more comfortable you can be using AI assistance. For example, a simple UI component with clear visual requirements is easy to verify, while a complex financial calculation with many edge cases might require more careful manual review.
Y-Axis: Feature Criticality
The vertical axis represents how critical the feature is to your application. Consider:
- Data integrity
- User safety
- Business impact
- Security implications
- Performance requirements
High-criticality features require more careful review and testing, regardless of how easy they are to verify. For instance, a banking transaction system needs more scrutiny than a weather widget, even if both are easy to test.
Real World Examples
High Criticality, Easy to Verify
- Input validation and sanitization with clear requirements
- Unit tests with obvious inputs and outputs
- Authentication flows that can be thoroughly tested
High Criticality, Hard to Verify
- Code that can modify data with complex business logic
- Security-critical code
- Functions that can modify data in a way that is not reversible
Low Criticality, Easy to Verify
- Documentation
- Visualizations of specific data e.g. username, email, etc.
- Simple utility functions
- UI animations that are purely decorative
Low Criticality, Hard to Verify
- Analytics that needs to join and map over multiple data sources
- Development tools
Making the Decision
When faced with a new feature, plot it on the matrix based on these axes. The further right and lower on the matrix, the more comfortable you can be using AI assistance. The further left and higher, the more you should consider writing the code yourself or using AI only for initial scaffolding.
Remember: This is a framework, not a rigid rule. Use your judgment and experience to make the final decision. The goal is to be efficient while maintaining code quality and reliability.
If you have other ways that you make this decision, I’d love to hear about them! Contact info below.