r/ExperiencedDevs • u/deafgamer_ • 18h ago
AI/LLM Increasingly unreadable technical specs and documentation due to heavy LLM usage?
I entered the industry in 2013. At the time, most documentation was technically readable by humans and if you didn't understand something, it was because you didn't understand the tooling being used or the commands and you needed to do more research on your own. Increasingly complex technical specs and commands were usually provided with diagrams, screenshots, long descriptions. I remember the most complex thing ever was working on robust CICD pipelines like Kafka or early AWS, but even that was conquerable by referring to the documentation and poring over Stackoverflow.
Fast forward to now. I got hired ~4 months ago after 10 months of looking. Everyone at my company heavily uses LLMs - to generate documentation, read documentation, write code. That's fine. But I can see the effect that even a few months have had on our documentation - due to the loop of using LLMS to read existing docs then generate the new set of docs for some new feature doing a +1 on the last, the documentation generated today has been absolutely unreadable for humans. Giant AWS CLI commands generated by LLMs that do all sorts of things with all sorts of flags. Tons of lambdas and user profiles to do some specific thing. Huge terraform files. Wall-of-text YML files with explanations intended to be read by LLMs, not humans. At this point I don't even try to bother understanding, I just chuck the entire thing into Claude and it understands what to do - or so it seems. I have no way of actually factchecking that because I can't take the time to understand the 10 flags it added to the command as I need to start working on the next thing. I'm uncomfortable that LLMs are started to be "trusted" more and more and if AI gets rugpulled I feel it would be hard for humans to convert all existing documentation to be human readable.
So far my biggest value in a post-AI world is my smell test. There's a LOT of things LLMs get wrong, and I can usually sniff it out pretty quick, and redirect it somewhere else instead. This usually comes from a place of "this is how I would have done it if I just needed it to work right away, but if I wanted more space optimization or to account for this edge case I would have done that other thing instead" so if LLMs get really far away from the mental guardrails I thought of myself, I'll get it back on track by providing the proper context.
Previously, I only used LLMs to generate code and some one-time scripts to do some tedious work. Begrudgingly due to the issue I presented above, I've started using it to read documentation that I don't understand. I'm still writing all of my own documentation, especially if I intend it to be read by a human in the future, like Confluence/Wiki documentation 100% generated by me, not a LLM.
The end result is I feel like a dinosaur these days. It doesn't help that I'm a bit removed from the code now, as I'm a full time manager, but I'm still trying to do some PR reviews with the time I can and direct some of the technical stuff if my team needs my input.