I resonate strongly with your framing. LLMs as suggestion engines, deterministic layer for execution.
I'm building something similar with security as the focus: deterministic policy that agents can't bypass (regardless of prompt injection). Same principle - deterministic enforcement guiding a probabalistic base.
Would love to hear more about your use case. What kinds of enterprise workflows are you targeting? Is security becoming a blocker?
A non-technical PM asked me (an early career SWE) to develop an agentic pipeline / tool that could ingest 1000+ COBOL programs related to a massive 30+ year old legacy system (many of which have multiple interrelated sub-routines) and spit out a technical design document that can help modernizing the system in the future.
- I have limited experience with architecture & design at this point in my career.
- I do not understand business context of a system that old and any of the decisions that occurred in that time.
- I have no business stakeholders or people capable of validating the output.
- I am the sole developer being tasked with this initiative.
- My current organization has next to no engineering standards or best practices.
No one in this situation is interested in these problems except me. My situation isn't unique with everyone high on AI looking to cram LLMs & agents into everything without any real explanation of what problem it solves or how to measure the outcome.
I admire you for thinking about this kind of issue, I wish I could work with more individuals who do :(
You can ask AI to focus on the functional aspects and create a design-only document. It can do that in chunks. You don't need to know about COBOL best practices now, that's an implementation detail. Is the plan to modernize the COBOL codebase or to rewrite in a different language?
First off, what you shared is cool, thank you. Especially considering it captures problems I need to address (token limitations, context transfer, managing how agents interact & execute their respective tasks).
My challenge specifically is that there is no real plan. It feels like this constant push to use these tools without any real clarity or objective. I know a lot of the job is about solving business problems, but no one asking me to do this has any idea or defined acceptance criteria to say the outputs are correct.
I also understand this is an enterprise / company issue, not that the problem is impossible or the idea itself is bad. Its just a common theme I am seeing where this stuff fails in enterprises because few are actually thinking how to apply it... as evidenced by the fact that I got more from your comment than I otherwise get attempting to collaborate in my own organization
I've been thinking about how ISO-9000 will be reconciled with LLMs? Will businesses abandon their ISO-9000 certifications in favor of "We use AI" or will ISO-9000 adapt in some way to the "need" for LLMs?
I resonate strongly with your framing. LLMs as suggestion engines, deterministic layer for execution.
I'm building something similar with security as the focus: deterministic policy that agents can't bypass (regardless of prompt injection). Same principle - deterministic enforcement guiding a probabalistic base.
Would love to hear more about your use case. What kinds of enterprise workflows are you targeting? Is security becoming a blocker?
My experience working at a large F500 company:
A non-technical PM asked me (an early career SWE) to develop an agentic pipeline / tool that could ingest 1000+ COBOL programs related to a massive 30+ year old legacy system (many of which have multiple interrelated sub-routines) and spit out a technical design document that can help modernizing the system in the future.
- I have limited experience with architecture & design at this point in my career.
- I do not understand business context of a system that old and any of the decisions that occurred in that time.
- I have no business stakeholders or people capable of validating the output.
- I am the sole developer being tasked with this initiative.
- My current organization has next to no engineering standards or best practices.
No one in this situation is interested in these problems except me. My situation isn't unique with everyone high on AI looking to cram LLMs & agents into everything without any real explanation of what problem it solves or how to measure the outcome.
I admire you for thinking about this kind of issue, I wish I could work with more individuals who do :(
Your task is certainly doable though.
You can ask AI to focus on the functional aspects and create a design-only document. It can do that in chunks. You don't need to know about COBOL best practices now, that's an implementation detail. Is the plan to modernize the COBOL codebase or to rewrite in a different language?
See what this skill does in Claude Code, you want something similar: https://github.com/glittercowboy/get-shit-done/blob/main/get...
First off, what you shared is cool, thank you. Especially considering it captures problems I need to address (token limitations, context transfer, managing how agents interact & execute their respective tasks).
My challenge specifically is that there is no real plan. It feels like this constant push to use these tools without any real clarity or objective. I know a lot of the job is about solving business problems, but no one asking me to do this has any idea or defined acceptance criteria to say the outputs are correct.
I also understand this is an enterprise / company issue, not that the problem is impossible or the idea itself is bad. Its just a common theme I am seeing where this stuff fails in enterprises because few are actually thinking how to apply it... as evidenced by the fact that I got more from your comment than I otherwise get attempting to collaborate in my own organization
> LLMs are probabilistic and non-deterministic
This is a polite way of saying unreliable and untrustworthy.
The problem facing enterprise is best understood by viewing LLMs as any other unreliable program.
> We’ve found that treating LLMs as suggestion engines rather than decision makers changes the architecture completely.
Figures. Look at the disruption LLM "suggestions" are inflicting on scientific journals, court cases and open source projects wordwide.
I've been thinking about how ISO-9000 will be reconciled with LLMs? Will businesses abandon their ISO-9000 certifications in favor of "We use AI" or will ISO-9000 adapt in some way to the "need" for LLMs?
If enterprises are deterministic, that’s what a coding LLMs are for. To create the deterministic part with the help of the LLM.
reminds me of this article > https://unstract.com/blog/understanding-why-deterministic-ou...