This seems less auditable though, because now there is more variability in the way something is installed. Now there are two layers to audit:
- What the agent is told to do in prose
- How the agent interprets those instructions with the particular weights/contexts/temperature at the moment.
I’m all for the prose idea, but wouldn’t want to trade determinism for it. Shell scripts can be statically analyzed. And also reviewed. Wouldn’t a better interaction be to use an LLM to audit the shell script, then hash the content?
Yes, this approach (substituting a markdown prompt for a shell script) introduces an interesting trade-off between "do I trust the programmer?" and "do I trust the LLM?" I wouldn't be surprised to see prompt-sharing become the norm as LLMs get better at following instructions and people get more comfortable using them.
- What the agent is told to do in prose
- How the agent interprets those instructions with the particular weights/contexts/temperature at the moment.
I’m all for the prose idea, but wouldn’t want to trade determinism for it. Shell scripts can be statically analyzed. And also reviewed. Wouldn’t a better interaction be to use an LLM to audit the shell script, then hash the content?