Most human programmers are bad at architecture. They’re making lots of mistakes too. Perfection is not a criterion for intelligence. In that case no human is intelligent.
Intelligence isn’t a one-dimensional axis either. Intelligence exists in many facets. Agents (so LLMs combined with tools, memory, etc.) overlap pretty much with one aspect that we perceive as intelligent. The main risk is basically the same as with humans of similar one-sided intellect: if you expect them to be equally intelligent in other areas, you will get disappointed or even hurt.
Most human programmers are bad at architecture. They’re making lots of mistakes too. Perfection is not a criterion for intelligence. In that case no human is intelligent.
Intelligence isn’t a one-dimensional axis either. Intelligence exists in many facets. Agents (so LLMs combined with tools, memory, etc.) overlap pretty much with one aspect that we perceive as intelligent. The main risk is basically the same as with humans of similar one-sided intellect: if you expect them to be equally intelligent in other areas, you will get disappointed or even hurt.