"So we've been talking about AI..." - the series, chapter 3 - Product Design
Make sure to take out the emojis, specially 🧠 from the chat result
The most striking benefits of AI I saw were around converting complex real world artifacts to structured formats. Audio, video, meeting transcriptions were the first showcases of these conversions and later on document processing, summarization and deep research.
One of the most impressive cases I worked with was to reduce lending documentation processing by 70% percent, with manageable precision and low recall of less than 30% of human interaction due to low quality document scans. These are bureaucratic processes depending on the country you live, which connects payments, lending, real estate ownership and usually have a high error and rework ratio.
Specialized models can be tuned to recognize legal processes and procedures are other important vector of optimization, establishing an active loop of the parts knowing how and when to react, reducing day to day document work on complex procedures and specially normalizing legal domains as people, work, federal, local laws, inheritance calculation and so on.
Fiscal and finance workflows are accelerated, as most of countries are relying on variable documents as invoices, and countries like Brazil have established formats. Connecting the day to day work to strategic and auditable structured data derived from these documents protect companies from fraud, enable them to connect with their banking partners and monitor payables and receivables with a consistent rate of success.
All of the above were doable with OCR, regular code, SQL and parsing for a long time. But to achieve scale, volume, reduce specialized code and implement quality indicators were possible only with the parallel processing and capabilities of image recognition models and large language models.
These kind of capabilities have to be known and familiar at the product design stage. Not only to gain efficiency but to improve user experience, reduce churn and risk. Have your product design team to be heavy AI users beyond text generation, visual creation.
Let your team use models as compressed search engines, ready to use knowledge base and be prepared for hallucination (plugging knowledge holes with fake information) and for the easy road of model fascination (confirmation bias).
Do not start with models that write PRD (product requirement documents) or specs from simple prompts. That will quickly lead to a wall of text that readers will probably use an LLM to summarize. There will be context and subtleties lost on that.
Intermission: AI backed products (RAG vs MCP)
At this stage, after a healthy product design cycle the discussion will is mostly around opportunity and execution choices.
Opportunity as in commercial and partnership deals with new players, aggressive pricing, new capabilities and the constant improvement of models offered by each big player.
Execution is balanced on fast, disposable and sustainable choices. How fast can you put it together and how to do it without breaking the bank ? Why avoid the trap of establishing the “AI Team” so the rest of the organization don't feel isolated and the effort is not durable ? How not to build a disposable product that will be given for free on ChatGPT marketplace ?
Engineering wise, the commonly used development model for AI backed prducts is to use a framework to abstract the model API like Langchain and instrument model interoperability, preventing vendor lock-in from the start and later on to combine features and orchestrate agents.
In terms of architecture, Agents, RAG and MCP are the main components of this ecosystem of extending and coordinating AI based components. Established vendors are releasing Agents and MCPs connected to their product on top of AI capabilities to create lock in.
I have released a new book: “Go for Gophers”. It is a progressive, hands on book for teams and engineers that are adopting Go and look for an idiomatic path forward. Quick, practical and loaded with useful examples. Check it out.
If you are a CTO, Tech Lead or Product Leader check The CTO Field Guide. It is a book to help product engineering leaders of all levels. If you need personalized help, check the mentoring program based on the book at https://ctofieldguide.com/mentoring.html.