· published on LinkedIn · translated from French

My top 19 key points of a GenAI project

The specifics of GenAI projects I wish I'd known two years ago.

As in any field in IT, there are specifics around GenAI projects.

Here’s my top 19 key points not to forget:

  1. De-risk as much as possible with POCs, ideally without code (provider playground or no-code tools like Flowise, Vectorshift).
  2. Test several agent architectures / ways of splitting responsibilities between agents (again, ideally without writing code).
  3. Agree on how responses are structured (with or without JSON? which format? Better not to lock the structure too strictly at this stage of the project).
  4. Improve the base for the system prompts, without over-refining them (the base should already be in place at this step), especially inter-agent communication.
  5. Choose your AI provider(s).
  6. Adjust the system prompt depending on the provider. E.g.: Claude XML, few-shot less effective, etc.
  7. Create/define/code the agents; choose a framework like LangChain (or not, but at least make a conscious choice).
  8. Certify the structuring of the outputs (structured_output, record_summary, output verification lib, etc.). Define the strategies: retry? auto-correction?
  9. Start automated testing (integration tests with LLM as Judge). At this stage, the risk is still high of having to drop parts of the project because of a provider’s new features or a functional pivot if the generative part doesn’t meet expectations.
  10. Depending on the system, you’re between 50 and 70% satisfactory responses at this point in the project; the goal is to reach +80 or 90% satisfaction (even if hard to measure).
  11. Use or build tools to make QA and debugging easier on deployed envs (internal playground, monitoring tools).
  12. Warm up API keys: good practice.
  13. Set up a Grader (evaluator agent). At this stage, you start generating results in QA or even beta, so it’s crucial not to lose information. Ideally a system that gives a clear view of the success rate of each use of the system.
  14. Start investing in failover systems or retry strategies (a non-negligible part of the success rate lies in the providers’ API success rates, which honestly aren’t great).
  15. Make system prompts dynamic depending on context (via RAG or other).
  16. Add a user rating system.

Security layer:

  1. Enforce data partitioning, physical if possible.
  2. Add protection layers to refuse to respond to prompt injections.
  3. Integrate a monitoring system to detect injections.

I really wish I’d had this top list 2 years ago.