Ivan 🥁 Nardini

In our latest article on To Data & Beyond, Eivind Kjosbakken discusses how you can fine-tune VLMs (visual large language models, often called VLLMs) like Qwen 2.5 VL 7B. He will first introduce you to a dataset of handwritten digits, which the base version of Qwen 2.5 VL struggles with. We will then inspect the dataset, annotate it, and use it to create a fine-tuned Qwen 2.5 VL, specialized in extracting hand-written text. 📌 The blog link is in the comment sections! ➡ If you want to master data science & AI beyond the basics, subscribe to the To Data & Beyond newsletter for in-depth tutorials and lessons: https://bit.ly/4lnAjqv #llm #opensource #largeLanguagemodels #python #datascience #machinelearning #ml #ai #artificialintelligence #genai #generativeai

62

1 Comment

𝗧𝗟;𝗗𝗥: Multi-agent AI mirrors distributed systems, facing ACR tradeoffs (Agency, Control, Reliability). Like the CAP theorem, you can't optimize all three. Anthropic's research shows the winning enterprise patterns. 𝗠𝘂𝗹𝘁𝗶-𝗔𝗴𝗲𝗻𝘁 𝗔𝗜 = 𝗗𝗶𝘀𝘁𝗿𝗶𝗯𝘂𝘁𝗲𝗱 𝗦𝘆𝘀𝘁𝗲𝗺𝘀 Everyone thinks agentic AI is magic (and hence the hype), but agentic AI is actually just distributed system architecture—orchestrating specialized agents across compute resources, managing state, handling failures, and coordinating workflows. The parallel isn't machine learning; 𝗶𝘁'𝘀 𝗺𝗶𝗰𝗿𝗼𝘀𝗲𝗿𝘃𝗶𝗰𝗲𝘀 𝗲𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝗶𝗻𝗴. Teams already handling distributed systems are perfectly positioned for multi-agent AI. 𝗔𝗖𝗥: 𝗧𝗵𝗲 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗖𝗔𝗣? Intercom's research (https://bit.ly/4jTNA8O) reveals a fundamental tradeoff mirroring distributed systems' CAP theorem (https://bit.ly/4lbZCeV):  • 𝗔𝗴𝗲𝗻𝗰𝘆 (autonomy): Independent decision-making  • 𝗖𝗼𝗻𝘁𝗿𝗼𝗹 (predictability): Constraining agent behavior  • 𝗥𝗲𝗹𝗶𝗮𝗯𝗶𝗹𝗶𝘁𝘆 (consistency): Consistent results across executions High-agency agents achieve only 20-30% reliability on complex tasks. But constrained, step-based agents hit 60%+ reliability—the enterprise sweet spot. 𝗔𝗻𝘁𝗵𝗿𝗼𝗽𝗶𝗰'𝘀 𝗣𝗿𝗼𝗱𝘂𝗰𝘁𝗶𝗼𝗻 𝗣𝗮𝘁𝘁𝗲𝗿𝗻𝘀 𝗳𝗼𝗿 𝗠𝘂𝗹𝘁𝗶 𝗔𝗴𝗲𝗻𝘁 𝘀𝘆𝘀𝘁𝗲𝗺𝘀 Their Research feature (https://bit.ly/408qpjZ) demonstrates distributed systems principles:  • 𝗢𝗿𝗰𝗵𝗲𝘀𝘁𝗿𝗮𝘁𝗼𝗿-𝗪𝗼𝗿𝗸𝗲𝗿 𝗣𝗮𝘁𝘁𝗲𝗿𝗻: Lead agent coordinates, subagents execute—like API gateways + microservices  • 𝗣𝗮𝗿𝗮𝗹𝗹𝗲𝗹 𝗣𝗿𝗼𝗰𝗲𝘀𝘀𝗶𝗻𝗴: Multiple agents with separate context windows—horizontal scaling of cognitive work  • 𝗦𝘁𝗮𝘁𝗲 𝗠𝗮𝗻𝗮𝗴𝗲𝗺𝗲𝗻𝘁: Memory persistence + checkpointing for failure recovery  • 𝗥𝗲𝘀𝗼𝘂𝗿𝗰𝗲 𝗘𝗰𝗼𝗻𝗼𝗺𝗶𝗰𝘀: 15x token usage requires careful allocation—distributed compute cost management 𝗢𝗽𝘁𝗶𝗺𝗮𝗹 𝗪𝗼𝗿𝗸𝗳𝗹𝗼𝘄𝘀 Multi-agent systems excel at:  • Breadth-first research (90.2% improvement)  • Information synthesis across context windows  • Tool-heavy integrations with specialized agents 𝗕𝗼𝘁𝘁𝗼𝗺 𝗹𝗶𝗻𝗲: Enterprise AI success comes from 𝗮𝗽𝗽𝗹𝘆𝗶𝗻𝗴 𝗱𝗶𝘀𝘁𝗿𝗶𝗯𝘂𝘁𝗲𝗱 𝘀𝘆𝘀𝘁𝗲𝗺𝘀 𝗲𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝗶𝗻𝗴 𝘁𝗼 𝗺𝘂𝗹𝘁𝗶-𝗮𝗴𝗲𝗻𝘁 𝗰𝗼𝗼𝗿𝗱𝗶𝗻𝗮𝘁𝗶𝗼𝗻. Microservices teams are already equipped to win. Get started with OSS AWS Strands Multi-agent (https://bit.ly/4e9hrsO) or Managed Amazon Web Services (AWS) Bedrock Agents (https://go.aws/43Z97qH). 

320

23 Comments

🚨 Embeddings are incredibly powerful — but they also come with a hard mathematical ceiling. Colleagues at Google DeepMind recently published On the Theoretical Limitations of Embedding-Based Retrieval. It’s a fascinating reminder that as we scale semantic search and RAG, some challenges are fundamental, not just engineering problems. 🔢 What the research shows For top-2 retrieval tasks, here’s the maximum corpus size that embeddings of different dimensions can theoretically represent: • 512-dim → ~500k docs • 768-dim → ~1.7M docs • 1024-dim → ~4M docs • 3072-dim → ~107M docs • 4096-dim → ~250M docs Beyond these thresholds, some query–document relationships simply can’t be represented — no matter how good the training or how much compute you use. 🍎 A simple but revealing example (LIMIT dataset) – Docs: short profiles like “Jon Durben likes Apples and Candy.” – Queries: “Who likes Apples?” – Relevance: all people who like that attribute. Sounds trivial. But when the dataset tests all possible combinations of relevance (Apples, Quokkas, Rabbits, etc.), the combinatorial explosion overwhelms embeddings. 👉 Results: – State-of-the-art embeddings (Gemini, Qwen, GritLM) score <20% recall@100 – Classic BM25 (sparse lexical) performs near-perfectly, since its effective dimensionality is far higher ⚙️ How the limit was measured Researchers ran “free embedding” experiments: – Directly optimized vectors with gradient descent (no language constraints) – Measured the largest corpus where retrieval was still solvable – Fit a curve → gave the thresholds above If even perfectly optimized vectors fail, real models can’t succeed either. 🚀 Why it matters As retrieval becomes more instruction-driven and web-scale, we’ll increasingly hit these ceilings. The future likely lies in hybrid architectures: ✅ Embeddings → fast coarse retrieval ✅ Cross-encoders → precise reranking ✅ Multi-vector or sparse models → richer representation 🔗 Full paper + dataset: https://buff.ly/bgAKB29 💡 Curious to hear from others: In your retrieval stacks — do you already combine embeddings with sparse or reranker approaches, or rely mostly on embeddings today?

51

2 Comments

Kimi-K2: Another DeepSeek moment? Maybe. Maybe not. But one thing is clear, it’s shifting how organizations are approaching with their AI strategy. With 1 trillion total parameters, Kimi-K2 is currently the largest open-source model available (we’re still waiting on Llama Behemoth). A few things that stood out to me: ➡️ It’s laser-focused on enterprise-grade challenges, especially on coding benchmarks, where it’s performing competitively or even outperforming the existing models, at least on LiveCodeBench. It's one of the top use cases across. ➡️No strings attached until you hit 100M DAUs or $20M in monthly revenue. After that, all they ask is to display their logo prominently in your UI. ➡️Perplexity hinted that they may use Kimi-K2 in post-training stages in the future. It’s a classic Innovator’s Dilemma playing out. It's funny to watch how startups are becoming the new incumbents so fast. #ExperienceFromTheField #WrittenByHuman

96

16 Comments

There’s a new breed of GenAI Application Engineers who can build more-powerful applications faster than was possible before, thanks to generative AI. Individuals who can play this role are highly sought-after by businesses, but the job description is still coming into focus. Let me describe their key skills, as well as the sorts of interview questions I use to identify them. Skilled GenAI Application Engineers meet two primary criteria: (i) They are able to use the new AI building blocks to quickly build powerful applications. (ii) They are able to use AI assistance to carry out rapid engineering, building software systems in dramatically less time than was possible before. In addition, good product/design instincts are a significant bonus. AI building blocks. If you own a lot of copies of only a single type of Lego brick, you might be able to build some basic structures. But if you own many types of bricks, you can combine them rapidly to form complex, functional structures. Software frameworks, SDKs, and other such tools are like that. If all you know is how to call a large language model (LLM) API, that's a great start. But if you have a broad range of building block types — such as prompting techniques, agentic frameworks, evals, guardrails, RAG, voice stack, async programming, data extraction, embeddings/vectorDBs, model fine tuning, graphDB usage with LLMs, agentic browser/computer use, MCP, reasoning models, and so on — then you can create much richer combinations of building blocks. The number of powerful AI building blocks continues to grow rapidly. But as open-source contributors and businesses make more building blocks available, staying on top of what is available helps you keep on expanding what you can build. Even though new building blocks are created, many building blocks from 1 to 2 years ago (such as eval techniques or frameworks for using vectorDBs) are still very relevant today. AI-assisted coding. AI-assisted coding tools enable developers to be far more productive, and such tools are advancing rapidly. Github Copilot, first announced in 2021 (and made widely available in 2022), pioneered modern code autocompletion. But shortly after, a new breed of AI-enabled IDEs such as Cursor and Windsurf offered much better code-QA and code generation. As LLMs improved, these AI-assisted coding tools that were built on them improved as well. Now we have highly agentic coding assistants such as OpenAI’s Codex and Anthropic’s Claude Code (which I really enjoy using and find impressive in its ability to write code, test, and debug autonomously for many iterations). In the hands of skilled engineers — who don’t just “vibe code” but deeply understand AI and software architecture fundamentals and can steer a system toward a thoughtfully selected product goal — these tools make it possible to build software with unmatched speed and efficiency. [Truncated due to length limit. Full post: https://lnkd.in/gsztgv2f ]

5,051

261 Comments