Code

I am excited to announce that my new book, LLM Customization and Fine-Tuning: Adaptation, Distillation, and Alignment, written with Weehyong Tok , is now available in Early Access (MEAP) from Manning Publications . 📚 A general-purpose model API is a fine place to start. Sooner or later, though, most teams need something an API cannot provide: a model that speaks their house terminology and calls their internal tools, that keeps sensitive data within their own boundaries, that meets a latency budget, and that behaves predictably after launch. That means adapting a model you own and run yourself. This book is a practical, end-to-end guide to doing exactly that and to keeping the result reliable in production. ...

The earlier cluster benchmark write-up asked which frontier models were fastest. This post asks the next question: once hardware, serving stack, and benchmark shape are held constant, what do deployment choices - weight quantization and KV-cache compression - actually do to real coding-agent behavior? The model family stays fixed; the things operators actually touch get varied. This grew out of a r/LocalLLaMA thread asking for real coding benchmarks across quantization levels rather than proxy metrics. The study plan: Qwen/Qwen3.6-27B as the primary sweep, Qwen3-Coder-30B-A3B-Instruct as the coding-specialist cross-check, and MiniMaxAI/MiniMax-M2.1 as the heavyweight extension. That last lane turned out to be the most instructive result of the study, and not in the way I expected. ...

Over the last several benchmark cycles, I kept coming back to the same practical question: once you hold the hardware and methodology constant, which large open models are actually pleasant to serve, which ones merely load, and which ones become operationally awkward the moment you move beyond a demo? This post is a technical deep dive into that question. Instead of presenting a generic leaderboard, I focus on the details that usually matter in real deployments: throughput under fixed traffic shapes, latency behavior, scaling across 8x and 16x H200 shapes, and the caveats that only show up when you try to run these models end to end. ...

5-Part Series: Part 0: Why build an OS from scratch? Part 1: Foundations Part 2: Communication Part 3: Concurrency Part 4 (this): Memory and beyond GitHub Repository: bahree/rust-microkernel - full source code and build scripts Docker Image: amitbahree/rust-microkernel - prebuilt dev environment with Rust, QEMU, and source code ...

5-Part Series: Part 0: Why build an OS from scratch? Part 1: Foundations Part 2: Communication Part 3 (this): Concurrency Part 4: Memory and beyond GitHub Repository: bahree/rust-microkernel — full source code and build scripts Docker Image: amitbahree/rust-microkernel — prebuilt dev environment with Rust, QEMU, and source code ...

GitHub Repository: bahree/nanoclaw - full source code NanoClaw is a headless AI assistant running on my personal server. It processes messages from WhatsApp, Telegram, and Slack, runs scheduled tasks, and manages conversations with Claude agents in isolated containers. It’s been incredibly useful, but it had one major pain point: no visibility into what it was doing or why. If something went wrong - a message didn’t get a reply, a task didn’t run - the only way to debug was to SSH into the server, tail logs, and piece together what happened. Not ideal when you’re on the go and your assistant just… stops responding. ...

5-Part Series: Part 0: Why build an OS from scratch? Part 1: Foundations Part 2 (this): Communication Part 3: Concurrency Part 4: Memory and beyond GitHub Repository: bahree/rust-microkernel - full source code and build scripts Docker Image: amitbahree/rust-microkernel - prebuilt dev environment with Rust, QEMU, and source code ...

5-Part Series: Part 0: Why build an OS from scratch? Part 1 (this): Foundations Part 2: Communication Part 3: Concurrency Part 4: Memory and beyond GitHub Repository: bahree/rust-microkernel — full source code and build scripts Docker Image: amitbahree/rust-microkernel — prebuilt dev environment with Rust, QEMU, and source code ...

5-Part Series: Part 0 (this): Why build an OS from scratch? Part 1: Foundations Part 2: Communication Part 3: Concurrency Part 4: Memory and beyond GitHub Repository: bahree/rust-microkernel — full source code and build scripts Docker Image: amitbahree/rust-microkernel — prebuilt dev environment with Rust, QEMU, and source code ...

TL;DR In this third part of our 4-part series on building language models from scratch, I explore the complete training infrastructure that transforms our clean historical data and custom tokenizer into working language models. Part 1 How to build a Large Language Model from Scratch - covered using the published model Part 2 Building LLMs from Scratch - Part 2: Data Collection & Custom Tokenizers - detailed data collection and custom tokenizer development. Here, we build the complete training pipeline from a custom GPT architecture through deployment-ready checkpoints. ...

TL;DR In this second part of our 4-part series on building language models from scratch, I explore the two foundational areas of LLM development: data collection and custom tokenizer creation. Part 1 - Building LLM from Scratch covered using the published model; here, we build the complete pipeline from raw historical documents to a custom tokenizer that understands archaic English, London geography, and period-specific terminology. ...

TL;DR In this post, I show how to build a working LLM from scratch and show a complete end-to-end pipeline from data gathering to training to deployment of a language model. For this project I concentrate on Old English and only related to London, using historical London texts (1500-1850). To show the flexibility, I built two language models which are identical in architecture and the only differs is their size and parameters (117M vs 354M). ...

1. Overview Rust has been gaining attention recently due to its unique combination of performance, safety, and modern programming features. Its strict ownership model eliminates common memory issues like null pointer dereferencing and data races, providing a secure environment for developers. At the same time, its expressive syntax and focus on developer productivity make it a strong contender for systems programming. Its growing ecosystem and community continue to expand its capabilities into areas like web development and embedded systems, ensuring a confident and secure learning experience for developers. ...

AI is advancing at an unprecedented pace, with Mixture of Experts (MoE) models being one set of model architectures at the forefront of this revolution. These architectures enable breakthroughs in efficiency and scalability by leveraging a modular design where only a subset of specialized “expert” networks are activated for each input. MoE architectures have become a cornerstone in building ultra-large-scale models like GLaM and Switch Transformers. Mixture of Experts (MoE) is an advanced machine learning architecture that lately has gained significance, particularly in the realm of #LLMs (large language models) and NNs (neural networks). In talking with many people about AI, I’ve found that MoE as a topic comes up often, with many folks either not understanding what it is or their understanding of it being incorrect. ...

1. A little background I have been meaning to automate the deployment of my blog post to a dev server (running locally) for a while, but I haven’t had the time to get around to it until now. In addition to deploying this, the dev server also had several constraints. It is one of the machines at home and is not exposed directly to the internet. I also don’t have any ports opened on the firewalls at home, which adds a few more interesting challenges. I have a server hosted online in one of the data centers, but I want to keep that for more ‘production’ things. ...

In today’s rapidly evolving tech world, mastering Generative AI isn’t just an advantage—it’s a necessity. Are you ready to harness its power to transform your business and solve real-world challenges? I’m excited to announce that my new book, Generative AI in Action, is now available in print and ebook formats from Manning Publications . 📖 Special Launch Offer 🌟 As a thank-you to my early supporters, I’m offering an exclusive discount. Use the code pbbahree at checkout to receive 45% off your purchase of Generative AI in Action in all formats (valid through Sept. 30, 2024)! ...

I have been running a couple of instances of Teslamate - one locally on a server at home and another in Azure in a Ubuntu VM (see 👉 this blog post for details). I have been backing up the data to a NAS and then an offsite backup for the local instance. For the Azure instance, I have been running various backups during the day and backing up the data to OneDrive. This allows me to have a data backup in case the VM crashes or the data gets corrupted. ...

We released Phi-3 recently, which builds on Phi-2 ( read more on that here ) and it is a great model to use for various tasks. In this post, we will show how to run Phi-3 locally including a demo of it running on a phone. There should have been a video here but your browser does not seem to support it. 1. What are Small Language Models (SLMs)? Before diving into running Phi-2 locally, let’s take a moment to understand the concept of small language models (SLMs) and their significance in natural language processing (NLP). A SLM is a type of AI model that has been trained on a massive dataset of text but is limited in terms of its size and capabilities compared to a Large Language Model (LLM). SLMs are designed to be more lightweight and efficient, making them suitable for various applications, including chatbots, language translation, and content generation. SLMs are much smaller than LLMs, with fewer parameters and a smaller dataset, so they have a lower computational cost, making them more suitable for edge or resource-constraint devices. ...

1. What are Small Language Models (SLMs)? Before diving into running Phi-2 locally, let’s take a moment to understand the concept of small language models (SLMs) and their significance in natural language processing (NLP). A SLM is a type of AI model that has been trained on a massive dataset of text but is limited in terms of its size and capabilities compared to a Large Language Model (LLM). SLMs are designed to be more lightweight and efficient, making them suitable for various applications, including chatbots, language translation, and content generation. SLMs are much smaller than LLMs, with fewer parameters and a smaller dataset, so they have a lower computational cost, making them more suitable for edge or resource-constraint devices. ...

🌐 As software continues to revolutionize the world, the advent of Generative AI is transforming the very fabric of software itself. My latest book, Generative AI in Action delves into this transformative journey. I am thrilled to announce the early release of my latest book, Generative AI in Action now available through Manning Early Access Program (MEAP) . This publication is a deep dive into the cutting-edge world of #GenerativeAI, #LLMs, #OpenAI, and #Azure #OpenAI, tailored specifically for enterprises. 🤘 ...

What does AI think of humans and our humor? I asked #AI - #CodeInterpreter plugin for #ChatGPT to express its experience as an AI working with humans as a meme. It wrote this code to create the meme below. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 import matplotlib.pyplot as plt import numpy as np # Create a new figure with a white background fig, ax = plt.subplots(figsize=(6, 6), dpi=80) fig.patch.set_facecolor('white') # Set up the text and meme caption meme_caption = "When I, an AI, try to understand human humor:" meme_text = "Error 404: Humor not found" # Add the meme caption and text to the figure ax.text(0.5, 0.7, meme_caption, horizontalalignment='center', verticalalignment='center', transform=ax.transAxes, color='black', fontsize=14, fontweight='bold') ax.text(0.5, 0.3, meme_text, horizontalalignment='center', verticalalignment='center', transform=ax.transAxes, color='black', fontsize=16, fontweight='bold') # Remove the axes for a clean meme look plt.axis('off') # Show the meme plt.show() AI's experience working with humans ...

What is Whisper from OpenAI? Whisper is a speech recognition model (ASR – automatic speech recognition) from OpenAI. The model itself is multi-task model and as a result in addition to speech recognition, can also do language identification and speech translation across a number of languages. The model is open sourced and it comes in 5 sizes. Of these, 4 have a english-only variant which seem to perform better if one only needs english. The model is also robust to noise, accents, background noise and technical language. Whisper achieves near SOTA performance with zero-shot translation from multiple-languages to English. ...

API Introduction Polite Fork Off As A Service (or PFOaaS) - https://pfoaas.desigeek.com/ is a modern REST API that solves the problem of one telling people to politely fork off. 😇 There are days when we all need such a service for various reasons, and I think it is a great way to release some pent-up frustration. 🖤 It is also a great way to get some laughs too. This is of course meant for hard code engineers, writing RPC free code 😜 ...

In the last week or so, all the range online has been #OpenAI’s new chatbot called #ChatGPT (you can read more details on ChatGPT here ). This also got me thinking, about how can we use #CoPilot more than just code. GitHub CoPilot as you might recall is your #AI powered pair-programmer. And as we can see below, it indeed is possible to use Codex as sort of a more general purpose usage. I start with the prompts on how one might use CoPilot – a function to read a file and return its contents as a string, just to show there isn’t anything different I am doing in using this. And then for general-purpose usage, I used the prompts in VSCode. ...

I recently needed to install Visual Studio 2022 on one my existing machines to debug a new zeroshot model that has a dependency on our Speech SDK. The Speech SDK is one of our key #AI services in Cognitive Services (as part of #AzureAI). I already had VSCode running, but in this case I need the bigger brother. After installing Visual Studio, I could not get any nuget packages to install; I could not even fetch anything and didn’t matter what I used - the package manager console in Visual Studio, PowerShell, etc. ...