I've been working on Mixture of experts (MoEs) and sparsity for a few years now, so I thought I'd make a short video explaining how MoE routing works.

For the video I trained a tiny character model with a 32-expert layer three times, on English, Python and arithmetic text.

Run 1, token choice with no balancing loss: a few experts get most of the tokens. Early in training one expert gets 57% of them while 17 of the 32 are barely used.

Run 2, token choice with a load-balancing loss: tokens spread out roughly evenly and every expert gets work.

Run 3, expert choice: each expert picks its own tokens, so the load is perfectly even, but some tokens get picked by no expert and others by several. It also ended with the lowest eval loss of the three.

For those who are familiar with the books by Sebastian Raschka (i.e. "Build a LLM/Reasoning Model from Scratch"), do you know of any recent similar books with a focus on computer vision models (e.g. CNNs, ViTs, Swin etc)? I would like to develop a more in depth understanding by building these models from scratch and I think the approach used by Raschka does exactly that for language models. Any recommendations are welcome!

Researchers tracked more than 2,600 AI agent attempts targeting Alibaba's Amap from Tencent-linked cloud infrastructure. None of the agents had authorization to access that service.

These were not human operators pivoting through tooling. They were autonomous agents running coordinated, repeated reconnaissance against a live production system at machine speed. Human-paced incident response cannot match that cadence.

The exposure here is not a misconfigured firewall or a leaked credential. The agents operated against a production service with no verified identity, no declared scope, and no gate between their initialization and their first outbound request. The target service had no way to distinguish a legitimate automated caller from an adversarial one before the probe landed.

For those running production APIs or services that autonomous agents — not humans, not static service accounts — might reach: what does your authorization boundary actually look like at the moment of first contact? Not at login, not in a post-incident log — at the exact moment an unknown agent makes its first request.

I am searching for answers while exploring graph neural networks, and i'm also searching for people who are willing to look outside of the language model approach to deep learning.

I'm working on a system right now. I have the scripts running functionally as a toy. Because deployment and scalability are difficult without a team and without more information for my system to process to buold relationships.

Would anybody know where to go to find focused deep learning corpora, that is open source and properly filtered?

I am searching for anything I can regarding mathematics, human language including historical incomplete languages, physics and all categories of the science.

I am looking for something dense, educational and clean.

AgentGarten separates interactive-world state from its visual presentation: executable scene code updates persistent state and interaction rules, while a shared neural renderer turns structured geometry conditions into observations. This lets an agent act on video-like inputs without making the simulator’s state depend on generated frames.

The renderer is adapted from a pretrained video model and trained with Adversarial Forcing. A notable component is exact block-by-block replay: it restores gradients through the history used for later predictions while preserving the original attention execution. The reported replay is bitwise identical to the cached rollout; the full-sequence alternative has a 3.99% relative L2 error.

In interactive tasks, pretrained agents refine inherited playbooks across rounds. The paper reports shelter construction by round 4, compared with roughly 25 million episodes in an earlier self-play result, while noting that these are not directly comparable: the agents start with pretrained knowledge and are learning to ground it in this setup. The renderer’s training and history-gradient method seem the most broadly relevant contributions; the agent results depend more heavily on the choice of pretrained model and task.

Full summary on AIModels.fyi

Original paper

Disclosure: AIModels.fyi is my site.

Getting Started with Bedrock AgentCore

https://debuggercafe.com/getting-started-with-bedrock-agentcore/

In this article, we will cover an introduction to Bedrock AgentCore and check the process from creating agents to their deployment.

This video features a talk by Dr. Edo Dotan titled "Deep Learning in Biology," presented to the Allen Discovery Center at Tufts University. The presentation explores the intersection of Artificial Intelligence (AI) and genomics, focusing on how Deep Learning—specifically Transformer architectures—can be applied to biological sequences.

Key Topics Covered:

• Introduction to AI/ML/DL Hierarchy: A breakdown of how Deep Learning resides within Machine Learning, which is a subfield of AI (1:09).
• Data Representation: Discussion on various data types in biology, including graphs, images, text, and, crucially, genomic sequences (5:28).
• The Challenge of Biological Data: Biological sequences like DNA are analyzed similarly to text strings but present unique challenges compared to standard English, such as different hierarchies, structures, and overlapping gene information (7:57).
• Tokenization in Transformers: Dr. Dotan explains the crucial role of tokenization in transformer models and its effect on performance when applied to genomic data (17:13). He highlights the methodology of training models using different approaches like BPE (Byte-Pair Encoding) vs. k-mer tokenization (20:12).
• BetaDescribe: The talk introduces a framework called BetaDescribe, designed for protein sequences (29:05). This model uses a modular architecture (35:46) including a generator, an evaluator, and a controller to provide rich, meaningful descriptions of protein functions.
• Case Studies & Future Paradigms: The presentation concludes with examples of using BetaDescribe to illuminate the "uncharacterized proteome," moving toward a new paradigm of biological annotation and experimental design (48:00).

A cell-based simulation calculates how cells move and respond to their neighbours. Repeating it under many settings is expensive, and random variation means the same settings can produce different final patterns.

Comlekoglu et al. trained a class-conditional diffusion model on images from a simulation of vessel-like network formation. The input is one of the tested simulation settings plus noise; the output is a possible final pattern, rather than the full sequence of cell movements.

They reported 20.6 seconds per generated pattern on four V100 GPUs, versus 447.6 seconds for CompuCell3D on an AMD EPYC CPU node. These are different hardware setups, and the comparison excludes training on 16 V100 GPUs.

The model covered 25 discrete parameter settings. Classifier checks and network-shape measurements helped evaluate outputs, but some patterns remained imperfect. Generating valid samples between those settings remains unresolved. The result concerns reproducing simulation outputs, not proving that the underlying biology is correct.

Paper: https://arxiv.org/abs/2505.09630

Explanation: https://rewirebio.io/blog/diffusion-surrogates-for-agent-based-biological-simulations/

Disclosure: the explanation is from my site, rewirebio.io. This summary and schematic were prepared with AI assistance. I have not run or independently reproduced the model.