Speeding Up Variable-Length Training with Dynamic Context Parallelism and NVIDIA Megatron Core
This post introduces Dynamic Context Parallelism (Dynamic-CP), a scheduling approach in NVIDIA Megatron Core used for LLM post-training or DiT pre-training. It dynamically selects the CP size per microbatch to…
Andrej Karpathy Open-Sources ‘Autoresearch’: A 630-Line Python Tool Letting AI Agents Run Autonomous ML Experiments on Single GPUs
Andrej Karpathy released autoresearch, a minimalist Python tool designed to enable AI agents to autonomously conduct machine learning experiments. The project is a stripped-down version of the nanochat LLM training…
Stop Tuning Hyperparameters. Start Tuning Your Problem.
. You’re three weeks into a churn prediction model, hunched over a laptop, watching a Bayesian optimization sweep crawl through its 200th trial. The validation AUC ticks from 0.847 to…
these LLMs are willing to commit academic fraud
Credit: Smith Collection/Gado/GettyAll major large language models (LLMs) can be used to either commit academic fraud or facilitate junk science, a test of 13 models has found.Still, some LLMs performed…
Engineers just created a “phonon laser” that could shrink your next smartphone
Engineers have taken a major step toward producing the smallest earthquakes ever created, shrinking seismic-style vibrations down to the scale of a microchip. The breakthrough centers on a device called…
Brain scans reveal how ketamine quickly lifts severe depression
Major depressive disorder (MDD) is a major global health problem and one of the leading causes of disability. About 30% of people diagnosed with depression develop treatment-resistant depression (TRD), meaning…
Too much screen time may be hurting kids’ hearts
More time using electronic devices or watching TV among children and young adults was linked with higher cardiometabolic disease risk, including high blood pressure, high cholesterol and insulin resistance, based…
NVIDIA Cosmos Reason 2 Brings Advanced Reasoning To Physical AI
NVIDIA today released Cosmos Reason 2, the latest advancement in open, reasoning vision language models for physical AI. Cosmos Reason 2 surpasses its previous version in accuracy and tops the…
Ensuring Balanced GPU Allocation in Kubernetes Clusters with Time-Based Fairshare
NVIDIA Run:ai v2.24 introduces time-based fairshare, a new scheduling mode that brings fair-share scheduling with time awareness for over-quota resources to Kubernetes clusters. This capability, built on the open source…
OpenAI Introduces Codex Security in Research Preview for Context-Aware Vulnerability Detection, Validation, and Patch Generation Across Codebases
OpenAI has introduced Codex Security, an application security agent that analyzes a codebase, validates likely vulnerabilities, and proposes fixes that developers can review before patching. The product is now rolling…