Resilient, Distributed AI Training at Scale — Google DeepMind

Decoupled DiLoCo is not only more resilient to failures, but is also practical for executing production-level, fully distributed pre-training. We successfully trained a 12 billion parameter model across four separate U.S. regions using 2-5 Gbps of wide-area networking (a level relatively achievable using existing internet connectivity between datacenter facilities, rather than requiring new custom network…

Read More

500k Biobank volunteers’ data listed for sale on Alibaba • The Register

Updated Details of volunteers of UK-based Biobank, which describes itself as the custodian of the world’s most comprehensive biomedical dataset, are for sale on Chinese ecommerce site Alibaba. The organization confirmed the data on roughly half a million volunteers was anonymized, but could not guarantee it would be impossible to identify individuals if it fell…

Read More