The Leaderboard Illusion
“We find that undisclosed private testing practices benefit a handful of providers who are able to test multiple variants before public release and retract scores if desired.”
“We find that undisclosed private testing practices benefit a handful of providers who are able to test multiple variants before public release and retract scores if desired.”
“From the start of 2024 to the present, the Android app marketplace went from hosting about 3.4 million apps worldwide to just around 1.8 million, according to a new analysis by app intelligence provider Appfigures.”
“The AI Assistant, first released in July 2023, has been downloaded over 22 million times but is rated only 2.3 out of 5.”
“Unless we secure $250,000 in committed funds, the OSL will shut down later this year.”
“We show that Amazon RDS for PostgreSQL multi-AZ clusters violate Snapshot Isolation, the strongest consistency model supported across all endpoints.”
“On Tuesday, the Karnataka High Court directed the Indian government to block Proton Mail, a popular email service known for its enhanced security, following a legal complaint filed by New Delhi-based M Moser Design Associates.”
“After compromising one device, attackers can use the same vulnerability to propagate to other devices and networks, furthering their impact and damage.”
“While Qwen2.5 was pre-trained on 18 trillion tokens, Qwen3 uses nearly twice that amount, with approximately 36 trillion tokens covering 119 languages and dialects.”
“In Portugal, Internet traffic dropped as the power grid failed, with traffic immediately dropping by half as compared to the previous week, falling to approximately 90% below the previous week within the next five hours.”
“This project yields important insights, and the risks (e.g. trauma etc.) are minimal. This means that suppressing publication is not proportionate to the importance of the insights the study yields.”
“Google pays Apple about $18 billion each year, and pays Mozilla about $450 million each year.”
“GitHub requested Fenix to identify “every specific file” in the repo that it considers infringing; Fenix responded with a statement that the “entire repository is infringing” and should be removed.”
“The outage began on April 23, after engineers conducting maintenance work mistakenly deleted critical storage connected to a key database, a CHS spokesperson said in a statement.”
“With that widget extension in place, as soon as I installed the VeryEvilNotify app on my security research device, the ‘Restore in Progress’ UI was shown, then failed with a prompt to restart the system.”
“Local book shops have managed to have a moment, thriving despite the increasing dominance of Amazon online by creating bonds with their local community and offering a shopping experience that Amazon can’t replicate.”
“This goes far beyond ignorance about the copyright implications of reverse engineering Nintendo binaries, and goes straight into outright deliberate, malicious code theft and copyright infringement.”
“It really demanded modern AI to formulate the three-dimensional structure very precisely to make this discovery.”
“An IMSI catcher is a tool that collects cellular signals and decodes packets to access and save off the IMSI.”
“All told, we believe Backblaze has resorted to aggressive accounting to inflate the results of an already loss-making business.”
“That means owners of the Nest thermostats released in 2011 and 2012 (as well as the version released in Europe in 2014) won’t receive updates, and their devices will no longer be supported in the Nest and Home apps.”
“The problem is, outside of anecdotal cases, no one has ever been able to find clear evidence that this is actually happening.”
“A 24-year-old Australian man who ordered uranium and plutonium to his parents’ apartment has been allowed to walk away from court on a two-year good behaviour bond.”
“Using difference-in-differences and employer policies as quasi-experimental variation, we estimate precise zeros: AI chatbots have had no significant impact on earnings or recorded hours in any occupation, with confidence intervals ruling out effects larger than 1%.”
“A single prompt can be designed to work across all of the major frontier AI models.”
“The company is saying it will start under $20,000 after the federal tax credit (providing that still exists next year).”
“In 2025, OSI excluded all votes for three candidates before publishing results.”
“I dutifully submitted feedback for “Wrong pin location or address”: And I got an email the next day saying that my edit was accepted: But it’s been 14 days and the address still maps incorrectly.”
“Microsoft has forbidden the use of its extensions outside of its own software products since at least September 2020, when the current licensing terms were published.”
“Synadia is attempting to convert a successful open source project into a closed, commercial product—and take the NATS community’s name, trust, and infrastructure with it.”
“The complainant discovered that, over a period of just 10 minutes, the game established a connection to external servers 150 times.”
“Pricing is $5 per million input tokens for text and $10 per million input tokens for images, and $40 per million output tokens for images.”
“Of interest to many Phoronix readers will be that GIM / SR-IOV support could be coming to client discrete GPUs, which has been a long sought feature for the Radeon graphics cards.”
“Similarly, consumers cannot fully benefit from alternative and cheaper offers as Apple prevents app developers from directly informing consumers of such offers.”
“So I chucked it into FontForge and yep, turns out the campaign used a pirated font the entire time!”
“I want to make it clear: all these findings prove that the bug is NOT an issue with Windows 11 24H2, as things like the way the stack is used by internal WinAPI functions are not contractual and they may change at any time, with no prior notice.”
“Yes, we would, as would many other parties,” he said.
“However, the drone equipped with the protective cage continued to fly stably even after the lightning strike.”
“Seeking answers via support channels, some were informed by an AI bot – reportedly named “Sam” – that their subscription was limited to a single active session, a restriction the company later confirmed was entirely fictitious.”
“We’ve made the decision to maintain our current approach to offering users third-party cookie choice in Chrome, and will not be rolling out a new standalone prompt for third-party cookies.”
“Supabase is currently used by two million developers who manage more than 3.5 million databases.”
“The digitising process erroneously combined ‘vegetative’ from one column of text with ’electron’ from another.”
“Replay attacks can be pretty easily thwarted by using an incrementing counter in the communications to nullify previous signals, ignoring signals with a counter value you’ve already seen.”
“Digging deeper I found test cases referencing Spegel and my previous employer, test cases that have been taken directly from my project.”
“Bluesky will proactively verify authentic and notable accounts and display a blue badge next to their names.”
“It is sort of a best case because the 30 percent applies to the network stack or communication part of it…”
“In March 2025, fossil fuels accounted for less than 50% (49.2%) of electricity generated, for the first month on record.”
“Thanks to a public leak of confidential military and police documents in March 2025, we can now attribute the campaign to the Royal Thai Armed Forces and/or the Royal Thai Police.”
“Compared to Cursor, Windsurf has fewer users, has been around for less time, has less brand recognition, and has diminishing prospects for future growth.”
“With this technique, the researchers enabled five people to see a new color, dubbed “olo,” which the study participants described as a “blue-green of unprecedented saturation.””
“Using int4 means each number is represented using only 4 bits – a 4x reduction in data size compared to BF16.”
“Let us just call this what it is. It is a grab for extra margin dollars.”
“SSL.com failed to conduct accurate domain validation control when utilizing the BR 3.2.2.4.14 DCV method (Email to DNS TXT Contact).”
“So if you as an app developer include such a 3rd party SDK in your app to make some money — you are part of the problem and I think you should be held responsible for delivering malware to your users, making them botnet members.”
“Before a phone is switched on and unlocked, certain data stored within is fully encrypted and harder to access without the user’s passcode.”
“The hashing audit eventually revealed that four files were corrupt (hash not matching).”
“The judge also ruled that the cops could, this one time, still use the evidence they obtained through this unconstitutional search.”
“OpenAI found that o3 hallucinated in response to 33% of questions on PersonQA, the company’s in-house benchmark for measuring the accuracy of a model’s knowledge about people.”
“It’s an obvious potential privacy issue. There’s nothing preventing a bad actor from screenshotting, say, a person’s Instagram Story and using ChatGPT to try to doxx them.”
“Normally this would be a simple problem to fix, just go to your Apple account settings and switch the region, but this too was denied, as I had the yearly Apple Music subscription.”
“the (unverified) reference implementation of SHA3 was hit with a bad CVE in 2022, which Python “inherited”, because it vendored that very same SHA3 implementation.”
“The judge did agree that Google violated the Sherman Act by monopolizing and unlawfully tying two parts of the adtech stack together, specifically DFP and AdX, and that Google is guilty of abuse of monopoly power in the publisher-side adtech.”
“We had lots of modeling that showed the payback around FSD [Full Self Driving] and Robotaxi was going to be slow. It was going to be choppy.”
“This block was the result of a communication error between Zoom’s domain registrar, Markmonitor, and GoDaddy Registry, which resulted in GoDaddy Registry mistakenly shutting down the zoom.us domain.”
“By exploiting a flaw in SSH protocol message handling, a malicious actor could gain unauthorized access to affected systems and execute arbitrary commands without valid credentials.”
“If you opt for a face-scan, Discord says the age estimation is performed on your device, so is not uploaded or stored. If you choose ID, the scan will be uploaded, but will be deleted after verification.”
“Snapchat had disclosed that it had removed approximately 60 accounts per month, or just over 700 total.”
“This evaluation was conducted in a relatively short time, and we only tested [o3] with simple agent scaffolds…”
“After nearly two years without revenue or committed paying customers, we realised we lacked a clear path to the necessary scale and returns.”
“Indeed, for most of the users I tested this against, it worked very well, and often times it was the top result.”
“There is a catch, however. Achieving that performance requires using Microsoft’s custom framework, bitnet.cpp, which only works with certain hardware at the moment.”
“Prior to this proposal by Apple, Google promoted a 90-day maximum lifetime, but they voted in favor of Apple’s proposal almost immediately after the voting period began.”
“The serious costs and widespread performance issues and outages caused by reckless scrapers has been on everyone’s mind in the sysadmin community as of late, and has been the subject of much discussion online.”
“Years ago it would’ve required a supercomputer and a PhD to do this stuff.”
“The attackers reportedly exploited outdated PHP code and deprecated MySQL functions in 4chan’s backend, particularly in a core script named yotsuba.php, which manages post submissions and moderation.”
“Once a compromised rule file enters a project repository, it corrupts all future code-generation sessions for every team member working in that project.”
“The transaction values Altera at $8.75 billion, just under half what Intel paid to acquire the company in 2015.”
“These are API-only models right now, not available through the ChatGPT interface (though you can try them out in OpenAI’s API playground).”
“Our engine is based on an early fork of vLLM from over a year ago.”
“Obviously getting hacked from downloading and running third-party code isn’t a novel vulnerability but the protocol has effectively created a low-friction path for less technical users to get exploited on their local machines.”
“An AI-powered bot pretending to be a human, lamenting AI-powered bots who pretend to be human, to gain human trust, so that it can covertly market AI-illustrated books.”
“While the RWD Cybertruck was said to have a starting price of under $40,000 back then (and then around $61,000 last year when it was available to order but never actually entered production), you’ll now have to pony up $71,985 for it, including $1,995 destination.”
“Because accurate timing is critical in many areas, from media, to the power grid, to 5G and communications, having a reliable terrestrial backup to GPS—especially one that can be hardened against different types of jamming attempts—may be important to our economy, communications and power grid.”
“From 2015 through 2023, the Baton Rouge-based insurer paid, on average, less than 9% of the charges billed by the breast center for more than 7,800 individual medical procedures — even though it had authorized all of them.”
“In a recent study, researchers found that about 5.2 percent of package suggestions from commercial models didn’t exist, compared to 21.7 percent from open source or openly available models.”
“Recall is an opt-in Windows feature that screenshots active windows every few seconds, analyzes them, and allows Windows 11 users to search text within the snapshots.”
“Anubis has been deployed by the United Nations.”
“Meta has complied with 94% of takedown requests issued by Israel since October 7, 2023.”
“My List” on Netflix randomly shuffles items and changes their covers every few hours.
“I’m told this comes as a shock to my managers and other Chrome team leaders. I’m told it’s not based on merit.”
“Debian, for example, has been working toward reproducible builds for more than a decade; it can now produce official live CDs of the current stable release that are reproducible.”
“Frustrated by the company’s business practices, users on the platform flooded its posts with backlash, ultimately prompting Adobe to delete all of its content.”
“In reality, however, Nate relied heavily on hundreds of human contractors in a call center in the Philippines to manually complete those purchases, the DOJ’s Southern District of New York alleges.”
“AI-related incidents are rising sharply, yet standardized RAI evaluations remain rare among major industrial model developers.”
“Due to budget constraints, the recent security audit focused on the most critical components of the PHP source code rather than the entire codebase.”
“In a post on X on Wednesday, Google DeepMind CEO Demis Hassabis said Google would add support for Anthropic’s Model Context Protocol, or MCP, to its Gemini models and SDK. He did not specify a timeline for when this would be done.”
“The Arroyo Engine will remain open-source and self-hostable.”
“Throughout my seven years at Meta I saw executives repeatedly undermine US national security and betray American values.”
“Even customers that adhere to the secret and top secret U.S. government classification levels will be able to use Gemini through an air-gapped version of Google Distributed Cloud that’s disconnected from the internet.”
“The pilot project followed 122 people for three years who received an unconditional sum of €1,200 every month. The control group included a further 1,580 people who did not receive any money.”
“By contrast, Whisky is based on CrossOver, but we don’t produce any bespoke fixes. I, quite frankly, do not have the requisite skills or time to do so.”
“The company published the numbers for those 4 quarters, but then, in the next quarter, Q1 2024, Tesla simply lamented that its solar installations had declined year over year but stopped publishing those figures.”
“Meta submitted a specially crafted, non-public variant of its Llama 4 AI model to an online benchmark that may have unfairly boosted its leaderboard position over rivals.”
“AT&T has announced that it will discontinue its email-to-text service on June 17, 2025.”
“Fortune 500 companies have unwittingly hired thousands of software engineers who claim to be American developers but are actually North Korean citizens using stolen or fake identities.”
“In a judgment handed down today, tribunal president Lord Justice Singh and Mr Justice Johnson said preventing disclosure even of the fact that a hearing was taking place would have represented ’the most fundamental interference with the principle of open justice’.”
“After 40 years, 30 million rules, 200 million dollars, 2000 person-years, and many promises, Cyc has failed to reach intellectual maturity, and may never will.”
“My sources assure me that this is NOT a clerical error and that Papan’s office confirms that the change is real.”
“Using AI effectively is now a fundamental expectation of everyone at Shopify.”
“The end came on Friday, when G/O fired the few remaining writers at Quartz and sold the carcass to a Canadian firm that appears mostly interested in the email list.”
“The European Commission is now finalizing a plan to simplify and potentially remove many of the regulatory requirements imposed by the continent’s complex and far-reaching General Data Protection Regulation, particularly those impacting small and medium-sized businesses.”
“A malicious attacker on the same network as the victim could fully compromise the target device without any user-interaction.”
“Our scanner has improved a lot since August, but because of regular engineering, not model improvements.”
“The Cloud is what happens when the cost of storing data is less than the cost of figuring out what to do with the crap.”
“A maximum severity remote code execution (RCE) vulnerability has been discovered impacting all versions of Apache Parquet up to and including 1.15.0.”
“If I receive an “e2e” encrypted email from gmail, I will not be able to read it without going to a Google service.”
“Now with macOS Sequoia, Apple has replaced rsync 2.6.9 with openrsync, an implementation of rsync which is not using any version of the GPL open source license.”
“Llama 4 refuses less on debated political and social topics overall (from 7% in Llama 3.3 to below 2%).”
“Microsoft fared a tad better in getting the dismissal of all DMCA 1202(b)(1) claims against it, whereas OpenAI succeeded only in getting the New York Times’ 1202(b)(1) claim, but not the claims of the Center for Investigative Reporting or the Daily News.”
“Charging an EV battery takes 30 to 40 minutes even for aggressive fast charging, and that time increases to over an hour in the winter.”
“Saccharin breaks the walls of bacterial pathogens, causing them to distort and eventually burst, killing the bacteria.”
“Late last year, LAist requested samples of MVP-Fx from Cal Fire, the U.S. Forest Service and Perimeter Solutions, which manufactures the product, for the purpose of running an independent analysis for heavy metals. All declined.”
“Once we have a sufficiently strong base of paying users to sustainably support our services, we plan to introduce a limited free tier to the public.”
“The issue isn’t that the license exists, it’s that it’s misrepresented as open source.”
“To that end, the Steering Committee is banning Anduril from recruiting on Discourse.”
“While members of the affected super funds are being urged to check their accounts for suspicious activity, media reports suggest that the threat actors targeted accounts in the pension drawdown phase, as those accounts can request lump-sum withdrawals.”
“another reminder of how OpenAI(and everyone else) trains on images that are very obviously someone else’s work.”
“Extra memory for the Altair could easily cost more than the computer itself,” recalled Gates.
“We predict that the impact of superhuman AI over the next decade will be enormous, exceeding that of the Industrial Revolution.”
“The models very rarely admitted to using the reward hacks in their Chain-of-Thought explanations, doing so less than 2% of the time in most of the testing scenarios.”
“Call metadata might seem harmless, but in the wrong hands, it becomes a powerful surveillance tool.”
“While our revenue continues to grow, Automattic operates in a highly competitive market, and technology is evolving at unprecedented levels.”
“The people taking the creatine supplement saw changes before they even started exercising, which leads us to believe that it wasn’t actual real muscle growth, but potentially fluid retention.”
“The results constitute the first empirical evidence that any artificial system passes a standard three-party Turing test.”
“The researchers found that in half of cases, conditions such as cardiac hypertrophy were not detected when patients were black women.”
“It wasn’t until 1975, when the Bletchley operation was declassified, that she could even tell her family what she had done during the war.”
“Because of Dave’s persistence and advocacy, millions of devices worldwide now ship algorithms like FQ-CoDel by default, to smooth connectivity.”
“In just the past two years, our team has encountered three separate GLIBC-specific compatibility issues, each directly impacting our products”
“This process is integrated in almost every commercial colour laser printer. This means that almost every printout contains coded information about the source device, such as the serial number.”