healthcarereimagined

JPMorganChase just released an Open Letter to their third party suppliers, and sounded the AI security alarm!

The financial giant sees what others are missing:

Companies rushed to deploy AI without understanding the consequences. The mandate was clear: innovate or die. But JP Morgan’s latest security assessment reveals that:

• 78% of enterprise AI deployments lack proper security protocols
• Most companies can’t explain how their AI makes decisions
• Security vulnerabilities have increased 3x since mass AI adoption

The problem? Speed > security.

JP Morgan’s CTO Pat Opet put it bluntly: “We’re seeing organizations deploy systems they fundamentally don’t understand.” The financial sector is particularly vulnerable – with trillions at stake.

What JP Morgan recommends:

→ Implement AI governance frameworks before deployment
→ Conduct regular red team exercises against AI systems
→ Establish clear model documentation standards
→ Create dedicated AI security response teams

JP Morgan itself has invested $2B in AI security measures while slowing certain deployments.

The hard truth:

The AI security debt is growing faster than our ability to pay it down. Companies that prioritize security now will emerge as leaders. Those that don’t may not survive the coming AI security reckoning.

An open letter to third-party suppliers

By Patrick Opet, Chief Information Security Officer

The modern ‘software as a service’ (SaaS) delivery model is quietly enabling cyber attackers and – as its adoption grows – is creating a substantial vulnerability that is weakening the global economic system.

• Software providers must prioritize security over rushing features. Comprehensive security should be built in or enabled by default.
• We must modernize security architecture to optimize SaaS integration and minimize risk.
• Security practitioners must work collaboratively to prevent the abuse of interconnected systems.

There is a growing risk in our software supply chain and we need your action

SaaS has become the default and is often the only format in which software is now delivered, leaving organizations with little choice but to rely heavily on a small set of leading service providers, embedding concentration risk into global critical infrastructure. While this model delivers efficiency and rapid innovation, it simultaneously magnifies the impact of any weakness, outage, or breach, creating single points of failure with potentially catastrophic systemwide consequences. Historically, software was distributed across diverse environments, each with unique security practices, inherently limiting the scale of any single breach. Today, an attack on one major SaaS or PaaS provider can immediately ripple through its customers. This fundamental shift demands our collective immediate attention.

At JPMorganChase, we’ve seen the warning signs firsthand. Over the past three years, our third-party providers experienced a number of incidents within their environments. These incidents across our supply chain required us to act swiftly and decisively, including isolating certain compromised providers, and dedicating substantial resources to threat mitigation.

Security must be prioritized

Risks extend beyond concentration alone. Fierce competition among software providers has driven prioritization of rapid feature development over robust security. This often results in rushed product releases without comprehensive security built in or enabled by default, creating repeated opportunities for attackers to exploit weaknesses. The pursuit of market share at the expense of security exposes entire customer ecosystems to significant risk and will result in an unsustainable situation for the economic system.

Security architecture must be modernized

Most critically, SaaS models are fundamentally reshaping how companies integrate services and data—a subtle yet profound shift eroding decades of carefully architected security boundaries. In the traditional model, security practices enforced strict segmentation between a firm’s trusted internal resources and untrusted external interactions using protocol termination, tiered access, and logical isolation. External interaction layers like APIs and websites were intentionally separated from a company’s core backend systems, applications, and data that powered them.

Modern integration patterns, however, dismantle these essential boundaries, relying heavily on modern identity protocols (e.g., OAuth) to create direct, often unchecked interactions between third-party services and firms’ sensitive internal resources. As a generic example, an AI-driven calendar optimization service integrating directly into corporate email systems through “read only roles” and “authentication tokens” can no doubt boost productivity when functioning correctly. Yet, if compromised, this direct integration grants attackers unprecedented access to confidential data and critical internal communications.

In practice, these integration models collapse authentication (verifying identity) and authorization (granting permissions) into overly simplified interactions, effectively creating single-factor explicit trust between systems on the internet and private internal resources. This architectural regression undermines fundamental security principles that have proven durability.

This problem is getting worse not better

Further compounding the risks are specific vulnerabilities intrinsic to this new landscape: inadequately secured authentication tokens vulnerable to theft and reuse; software providers gaining privileged access to customer systems without explicit consent or transparency; and opaque fourth-party vendor dependencies silently expanding this same risk upstream. Critically, the explosive growth of new value-bearing services in data management, automation, artificial intelligence, and AI agents amplifies and rapidly distributes these risks, bringing them directly to the forefront of every organization.

This weakness is known to attackers who are now actively targeting trusted integration partners—Microsoft Threat Intelligence recently authored a blog post that Chinese state actors were shifting tactics to target “common IT solutions like remote management tools and cloud applications to gain initial access” to their downstream customers.

Call to action

We stand at a critical juncture. Providers must urgently reprioritize security, placing it equal to or above launching new products. ‘Secure and resilient by design’ must go beyond slogans—it requires continuous, demonstrable evidence that controls are working effectively, not simply relying on annual compliance checks. Customers should be afforded the benefit of secure by default configurations, transparency to risks, and management of the controls they need to operate safely within a SaaS delivery model. The ecosystem must address trustworthy integration. There are some solutions available today, like confidential computing, customer self-hosting, and bring your own cloud, which all give organizations stronger controls to protect their data while enabling them to benefit from SaaS solutions.  

We must establish new security principles and implement robust controls that enable the swift adoption of cloud services while protecting customers from their providers’ vulnerabilities. Traditional measures like network segmentation, tiering, and protocol termination were durable in legacy principles but may no longer be viable today in a SaaS integration model. Instead, we need sophisticated authorization methods, advanced detection capabilities, and proactive measures to prevent the abuse of interconnected systems.

The most effective way to begin change is to reject these integration models without better solutions. I hope you’ll join me in recognizing this challenge and responding decisively, collaboratively, and immediately.

Patrick Opet, Chief Information Security Officer, JPMorganChase

Article link: https://www.jpmorgan.com/technology/technology-blog/open-letter-to-our-suppliers

By Gary Hamel

Apr 14, 2015

Pope Francis has made no secret of his intention to radically reform the administrative structures of the Catholic church, which he regards as insular, imperious, and bureaucratic. He understands that in a hyper-kinetic world, inward-looking and self-obsessed leaders are a liability.

Last year, just before Christmas, the Pope addressed the leaders of the Roman Curia — the Cardinals and other officials who are charged with running the church’s byzantine network of administrative bodies. The Pope’s message to his colleagues was blunt. Leaders are susceptible to an array of debilitating maladies, including arrogance, intolerance, myopia, and pettiness. When those diseases go untreated, the organization itself is enfeebled. To have a healthy church, we need healthy leaders.

Through the years, I’ve heard dozens of management experts enumerate the qualities of great leaders. Seldom, though, do they speak plainly about the “diseases” of leadership. The Pope is more forthright. He understands that as human beings we have certain proclivities — not all of them noble. Nevertheless, leaders should be held to a high standard, since their scope of influence makes their ailments particularly infectious.

The Catholic Church is a bureaucracy: a hierarchy populated by good-hearted, but less-than-perfect souls. In that sense, it’s not much different than your organization. That’s why the Pope’s counsel is relevant to leaders everywhere.

With that in mind, I spent a couple of hours translating the Pope’s address into something a little closer to corporate-speak. (I don’t know if there’s a prohibition on paraphrasing Papal pronouncements, but since I’m not Catholic, I’m willing to take the risk.)

Herewith, then, the Pope (more or less):

____________________

The leadership team is called constantly to improve and to grow in rapport and wisdom, in order to carry out fully its mission. And yet, like any body, like any human body, it is also exposed to diseases, malfunctioning, infirmity. Here I would like to mention some of these “[leadership] diseases.” They are diseases and temptations which can dangerously weaken the effectiveness of any organization.

1. The disease of thinking we are immortal, immune, or downright indispensable, [and therefore] neglecting the need for regular check-ups. A leadership team which is not self-critical, which does not keep up with things, which does not seek to be more fit, is a sick body. A simple visit to the cemetery might help us see the names of many people who thought they were immortal, immune, and indispensable! It is the disease of those who turn into lords and masters, who think of themselves as above others and not at their service. It is the pathology of power and comes from a superiority complex, from a narcissism which passionately gazes at its own image and does not see the face of others, especially the weakest and those most in need. The antidote to this plague is humility; to say heartily, “I am merely a servant. I have only done what was my duty.”
2. Another disease is excessive busyness. It is found in those who immerse themselves in work and inevitably neglect to “rest a while.” Neglecting needed rest leads to stress and agitation. A time of rest, for those who have completed their work, is necessary, obligatory and should be taken seriously: by spending time with one’s family and respecting holidays as moments for recharging.
3. Then there is the disease of mental and [emotional] “petrification.” It is found in leaders who have a heart of stone, the “stiff-necked;” in those who in the course of time lose their interior serenity, alertness and daring, and hide under a pile of papers, turning into paper pushers and not men and women of compassion. It is dangerous to lose the human sensitivity that enables us to weep with those who weep and to rejoice with those who rejoice! Because as time goes on, our hearts grow hard and become incapable of loving all those around us. Being a humane leader means having the sentiments of humility and unselfishness, of detachment and generosity.
4. The disease of excessive planning and of functionalism. When a leader plans everything down to the last detail and believes that with perfect planning things will fall into place, he or she becomes an accountant or an office manager. Things need to be prepared well, but without ever falling into the temptation of trying to eliminate spontaneity and serendipity, which is always more flexible than any human planning. We contract this disease because it is easy and comfortable to settle in our own sedentary and unchanging ways.
5. The disease of poor coordination. Once leaders lose a sense of community among themselves, the body loses its harmonious functioning and its equilibrium; it then becomes an orchestra that produces noise: its members do not work together and lose the spirit of camaraderie and teamwork. When the foot says to the arm: ‘I don’t need you,’ or the hand says to the head, ‘I’m in charge,’ they create discomfort and parochialism.
6. There is also a sort of “leadership Alzheimer’s disease.” It consists in losing the memory of those who nurtured, mentored and supported us in our own journeys. We see this in those who have lost the memory of their encounters with the great leaders who inspired them; in those who are completely caught up in the present moment, in their passions, whims and obsessions; in those who build walls and routines around themselves, and thus become more and more the slaves of idols carved by their own hands.
7. The disease of rivalry and vainglory. When appearances, our perks, and our titles become the primary object in life, we forget our fundamental duty as leaders—to “do nothing from selfishness or conceit but in humility count others better than ourselves.” [As leaders, we must] look not only to [our] own interests, but also to the interests of others.
8. The disease of existential schizophrenia. This is the disease of those who live a double life, the fruit of that hypocrisy typical of the mediocre and of a progressive emotional emptiness which no [accomplishment or] title can fill. It is a disease which often strikes those who are no longer directly in touch with customers and “ordinary” employees, and restrict themselves to bureaucratic matters, thus losing contact with reality, with concrete people.
9. The disease of gossiping, grumbling, and back-biting.This is a grave illness which begins simply, perhaps even in small talk, and takes over a person, making him become a “sower of weeds” and in many cases, a cold-blooded killer of the good name of colleagues. It is the disease of cowardly persons who lack the courage to speak out directly, but instead speak behind other people’s backs. Let us be on our guard against the terrorism of gossip!
10. The disease of idolizing superiors. This is the disease of those who court their superiors in the hope of gaining their favor. They are victims of careerism and opportunism; they honor persons [rather than the larger mission of the organization]. They think only of what they can get and not of what they should give; small-minded persons, unhappy and inspired only by their own lethal selfishness. Superiors themselves can be affected by this disease, when they try to obtain the submission, loyalty and psychological dependency of their subordinates, but the end result is unhealthy complicity.
11. The disease of indifference to others. This is where each leader thinks only of himself or herself, and loses the sincerity and warmth of [genuine] human relationships. This can happen in many ways: When the most knowledgeable person does not put that knowledge at the service of less knowledgeable colleagues, when you learn something and then keep it to yourself rather than sharing it in a helpful way with others; when out of jealousy or deceit you take joy in seeing others fall instead of helping them up and encouraging them.
12. The disease of a downcast face. You see this disease in those glum and dour persons who think that to be serious you have to put on a face of melancholy and severity, and treat others—especially those we consider our inferiors—with rigor, brusqueness and arrogance. In fact, a show of severity and sterile pessimism are frequently symptoms of fear and insecurity. A leader must make an effort to be courteous, serene, enthusiastic and joyful, a person who transmits joy everywhere he goes. A happy heart radiates an infectious joy: it is immediately evident! So a leader should never lose that joyful, humorous and even self-deprecating spirit which makes people amiable even in difficult situations. How beneficial is a good dose of humor! …
13. The disease of hoarding. This occurs when a leader tries to fill an existential void in his or her heart by accumulating material goods, not out of need but only in order to feel secure. The fact is that we are not able to bring material goods with us when we leave this life, since “the winding sheet does not have pockets” and all our treasures will never be able to fill that void; instead, they will only make it deeper and more demanding. Accumulating goods only burdens and inexorably slows down the journey!
14. The disease of closed circles, where belonging to a clique becomes more powerful than our shared identity. This disease too always begins with good intentions, but with the passing of time it enslaves its members and becomes a cancer which threatens the harmony of the organization and causes immense evil, especially to those we treat as outsiders. “Friendly fire” from our fellow soldiers, is the most insidious danger. It is the evil which strikes from within. As it says in the bible, “Every kingdom divided against itself is laid waste.”
15. Lastly: the disease of extravagance and self-exhibition. This happens when a leader turns his or her service into power, and uses that power for material gain, or to acquire even greater power. This is the disease of persons who insatiably try to accumulate power and to this end are ready to slander, defame and discredit others; who put themselves on display to show that they are more capable than others. This disease does great harm because it leads people to justify the use of any means whatsoever to attain their goal, often in the name of justice and transparency! Here I remember a leader who used to call journalists to tell and invent private and confidential matters involving his colleagues. The only thing he was concerned about was being able to see himself on the front page, since this made him feel powerful and glamorous, while causing great harm to others and to the organization.

Friends, these diseases are a danger for every leader and every organization, and they can strike at the individual and the community levels.

____________________

So, are you a healthy leader? Use the Pope’s inventory of leadership maladies to find out. Ask yourself, on a scale of 1 to 5, to what extent do I . . .

• Feel superior to those who work for me?
• Demonstrate an imbalance between work and other areas of life?
• Substitute formality for true human intimacy?
• Rely too much on plans and not enough on intuition and improvisation?
• Spend too little time breaking silos and building bridges?
• Fail to regularly acknowledge the debt I owe to my mentors and to others?
• Take too much satisfaction in my perks and privileges?
• Isolate myself from customers and first-level employees?
• Denigrate the motives and accomplishments of others?
• Exhibit or encourage undue deference and servility?
• Put my own success ahead of the success of others?
• Fail to cultivate a fun and joy-filled work environment?
• Exhibit selfishness when it comes to sharing rewards and praise?
• Encourage parochialism rather than community?
• Behave in ways that seem egocentric to those around me?

As in all health matters, it’s good to get a second or third opinion. Ask your colleagues to score you on the same fifteen items. Don’t be surprised if they say, “Gee boss, you’re not looking too good today.” Like a battery of medical tests, these questions can help you zero in on opportunities to prevent disease and improve your health. A Papal leadership assessment may seem like a bit of a stretch. But remember: the responsibilities you hold as a leader, and the influence you have over others’ lives, can be profound. Why not turn to the Pope — a spiritual leader of leaders — for wisdom and advice?

Article link: https://hbr.org/2015/04/the-15-diseases-of-leadership-according-to-pope-francis

Gary Hamel is a visiting professor at London Business School and the founder of the Management Lab. He is a coauthor of Humanocracy: Creating Organizations as Amazing as the People Inside Them(Harvard Business Review Press, 2020).

Fears about potential future existential risk are blinding us to the fact AI systems are already hurting people here and now.

October 30, 2023

This is an excerpt from Unmasking AI: My Mission to Protect What Is Human in a World of Machines by Joy Buolamwini, published on October 31 by Random House. It has been lightly edited. 

The term “x-risk” is used as a shorthand for the hypothetical existential risk posed by AI. While my research supports the idea that AI systems should not be integrated into weapons systems because of the lethal dangers, this isn’t because I believe AI systems by themselves pose an existential risk as superintelligent agents. 

AI systems falsely classifying individuals as criminal suspects, robots being used for policing, and self-driving cars with faulty pedestrian tracking systems can already put your life in danger. Sadly, we do not need AI systems to have superintelligence for them to have fatal outcomes for individual lives. Existing AI systems that cause demonstrated harms are more dangerous than hypothetical “sentient” AI systems because they are real. 

Related Story

Joy Buolamwini: “We’re giving AI companies a free pass”

The pioneering AI researcher and activist shares her personal journey in a new book, and explains her concerns about today’s AI systems.

One problem with minimizing existing AI harms by saying hypothetical existential harms are more important is that it shifts the flow of valuable resources and legislative attention. Companies that claim to fear existential risk from AI could show a genuine commitment to safeguarding humanity by not releasing the AI tools they claim could end humanity. 

I am not opposed to preventing the creation of fatal AI systems. Governments concerned with lethal use of AI can adopt the protections long championed by the Campaign to Stop Killer Robots to ban lethal autonomous systems and digital dehumanization. The campaign addresses potentially fatal uses of AI without making the hyperbolic jump that we are on a path to creating sentient systems that will destroy all humankind.

Though it is tempting to view physical violence as the ultimate harm, doing so makes it easy to forget pernicious ways our societies perpetuate structural violence. The Norwegian sociologist Johan Galtung coined this term to describe how institutions and social structures prevent people from meeting their fundamental needs and thus cause harm. Denial of access to health care, housing, and employment through the use of AI perpetuates individual harms and generational scars. AI systems can kill us slowly.

Given what my “Gender Shades” research revealed about algorithmic bias from some of the leading tech companies in the world, my concern is about the immediate problems and emerging vulnerabilities with AI and whether we could address them in ways that would also help create a future where the burdens of AI did not fall disproportionately on the marginalized and vulnerable. AI systems with subpar intelligence that lead to false arrests or wrong diagnoses need to be addressed now. 

When I think of x-risk, I think of the people being harmed now and those who are at risk of harm from AI systems. I think about the risk and reality of being “excoded.” You can be excoded when a hospital uses AI for triage and leaves you without care, or uses a clinical algorithm that precludes you from receiving a life-saving organ transplant. You can be excoded when you are denied a loan based on algorithmic decision-making. You can be excoded when your résumé is automatically screened out and you are denied the opportunity to compete for the remaining jobs that are not replaced by AI systems. You can be excoded when a tenant-screening algorithm denies you access to housing. All of these examples are real. No one is immune from being excoded, and those already marginalized are at greater risk.

This is why my research cannot be confined just to industry insiders, AI researchers, or even well-meaning influencers. Yes, academic conferences are important venues. For many academics, presenting published papers is the capstone of a specific research exploration. For me, presenting “Gender Shades” at New York University was a launching pad. I felt motivated to put my research into action—beyond talking shop with AI practitioners, beyond the academic presentations, beyond private dinners. Reaching academics and industry insiders is simply not enough. We need to make sure everyday people at risk of experiencing AI harms are part of the fight for algorithmic justice.

Read our interview with Joy Buolamwini here. 

Article link: https://www.technologyreview.com/2023/10/30/1082656/focus-on-existing-ai-harms/?

Check out the Five Takeaways from #HIMSS25 to watch #FEHRM Director William Tinston share the latest about the #FederalEHR and what’s ahead in the evolving #healthcare landscape. https://lnkd.in/em_PSw6y.

Article link: https://www.linkedin.com/posts/fehrm_himss25-fehrm-federalehr-activity-7319009213454708736-3Y_N?

AI technologies have had an undeniable impact on the information landscape.

If deployed responsibly, AI can facilitate access to accurate, reliable information, foster free expression, and contribute to healthy and vibrant information ecosystems.

But how do we make this vision a reality?

The UN Security Council took up these questions in a recent informal meeting on “Harnessing safe, inclusive and trustworthy AI for the maintenance of international peace and security.”

One of the focus areas of the meeting was AI’s impact on information integrity.

Two key risks were highlighted:

1. The role of AI in curating and creating information, and people’s growing reliance on this technology to shape their understanding of the world.

At present, tech companies continue to integrate AI into our everyday applications at breakneck speed but genAI tools cannot uniformly be relied on as sources of accurate information.

Ongoing studies and testing show that they frequently do not distinguish between rigorous science on the one hand and dirty data or outright nonsense on the other.

And yet, people are increasingly accessing this convenient and flawed data without being equipped to assess its veracity or reliability. This can contribute to a deeply concerning trend: thelack of trust in any information source and in the information ecosystem more broadly.

People just don’t know what is real and what to believe.

At pivotal societal moments and especially in fragile settings, this can feed into instability and unrest, sometimes with disastrous consequences.

We are, in effect, guinea pigs in an information experiment in which the resilience of our societies is being put to the test.

2. The misuse of AI in facilitating the creation and dissemination of false and hateful information at scale.

This can affect the peace and security of communities and countries.

GenAI tools have made it immeasurably easier for a broad range of actors to spread false claims for financial or strategic gain.

It now costs next to nothing to quickly and easily create a flood of convincing lies and hate, with minor human intervention.

To target specific groups and individuals, actors draw on our personal data that has been sold as part of the advertising supply chain.

Tactics like this can trigger diplomatic crises, incite unrest, and undermine understanding of realities on the ground.

AI-generated deep fake images, audio and video have been weaponized in conflicts from Ukraine, to Gaza, to Sudan.

We’re also now seeing AI used openly and explicitly to generate content designed to undermine social cohesion.

Content that demonises or dehumanises women, refugees, migrants, and minorities.

Content that is anti-Semitic, Islamophobic, racist and xenophobic.

The UN has experienced grave impacts on our missions and priorities.

Peace and humanitarian operations are targeted with false and hateful narratives, negatively affecting the safety and security of UN personnel and making the tasks of implementing their mandates harder.

In turn, this is impacting communities small and large, jeopardizing their stability.

What are some of the concrete steps needed to address these areas of risk?

The UN system has been ramping up efforts.

This includes the 3R approach:

research, risk assessment and response.

Through research we can better understand information risks in complex, multiplatform, multilingual environments.

Our work includes outreach with affected communities to hear and understand their perceptions and concerns.

UN entities then assess the risk posed to UN mandate delivery and thematic priorities.

We use this insight to design appropriate mitigation and response measures.

Most immediately, this means quick and proactive communication to address information voids.

This can be done through our own channels and also via coalitions with community leaders, influencers and civil society groups.

Of course, these efforts require resources, including for skills training and technology.

We also continue to support the efforts of the international community, including through our work to implement the UN Global Principles for Information Integrity.

The Principles provide a framework for action to strengthen the information ecosystem — and are a resource to UN member countries in meeting the commitments agreed in the Global Digital Compact.

The Principles contain recommendations for a wide range of stakeholders around five pillars: societal trust and resilience; healthy incentives; public empowerment; independent, free and pluralistic media; and transparency and research.

The United Nations Global Principles for Information Integrity

Three key recommendation areas are highlighted below.

1. Countries play a central role in shaping information spaces, beginning with obligations to respect, protect and promote human rights, in particular the right to freedom of expression.

This means that regulatory measures to address information integrity comply with applicable international law and are carried out with the full participation of civil society.

Freedom of expression requires not only that people are able to express their views, but that they are able to seek and receive ideas and information of all kinds.

It is exponentially harder to do this when you’re in a polarized, opaque information environment crowded with lies and hate.

In such a landscape, guardrails allow for more free speech, not less, and protect people who feel unsafe in online spaces, giving voice to those otherwise silenced.

Guardrails can help enable inclusive access to information.

And they can support and encourage innovation and help foster public trust in fast-emerging AI technologies.

There is growing awareness among the communities the UN serves that the sooner guardrails are established, the less the risk for us all.

2. Information integrity is not possible without a free, independent and pluralistic media.

Professional journalism requires considerable investment. Yet the advertising-driven business model which long supported independent media has drastically eroded.

AI has prompted new concerns. Quality journalism is being scraped and summarized or used to train AI without permission or compensation.

GenAI search summaries replacing standard search results can reduce web traffic to news sites, further affecting revenue streams.

Many news outlets, particularly at the local level, just can’t compete. For them it is not just a matter of relevance. It is a matter of survival.

The result is news deserts. The vacuum is filled by AI-generated articles or downgraded versions of journalistic content, or malicious content imitating news.

This can be particularly destabilising in conflict and crisis zones.

Responses must therefore urgently support sustainable business models for public interest media.

They must also ensure better protection for media workers, along with researchers, academics and civil society, who are under attack around the world.

3. There is an urgent global need for public empowerment in the information ecosystem, including through measures to boost resilience.

Media and digital literacy skills can allow people to navigate information spaces safely and effectively.

Countries can prioritize the literacy needs of groups in vulnerable and marginalized situations — who are often those most adversely affected by information risks.

They can also undertake literacy efforts around specific problems related to AI, keeping up with new and emerging technologies

Public awareness must improve around online rights, how AI works, and how personal data is used.

The urgency of the challenge requires multi stakeholder coalitions with collaborations between countries to support capacity building and increase global resilience.

This is the only viable path to an information ecosystem in which AI innovation is harnessed for information integrity, human rights and international peace and security.

The text above is adapted from a briefing to the Security Council by Charlotte Scaddan, Senior Adviser on Information Integrity, UN Global Communications

Article link: https://medium.com/we-the-peoples/information-integrity-in-the-ai-age-51494645e765

The Indian Health Service (IHS) is a federally mandated health system providing care to approximately 2.6 million American Indian and Alaska Native people across 574 federally recognized Tribes. Many IHS facilities are in rural or remote areas, often providing the only available healthcare. Unlike other American health systems, IHS is not a benefit or entitlement – it is the product of treaties between the United States and sovereign Tribal Nations.

In FY2023, IHS spent approximately $4,078 per person on care, compared to a national average expenditure of $14,570 per person. Despite challenges, including aging facilities and staffing shortages, IHS delivers remarkable care. Staffed by dedicated Native and allied physicians, nurses, public health workers, and technicians, these facilities are models of resilience. They frequently incorporate traditional healers and language keepers into care delivery, finding innovative ways to meet complex health needs.

Federal decisions, like those discussed by Health Secretary Robert F. Kennedy Jr. during his “Make America Healthy Again” tour, directly impact the resources and policies that shape the IHS and its ability to serve Tribal communities effectively. Additional resources at IHS, combined with the cultivation and empowerment of Indigenous healthcare professionals and Tribal leadership, could be transformative. We hope that the pleas from our tribal members resonated with Secretary Kennedy and leads to meaningful support for IHS.

Read the New York Times article https://bit.ly/RFKonNavajo.

NativesDoingTheirPart #NativeAmericanHealth #IndigenousHealth #NativeAmerican #AmericanIndian #AlaskanNative #HealthyTribes #IndigenousPeople #NativeHealth #TribalHealth

Secretary Kennedy The New York Times CDC Indian Country Today Indian Health Service Johns Hopkins Bloomberg School of Public Health National Indian Health Board National Native News Native News Online Urban Indigenous Collective Urban Indian Health Institute Johns Hopkins Department of International Health

Fuzziness may rule the quantum realm, but it can be manipulated to our advantage.

By Sophia Chen

February 29, 2024

When two black holes spiral inward and collide, they shake the very fabric of space, producing ripples in space-time that can travel for hundreds of millions of light-years. Since 2015, scientists have been observing these so-called gravitational waves to help them study fundamental questions about the cosmos, including the origin of heavy elements such as gold and the rate at which the universe is expanding. 

But detecting gravitational waves isn’t easy. By the time they reach Earth and the twin detectors of the Laser Interferometer Gravitational-Wave Observatory (LIGO), in Louisiana and Washington state, the ripples have dissipated into near silence. LIGO’s detectors must sense motions on the scale of one ten-thousandth the width of a proton to stand a chance. 

LIGO has confirmed 90 gravitational wave detections so far, but physicists want to detect more, which will require making the experiment even more sensitive. And that is a challenge. 

“The struggle of these detectors is that every time you try to improve them, you actually can make things worse, because they are so sensitive,” says Lisa Barsotti, a physicist at the Massachusetts Institute of Technology.

Nevertheless, Barsotti and her colleagues recently pushed past this challenge, creating a device that will allow LIGO’s detectors to detect far more black hole mergers and neutron star collisions. The device belongs to a growing class of instruments that use quantum squeezing—a practical way for researchers dealing with systems that operate by the fuzzy rules of quantum mechanics to manipulate those phenomena to their advantage. 

Physicists describe objects in the quantum realm in terms of probabilities—for example, an electron is not located here or there but has some likelihood of being in each place, locking into one only when its properties are measured. Quantum squeezing can manipulate the probabilities, and researchers are increasingly using it to exert more control over the act of measurement, dramatically improving the precision of quantum sensors like the LIGO experiment.  

“In precision sensing applications where you want to detect super-small signals, quantum squeezing can be a pretty big win,” says Mark Kasevich, a physicist at Stanford University who applies quantum squeezing to make more precise magnetometers, gyroscopes, and clocks with potential applications for navigation. Creators of commercial and military technology have begun dabbling in the technique as well: the Canadian startup Xanadu uses it in its quantum computers, and last fall, DARPA announced Inspired, a program for developing quantum squeezing technology on a chip. Let’s take a look at two applications where quantum squeezing is already being used to push the limits of quantum systems.

Taking control of uncertainty

The key concept behind quantum squeezing is the phenomenon known as Heisenberg’s uncertainty principle. In a quantum-mechanical system, this principle puts a fundamental limit on how precisely you can measure an object’s properties. No matter how good your measurement devices are, they will suffer a fundamental level of imprecision that is part of nature itself. In practice, that means there’s a trade-off. If you want to track a particle’s speed precisely, for example, then you must sacrifice precision in knowing its location, and vice versa. “Physics imposes limits on experiments, and especially on precision measurement,” says John Robinson, a physicist at the quantum computing startup QuEra. 

By “squeezing” uncertainty into properties they aren’t measuring, however, physicists can gain precision in the property they want to measure. Theorists proposed using squeezing in measurement as early as the 1980s. Since then, experimental physicists have been developing the ideas; over the last decade and a half, the results have matured from sprawling tabletop prototypes to practical devices. Now the big question is what applications will benefit. “We’re just understanding what the technology might be,” says Kasevich. “Then hopefully our imagination will grow to help us find what it’s really going to be good for.” 

LIGO is blazing a trail to answer that question, by enhancing the detectors’ ability to measure extremely tiny distances. The observatory registers gravitational waves with L-shaped machines capable of sensing tiny motions along their four-kilometer-long arms. At each machine, researchers split a laser beam in two, sending a beam down each arm to reflect off a set of mirrors. In the absence of a gravitational wave, the crests and troughs of the constituent light waves should completely cancel each other out when the beams are recombined. But when a gravitational wave passes through, it will alternately stretch and compress the arms so that the split light waves are slightly out of phase.

The resulting signals are subtle, though—so subtle that they risk being drowned out by the quantum vacuum, the irremovable background noise of the universe, caused by particles flitting in and out of existence. The quantum vacuum introduces a background flicker of light that enters LIGO’s arms, and this light pushes the mirrors, shifting them on the same scale as the gravitational waves LIGO aims to detect.

Barsotti’s team can’t get rid of this background flicker, but quantum squeezing allows them to exert limited control over it. To do so, the team installed a 300-meter-long cavity in each of LIGO’s two L-shaped detectors. Using lasers, they can create an engineered quantum vacuum, in which they can manipulate conditions to increase their level of control over either how bright the flicker can be or how randomly it occurs in time. Detecting higher-frequency gravitational waves is harder when the rhythm of the flickering is more random, while lower-frequency gravitational waves get drowned out when the background light is brighter. In their engineered vacuum, noisy particles still show up in their measurements, but in ways that don’t do as much to disturb the detection of gravitational waves.“ You can [modify] the vacuum by manipulating it in a way that is useful to you,” she explains. 

The innovation was decades in the making: through the 2010s, LIGO incorporated incrementally more sophisticated forms of quantum squeezing based on theoretical ideas developed in the 1980s. With these latest squeezing innovations, installed last year, the collaboration expects to detect gravitational waves up to 65% more frequently than before.

Quantum squeezing has also improved precision in timekeeping. Working at the University of Colorado Boulder with physicist Jun Ye, a pioneer in atomic clock technology, Robinson and his team made a clock that will lose or gain at most a second in 14 billion years. These super-precise clocks tick slightly differently in different gravitational fields, which could make them useful for sensing how Earth’s mass redistributes itself as a result of seismic or volcanic activity. They could also potentially be used to detect certain proposed forms of dark matter, the hypothesized substance that physicists think permeates the universe, pulling on objects with its gravity. 

The clock Robinson’s team developed, a type called an optical atomic clock, uses 10,000 strontium atoms. Like all atoms, strontium emits light at specific signature frequencies as electrons around the atom’s nucleus jump between different energy levels. A fixed number of crests and troughs in one of these light waves corresponds to a second in their clock. “You’re saying the atom is perfect,” says Robinson. “The atom is my reference.” The “ticking” of this light is far steadier than the vibrating quartz crystal in a wristwatch, for example, which expands and contracts at different temperatures to tick at different rates.

In practice, the tick in the Robinson team’s clock comes not from the light the electrons emit but from how the whole system evolves over time. The researchers first put each strontium atom in a “superposition” of  two states: one in which the atom’s electrons are all at their lowest energy levels and another in which one of the electrons is in an excited state. This means each atom has some probability of being in either state but is not definitively in either one—similar to how a coin flipping in the air has some probability of being either heads or tails, but is neither.

Then they measure how many atoms are in each state. The act of measurement puts the atoms definitively in one state or the other, equivalent to letting the flipping coin land on a surface. Before they measure the atoms, even if they intend to wind up with a 50-50 mixture, they cannot precisely dictate how many atoms will end up in each state. That’s because in addition to the system’s change over time, there is also inherent uncertainty in the state of the individual atoms. Robinson’s team uses quantum squeezing to more reliably determine their final states by reducing these intrinsic fluctuations. Specifically, they manipulate the uncertainties in the direction of each atom’s spin, a property of many quantum particles that has no classical counterpart. Squeezing improved the clock’s precision by a factor of 1.5.

To be sure, gravitational waves and ultra-precise clocks are niche academic applications. But there is interest in adapting the approach to other, potentially more mainstream uses, including quantum computers, navigation, and microscopy.

The increased use of quantum squeezing is part of a wider technological trend toward higher precision—one that encompasses cramming more transistors on chips, studying the universe’s most elusive particles, and clocking the fleeting time it takes for an electron to leave a molecule. Squeezing benefits only measurements so subtle that the randomness of quantum mechanics contributes significant noise. But it turns out that physicists have more control than they think. They may not be able to remove the randomness, but they can engineer where it shows up.

Article link: https://www-technologyreview-com.cdn.ampproject.org/c/s/www.technologyreview.com/2024/02/29/1089092/how-scientists-are-using-quantum-squeezing-to-push-the-limits-of-their-sensors/amp/

Only a year ago, Pat Gelsinger intended for Intel to compete with Nvidia, but the new CEO, Lip-Bu Tan, is burying that ambition.
He does not aim at the Client Computing Group when discussing funding new engineering ideas and innovations.
The Day 1 principle is a start-up philosophy; the new AI strategy after Gaudi has been virtually written off.

And it makes sense. While Intel was playing CEO roulette, Nvidia increased its R&D budget dramatically, and the AI giant did not need to fund other hobby projects like Intel. To catch Nvidia, Intel needed a big bet, which Gaudi never was, and now it is too late.

In 2024, Intel was spending 45% of its operating budget on Intel Foundry, and what was left of the R&D budget was prioritised for the Client Computing Group.

This is insufficient to compete with Nvidia, so the Intel Periscope has turned towards the two smaller vessels involved in XPU designs.

If the New Intel and the 2025 Vision are important to you, read more here: https://lnkd.in/drXUPZXJ

Article link; https://www.linkedin.com/posts/clausaasholm_only-a-year-ago-pat-gelsinger-intended-for-activity-7314629995451064321-RGiv?

The Lip-Bu Tan 2025 Vision of The New Intel

CLAUS AASHOLM

APR 05, 2025

Intel’s new CEO, Lip-Bu Tan, brought many new words that can give us insight into Intel’s strategy. While words don’t easily move supertankers like Intel, words are important, and all strategy and transformation begin with words. 

Under Pat Gelsinger, there were several strategic pivots. While he was the master of creative external CapEx deals to fund Intel Foundry, the strategy was retrenched to a focus on the x86 franchise and the Client Computing group under Michelle Johnston Holthaus as she protected her CCG tribe.

The appointment of Lip-Bu Tan does not continue the current Intel Strategy, and it is vital to understand what the new CEO brings to the party. You can think what you want about Intel, but it still powers most of the world’s computing even though it is struggling to do so profitably. Intel is still a key factor in the development of the semiconductor industry.

In his first keynote, Lip-Bu Tan directly discussed not being happy with where Intel is. While this is not the same as signalling a strategic change, his statements about ruthlessly focusing on the truth are. This is an acceptance of the Intel culture of “glossing over” rather than facing the brutal truths. 

The most radical idea that Lip-Bu Tan introduced was that Intel needs to adopt a day 1 mentality. While it sounds like more words, the implications of a “Day 1 mentality” can mean dramatic change for the Intel organisation and culture, as Intel is currently the poster child of a “Day 2” company.

Amazon’s “Day One”: A Philosophy of Perpetual Innovation

At the heart of Amazon’s relentless drive lies its “Day One” philosophy, which emphasises maintaining a startup-like mentality, even within a massive corporation. Introduced by Jeff Bezos in his 1997 shareholder letter, “Day One” is about constant curiosity, nimbleness, and a willingness to experiment. It’s the antithesis of “Day Two,” where complacency sets in, and companies become slow, bureaucratic, and less customer-focused.

Bezos understood that customer obsession is the key to maintaining “Day One” vitality. As customers always seek better, faster, and more innovative solutions, companies must continuously adapt and evolve to meet their needs. As tech reporter Alex Kantrowitz explains in his book “Always Day One,” this philosophy encourages building for the future, not resting on past achievements. It means treating even a sprawling enterprise like a small, agile startup, fostering a culture of innovation and a relentless focus on the customer. In essence, “Day One” is about embracing change, challenging the status quo, and always acting with the urgency and passion of a company just starting.

Lip-Bu keeps flirting with “Day 1” as he calls for customer focus.

“My number one priority was spending time with customers, and they gave me honest feedback. We have a lot of work ahead of us.” 

While talking to customers and getting feedback is essential, it is not the same as becoming customer-centric. A company like Apple is customer-centric and has delivered the most successful consumer product ever, even though nobody ever asked for an iPhone.

Talking to customers isn’t enough; you’ll need to understand their future situation and latent needs. This is not Intel’s traditional starting point, as the company has its cultural roots deep in products and technology, and Lip-Bu Tan wants to challenge that.

When discussing his background, Lip-Bu Tan highlights times when he successfully changed the culture of the companies he was involved with. This signals that cultural change is high on the agenda, and he believes he is the right person to drive it.

This also reveals that Lip-Bu Tan believes that corporate culture can be designed within a reasonable timeframe, which sounds more straightforward than it is. 

Anybody who has worked in a company with several acquisitions knows that the acquired culture does not die. It lives on in the acquired people and influences the existing culture. 

One of the key features of a large corporation is what I call management-dampening. All top management decisions must pass through the hierarchical layers in a sense-making process. The simple CEO decisions have complex impacts on the organisation, and each level needs to adapt to them in a way that makes sense for that layer and their internal and external customers. 

The company’s suspension system ensures that radical changes don’t break the system and prevent change from happening. 

I have seen several examples of how the management-dampening system has saved (prevented) senior management’s panic-flip-flopping decisions. Unfortunately, it also dampens necessary changes and complicates cultural change. 

An overview of the “Day One” philosophy can be seen below, along with the key message.

As Lip-Bu Tan talks about the truth, it is necessary to highlight Intel’s current approach to these principles. While you might disagree with my assessment, it is hard to characterise Intel as anywhere near a “Day One” company.

Lip-Bu Tan recognises this as he states that he is unhappy with things and wants to correct past mistakes. He wants to create the “New Intel”, an engineering-first company, suggesting that Intel is currently a “something else-first” company. 

The truth

The foundation of everything I do starts with a data-driven assessment of the current situation. While I know it is easier to sell happy data, it is a principle of mine to stay clear of liking and disliking data. 

There can be many truths, and data does respond to torture. Still, it is always in the company’s best interest to begin the strategic formulation based on an honest assessment of the situation.

Happy data rarely creates happy endings.

Intel is undoubtedly in a difficult spot, and Lip-Bu Tan’s ambitions are at odds with reality. Scarcity creates tension between different objectives but is innovation’s greatest friend. 

As I pointed out in The Corporate Sandwich Model, Intel externally has a problem with the truth. It is all rah-rah and up and to the right. This might be different internally, but I doubt it.

It’s time to explore the tension between reality and the Intel 2025 Vision. 

The x86 Franchise

The most apparent change in strategy from Gelsinger to Holthaus was the introduction of the “x86 franchise”. That the Queen of the Client Computing Group praised the holiest of relics in CCG was unsurprising and looked like the usual internal divisional power game. Lip-Bu Tan adds colour to the x86 concept and positions it more as an asset in the Foundry strategy as part of the differentiation strategy: IP and localisation. The x86 is also positioned in the future AI strategy built on the smoulders of Gaudi that Holthaus effectively killed by calling it “difficult” to use

The Innovation perspective.

Intel is trying to reduce operating costs to achieve positive cash flow. However, it is becoming evident that Intel’s current AI data centre product strategy is non-existent. 

In the 2024 Intel Vision keynote, Pat Gelsigner called Gaudi the only alternative to Nvidia H100 in data centre learning applications. As Michelle Johnston Holthaus took the co-CEO chair, she demonstrated the rift between the client group and the other divisions and called Gaudi challenging. This was a virtual death sentence, and Lip-Bu Tan is not reviving the architecture. 

He clearly stated that Intel is unhappy with its current position and needs to develop a new approach. Intel needs higher performance, lower power and lower cost. In one year, The focus shifted from competing with Nvidia to competing with Broadcom and Marvell in XPU accelerators in inference rather than Nvidia in Learning and inference. 

This also correlates with his statements about becoming an outside-in company that starts with the customer’s problem rather than the hardware in the old inside-out approach. 

Nvidia’s AI strategy builds on a well-executed product strategy, while Broadcom and Marvell are working on customer problems. Customers are telling Li-Bu Tan that they would love an Intel AI Platform (cheaper, faster, lower power—surprise!), and he has started deep-dive dives into reimagining the products. 

This is the key area to win as AI accelerates tech adoption and is the only driver behind the upturn. Other focus areas, such as photonics, quantum, and robotics, are on the agenda, but the data centre AI needs to be revived.

The competitive situation in AI is hopeless for Intel.

Excluding Nvidia revenue makes it even more embarrassing for Intel. For now, it is evident that Intel is not aiming at Nvidia and learning but at the competitors in sight that are focusing on the accelerator designs.

Article link; https://clausaasholm.substack.com/p/from-a-something-else-first-to-an?

What the firm found challenges some basic assumptions about how this technology really works.

By Will Douglas Heaven

March 27, 2025

The AI firm Anthropic has developed a way to peer inside a large language model and watch what it does as it comes up with a response, revealing key new insights into how the technology works. The takeaway: LLMs are even stranger than we thought.

The Anthropic team was surprised by some of the counterintuitive workarounds that large language models appear to use to complete sentences, solve simple math problems, suppress hallucinations, and more, says Joshua Batson, a research scientist at the company.

It’s no secret that large language models work in mysterious ways. Few—if any—mass-market technologies have ever been so little understood. That makes figuring out what makes them tick one of the biggest open challenges in science.

But it’s not just about curiosity. Shedding some light on how these models work exposes their weaknesses, revealing why they make stuff up and why they can be tricked into going off the rails. It helps resolve deep disputes about exactly what these models can and can’t do. And it shows how trustworthy (or not) they really are.

Batson and his colleagues describe their new work in two reports published today. The first presents Anthropic’s use of a technique called circuit tracing, which lets researchers track the decision-making processes inside a large language model step by step. Anthropic used circuit tracing to watch its LLM Claude 3.5 Haiku carry out various tasks. The second (titled “On the Biology of a Large Language Model”) details what the team discovered when it looked at 10 tasks in particular.

“I think this is really cool work,” says Jack Merullo, who studies large language models at Brown University in Providence, Rhode Island, and was not involved in the research. “It’s a really nice step forward in terms of methods.”

Circuit tracing is not itself new. Last year Merullo and his colleagues analyzed a specific circuit in a version of OpenAI’s GPT-2, an older large language model that OpenAI released in 2019. But Anthropic has now analyzed a number of different circuits inside a far larger and far more complex model as it carries out multiple tasks. “Anthropic is very capable at applying scale to a problem,” says Merullo.

Eden Biran, who studies large language models at Tel Aviv University, agrees. “Finding circuits in a large state-of-the-art model such as Claude is a nontrivial engineering feat,” he says. “And it shows that circuits scale up and might be a good way forward for interpreting language models.”

Circuits chain together different parts—or components—of a model. Last year, Anthropic identified certain components inside Claudethat correspond to real-world concepts. Some were specific, such as “Michael Jordan” or “greenness”; others were more vague, such as “conflict between individuals.” One component appeared to represent the Golden Gate Bridge. Anthropic researchers found that if they turned up the dial on this component, Claude could be made to self-identify not as a large language model but as the physical bridge itself.

The latest work builds on that research and the work of others, including Google DeepMind, to reveal some of the connections between individual components. Chains of components are the pathways between the words put into Claude and the words that come out.  

“It’s tip-of-the-iceberg stuff. Maybe we’re looking at a few percent of what’s going on,” says Batson. “But that’s already enough to see incredible structure.”

Growing LLMs

Researchers at Anthropic and elsewhere are studying large language models as if they were natural phenomena rather than human-built software. That’s because the models are trained, not programmed.

“They almost grow organically,” says Batson. “They start out totally random. Then you train them on all this data and they go from producing gibberish to being able to speak different languages and write software and fold proteins. There are insane things that these models learn to do, but we don’t know how that happened because we didn’t go in there and set the knobs.”

Sure, it’s all math. But it’s not math that we can follow. “Open up a large language model and all you will see is billions of numbers—the parameters,” says Batson. “It’s not illuminating.”

Anthropic says it was inspired by brain-scan techniques used in neuroscience to build what the firm describes as a kind of microscope that can be pointed at different parts of a model while it runs. The technique highlights components that are active at different times. Researchers can then zoom in on different components and record when they are and are not active.

Take the component that corresponds to the Golden Gate Bridge. It turns on when Claude is shown text that names or describes the bridge or even text related to the bridge, such as “San Francisco” or “Alcatraz.” It’s off otherwise.

Yet another component might correspond to the idea of “smallness”: “We look through tens of millions of texts and see it’s on for the word ‘small,’ it’s on for the word ‘tiny,’ it’s on for the French word ‘petit,’ it’s on for words related to smallness, things that are itty-bitty, like thimbles—you know, just small stuff,” says Batson.

Having identified individual components, Anthropic then follows the trail inside the model as different components get chained together. The researchers start at the end, with the component or components that led to the final response Claude gives to a query. Batson and his team then trace that chain backwards.

Odd behavior

So: What did they find? Anthropic looked at 10 different behaviors in Claude. One involved the use of different languages. Does Claude have a part that speaks French and another part that speaks Chinese, and so on?

The team found that Claude used components independent of any language to answer a question or solve a problem and then picked a specific language when it replied. Ask it “What is the opposite of small?” in English, French, and Chinese and Claude will first use the language-neutral components related to “smallness” and “opposites” to come up with an answer. Only then will it pick a specific language in which to reply. This suggests that large language models can learn things in one language and apply them in other languages.

Anthropic also looked at how Claude solved simple math problems. The team found that the model seems to have developed its own internal strategies that are unlike those it will have seen in its training data. Ask Claude to add 36 and 59 and the model will go through a series of odd steps, including first adding a selection of approximate values (add 40ish and 60ish, add 57ish and 36ish). Towards the end of its process, it comes up with the value 92ish. Meanwhile, another sequence of steps focuses on the last digits, 6 and 9, and determines that the answer must end in a 5. Putting that together with 92ish gives the correct answer of 95.

And yet if you then ask Claude how it worked that out, it will say something like: “I added the ones (6+9=15), carried the 1, then added the 10s (3+5+1=9), resulting in 95.” In other words, it gives you a common approach found everywhere online rather than what it actually did. Yep! LLMs are weird. (And not to be trusted.)

This is clear evidence that large language models will give reasons for what they do that do not necessarily reflect what they actually did. But this is true for people too, says Batson: “You ask somebody, ‘Why did you do that?’ And they’re like, ‘Um, I guess it’s because I was— .’ You know, maybe not. Maybe they were just hungry and that’s why they did it.”

Biran thinks this finding is especially interesting. Many researchers study the behavior of large language models by asking them to explain their actions. But that might be a risky approach, he says: “As models continue getting stronger, they must be equipped with better guardrails. I believe—and this work also shows—that relying only on model outputs is not enough.”

A third task that Anthropic studied was writing poems. The researchers wanted to know if the model really did just wing it, predicting one word at a time. Instead they found that Claude somehow looked ahead, picking the word at the end of the next line several words in advance.  

For example, when Claude was given the prompt “A rhyming couplet: He saw a carrot and had to grab it,” the model responded, “His hunger was like a starving rabbit.” But using their microscope, they saw that Claude had already hit upon the word “rabbit” when it was processing “grab it.” It then seemed to write the next line with that ending already in place.

This might sound like a tiny detail. But it goes against the common assumption that large language models always work by picking one word at a time in sequence. “The planning thing in poems blew me away,” says Batson. “Instead of at the very last minute trying to make the rhyme make sense, it knows where it’s going.”

“I thought that was cool,” says Merullo. “One of the joys of working in the field is moments like that. There’s been maybe small bits of evidence pointing toward the ability of models to plan ahead, but it’s been a big open question to what extent they do.”

Anthropic then confirmed its observation by turning off the placeholder component for “rabbitness.” Claude responded with “His hunger was a powerful habit.” And when the team replaced “rabbitness” with “greenness,”Claude responded with “freeing it from the garden’s green.”

Anthropic also explored why Claude sometimes made stuff up, a phenomenon known as hallucination. “Hallucination is the most natural thing in the world for these models, given how they’re just trained to give possible completions,” says Batson. “The real question is, ‘How in God’s name could you ever make it not do that?’”

The latest generation of large language models, like Claude 3.5 and Gemini and GPT-4o, hallucinate far less than previous versions, thanks to extensive post-training (the steps that take an LLM trained on text scraped from most of the internet and turn it into a usable chatbot). But Batson’s team was surprised to find that this post-training seems to have made Claude refuse to speculate as a default behavior. When it did respond with false information, it was because some other component had overridden the “don’t speculate” component.

This seemed to happen most often when the speculation involved a celebrity or other well-known entity. It’s as if the amount of information available on a subject pushed the speculation through, despite the default setting. When Anthropic overrode the “don’t speculate” component to test this, Claude produced lots of false statements about individuals, including claiming that Batson was famous for inventing the Batson principle (he isn’t).

Still unclear

Because we know so little about large language models, any new insight is a big step forward. “A deep understanding of how these models work under the hood would allow us to design and train models that are much better and stronger,” says Biran.

But Batson notes there are still serious limitations. “It’s a misconception that we’ve found all the components of the model or, like, a God’s-eye view,” he says. “Some things are in focus, but other things are still unclear—a distortion of the microscope.”

And it takes several hours for a human researcher to trace the responses to even very short prompts. What’s more, these models can do a remarkable number of different things, and Anthropic has so far looked at only 10 of them.

Batson also says there are big questions that this approach won’t answer. Circuit tracing can be used to peer at the structures inside a large language model, but it won’t tell you how or why those structures formed during training. “That’s a profound question that we don’t address at all in this work,” he says.

But Batson does see this as the start of a new era in which it is possible, at last, to find real evidence for how these models work: “We don’t have to be, like: ‘Are they thinking? Are they reasoning? Are they dreaming? Are they memorizing?’ Those are all analogies. But if we can literally see step by step what a model is doing, maybe now we don’t need analogies.”

Article link: https://www-technologyreview-com.cdn.ampproject.org/c/s/www.technologyreview.com/2025/03/27/1113916/anthropic-can-now-track-the-bizarre-inner-workings-of-a-large-language-model/amp/