Missing sense: Why physical AI can see everything and feel nothing

Reach into your pocket and find your keys without looking. You just did something no robot can reliably do. You couldn’t see into your pocket, yet your fingers sorted keys from coins and lint, registered the weight and the cut edges, and closed around the right object. That was touch, quietly doing the kind of work that sight simply cannot.

Artificial intelligence (AI) has learned to see. It has learned to listen. Increasingly, it can reason, plan, converse, and generate sophisticated solutions to complex problems. Yet despite astonishing advances in foundational models and robotic computer vision, the next generation of intelligent machines remains constrained by a surprisingly simple limitation: ask a robot to pick up a paper cup, a grape, or a crumpled receipt, and the illusion breaks. It hesitates. It crushes the cup or drops the grape. The brain got smart far faster than the hands did.

This limitation isn’t because robots lack intelligence. It’s because they lack touch. For decades, AI has largely been about perception—understanding the world through images, language, and sound. Physical AI systems—that must operate and interact directly in the real world—reshape the challenge entirely. Instead of simply interpreting the world, intelligent machines must interact with it. They must grasp, manipulate, assemble, lift, carry, and collaborate efficiently and safely alongside people.

The gating modality for physical AI, therefore, is no longer being able to see. It’s being able to feel. Vision gave robots a view of the world. Touch, or multimodal sensing, is what will finally let them reach into it.

Figure 1 Physical AI doesn’t stop at perception. Multimodal sensing integrates vision, touch, and edge AI to enable robots to understand, adapt to, and safely interact with the physical world in real time. Source: Synaptics

The shift from perception to interaction

The last decade has been dominated by extraordinary progress in perception technologies. Computer vision systems now recognize objects with remarkable accuracy. Large language models (LLMs) can break down complicated tasks into logical sequences of actions. Robots increasingly understand what they are looking at.

Seeing and understanding, however, is not the same as taking action. A warehouse robot may correctly identify hundreds of products on a shelf yet struggle to remove a flexible package without crushing it. A humanoid robot may recognize a wine glass instantly but fail to grasp it securely and with the carefulness required of a fragile object.

To understand the shift from perception to interaction, it helps to contrast two flavors of autonomy by comparing autonomous vehicles with robotic manipulation. A self-driving car spends nearly all its effort avoiding contact. For an autonomous vehicle, touching something usually represents failure.

A robotic hand exists for precisely the opposite purpose. Its job is to make contact deliberately, continuously, and intelligently. Every successful grasp depends not simply on locating an object, but on understanding how that object responds when the instant contact is made. That information cannot be seen. It must be felt.

Vision plans, touch executes

Researchers increasingly describe robotic manipulation with a simple phrase: vision is for planning, and touch is for execution. Vision excels at global understanding. Cameras determine where objects are located, estimate pose, classify materials, and plan trajectories. They provide the strategic overview necessary for intelligent action.

But the moment robotic digits close around an object, vision begins to fail. The contact point disappears behind the hand itself. Cameras cannot observe friction. They cannot determine whether a paper cup is slightly damp, whether a cardboard box is heavier than expected, or whether a glass has begun slipping between two fingertips.

These are not visual problems. They are tactile problems. Humans solve them effortlessly. We tighten our grip on a slippery glass before it falls. We pick up an egg and instinctively apply exactly enough force—not too little, not too much. We rarely think about these abilities because our nervous system performs them automatically. For robots, they remain among the hardest problems in engineering.

Figure 2 Touch happens fast and intelligence happens faster. In just 70 milliseconds, multimodal touch sensing and edge AI transform raw tactile signals into intelligent grip control for safer, more dexterous robots. Source: Synaptics

The next robotics frontier: The science of the “slip”

Picture a condensation-slicked glass of water in your hand, eyes closed. As it begins to slide, you don’t need to see it to react. Your fingertips pick up a faint change in vibration and pressure, and your brain issues a “smart squeeze”—just enough extra force to arrest the slide without shattering the glass. The whole loop, from sensation to correction, runs below the level of conscious thought.

That loop is one of the hardest things to reproduce in a robot. Machines can compute trajectories, yet they remain clumsy about the feel of friction. The traditional answer was pre-programmed rigidity: apply a fixed grip force and hope the object—steel or sponge—cooperates.

Today’s physical AI system developers, however, look for something more subtle: the moment, invisible to the eye, when a stable hold begins to come apart. Contrary to intuition, objects rarely transition instantly from being secure to being dropped. Slip begins gradually.

At microscopic scales, every surface consists of tiny peaks and valleys. As force builds across a contact patch, the outer regions begin slipping while the center continues gripping. Engineers call this incipient slip, and detecting it early transforms robotic control. Instead of reacting after failure, robots can intervene before failure occurs.

This failure intervention requires two complementary sensing systems that mirror the layered design of human skin. A slower channel continuously measures pressure, shear forces, and load distribution across the fingertip. A much faster channel listens to high-frequency vibrations that signal the earliest stages of slipping. Together they create something remarkably similar to the layered sensing architecture found in human skin. One channel feels pressure, the other hears friction.

Only by combining both can a robot understand what’s actually happening at the point of contact.

Figure 3 Physical AI is a continuous feedback loop, not a one-time decision. True physical AI is possible when vision, touch, and edge intelligence work together in a continuous cycle of sensing, thinking, acting, and adapting. Source: Synaptics

Why touch changes everything

A useful multimodal sensing system must answer four fundamental questions.

First, has contact actually occurred? Millimeter-scale positioning errors matter. A robot must know the precise instant it touches an object rather than empty space.

Second, how much force is being applied? Picking up a steel bracket requires entirely different forces than handling fruit, laboratory samples, or medical devices.

Third, is the object beginning to slip? Perhaps the most important capability of all is detecting incipient slip—the microscopic changes that occur before an object falls.

Fourth, how does it feel? While the internet contains unlimited visual data, it contains almost no tactile experience data. No image can tell an AI application what wet glass feels like, how fabric stretches, or how friction changes as an object rotates within a grasp. The only way to learn these interactions is through physical contact. Touch provides ground truth: rich, labeled physical data about what real contact feels like, which is precisely what physical AI systems require.

Intelligence belongs at the edge

Touch also changes where intelligence must live. Unlike language models, tactile decisions cannot wait hundreds of milliseconds for cloud processing or centralized computation. If a glass begins slipping, every millisecond matters. Human grip corrections occur in roughly 70 milliseconds. A robot that must transmit raw sensor data to a central processor before deciding how to respond has already lost valuable time.

Instead, tactile intelligence must move to the edge. Each fingertip becomes an intelligent sensing node, interpreting pressure, force, and vibration locally before sending only meaningful events to higher-level control systems. This architecture mirrors biology: individual nerve endings process local information before communicating with the brain. The brain receives distilled information about events that matter rather than monitoring every sensory receptor continuously.

Physical AI increasingly requires the same distributed architecture. Vision determines the objective. Local intelligence manages contact. Higher-level AI coordinates the task. Touch increases the likelihood that each task is executed successfully.

Physical AI architecture: Reliability is more important than resolution

Much of today’s discussion around robotic touch as the path to dexterous machines focuses on sensor resolution. Resolution matters, but reliability matters more. Building an impressive laboratory demonstration that executes large numbers of touch events is relatively straightforward. Building a tactile sensor that performs accurately after millions of grasps in factories, warehouses, and hospitals is vastly more difficult.

Every tactile technology has weaknesses. Soft materials wear. Temperature changes electrical characteristics. Motor noise contaminates tiny signals. Elastomers gradually develop hysteresis, remembering previous deformations instead of returning instantly to baseline.

Optical sensors produce exceptionally rich data but require bulky camera systems. Magnetic sensing performs beautifully until exposed to external magnetic fields. Piezoelectric materials detect vibration extraordinarily well but cannot measure static force.

The challenge is not finding a perfect sensing modality. The challenge is engineering reliable systems that compensate for imperfections while surviving years of real-world use. Success requires expertise that extends well beyond robotics to materials science, signal processing, embedded intelligence, silicon design and manufacturing, firmware, and systems engineering, all working together as a single discipline. Ultimately, reliability—not raw sensitivity—is what determines whether physical AI scales beyond research laboratories.

Why this moment matters

The urgency surrounding physical AI development is driven by timing. In industrial applications, for example, humanoid robots are moving from prototypes toward mass commercial deployment according to industry research analysts. Conservative estimates from Goldman Sachs Research predict 1.4 million active humanoids will be working in global manufacturing, warehouses, and logistics by 2035. Morgan Stanley projects an even higher trajectory, citing massive production scale and adoption of 13 million service robots working along humans in the same timeframe.

The opportunity extends even further. Prosthetic limbs require tactile feedback to restore natural function. Data collection systems need tactile sensing to train future manipulation models.

Large-area robotic skin improves safe collaboration between humans and machines. Every application depends upon extracting trustworthy physical information from an inherently noisy world, and regardless of form factor, every Physical AI system eventually encounters the same bottleneck: hands or more precisely, contact.

Whether a machine uses five-fingered humanoid hands, industrial grippers, or specialized end effectors, it must regulate force, detect slip, and manipulate objects safely. Touch, therefore, becomes a foundational capability rather than an optional enhancement.

From digital to physical intelligence: Relying on touch for physical AI

History suggests that every major computing revolution has been enabled by a new interface. The graphical user interface transformed personal computing. The touchscreen transformed mobile computing. Touch now appears poised to transform robotics.

Just as computer vision unlocked autonomous perception, tactile intelligence will unlock autonomous manipulation. The companies that succeed in this technology space will not simply build better sensors. They will build complete sensing systems that combine advanced materials, embedded intelligence, signal processing, edge AI, robust manufacturing, and decades of experience extracting reliable information from imperfect physical environments. That combination—not any single breakthrough—will define the next generation of physical AI.

Physical AI is often described as giving machines a brain. In reality, we have spent the past decade building remarkably capable brains. What robots still lack is a nervous system: the ability to feel force, sense friction, recognize instability before failure, and transform contact into understanding.

The future of intelligent machines will not be determined solely by larger models or more powerful processors. It will be determined by whether those machines can interact with the physical world with the same confidence that humans have.

Vision gave AI the ability to understand the world. Touch will give it the ability to change it.

Satish Ganesan joined Synaptics in November 2019 and serves as senior VP and GM of Edge Interface & Sensing Division and Chief Strategy Officer. Prior to Synaptics, Satish served as chief product officer of Keyssa, a wireless startup focused on short-range connectivity. He also held several executive positions at Broadcom and Xilinx.

Source link