In-Context Operator Networks (ICONs) are models that learn operators across different types of PDEs using a few-shot, in-context approach. Although they show successful generalization to various PDEs, existing methods treat each data point as a single token, and suffer from computational inefficiency when processing dense data, limiting their application in higher spatial dimensions. In this work, we propose Vision In-Context Operator Networks (VICON), incorporating a vision transformer architecture that efficiently processes 2D functions through patch-wise operations. We evaluated our method on three fluid dynamics datasets, demonstrating both superior performance (reducing scaled $L^2$ error by $40\%$ and $61.6\%$ for two benchmark datasets for compressible flows, respectively) and computational efficiency (requiring only one-third of the inference time per frame) in long-term rollout predictions compared to the current state-of-the-art sequence-to-sequence model with fixed timestep prediction: Multiple Physics Pretraining (MPP). Compared to MPP, our method preserves the benefits of in-context operator learning, enabling flexible context formation when dealing with insufficient frame counts or varying timestep values.

Event cameras that asynchronously output low-latency event streams provide great opportunities for state estimation under challenging situations. Despite event-based visual odometry having been extensively studied in recent years, most of them are based on monocular and few research on stereo event vision. In this paper, we present ESVIO, the first event-based stereo visual-inertial odometry, which leverages the complementary advantages of event streams, standard images and inertial measurements. Our proposed pipeline achieves temporal tracking and instantaneous matching between consecutive stereo event streams, thereby obtaining robust state estimation. In addition, the motion compensation method is designed to emphasize the edge of scenes by warping each event to reference moments with IMU and ESVIO back-end. We validate that both ESIO (purely event-based) and ESVIO (event with image-aided) have superior performance compared with other image-based and event-based baseline methods on public and self-collected datasets. Furthermore, we use our pipeline to perform onboard quadrotor flights under low-light environments. A real-world large-scale experiment is also conducted to demonstrate long-term effectiveness. We highlight that this work is a real-time, accurate system that is aimed at robust state estimation under challenging environments.

You might know motion capture as the tech that transformed Andy Serkis into Gollum, but now it can transform everyday people into animated avatars in virtual worlds, and all in real-time. Motion capture--which uses body sensors, ultra-precise cameras, and modeling software to create 3D animations from real-life human movement--is now taking on location-based virtual reality, or LBVR. PCWorld visited a leading motion capture company called Vicon in Oxford, England to learn how mocap has evolved to take on this new frontier in entertainment. If you've watched behind-the-scenes footage of how motion capture (or mocap) works, you've probably seen actors in skintight lycra suits covered with golf ball-sized sensors. Normally, dozens of infrared cameras track these sensors to model an actor's movements.

Modern motion-capture systems are the product of a century of tinkering, innovation and computational advances. Mocap was born a lifetime before Gollum hit the big screen in The Lord of the Rings, and ages before the Cold War, Vietnam War or World War II. It was 1915, in the midst of the First World War, when animator Max Fleischer developed a technique called rotoscoping and laid the foundation for today's cutting-edge mocap technology. Rotoscoping was a primitive and time-consuming process, but it was a necessary starting point for the industry. In the rotoscope method, animators stood at a glass-topped desk and traced over a projected live-action film frame-by-frame, copying actors' or animals' actions directly onto a hand-drawn world.

Epic Games has been obsessed with real-time motion capture for years, but the company is now trying to take its experiments with the technology one step further. Enter "Siren," a digital personality that it created alongside a few prominent firms in the gaming industry: Vicon, Cubic Motion, 3Lateral and Tencent (which just became a major investor in Ubisoft). The crazy thing about Siren is that she comes to life using live mocap tech, powered by software from Vicon, that can make her body and finger movements be captured and live-streamed into an Unreal Engine project. Back in 2016, Epic Games teased the live motion-capture technology first used for Hellblade, which was stunning and showed the potential of the tech. With this new iteration, though, the company says it hopes to take "live-captured digital humans to the next level."

Epic Games, CubicMotion, 3Lateral, Tencent, and Vicon took a big step toward creating believable digital humans today with the debut of Siren, a demo of a woman rendered in real-time using Epic's Unreal Engine 4 technology. The move is a step toward transforming both films and games using digital humans who look and act like the real thing. The tech, shown off at Epic's event at the Game Developers Conference in San Francisco, is available for licensing for game or film makers. Cubic Motion's computer vision technology empowered producers to conveniently and instantaneously create digital facial animation, saving the time and cost of digitally animating it by hand. "Everything you saw was running in the Unreal Engine at 60 frames per second," said Epic Games chief technology officer Kim Libreri, during a press briefing on Wednesday morning at GDC. "Creating believable digital characters that you can interact with and direct in real-time is one of the most exciting things that has happened in the computer graphics industry in recent years."