Skip to main content

3D News

Dear 3D Vision Fans, We have enjoyed sharing 3D Vision content here over the past eight years, and seeing the fantastic images and videos you’ve uploaded. Thanks for making this such a fun destination for 3D enthusiasts. Usage of this site has been declining, and therefore, we are going to transition the 3D photo function of this site over to The last day that you will be able to access photos is Aug. 1, 2018. In addition, the 3D video...

Recent Blog Entries

San Francisco’s Museum of Modern Art wants to bring you closer to paintings. How close? Literally inside.

The iconic museum is running an exhibit of surrealist Rene Magritte that also offers visitors an augmented reality trip into his provocative, often jarring, art.

People can step in front of AR pieces inspired by the artist to enter and interact with famous interpretations of works such as “Shéhérazade,” which depicts an exotic woman behind pearls in the sky — or a viewer taking her place.

AR inspired by Magritte’s “Shéhérazade,” 1950.

The monitors are designed as window frames reminiscent of one of Magritte’s most famous paintings, “Where Euclid Walked,” which shows an easel painting in front of a window. The original painting begs a contemplation of what is real versus what is perceived — much like AR.

Created with frog design, a global design firm based in San Francisco, the interactive portion of the show uses these traditionally appearing window frames packing NVIDIA GPU computing power and front-facing cameras for visitors to peer into and interact with the artist’s world.

AR inspired by “Le Blanc Seeing,” 1965. Play in the Artwork

The first AR stop, inspired by “Le Blanc Seing,” displays a children’s storybook-like forest scene in which visitors have to position themselves slowly and carefully for the cameras and sensors to allow them into the forest image and be seen. The intent is to force visitors to slow down and really study the interaction to fit in visibly between trees.

Some windows present Magritte-like puzzles for visitors to solve. In one, people’s images were not fed into the monitor they were facing but instead an adjacent one, causing confusion and interactions with fellow museum visitors to figure it out. Sometimes people would ask the stranger on the other end for a photo. Voila, just the type of interactivity that was aimed for.

Images of visitors appear in unexpected locations.

“The paintings are actually looking back and seeing the people — they are turned from visitors to participants,” said Charles Yust, a principal design technologist at frog.

The exhibit itself spills across eight different rooms in the recently expanded downtown museum. Among the more than 70 works of art are some of the Belgian artist’s best known pieces, which show his highly idiosyncratic perspective, such as the painting “Personal Values,” from 1952, which depicts ordinary objects — like a comb and a glass — claustrophobically oversized into a bedroom. Also on display were examples of his famous bowler hat paintings, such as the “Happy Donor,” from 1966, and “The Son of Man,” from 1964.

While the show is a big one for the museum, what sets it apart is its use of AR — which harnesses StereoLabs Zed Mini AR cameras for video and depth perception and NVIDIA Jetson TX2 for processing the real-time interactions.

Jetson for Museum Pieces

Frog design set out to build the interactive monitors as pieces of artwork themselves. The global design firm worked closely with the museum’s staff to conceptualize these AR displays.

The AR exhibit was designed to deepen understanding of Magritte’s work and get visitors to interact with one another in moments of confusion — even consternation — and reflection at times, in the spirit of Magritte’s art.

“We know that playful interactive experiences can create a strong connection between visitors and the subject matter,” said Chad Coerver, chief content officer at the SFMOMA. “We have been working really hard to think about how tech can advance the museum’s mission and connect with the Bay Area around us.”

The show, which runs at the SFMOMA through Oct. 28, includes more than 20 pieces by the Belgian artist that have never previously been shown in the U.S.

The post Augmented Reality Lets You Slip Inside Magritte’s Surreal Scenes at SFMOMA appeared first on The Official NVIDIA Blog.

Three pioneering research teams supported by NVIDIA AI Labs are presenting key findings this week at the International Conference on Machine Learning, a major AI show taking place in Stockholm.

Known as NVAIL, our program provides these research partners with access to powerful GPU computing resources.

Researchers from Tsinghua University and Georgia Tech are exploring ways to detect vulnerabilities in neural networks that use graph structured data. An Oxford University team is training multiple AI agents to efficiently operate together in the same environment. And Carnegie Mellon University researchers are determining how a neural network can more quickly learn the optimal path around a space.

Strengthening Neural Networks Against Attack

If shown an image of a watermelon that had some scrambled pixels laid over it, a human would still be able to easily identify it. But that can be enough to fool a neural network into misclassifying the pictured object as an elephant instead — a kind of adversarial attack hackers can use to manipulate the algorithm.

While existing research on adversarial attacks has focused on images, a joint paper from Georgia Tech, Ant Financial and Tsinghua University shows for the first time that this vulnerability extends to neural networks for graph data as well.

Graph structured data consists of nodes, which store data, and edges, which connect nodes to one another. The researchers experimented by adding and deleting edges to see where the neural network begins to perform badly in response to edge modifications.

Social network data, like the graph of how a single user is connected to a web of Facebook friends, is one example of graph structured data. Another is data on money transactions between individuals — such as records of who has sent money to whom.

Fooling a graph neural network that looks at financial data could result in a fraudulent transaction being labeled as legitimate. “If such models are not robust, if that’s easy to be attacked, that raises more concerns about using these models,” said Hanjun Dai, Ph.D. student at Georgia Tech and lead author on the paper.

The team used the cuDNN software library and ran their experiments on Tesla and GeForce GTX 1080 Ti GPUs. While the paper focuses on investigating the problem of adversarial attacks on graph structured data, the goal is for future research to propose solutions to strengthen graph neural networks so they provide reliable results despite attempted attacks.

Teamwork Makes the Neural Net Work

Driving is a multiplayer activity. Though each driver only has control over a single vehicle, the driver’s actions affect everyone else on the road. The person behind the wheel must also consider the actions of fellow motorists when deciding what to do.

Translating this kind of multilayered understanding into AI is a challenge.

An AI agent takes in information and feedback from its environment to learn and make decisions. But when there are multiple agents operating in the same space, researchers are tasked with teaching each AI to understand how the other agents affect the final outcome.

If an agent can’t reason about the behavior of others, it wouldn’t be able to properly reconcile its observations.

“For instance, it could find itself in exactly the same situation as earlier, take the same action and something different could happen,” said Oxford University doctoral student Tabish Rashid, a co-author on a paper that will be presented at ICML. “That causes conflicting learning to happen. It makes it difficult to learn what to do.”

This problem can be avoided during training, where researchers can allow multiple agents to communicate with one another and know other agents’ actions. But in the real world, one AI agent will not always have communication with or insight into the plans of others — so it must be able to act independently.

The Oxford researchers proposed a novel method that takes advantage of the training setting. Using the strategy game StarCraft II, they trained several agents together in an environment where agents could share information freely. After this centralized training, the agents were tested on how well they could perform independently.

Co-author Mikayel Samvelyan, former master’s student at Oxford, said this approach transfers well beyond the research setting: “You can train agents in a simulator and then use the strategies they learned in the real world.”

The team used an NVIDIA DGX-1 AI supercomputer and several GeForce GTX 1080 Ti GPUs for their work.

Planning the Perfect Path

Watching a cleaning robot wend its way around a swimming pool can be a mildly entertaining pastime on a lazy summer day. But is it taking the most efficient path around the pool to save time and energy?

Neural networks can help robots learn the optimal path around an environment faster and with less input information. A research group at Carnegie Mellon authored a paper outlining a path-finding model that’s simpler to train and more generic than current algorithms.

This makes it easier for developers to take the same base model, quickly apply it to different solutions, and optimize it. Applications for path-finding are diverse, ranging from household robots to factory robots, drones and autonomous vehicles.

Using 2D and 3D mazes, the team trained the neural network, powered by the NVIDIA DGX-1, an essential tool for accelerating deep learning research. Out in the world, an AI may not always have a map or know the structure of an environment beforehand — so the model was developed to learn just from images of the environment.

“Navigation is one of the core components for pretty much any intelligent system,” said Ruslan Salakhutdinov, computer science professor at Carnegie Mellon. Path planning networks like this one could become a building block that developers plug into larger robotic systems, he said.

Attendees of ICML, which runs July 10-15, can hear about each of these projects at the conference. Come by the NVIDIA booth (B02:12, Hall B) to connect with our AI experts, take a look at the new DGX-2 supercomputer and check out the latest demos.

The post Smorgasbord of AI Research Gets Set Out in Stockholm by NVAIL Partners appeared first on The Official NVIDIA Blog.