Goto

Collaborating Authors

 graffiti


Learning from Pattern Completion: Self-supervised Controllable Generation

Neural Information Processing Systems

The human brain exhibits a strong ability to spontaneously associate different visual attributes of the same or similar visual scene, such as associating sketches and graffiti with real-world visual objects, usually without supervising information. In contrast, in the field of artificial intelligence, controllable generation methods like ControlNet heavily rely on annotated training datasets such as depth maps, semantic segmentation maps, and poses, which limits the method's scalability. Inspired by the neural mechanisms that may contribute to the brain's associative power, specifically the cortical modularization and hippocampal pattern completion, here we propose a self-supervised controllable generation (SCG) framework. Firstly, we introduce an equivariance constraint to promote inter-module independence and intra-module correlation in a modular autoencoder network, thereby achieving functional specialization. Subsequently, based on these specialized modules, we employ a self-supervised pattern completion approach for controllable generation training. Experimental results demonstrate that the proposed modular autoencoder effectively achieves functional specialization, including the modular processing of color, brightness, and edge detection, and exhibits brain-like features including orientation selectivity, color antagonism, and center-surround receptive fields. Through self-supervised training, associative generation capabilities spontaneously emerge in SCG, demonstrating excellent zero-shot generalization ability to various tasks such as superresolution, dehaze and associative or conditional generation on painting, sketches, and ancient graffiti. Compared to the previous representative method ControlNet, our proposed approach not only demonstrates superior robustness in more challenging high-noise scenarios but also possesses more promising scalability potential due to its self-supervised manner. Codes are released on Github and Gitee.


AI based signage classification for linguistic landscape studies

arXiv.org Artificial Intelligence

Linguistic Landscape (LL) research traditionally relies on manual photography and annotation of public signages to examine distribution of languages in urban space. While such methods yield valuable findings, the process is time-consuming and difficult for large study areas. This study explores the use of AI powered language detection method to automate LL analysis. Using Honolulu Chinatown as a case study, we constructed a georeferenced photo dataset of 1,449 images collected by researchers and applied AI for optical character recognition (OCR) and language classification. We also conducted manual validations for accuracy checking. This model achieved an overall accuracy of 79%. Five recurring types of mislabeling were identified, including distortion, reflection, degraded surface, graffiti, and hallucination. The analysis also reveals that the AI model treats all regions of an image equally, detecting peripheral or background texts that human interpreters typically ignore. Despite these limitations, the results demonstrate the potential of integrating AI-assisted workflows into LL research to reduce such time-consuming processes. However, due to all the limitations and mis-labels, we recognize that AI cannot be fully trusted during this process. This paper encourages a hybrid approach combining AI automation with human validation for a more reliable and efficient workflow.


Learning from Pattern Completion: Self-supervised Controllable Generation

Neural Information Processing Systems

The human brain exhibits a strong ability to spontaneously associate different visual attributes of the same or similar visual scene, such as associating sketches and graffiti with real-world visual objects, usually without supervising information. In contrast, in the field of artificial intelligence, controllable generation methods like ControlNet heavily rely on annotated training datasets such as depth maps, semantic segmentation maps, and poses, which limits the method's scalability. Inspired by the neural mechanisms that may contribute to the brain's associative power, specifically the cortical modularization and hippocampal pattern completion, here we propose a self-supervised controllable generation (SCG) framework. Firstly, we introduce an equivariance constraint to promote inter-module independence and intra-module correlation in a modular autoencoder network, thereby achieving functional specialization. Subsequently, based on these specialized modules, we employ a self-supervised pattern completion approach for controllable generation training. Experimental results demonstrate that the proposed modular autoencoder effectively achieves functional specialization, including the modular processing of color, brightness, and edge detection, and exhibits brain-like features including orientation selectivity, color antagonism, and center-surround receptive fields.


Refutation of Spectral Graph Theory Conjectures with Search Algorithms)

arXiv.org Artificial Intelligence

We are interested in the automatic refutation of spectral graph theory conjectures. Most existing works address this problem either with the exhaustive generation of graphs with a limited size or with deep reinforcement learning. Exhaustive generation is limited by the size of the generated graphs and deep reinforcement learning takes hours or days to refute a conjecture. We propose to use search algorithms to address these shortcomings to find potentially large counter-examples to spectral graph theory conjectures in seconds. We apply a wide range of search algorithms to a selection of conjectures from Graffiti. Out of 13 already refuted conjectures from Graffiti, our algorithms are able to refute 12 in seconds. We also refute conjecture 197 from Graffiti which was open until now.


Generative AI can turn your most precious memories into photos that never existed

MIT Technology Review

"It's very easy to see when you've got the memory right, because there is a very visceral reaction," says Pau Garcia, founder of Domestic Data Streamers. Dozens of people have now had their memories turned into images in this way via Synthetic Memories, a project run by Domestic Data Streamers. The studio uses generative image models, such as OpenAI's DALL-E, to bring people's memories to life. Since 2022, the studio, which has received funding from the UN and Google, has been working with immigrant and refugee communities around the world to create images of scenes that have never been photographed, or to re-create photos that were lost when families left their previous homes. Now Domestic Data Streamers is taking over a building next to the Barcelona Design Museum to record people's memories of the city using synthetic images.


TITAA #35: Witch Elms and Barrows - by Lynn Cherny

#artificialintelligence

"Who put Bella down the Wych Elm - Hagley Wood?" A famous unsolved murder mystery memorialized by graffiti in England, I ran across it twice this month. The first instance of this graffiti was seen on the wall in Birmingham Fruit Market, in chalk, on March 30, 1944. Then it morphed into "Who put Luebella in the Wych Elm" on March 31 (reddit source). In 1999, a version appeared on the obelisk on Wychbury Hill as seen above and has remained ever since, even after restoration of the obelisk. The graffiti, by unknown writers, evidently refers to the body of a murdered woman found in an elm in Hagley Wood in April 1943. Robert Hart, of Wollescote, Stourbridge, told the Coroner and jury how at midday on Sunday, 18 April, he and three other lads when birdsnesting in the wood.


Extended Version of GTGraffiti: Spray Painting Graffiti Art from Human Painting Motions with a Cable Driven Parallel Robot

arXiv.org Artificial Intelligence

We present GTGraffiti, a graffiti painting system from Georgia Tech that tackles challenges in art, hardware, and human-robot collaboration. The problem of painting graffiti in a human style is particularly challenging and requires a system-level approach because the robotics and art must be designed around each other. The robot must be highly dynamic over a large workspace while the artist must work within the robot's limitations. Our approach consists of three stages: artwork capture, robot hardware, and planning & control. We use motion capture to capture collaborator painting motions which are then composed and processed into a time-varying linear feedback controller for a cable-driven parallel robot (CDPR) to execute. In this work, we will describe the capturing process, the design and construction of a purpose-built CDPR, and the software for turning an artist's vision into control commands. Our work represents an important step towards faithfully recreating human graffiti artwork by demonstrating that we can reproduce artist motions up to 2m/s and 20m/s$^2$ within 9.3mm RMSE to paint artworks. Changes to the submitted manuscript are colored in blue.


Art with AI: Turning photographs into artwork with Neural Style Transfer

#artificialintelligence

Please Note: I reserve the rights of all the media used in this blog -- photographs, animations, videos, etc. they are my work (except the 7 mentioned artworks by artists which were used as style images). GIFs might take a while to load, please be patient. If that is the case please open in browser instead. The world today doesn't make sense, so why should I paint pictures that do? -- Pablo Picasso Here are the results, some combinations produced astounding artwork. Here's an image of a bride & graffiti, combining them results in an output similar to doodle painting. Here, you can see the buildings being popped up in the background.


Meet the researchers bringing bizarre AI creations to life

#artificialintelligence

"The larger goal of the project is to democratize AI for the public. Currently, the creative usage of AI is within the reach of AI practitioners," computer scientist Pinar Yanardag, the MIT postdoctoral researcher who founded the group, told Futurism. Most of the projects used machine learning algorithms that, upon being trained with thousands of examples of pizzas and perfume recipes, spat out unique creations that were sort of like the thing they were supposed to create, but just a little bit off. Often, the results were outlandish and alien. For instance, several pizzas called for fictional ingredients or skipped crucial steps in the baking process.


Meet the researchers bringing bizarre AI creations to life

#artificialintelligence

Time to throw on your oversized graduation cap and ruffly shirt, spritz yourself with some of your new "Hivinga" perfume, and meet your better half for some sweet potato, beans, and brie pizza covered with "snipped caramel cheese." Things that don't seem quite right here. Oh, right, that's because all these date night tips came from a series of artificial intelligence algorithms custom-built to generate new clothing designs, pizza recipes, and perfumes. Whatever bizarre (and, frankly, unappetizing) ideas the AI systems conjured were built by a group of MIT researchers and students called How to Generate (Almost) Anything. The team builds and trains algorithms that can tackle seemingly creative tasks to come up with brand new designs for each week's theme.