iiinova — where your digital self keeps evolving

That’s definitely a valid concern, and I believe we need to rethink how LLMs and other generative models are deployed and used in practice. We do explicitly discuss some of these considerations.

As a conlanger myself, I was mainly curious to explore whether LLMs could be used as a creative assistant for humans, as well as procedural generation in games with unbounded worlds. I hope this gets more people interested in conlanging and experimenting themselves.

120

Thanks for the heads-up, fixing this.

ConlangCrafter could potentially be used in pedagogy, typological and NLP work, and many entertainment applications. Imagine a video game where aliens can speak countless new procedurally-generated languages.

260

To enhance consistency and diversity, our pipeline incorporates randomness injection and self-refinement mechanisms. This is measured by our novel evaluation framework, providing rigorous evaluation for the new task of computational conlanging.

240

The ConlangCrafter pipeline harnesses an LLM to generate a description of a constructed language and self refines it in the process. We decompose language creation into phonology, grammar, and lexicon, and then translate sentences while constructing new needed grammar points.

182

Conlangs (Constructed Languages), from Tolkien’s Elvish to Esperanto, have long been created for artistic, philosophical, or practical purposes. As generative AI proves its creative power, we ask: Can it also take on the laborious art of conlang creation?

260

The number of languages in the world just got a lot higher! At least constructed ones. Meet ConlangCrafter - a pipeline for creating novel languages with LLMs. A Japanese-Esperanto creole? An alien cephalopod color-based language? Enter your idea and see a conlang emerge. 🧵👇

33811

At inference time, we inject the appearance of the observed view to get consistent novel views. This also enables cool applications like appearance-conditioned NVS! (4/5)

100

To learn from this data, we use a novel multi-view diffusion architecture adapted from CAT3D, modeling appearance variations with a bottleneck encoder applied to VAE latents and disambiguating scene scale via warping. (3/5)

100

Photos like the ones below differ in global appearance (day vs. night, lighting), aspect ratio, and even weather. But they give clues to how scenes are build in 3D. (2/5)

100

Morris Alper @malper.bsky.social · 14/06/2025

💥New preprint! WildCAT3D uses tourist photos in-the-wild as supervision to learn to generate novel, consistent views of scenes like the one shown below. h/t Tom Monnier and all collaborators (1/5)

150

Disappointing that arXiv doesn't allow XeLaTex/LuaLaTex submissions, which have the least broken multilingual support of LaTeX compilers. The web shouldn't be limited to English in 2025!

Finally we show that ProtoSnap-aligned skeletons can be used as conditions for a ControlNet model to generate synthetic OCR training data. By controlling the shapes of signs in training, we can achieve SOTA on cuneiform sign recognition. (Bottom: synthetic generated sign images)

100

Our results show that ProtoSnap effectively aligns wedge-based skeletons to scans of real cuneiform signs, with global and local refinement steps. We provide a new expert-annotated test set to quantify these results.

100

ProtoSnap uses features from a fine-tuned diffusion model to optimize for the correct alignment between a skeleton matched with a prototype font image and a scanned sign. Perhaps surprising that image generation models can be applied to this sort of discriminative task!

100

We tackle this by directly measuring the internal configuration of characters. Our approach ProtoSnap "snaps" a prototype (font)-based skeleton onto a scanned cuneiform sign using a multi-stage pipeline with SOTA methods from computer vision and generative AI.

110

Some prior work has tried to classify scans of signs categorically, but signs' shapes differ drastically in different time periods and regions making this less effective. E.g. both signs below are AN, from different eras. (Top: font prototype; bottom: scan of sign real tablet)

100

Arguably the most ancient writing system in the world (since ~3300 BCE), cuneiform inscriptions in ancient languages (e.g. Sumerian, Akkadian) are numerous but hard to read due to the complex writing system, wide variation in sign shapes, and physical nature as imprints in clay.

100

Cuneiform at #ICLR2025! ProtoSnap finds the configuration of wedges in scanned cuneiform signs for downstream applications like OCR. A new tool for understanding the ancient world! tau-vailab.github.io/ProtoSnap/ h/t Rachel Mikulinsky @ShGordin @ElorHadar and all collaborators. 🧵👇

171

000

100

110

100

Kush Jain @kjain14.bsky.social · 19/12/2024

100

Reposted by Morris Alper

Thrilled to announce our new work TestGenEval, a benchmark that measures unit test generation and test completion capabilities. This work was done in collaboration with the FAIR CodeGen team. Preprint: arxiv.org/abs/2410.00752 Leaderboard: testgeneval.github.io/leaderboard....

1177

Morris Alper @malper.bsky.social · 20/12/2024

Great news! BERT-like models are extremely useful and imo unfairly overlooked in the recent GenAI hype cycle. Looking forward to playing with this

000

WAFFLE: Multimodal Floorplan Understanding in the Wild

We look forward to progress on architectural tasks being benchmarked and accelerated by WAFFLE! See our project page for more details and links to our paper, code, and data: tau-vailab.github.io/WAFFLE/

tau-vailab.github.io

000

We show that our dataset serves as a new, challenging benchmark for common floorplan understanding tasks such as semantic segmentation. We also show it can be used to enable new tasks such as floorplan generation conditioned on building type and boundary.

100

We use modern foundation models (LLMs, vision-language models) to filter and structure raw, noisy open data to identify floorplan images and extract structured metadata, including global properties (e.g. floorplan type) and grounded architectural features within images.

100

WAFFLE (WikipediA-Fueled FLoorplan Ensemble) is a multimodal dataset of ~20K diverse floorplans, of many building types (e.g. homes, churches, hospitals, schools, ...), regions, eras, and data formats, along with structured metadata.

100

WAFFLE: Multimodal Floorplan Understanding in the Wild

Project page: tau-vailab.github.io/WAFFLE/ Paper: arxiv.org/abs/2412.00955 Architecture is complicated and automatic methods could help design and maintain buildings. But current datasets are very limited (e.g. apartments from one country). That's where WAFFLE comes in!

arxiv.org

Buildings are a central feature of human culture and are increasingly being analyzed with computational methods. However, recent works on computational building understanding have largely focused on n...

100

Bite into WAFFLE 🧇, our new multimodal floorplan dataset and paper - now accepted to #WACV2025! Work with Keren Ganon, Rachel Mikulinsky, Hadar Elor. More info below👇

110

WAFFLE: Multimodal Floorplan Understanding in the Wild

tau-vailab.github.io

000

100

100

100

WAFFLE: Multimodal Floorplan Understanding in the Wild

arxiv.org

100

Morris Alper @malper.bsky.social · 06/12/2024

Is there any good way to flag or address ethical issues with a paper post-publication?

Morris Alper @malper.bsky.social · 02/12/2024

What's a better alternative? Ending without some formality to signal that you're done seems too abrupt.

200

Morris Alper @malper.bsky.social · 01/12/2024

Playing with Suno AI song generation again, v3.5 seems to be more consistently catchy and better at (non-English) languages. This is not bad and funny how it sounds like a lot of Israeli prog-rock songs... suno.com/song/1e21b93...

suno.com

רובוט בודד by @noblekalimba435 | Suno

experimental, rock, prog-rock, electric song. Listen and make your own with Suno.

020

Morris Alper @malper.bsky.social · 30/11/2024

Good question! Semantic hierarchies should form a multi-branched DAG structure, while our benchmark and hierarchical retrieval is using the simplifying assumption of a single, linear hierarchy per image. I'd hope to see extensions of this taking multiple branching into account.