r/Multimodal • • Jul 30 '21

DALL·E mini is now available

Thumbnail
huggingface.co
9 Upvotes

r/Multimodal • • Jun 29 '21

GitHub Copilot · Your AI pair programmer

Thumbnail
copilot.github.com
5 Upvotes

r/Multimodal • • Mar 17 '21

Multilingual Multimodal Pre-training for Zero-Shot Cross-Lingual Transfer of Vision-Language Models

Thumbnail
arxiv.org
5 Upvotes

r/Multimodal • • Feb 25 '21

Text-to-image Google Colab notebook "Aleph-Image: CLIPxDAll-E" has been released. This notebook uses OpenAI's CLIP neural network to steer OpenAI's DALL-E image generator to try to match a given text description.

Thumbnail
self.machinelearning
5 Upvotes

r/Multimodal • • Feb 22 '21

r/Multimodal Lounge

5 Upvotes

A place for members of r/Multimodal to chat with each other


r/Multimodal • • Dec 08 '23

New Multimodal Model Coin-CLIP for Coin Identification/Recognition

4 Upvotes

Coin-CLIP breezedeus/coin-clip-vit-base-patch32 is built upon OpenAI's CLIP (ViT-B/32) model and fine-tuned on a dataset of more than 340,000 coin images using contrastive learning techniques. This specialized model is designed to significantly improve feature extraction for coin images, leading to more accurate image-based search capabilities. Coin-CLIP combines the power of Visual Transformer (ViT) with CLIP's multimodal learning capabilities, specifically tailored for the numismatic domain.

Key Features:

  • State-of-the-art coin image retrieval;

  • Enhanced feature extraction for numismatic images;

  • Seamless integration with CLIP's multimodal learning.

To further simplify the use of the Coin-CLIP model, I created https://github.com/breezedeus/Coin-CLIP , which provides tools for quickly building a coin image retrieval engine.

Try this online Demo for American Coin Images:

https://huggingface.co/spaces/breezedeus/USA-Coin-Retrieval


r/Multimodal • • Apr 04 '22

Pathways Language Model (PaLM): Scaling to 540 Billion Parameters for Breakthrough Performance

Thumbnail
ai.googleblog.com
4 Upvotes

r/Multimodal • • Sep 06 '21

Finetuned Language Models Are Zero-Shot Learners

Thumbnail arxiv.org
4 Upvotes

r/Multimodal • • Jun 29 '21

Multimodal Few-Shot Learning with Frozen Language Models

Thumbnail
arxiv.org
4 Upvotes

r/Multimodal • • Apr 23 '21

VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text

Thumbnail
arxiv.org
4 Upvotes

r/Multimodal • • Apr 14 '21

AI model sizes will continue to grow, by 2023 NVIDIA believes that models will have 100 trillion or more connections. Models of that size will exceed the technical capabilities of existing platforms.

Thumbnail
forbes.com
4 Upvotes

r/Multimodal • • Mar 09 '21

New Google Colab notebook "Aleph2Image Modified by kingchloexx for Image+Text to Image - Colaboratory" by kingchloexx. This notebook is for editing an existing image using a text description. Example: Text "green fur" with "plus" operation.

Thumbnail
gallery
4 Upvotes

r/Multimodal • • Mar 04 '21

Multimodal Neurons in Artificial Neural Networks

Thumbnail
openai.com
4 Upvotes

r/Multimodal • • Mar 03 '21

WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learning

Thumbnail
arxiv.org
4 Upvotes

r/Multimodal • • Mar 02 '21

We used Big sleep to see if it could design our logo

Thumbnail
labelf.ai
4 Upvotes

r/Multimodal • • Mar 02 '21

Text-to-image for text "Gwen Stefani at The Great Pyramid of Giza" plus an input image using Google Colab notebook Aphantasia

Thumbnail
gallery
4 Upvotes

r/Multimodal • • Mar 02 '21

"M6: A Chinese Multimodal Pretrainer", Lin et al 2021 {Alibaba} (1.9TB images/0.29TB text for 100b-parameter text-image Transformer)

Thumbnail
arxiv.org
4 Upvotes

r/Multimodal • • Feb 28 '21

DALL-E x CLIP - "The Industrial Revolution and its consequences."

Post image
4 Upvotes

r/Multimodal • • Feb 28 '21

Article about a Twitter bot that uses GPT-2 to invent heavy metal band album names and The Big Sleep to generate the album artwork: "Evil Chicken is my new favorite band — but they don’t exist"

Thumbnail
thenextweb.com
4 Upvotes

r/Multimodal • • Feb 25 '21

A Straightforward Framework For Video Retrieval Using CLIP

Thumbnail
arxiv.org
4 Upvotes

r/Multimodal • • Feb 24 '21

For developers: OpenAI has released the encoder and decoder for the discrete VAE used for DALL-E.

Thumbnail
self.mediasynthesis
4 Upvotes

r/Multimodal • • Feb 22 '21

De Deyne et al. (2021) Visual and Affective Multimodal Models of Word Meaning in Language and Mind

Thumbnail
onlinelibrary.wiley.com
4 Upvotes

r/Multimodal • • Feb 22 '21

DALL·E: Creating Images from Text

Thumbnail
openai.com
4 Upvotes

r/Multimodal • • Jun 04 '26

Anyone tried running a long-horizon coding task on an open-weights multimodal model?

3 Upvotes

Been watching the open-weights multimodal space for a while and M3 caught my eye because it's the first one I've seen combine native multimodality, million-token context, and agentic coding in a single open package — usually you have to pick two out of three from closed models.

The part I keep coming back to is the MSA architecture handling the 1M token context without RAG or chunking. For immigration law docs where a single review package can run 60 pages of evidence, not having to split and reassemble context feels like it actually solves a real bottleneck rather than working around one.

For those who've tested M3 on cross-modal tasks — does the native multimodal integration hold up in practice, or does it feel like vision bolted on after the fact? Specifically curious about how it handles parsing mixed PDF/image stacks compared to stitching separate pipelines together.