r/Multimodal • u/bakztfuture • Jul 30 '21
r/Multimodal • u/bakztfuture • Jun 29 '21
GitHub Copilot · Your AI pair programmer
r/Multimodal • u/bakztfuture • Mar 17 '21
Multilingual Multimodal Pre-training for Zero-Shot Cross-Lingual Transfer of Vision-Language Models
r/Multimodal • u/Wiskkey • Feb 25 '21
Text-to-image Google Colab notebook "Aleph-Image: CLIPxDAll-E" has been released. This notebook uses OpenAI's CLIP neural network to steer OpenAI's DALL-E image generator to try to match a given text description.
r/Multimodal • u/bakztfuture • Feb 22 '21
r/Multimodal Lounge
A place for members of r/Multimodal to chat with each other
r/Multimodal • u/breezedeus • Dec 08 '23
New Multimodal Model Coin-CLIP for Coin Identification/Recognition
Coin-CLIP breezedeus/coin-clip-vit-base-patch32 is built upon OpenAI's CLIP (ViT-B/32) model and fine-tuned on a dataset of more than 340,000 coin images using contrastive learning techniques. This specialized model is designed to significantly improve feature extraction for coin images, leading to more accurate image-based search capabilities. Coin-CLIP combines the power of Visual Transformer (ViT) with CLIP's multimodal learning capabilities, specifically tailored for the numismatic domain.
Key Features:
State-of-the-art coin image retrieval;
Enhanced feature extraction for numismatic images;
Seamless integration with CLIP's multimodal learning.
To further simplify the use of the Coin-CLIP model, I created https://github.com/breezedeus/Coin-CLIP , which provides tools for quickly building a coin image retrieval engine.
Try this online Demo for American Coin Images:
r/Multimodal • u/bakztfuture • Apr 04 '22
Pathways Language Model (PaLM): Scaling to 540 Billion Parameters for Breakthrough Performance
r/Multimodal • u/bakztfuture • Sep 06 '21
Finetuned Language Models Are Zero-Shot Learners
arxiv.orgr/Multimodal • u/bakztfuture • Jun 29 '21
Multimodal Few-Shot Learning with Frozen Language Models
r/Multimodal • u/bakztfuture • Apr 23 '21
VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text
r/Multimodal • u/bakztfuture • Apr 14 '21
AI model sizes will continue to grow, by 2023 NVIDIA believes that models will have 100 trillion or more connections. Models of that size will exceed the technical capabilities of existing platforms.
r/Multimodal • u/Wiskkey • Mar 09 '21
New Google Colab notebook "Aleph2Image Modified by kingchloexx for Image+Text to Image - Colaboratory" by kingchloexx. This notebook is for editing an existing image using a text description. Example: Text "green fur" with "plus" operation.
r/Multimodal • u/bakztfuture • Mar 04 '21
Multimodal Neurons in Artificial Neural Networks
r/Multimodal • u/bakztfuture • Mar 03 '21
WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learning
r/Multimodal • u/bakztfuture • Mar 02 '21
We used Big sleep to see if it could design our logo
r/Multimodal • u/Wiskkey • Mar 02 '21
Text-to-image for text "Gwen Stefani at The Great Pyramid of Giza" plus an input image using Google Colab notebook Aphantasia
r/Multimodal • u/bakztfuture • Mar 02 '21
"M6: A Chinese Multimodal Pretrainer", Lin et al 2021 {Alibaba} (1.9TB images/0.29TB text for 100b-parameter text-image Transformer)
r/Multimodal • u/bakztfuture • Feb 28 '21
DALL-E x CLIP - "The Industrial Revolution and its consequences."
r/Multimodal • u/Wiskkey • Feb 28 '21
Article about a Twitter bot that uses GPT-2 to invent heavy metal band album names and The Big Sleep to generate the album artwork: "Evil Chicken is my new favorite band — but they don’t exist"
r/Multimodal • u/bakztfuture • Feb 25 '21
A Straightforward Framework For Video Retrieval Using CLIP
r/Multimodal • u/Wiskkey • Feb 24 '21
For developers: OpenAI has released the encoder and decoder for the discrete VAE used for DALL-E.
r/Multimodal • u/camrobjones • Feb 22 '21
De Deyne et al. (2021) Visual and Affective Multimodal Models of Word Meaning in Language and Mind
r/Multimodal • u/Away-Control-2008 • Jun 04 '26
Anyone tried running a long-horizon coding task on an open-weights multimodal model?
Been watching the open-weights multimodal space for a while and M3 caught my eye because it's the first one I've seen combine native multimodality, million-token context, and agentic coding in a single open package — usually you have to pick two out of three from closed models.
The part I keep coming back to is the MSA architecture handling the 1M token context without RAG or chunking. For immigration law docs where a single review package can run 60 pages of evidence, not having to split and reassemble context feels like it actually solves a real bottleneck rather than working around one.
For those who've tested M3 on cross-modal tasks — does the native multimodal integration hold up in practice, or does it feel like vision bolted on after the fact? Specifically curious about how it handles parsing mixed PDF/image stacks compared to stitching separate pipelines together.