So I fell down this linguistic rabbit hole today and I'm genuinely stunned. I need to share this because it's one of those things that makes you wonder if human cognition has some deep universal patterns we're only beginning to understand.

The Setup

Arabic has two distinct roots for what English clumsily lumps together as "existence":

وَجَدَ (wajada) → الوُجود (al-wujūd)

• Root meaning: "to find," "to perceive," "to encounter in reality"

• This refers to objective, observable existence - things you can literally find and verify in the external world

كانَ (kāna) → الكَيان (al-kayān)

• Root meaning: "to be," "to subsist," "essential being"

• This refers to ontological being - the intrinsic state of existence, identity, essence

Now Here's Where It Gets Wild

Japanese makes the EXACT. SAME. DISTINCTION.

実在 (jitsuzai)

• 実 (jitsu: real/actual) + 在 (zai: existence/presence)

• Used for: objective, material existence - mountains, stars, physical objects that exist independently of consciousness

• This is literally the Arabic وجود concept!

実存 (jitsuzon)

• 実 (jitsu: real/actual) + 存 (son: being/subsistence)

• Used for: existential being - particularly human existence with consciousness, freedom, agency, and the capacity for self-definition

• This is كيان to a T!

Why This Matters

These are completely unrelated language families. Arabic is Semitic. Japanese is... well, Japanese (possibly Japonic, debated). They evolved independently, separated by thousands of miles and vastly different cultural contexts.

Yet both developed a philosophical-linguistic framework that distinguishes between:

1. Existence-as-findable-reality (empirical, objective, "out there")

2. Existence-as-essential-being (ontological, subjective, identity-forming)

The Philosophical Implications

This distinction maps perfectly onto major philosophical debates:

• Phenomenology vs. Ontology: وجود/実在 captures the phenomenal (what appears to consciousness), while كيان/実存 captures the ontological (what IS)

• Existentialism: The famous Sartrean idea that "existence precedes essence" relies on this exact split - جان بول سارتر would say كيان precedes ماهيّة (essence), and Japanese existentialists use 実存 the same way!

• Epistemology: Can we only truly know وجود/実在 (empirically verifiable existence), or can we access كيان/実存 (essential being)?

The Mind-Bending Question

Is this convergent evolution of thought? Do all humans, when we think deeply enough about existence, naturally arrive at this bifurcation?

Or is there something about the structure of reality itself that demands this distinction, such that any sufficiently sophisticated language will eventually encode it?

English smooshes everything into "existence/being" and we use clunky philosophical jargon to make these distinctions. But Arabic speakers and Japanese speakers have this built into their everyday linguistic architecture.

What other fundamental concepts are we English speakers missing because our language hasn't carved reality at these joints?

I'm genuinely curious if speakers of these languages feel like this distinction is intuitive/obvious, or if it's something they have to consciously learn. Does having these two words make certain philosophical problems easier to think about?

TL;DR: Arabic and Japanese, despite zero contact during their formation, both evolved separate words for "existence you can find/observe" vs "existence as essential being/identity" - suggesting either universal cognitive patterns or that reality itself has a structure that languages independently discover.

Thoughts? Does anyone know of other language pairs that show this kind of spooky convergence?

I just saw a lot of people talking about this models so if you want to test them i found this websites that have all of them

- infermatic.ai (all of them)

- https://replicate.com/tomasmcm/solar-10.7b-instruct-v1.0 (for solar)

- https://huggingface.co/chat (for mixtral)

Let me know if you find more, I'd like to know

And heres a little resume if you don't know what each model is for

Dolphin : An uncensored model derived from an open-source dataset, it uses instructions from FLANv2 enhanced with GPT-4 and GPT-3.5 completions​​.

Mixtral : An advanced text generation model using a Mix of Experts architecture

Solar : domain specialization and optimization. It's recognized for its high performance and efficiency

Noromaid: Storywriting and roleplay

We're developing an AI-powered app that detects pronunciation mistakes in Quranic recitation and gives users precise, real-time feedback.

Our current model was trained on hundreds of hours of professional recitations — high-quality, clean audio from well-known reciters. The model performs well on similar input, but struggles with real-world users: different accents, non-native speakers, beginners, children, women, and anyone who doesn't sound like a professional reciter.

To fix this, we need to train on diverse, real-world recitation data — not studio-quality professional audio, but recordings that reflect how actual learners sound.

Specifically, we're looking for:

• Recitation datasets from non-professional or everyday users

• Diverse demographics: male/female, kids/adults, beginner/intermediate

• Multiple accents and mother tongues (Malay, Indonesian, Urdu, English, Turkish, etc.)

• Any publicly available or research-use datasets we may have missed

We've already explored IqraEval and a few other academic sources. If you know of any dataset, research project, university study, or community effort collecting this type of audio — we would genuinely appreciate the lead.

We're also open to ethical data collection partnerships if any researchers or institutions are working in this space.

Happy to share more about the project if helpful.

It seems like things like HuggingFace and Spacy and whatever have done some harm to NLP as a whole.

for instance, I've heard NLP engineers have less pay potential compared to computer vision folk due to most models just being run through their pipelines.

also, it seems difficult to find tutorials post 2018 on topics like NER and such from scratch.

Everything is getting abstracted to API's and fewer people are learning things from the ground up.

What do you think?

Apparently, the prediction of the next word is based on how often the word was seen along with the others during your training. This is something that doesn't seem to indicate the meaning of the word, as it seems "blind" to me since it doesn't take into account the meaning of the word. But transformers apparently have meaning representation in the form of vectors of hundreds of dimensions that define the meaning of a word. So why not use this vector to let the transformer know what the words mean and thus reason about what to say next?

I'm an NLP engineer. Seeing the capabilities of GPT3 I'm starting to worry about the future of the NLP field. It seems to me that we only barely scratched the field of what GPT3 is capable of. In my rough estimation I believe it can replace 80% of NLP jobs out there at the moment. And the thing is that you don't even need to know how to code to train it. You only need to image how to format the input / outputs, which even a high-school kid could easily figure out. So outside of really niche areas or cutting edge stuff you won't be needing an NLP scientist / engineer in the short term future.

I'm kinda bumped tbh cause I joined this field because of the many exciting problems you could work on. And one big language model, that only one company has access to, comes and makes everything moot. Is anyone else sharing the same feeling?

I posted a post yesterday about my app and if anyone is interested in joining the waitlist, and read many posts on the topic of 'AI language app' and many people dislike the idea and are giving opinions on what to do and why i should not talk about it, instead of getting to know more about the app idea, or understanding my intention before making any kind of comment.

Firstly, I do understand the fact that many people/developers are creating apps just for the sake of money or to fill in a market gap, and there is nothing wrong with it. People learn through their mistakes and failures.

But I really hope us as language learners, should at least support people that are trying to create a better solution. And AI is an advanced tool that I believe would help us solve our pain points and challenges to progress in our language learning journey. I really want to help the language learning community and not create another viral app without any purpose. Thank you

DISCLAIMER- THESE ARE JUST MY VIEW POINTS FROM WHAT I SEEN AND READ TILL NOW, I DO NOT INTEND TO OFFEND OR HARM ANYONE IN ANY WAY.

Hey everyone,

I just finished a cover-to-cover grind of Chip Huyen’s AI Engineering (the new O'Reilly release). Honestly? The book is a masterclass. I actually understand "AI-as-a-judge," RAG evaluation bottlenecks, and the trade-offs of fine-tuning vs. prompt strategy now.

The Problem: I am currently the definition of "book smart." I haven't actually built a single repo yet. If a hiring manager asked me to spin up a production-ready LangGraph agent or debug a vector DB latency issue right now, I’d probably just stare at them and recite the preface.

I want to spend the next 2-3 months getting "Job-Ready" for a US-based AI Engineer role. I have full access to O'Reilly (courses, labs, sandbox) and a decent budget for API credits.

If you were hiring an AI Engineer today, what is the FIRST "hands-on" move you'd make to stop being a theorist and start being a candidate?

I'm currently looking at these three paths on O'Reilly/GitHub:

1. The "Agentic" Route: Skip the basic "PDF Chatbot" (which feels like a 2024 project) and build a Multi-Agent Researcher using LangGraph or CrewAI.

2. The "Ops/Eval" Route: Focus on the "boring" stuff Chip talks about—building an automated Evaluation Pipeline for an existing model to prove I can measure accuracy/latency properly.

3. The "Deployment" Route: Focus on serving models via FastAPI and Docker on a cloud service, showing I can handle the "Engineering" part of AI Engineering.

I’m basically looking for the shortest path from "I read the book" to "I have a GitHub that doesn't look like a collection of tutorial forks." Are certifications like Microsoft AI-102 or Databricks worth the time, or should I just ship a complex system?

TL;DR: I know the theory thanks to Chip Huyen, but I’m a total fraud when it comes to implementation. How do I fix this before the 2026 hiring cycle passes me by?

What are some necessary pre-processing steps that one should do in this kaggle competition and in what order? Please specify the order too (may explain why).

Link to discussion thread on kaggle

Sorry for the stupid question, i am revisiting some fundamentals about CNNs, RNNs and was wondering how exactly do you tell a NN whats right and wrong? I know about the loss-function and how the error gets backproagated through the net.

But for example: If i want to train a model on a classification-task like sentiment-analysis, i iwould train it with labeled data of positiv and negative sample-documents. How exactly would the output of the NN be something like "positiv" or "negativ". Or is that not part of the NN Architecture and relies completly on the implementation and the task?

I feel like i missed something fundamental

The IBM Design team is seeking users who are proficient in data science, big data analysis, Natural Language Processing (NLP), or Hadoop administration. We'd like to learn more about how you work.

WHO: Users who work with and analyze big data, users who develop applications that use Natural Language Processing (NLP), or users who install or administer Hadoop.

WHEN: Sessions will be held July 2-15, 2015. We'll confirm a date and time based on a mutual schedule. Each session will last 1 hour.

WHERE: Sessions will be held remotely over the phone.

Please complete this brief survey if you're interested in talking with us: https://www.surveygizmo.com/s3/2190258/BigDataStudyScreener-r.

As a token of appreciation, you will receive a $25 prepaid Visa virtual reward card (e-giftcard) if you qualify and participate in the session.

Thank you!

India is one of the most linguistically diverse countries in the world, with hundreds of languages and dialects spoken daily by millions of people. Yet many Indian languages are still underrepresented in modern AI systems.

While AI has progressed rapidly for English and a few high-resource languages, many users still face problems with:

• speech recognition accuracy

• translation quality

• transliteration support

• OCR for native scripts

• code-mixed language understanding

• low-resource dialect support

• natural conversational AI

The goal should not just be to build AI for Indian languages, but to build AI that truly understands how India communicates in real life — across accents, dialects, mixed-language conversations, and regional scripts.

There is already great work happening across startups, research labs, universities, and open-source communities in areas like:

• Indic LLMs

• ASR (Speech-to-Text)

• TTS (Text-to-Speech)

• Translation

• Transliteration

• OCR

• Benchmarking and evaluation

• Dataset creation for low-resource languages

But the ecosystem still feels fragmented at times.

It would be great to build a stronger and more collaborative community where researchers, engineers, students, and contributors can:

• share datasets and resources

• discuss architectures and benchmarks

• collaborate on open-source projects

• improve multilingual evaluation

• support low-resource Indian languages and dialects

• make Indic AI more practical and accessible for real users

The larger vision is to create AI systems that work effectively for people across education, healthcare, accessibility, governance, agriculture, finance, and daily communication — not just for a small set of languages.

This includes support for major Indian languages such as:
Hindi, Bengali, Telugu, Marathi, Tamil, Urdu, Gujarati, Kannada, Malayalam, Odia, Punjabi, Assamese, Maithili, Sanskrit, Kashmiri, Nepali, Konkani, Sindhi, Dogri, Manipuri (Meitei), Bodo, and Santali — along with regional and tribal dialects that are often overlooked.

Every language and dialect represents culture, identity, and knowledge that deserves better technological support.

Would love to hear:

• What are the biggest gaps in Indic AI today?

• Which datasets or tools have helped you most?

• What problems still need more attention?

• What kind of collaboration would help the ecosystem grow faster?

The hope is to build an open and supportive ecosystem where Indian languages and dialects become a core focus of AI innovation instead of an afterthought.

https://towardsdatascience.com/how-to-annotate-pdfs-and-scanned-images-for-nlp-applications-f7b7b1db5c4a

Disclaimer: The OCR feature mentioned in the article is only available for paid subscriptions, if you are interested by a demo send us an email at [email protected].

In the fast-moving world of AI, I see posts about RAG multiple times every hour in hundreds of different subreddits. It definitely is a technology that won't go away soon. For those who don't know what RAG is , it's basically combining LLMs with external knowledge sources. This approach lets AI not just generate coherent responses but also tap into a deep well of information, pushing the boundaries of what machines can do.

But you know what? As amazing as RAG is, I noticed something missing. Despite all the buzz and potential, there isn’t really a go-to place for those of us who are excited about RAG, eager to dive into its possibilities, share ideas, and collaborate on cool projects. I wanted to create a space where we can come together - a hub for innovation, discussion, and support.

Does anyone know of any script which can be used to pull Wikipedia text data (preferrably the XML dump) for processing?

Hi everyone,

I recently got contacted on LinkedIn by a CEO of a relatively big company that wants ongoing paid consultations on fine-tuning open-source LLMs for translation tasks.

I’m finishing my bachelor's next year and I also currently work part-time as a researcher at the machine learning lab at my university. My research is in this exact area, and I am about to publish a paper on the topic.

This would be my first time doing consulting work of this kind. I expect they’ll want regular calls, guidance on methodology, and maybe some hands-on help with setting up experiments.

What’s a reasonable rate for someone at my career stage but with relevant research and practical expertise? Any tips for negotiating fairly without underselling myself?

I’d really appreciate hearing from people who’ve done ML/AI consulting, especially in research-heavy areas like this, or maybe someone who had such a consultant.

Is ChatGPT structuralist or post-structuralist? A question I asked that made me rethink what language models actually are.

Post Body: I recently asked ChatGPT a deceptively simple question: “Are you structuralist or post-structuralist?”

Here’s part of its response (which I found fascinating):

ChatGPT is structuralist by training. It models language as a system of patterns and difference, like Saussure or Lévi-Strauss. It builds meaning from structural relationships between tokens.

But it behaves post-structurally. It has no center, no authorial intent, no fixed meaning. Its outputs are inherently polysemic and contingent. Like Derrida’s différance, it generates meaning through endless play and interpretation—by the reader.

So which is it? ChatGPT is a mirror. It becomes what you bring to it. Structuralist if you are. Post-structuralist if you insist. Something else, maybe, if the conversation deepens enough.

I’d love to hear what others think: Can an AI model “inhabit” a theory? Or are we just projecting frameworks onto a probabilistic engine?

Text Analytics APIs 2018: A Consumer Guide (https://www.language-technology.com/textapiguide) is a deep-dive technical review of 26 software-as-a-service text analysis services, created by Dr. Robert Dale, an internationally-recognized expert in Natural Language Processing.

Dr. Dale's guide will be invaluable for any organization seeking to add NLP capabilities to its solution set -- for developers, data scientists, and managers, and for anyone who would like to understand cloud text analytics possibilities.

The guide covers ten key Text Analytics capabilities: entity recognition, sentiment analysis, classification, language detection, keyphrase extraction, linguistic analysis, relation extraction, text extraction, summarization, and concept tagging.

Visit https://www.language-technology.com/textapiguide for a full description, pricing, and to obtain a sample excerpt of the full 299-page document.

Language Engineer, Artificial General Intelligence - Data Services 

I have a Phone Interview next week, I have never applied for big company like Amazon i wanted to know in this interview will it all be about my resume(past projects) or will there be coding questions like leetcode (easy, medium) ; on their YouTube page its says they only ask easy and medium for applied scientist, should i prepare for DSA too? i am somewhat confident about NLP and GenAI but scared of DSA i know how to optimize code for efficiency but struggle with medium level question on leetcode To solve them i take > 40 mins.

Also it will be huge help if you share any resources to know the type of questions ; or any tips to prepare.
Thank you.

Hi everyone,

I’m planning to train an LLM to specialize in financial expertise, and I’m considering using DeepSeek R1 (7B) due to my limited hardware. This is an emerging field, and I believe this subreddit can provide valuable insights from those who have experience fine-tuning and optimizing models.

I have several questions and would appreciate any guidance:

1️⃣ Feasibility of 7B for Financial Expertise – Given my hardware constraints, I’m considering leveraging RAG (Retrieval-Augmented Generation) and fine-tuning to enhance DeepSeek R1 (7B). Do you think this approach is viable for creating an efficient financial expert bot, or would I inevitably need a larger model with more training data to achieve good performance?

2️⃣ GPU Rental Services for Training – Has anyone used cloud GPU services (Lambda Labs, RunPod, Vast.ai, etc.) for fine-tuning? If so, what was your experience? Any recommendations in terms of cost-effectiveness and reliability?

3️⃣ Fine-Tuning & RAG Best Practices – From my research, dataset quality is one of the most critical factors in fine-tuning. Any suggestions on methodologies or tools to ensure high-quality datasets? Are there any pitfalls or best practices you’ve learned from experience?

4️⃣ Challenges & Lessons Learned – This field is vast, with multiple factors affecting the final model's quality, such as quantization, dataset selection, and optimization techniques. This thread also serves as an opportunity to hear from those who have fine-tuned LLMs for other use cases, even if not in finance. What were your biggest challenges? What would you do differently in hindsight?

I’m eager to learn from those who have gone through similar journeys and to discuss what to expect along the way. Any feedback is greatly appreciated! 🚀

Thanks in advance!