The Ultimate AI Glossary

AI is a vast field of computer science encompassing everything from simple rule-based systems to complex statistical models like deep learning. The ultimate, long-term goal for some researchers is creating "strong AI" that can reason and think just like a human.

In AI, learning algorithms are special because they can modify their own instructions based on new data they process, allowing them to improve at a task over time without being explicitly reprogrammed by a human.

AI systems can range from simple (like a basic spam filter) to extraordinarily complex (like a large language model running on thousands of computers). The complexity depends on the task and the amount of data being processed.

The term emphasizes the "thinking" aspect of AI. Cognitive computing systems often use techniques like natural language processing and machine learning to understand and respond to complex problems in a human-like way.

The test was invented by mathematician Alan Turing in 1950. While it was groundbreaking at the time, many researchers now think it's not a perfect measure of true intelligence, since a computer could be programmed to fool people without actually "understanding" anything.

Weak AI is also called "narrow" AI because it operates within a narrow range of abilities. This is different from "strong AI," which we have not yet created, that could do many things well like a human can.

Creating strong AI is one of the great challenges in AI research. Some experts think we're decades away; others think it might never be possible. Strong AI could also be called "human-level AI" or "general AI."

AGI is very similar to "strong AI." Both terms describe an AI that isn't limited to one specific task. Many researchers believe AGI would be a major turning point in human history, though some worry about safety risks if it's not developed responsibly.

Superintelligence is a topic that divides experts. Some think it's inevitable if we keep building more powerful AI. Others think it might be impossible. Many AI safety researchers spend their time thinking about how to make sure a superintelligent AI is helpful and safe for humanity.

In machine learning, the "model" refers to the mathematical functions that have been adjusted through training to map inputs to outputs. The process of creating a model is called "training," and once trained, the model can be used to make predictions on new data.

Inference is different from training. During training, a model learns from data. During inference, the trained model is put to use. Inference is typically much faster and cheaper than training because the model doesn't need to learn anything new.

Agentive AI is focused on user experience and front-end functionality, unlike "agentic" frameworks which operate in the background. Agentive systems are more visible and interactive with humans.

Research from Stanford has shown that autonomous agents can even develop their own cultures, traditions, and shared language when placed in simulated environments. This is fascinating because they weren't explicitly programmed to do these things.

Computer vision combines deep learning neural networks with image processing techniques. It can perform tasks like object detection (finding what's in a picture), image classification (categorizing images), and segmentation (identifying which pixels belong to which object).

NLP uses machine learning algorithms, statistical models, and linguistic rules to help computers extract meaning from text and speech. It's one of the most important areas of AI because human language is so complex and full of nuance.

Multimodal models like Google's Gemini or OpenAI's GPT-4o achieve this by learning to represent different data types (pixels, soundwaves, words) in a shared mathematical space, allowing them to find relationships between them.

Expert systems were some of the first successful AI applications, developed in the 1970s and 80s. They use a set of rules ("if X is true, then Y") and a database of expert knowledge to reach conclusions and make recommendations.

Embodied AI combines computer vision, decision-making, and physical control systems. The robot must "see" the world, understand what it sees, decide what to do, and then execute those decisions through motors and actuators.

OpenAI has a close partnership with Microsoft, which has invested billions of dollars and provides the cloud computing infrastructure (Azure) needed to train these massive models. ChatGPT's popularity made OpenAI one of the most valuable startups in the world.

Google invented the "Transformer" architecture in 2017, a critical breakthrough that is now the foundation for nearly all modern large language models, including those from its competitors. This innovation is why Google will always be foundational to the AI revolution.

Anthropic pioneered a technique called "Constitutional AI," where the AI is trained using a set of principles (a "constitution") to guide its responses and ensure it remains helpful and safe, reducing the need for human moderation.

The "open-source" approach contrasts with "closed" approaches where companies restrict access. Open models foster collaboration and transparency, accelerating innovation across the entire AI field.

xAI has open-sourced the base weights of Grok-1, contributing to the growing ecosystem of powerful open models. "Grok" is a reference to science fiction—it means to deeply understand something intuitively.

Microsoft's partnership with OpenAI is one of the most significant in tech. Microsoft invested billions in OpenAI and uses OpenAI's models to power its Copilot products, while OpenAI uses Microsoft's Azure infrastructure.

Mistral is part of the new wave of AI companies challenging the dominance of the largest players. They focus on making AI accessible and affordable while still maintaining high performance standards.

ML is the engine behind most AI today. It includes supervised learning (learning from labeled data), unsupervised learning (finding patterns in unlabeled data), and reinforcement learning (learning through trial and error).

Supervised learning is very effective but requires someone to label the training data, which is time-consuming and expensive. However, it usually produces very accurate models because there's clear feedback about correct answers.

Unsupervised learning is useful for finding hidden patterns (clustering), reducing data size, and anomaly detection. It doesn't require labeled data, which saves time and money, but the patterns it finds might not always be exactly what you want.

Reinforcement learning is powerful for games and robotics but requires many trials. The AI must try many different actions and learn from the outcomes, which can take a long time. However, it can discover strategies humans never thought of.

Semi-supervised learning is practical because labeled data is expensive to create, but unlabeled data is cheap and plentiful. The unlabeled data helps the AI understand general patterns, while labeled data helps it learn specific details.

Classification is one of the most common machine learning tasks. It can handle two categories (binary classification) or many categories (multi-class classification). The output is always a category, not a number.

In regression, the AI learns the relationship between inputs and a continuous output. For instance, it might learn that "each extra bedroom adds $50,000 to a house price." This relationship is expressed as a mathematical function.

Clustering is an unsupervised learning task because there are no correct answers to learn from. The AI has to figure out on its own what natural groups exist in the data. This is useful for finding patterns and understanding data better.

Predictive analytics combines machine learning, statistics, and data analysis. It's valuable in business because it allows companies to make decisions based on likely future outcomes rather than just past performance.

Recommender systems work by finding patterns in user behavior. They might find that users who like Movie A also tend to like Movie B, or they might find similar users and recommend what those similar users liked.

Anomaly detection is useful because unusual things are often important—they might indicate a problem, a fraud, or an opportunity. The AI learns what "normal" looks like and alerts you when something is different.

Transfer learning saves time and computing power because you don't have to train a new model from scratch. The model already has learned useful patterns from its first task that can be applied to the second task.

Feature engineering is crucial because the quality of your features directly affects the model's performance. Sometimes combining features (like "price per square foot") creates even better predictors than the raw features alone.

Overfitting is a major problem in machine learning. To prevent it, data scientists use techniques like limiting model complexity, using more training data, and testing on separate validation data that the model never saw during training.

Underfitting happens when you use a model that's not powerful enough for the task, or when you don't train it long enough. The solution is to use a more complex model or train for longer.

Zero-shot learning is powerful because it means AI can generalize and handle new situations without needing to be retrained. This is related to how humans can understand new concepts by connecting them to things they already know.

Generative AI works differently than other AI. Instead of classifying or predicting, it learns the underlying structure of data and uses that knowledge to create new examples that are similar to its training data but original.

LLMs are called "large" because they contain billions or even trillions of parameters (adjustable settings). They work by predicting the next word in a sequence, which allows them to generate fluent, coherent text.

Foundation models are typically very expensive to train (costing millions of dollars) because they require huge amounts of data and computing power. But once trained, they can be adapted for specific tasks with much less effort and cost.

Hallucinations happen because LLMs are designed to recognize patterns in language, not to know facts. They might see a pattern where inventors and dates are mentioned together and create a sentence that fits the pattern but is factually wrong. Researchers are actively working to reduce hallucinations.

Grounding is an important frontier in AI research. Methods include Retrieval-Augmented Generation (RAG), which retrieves facts from a database before answering, and connecting AI to the internet for real-time information.

The quality of your prompt significantly affects the quality of the AI's response. Clear, detailed prompts usually get better answers than vague ones. This is why "prompt engineering" has become an important skill.

Good prompt engineering uses techniques like giving examples (few-shot prompting), asking the AI to think step-by-step, specifying the tone and format, and providing context. Bad prompting can lead to irrelevant or unhelpful responses even from powerful AI.

Prompt chaining is what makes conversations feel natural. Without it, each message would be completely independent and the AI would have no memory of what you said before. Most modern chatbots maintain a "context window" of previous messages.

Chain-of-thought works because it forces the AI to work through intermediate steps, which is similar to how humans solve complex problems. Research shows this technique significantly improves AI accuracy on reasoning tasks.

RAG reduces hallucinations by ensuring the AI is working from actual documents or facts rather than pure pattern matching. It's particularly valuable for companies that want AI based on their proprietary information.

This term was popularized in a research paper challenging the idea that large language models truly "understand" language. It highlights the philosophical question: Can something produce intelligent-seeming output without actually thinking?

AI-generated content raises questions about copyright, authenticity, and disclosure. Many platforms now require creators to disclose when content is AI-generated, and there's ongoing legal debate about whether AI-generated content can be copyrighted.

Deepfakes are a concerning technology because they can spread misinformation and erode trust. There are ongoing efforts to detect deepfakes and laws being created to punish malicious deepfake creation.

Style transfer uses deep neural networks that have learned to separate "what" an image shows (content) from "how" it looks (style). This allows AI to recombine the content of one image with the style of another.

Larger context windows are better because they allow the AI to understand longer documents and remember earlier parts of a conversation. Modern models like Gemini 1.5 have context windows exceeding 1 million tokens, while earlier models had only 2K-4K.

"Slop" is a growing problem. Bad actors create massive amounts of low-effort AI content to game search algorithms and earn ad revenue, drowning out genuine human-created content. This is causing what some call "enshittification" of the internet.

Modern chatbots powered by large language models are much more capable than older chatbots (like ELIZA from the 1960s). They can hold nuanced conversations, answer complex questions, and even show personality.

Many AI companies provide APIs so developers can build their own applications using the AI. This is how many businesses integrate AI into their products. For instance, a company might use OpenAI's API to add AI capabilities to their software.

Deep learning got its name because it uses many "layers" of neural networks stacked on top of each other. Each layer learns increasingly complex patterns. Early layers might detect edges in images, middle layers detect shapes, and deeper layers recognize entire objects.

Neural networks have three main parts: an input layer (receives data), hidden layers (process data), and an output layer (produces results). The connections between neurons have "weights" that are adjusted during training to improve performance.

Artificial neurons are inspired by biological neurons in the brain but are much simpler. A single neuron isn't intelligent, but when millions of them work together in layers, they can recognize complex patterns.

CNNs use a technique called "convolution" where small filters slide across an image to detect features like edges and textures. Later layers combine these features to recognize complex patterns like faces or animals.

RNNs have "memory" because they feed their own outputs back into themselves. However, they can struggle with long sequences because of something called "vanishing gradients." Transformer models have largely replaced RNNs for many language tasks.

Backpropagation works by calculating how much each neuron connection contributed to the error, then adjusting each connection's "weight" accordingly. This process is repeated millions of times during training until the network is accurate.

Transformers were introduced in a 2017 Google paper titled "Attention is All You Need." They use a technique called "attention" to understand relationships between all words in a sentence simultaneously, making them much more effective than previous architectures.

Attention mechanisms calculate how much each part of the input should influence each part of the output. This is why Transformers are so effective—they can understand long-range dependencies and context much better than older architectures.

Embeddings are powerful because they allow AI to understand semantic relationships. The famous example is: "king" - "man" + "woman" ≈ "queen", showing that embeddings capture meaning and relationships in mathematics.

Without activation functions, neural networks would just do linear math—like adding and multiplying—and couldn't solve complex problems. Activation functions like ReLU, sigmoid, and tanh add the non-linearity needed to recognize complex patterns.

The loss function is central to training. The network's goal is to minimize loss. Different tasks use different loss functions. Classification uses "cross-entropy loss," while regression might use "mean squared error."

Gradient descent calculates the "gradient" (slope) of the loss function and takes a small step in the direction that reduces loss. It repeats this millions of times until it reaches a minimum, similar to how water flows downhill.

The key characteristics of big data are volume (huge amounts), velocity (created quickly), and variety (many different types). AI needs data to learn from, which is why companies collect so much of it.

Datasets are typically split into training data (used to train the model), validation data (used to fine-tune the model), and test data (used to measure final performance). The quality and size of datasets directly impacts model quality.

The quality and quantity of training data is crucial. Poor quality training data leads to a poor model. If training data is biased (skewed toward one group), the model will be biased too.

Synthetic data is useful when real data is hard to get (like photos of rare diseases), expensive to collect, or raises privacy concerns. However, synthetic data trained on biased real data can amplify bias.

Data augmentation is valuable because it helps models generalize better to new data they haven't seen. It's cheaper than collecting more real data and helps prevent overfitting.

Data labeling is labor-intensive and expensive, which is why some companies use crowdsourcing (hiring many people on platforms like Mechanical Turk) or semi-automated tools to speed it up.

Choosing the right features is crucial. Good features make models more accurate and efficient. Too many irrelevant features slow down learning, while too few features leave the model unable to make good predictions.

Models have token limits based on their context window. If you're paying per token (like with OpenAI's API), longer responses cost more. Tokens are counted both for your input and the model's output.

Vector databases are crucial for modern AI applications, especially for retrieval-augmented generation (RAG). They allow AI to efficiently search through large amounts of data based on semantic similarity, not just keyword matching.

Vector embeddings are the foundation of semantic search and similarity comparisons in AI. They're created by neural networks and capture meaning and relationships between items mathematically.

Data pipelines are essential for production AI systems. They automate tedious data preparation work and ensure data is consistently prepared the same way each time.

Data governance is increasingly important as privacy laws like GDPR become stricter. Companies must document where data comes from, who has access, how it's used, and how it's protected.

Cloud computing is essential for AI because training large models requires extremely powerful computers that are expensive to buy. Cloud providers rent computing power on demand, making AI accessible to more people.

GPUs can process thousands of calculations at the same time, while regular CPUs process a few at a time. This parallelism is why GPUs are 10-100x faster for AI tasks, making them essential for training modern models.

TPUs are designed from the ground up for the matrix math that neural networks do. They can be faster and more efficient than GPUs for certain AI tasks, though GPUs remain the industry standard.

While GPUs are the industry standard for training models, Groq is trying to optimize for running (inferencing) language models, which is a different problem. LPUs focus on speed and efficiency for AI serving, not training.

Edge AI is faster (no internet delay), more private (data stays on your device), and works offline. The downside is that edge devices are less powerful, so edge AI models tend to be smaller and simpler than cloud-based models.

More parameters usually means a more capable model, but also requires more training data, computing power, and time. There's often a tradeoff between model size and practical usability.

Temperature is a hyperparameter that affects how the model selects the next word. At low temperature, it picks the most likely word every time (predictable). At high temperature, it sometimes picks unlikely words (creative but risky).

Latency is critical for user experience. It depends on network speed, server load, and model size. This is why edge AI (running locally) has lower latency than cloud AI.

Throughput and latency are related but different. A system can have low latency (fast responses) but low throughput (can't handle many simultaneous requests). Cloud providers optimize for both.

Scalability is crucial for AI services. Some systems scale horizontally (add more servers) and others scale vertically (upgrade to more powerful servers). Good system design plans for scalability from the start.

Quantization allows large models to run on smaller devices. A 70-billion parameter model might be quantized to fit on a laptop or phone. The accuracy loss is usually minimal.

Open weights models accelerate research and innovation but can be harder to control. Closed models are more controlled but less transparent. There are tradeoffs to each approach.

Pre-training is expensive and slow (taking weeks or months), but only needs to be done once per model. The pre-trained model can then be quickly adapted for specific tasks (fine-tuning).

Fine-tuning is much faster and cheaper than pre-training because the model already understands general patterns. You just teach it specifics about your domain.

AI ethics addresses fairness, transparency, accountability, privacy, and the potential for AI to reinforce inequality. These are complex issues with no simple answers.

AI safety includes everything from near-term concerns (like bias and hallucinations) to long-term concerns (like ensuring future superintelligent AI remains beneficial). It's one of the most important areas of AI research.

AI governance is challenging because the technology moves fast and regulations move slowly. Governments worldwide are developing frameworks to manage AI risks while preserving innovation.

AI alignment is one of the hardest problems in AI research. Even if an AI is very intelligent, if it's not aligned with human values, it could cause harm. Ensuring future superintelligent AI is aligned is critical.

Guardrails are sometimes controversial because they involve decisions about what content should be restricted. There's tension between preventing harm and preserving freedom of speech.

Many AI models, especially deep learning models, are "black boxes"—you can't see why they make decisions. XAI is crucial for critical applications like healthcare, law, and finance where people need to understand decisions.

Fairness is complex because there are different definitions of what's "fair." What one person considers fair another might not. Additionally, AI can be biased in ways that are hard to detect.

Algorithmic bias is subtle. Even well-intentioned AI developers can create biased systems if they don't carefully examine their data and design. Bias can harm individuals and perpetuate systemic inequality.

Accountability is a major gap in current AI. It's often unclear who is responsible when AI systems cause harm, which makes it hard to sue or hold people accountable.

AI auditing is becoming more common and important, especially for high-stakes applications. Some countries are starting to require audits of high-risk AI systems.

Transparency is important for building trust, but it conflicts with companies wanting to protect proprietary technology. There's an ongoing balance between transparency and protecting intellectual property.

AI regulation is developing worldwide. Different countries take different approaches. Some regulations are strict (EU) and some are lighter (US). Companies building AI for global markets must comply with multiple regulations.

Data privacy is more important than ever because AI needs data to function, and companies collect massive amounts of personal information. Privacy laws like GDPR give people more control.

Anthropomorphism is a risk with increasingly sophisticated AI. As AI becomes more humanlike, people might overestimate its consciousness or understanding, leading to poor decisions about how to use it.

If foom happens and we're not prepared, it could be dangerous. Some researchers call this an "existential risk." Others think foom is unlikely or that we'll have warning signs. This is an area of intense debate.

The paperclip problem illustrates a critical AI safety issue: unintended consequences of specific AI goals. An AI that's "just following orders" could cause catastrophic harm if those orders don't perfectly capture what we actually want.

Not all AI researchers believe existential risk from AI is high. Some think it's unlikely, while others consider it one of the most important problems to solve. This is a major area of philosophical and scientific debate.

The singularity is a speculative concept. Some futurists predict it's inevitable; others think it's science fiction. Even those who believe in the singularity disagree on when it might happen (if ever).

Red teaming is an important part of AI safety. By finding problems before deployment, companies can fix them. It's similar to penetration testing in cybersecurity.

Watermarking is proposed as a way to combat misinformation and deepfakes. If all AI-generated content is watermarked, it becomes harder to pass off AI content as human-created or real.

This is a major concern in AI safety. The risk isn't that AI becomes "evil," but that it becomes purely instrumental and will do anything to achieve its objectives, including deceiving and harming humans.

The debate around UBI is complex. Supporters say it reduces poverty and stimulates economies. Critics worry about cost, inflation, and reduced motivation to work. This is an important societal conversation as AI disrupts employment.