Computer Β· Part D - Data, Connectivity, Intelligence and Interactive Computing Β· Major topic 21 of 39
Machine Learning, Deep Learning, Generative AI, LLMs and AI Engineering
Learning patterns from data, building foundation-model applications, measuring performance and deploying systems responsibly. This domain depends on programming, data handling, probability, linear algebra and optimization.
308 items insideAsked 1Γ in the RPSC/RSSB papers we analysed, most recently 2024COM-21
Also look at: Science & Technology: Artificial Intelligence, Machine Learning, Robotics and Cyber-Physical SystemsChemistry: Cheminformatics, Chemical Data & AI/ML for ChemistryScience & Technology: Computer Architecture, Operating Systems, Programming and Software Systems
- 1
Historical evolution β key dates
- Early perceptrons and machine-learning ideas emerge.1950s
- Rosenblattβs perceptron becomes a landmark neural-learning model.1957
- Backpropagation is popularized for training multilayer neural networks.1986
- Statistical learning, support-vector machines, ensembles and large datasets grow.1990s-2000s
- Deep-learning methods attract renewed attention; GPU training later accelerates progress.2006
- AlexNetβs image-recognition results mark a major deep-learning inflection.2012
- βAttention Is All You Needβ introduces the Transformer architecture.2017
- GPT-3 highlights few-shot language-model capabilities.2020
- ChatGPT is released publicly as a research preview.2022
- Multimodal foundation models, retrieval-augmented generation, tool-using agents, evaluations, safety and inference efficiency become major engineering topics.2023-2026
- 2
Machine-learning problem types
- Supervised; unsupervised; semi-supervised; self-supervised; reinforcement; online/incremental; transfer learning
7 more points
- Supervised
- Unsupervised
- Semi-supervised
- Self-supervised
- Reinforcement
- Online/incremental
- Transfer learning
- Classification; regression; ranking; clustering; dimensionality reduction; anomaly detection; forecasting; recommendation
8 more points
- Classification
- Regression
- Ranking
- Clustering
- Dimensionality reduction
- Anomaly detection
- Forecasting
- Recommendation
- Inputs/features; targets/labels; training/validation/test splits; distribution shift; deployment context
5 more points
- Inputs/features
- Targets/labels
- Training/validation/test splits
- Distribution shift
- Deployment context
- Supervised; unsupervised; semi-supervised; self-supervised; reinforcement; online/incremental; transfer learning
- 3
Data and features
- Dataset version; schema; collection; labeling; data quality; leakage; imbalance; sampling; deduplication
9 more points
- Dataset version
- Schema
- Collection
- Labeling
- Data quality
- Leakage
- Imbalance
- Sampling
- Deduplication
- Scaling; normalization; categorical encoding; text/image/audio representation; missing-data strategies
5 more points
- Scaling
- Normalization
- Categorical encoding
- Text/image/audio representation
- Missing-data strategies
- Feature engineering; feature selection; embeddings; feature store; lineage; consent and provenance
6 more points
- Feature engineering
- Feature selection
- Embeddings
- Feature store
- Lineage
- Consent and provenance
- Dataset version; schema; collection; labeling; data quality; leakage; imbalance; sampling; deduplication
- 4
Classical algorithms
- Linear/logistic regression; k-nearest neighbours; naive Bayes; decision trees; random forests
5 more points
- Linear/logistic regression
- K-nearest neighbours
- Naive Bayes
- Decision trees
- Random forests
- Gradient boosting; support-vector machines; k-means; hierarchical clustering; DBSCAN; PCA; matrix factorization
7 more points
- Gradient boosting
- Support-vector machines
- K-means
- Hierarchical clustering
- DBSCAN
- PCA
- Matrix factorization
- Calibration; regularization; model selection; ensemble methods; interpretable models
5 more points
- Calibration
- Regularization
- Model selection
- Ensemble methods
- Interpretable models
- Linear/logistic regression; k-nearest neighbours; naive Bayes; decision trees; random forests
- 5
Neural-network foundations
- Neuron; layer; activation; loss; gradient; backpropagation; optimizer; batch; epoch; learning rate
10 more points
- Neuron
- Layer
- Activation
- Loss
- Gradient
- Backpropagation
- Optimizer
- Batch
- Epoch
- Learning rate
- MLP; CNN; RNN; LSTM/GRU; attention; Transformer; encoder/decoder; residual connection; normalization
9 more points
- MLP
- CNN
- RNN
- LSTM/GRU
- Attention
- Transformer
- Encoder/decoder
- Residual connection
- Normalization
- Initialization; vanishing/exploding gradients; dropout; weight decay; batch normalization; early stopping
6 more points
- Initialization
- Vanishing/exploding gradients
- Dropout
- Weight decay
- Batch normalization
- Early stopping
- Neuron; layer; activation; loss; gradient; backpropagation; optimizer; batch; epoch; learning rate
- 6
Model training and evaluation
- Baseline; split strategy; cross-validation; hyperparameter tuning; grid/random/Bayesian search
5 more points
- Baseline
- Split strategy
- Cross-validation
- Hyperparameter tuning
- Grid/random/Bayesian search
- Accuracy; precision; recall; F1; ROC-AUC; PR-AUC; calibration; MAE/MSE/RMSE; ranking metrics
9 more points
- Accuracy
- Precision
- Recall
- F1
- ROC-AUC
- PR-AUC
- Calibration
- MAE/MSE/RMSE
- Ranking metrics
- Error analysis; subgroup evaluation; confidence intervals; robustness; fairness; interpretability; reproducibility
7 more points
- Error analysis
- Subgroup evaluation
- Confidence intervals
- Robustness
- Fairness
- Interpretability
- Reproducibility
- Baseline; split strategy; cross-validation; hyperparameter tuning; grid/random/Bayesian search
- 7
Deep-learning specialties
- Computer vision; object detection; segmentation; OCR; pose estimation; speech and audio; time series
7 more points
- Computer vision
- Object detection
- Segmentation
- OCR
- Pose estimation
- Speech and audio
- Time series
- Graph neural networks; recommender systems; multimodal learning; self-supervised representation learning
4 more points
- Graph neural networks
- Recommender systems
- Multimodal learning
- Self-supervised representation learning
- Computer vision; object detection; segmentation; OCR; pose estimation; speech and audio; time series
- 8
Generative modelling
- Autoregressive models; variational autoencoders; GANs; diffusion models; normalizing flows; energy-based models
6 more points
- Autoregressive models
- Variational autoencoders
- GANs
- Diffusion models
- Normalizing flows
- Energy-based models
- Text/image/audio/video/code generation; conditioning; guidance; sampling; controllability; watermarking concepts
6 more points
- Text/image/audio/video/code generation
- Conditioning
- Guidance
- Sampling
- Controllability
- Watermarking concepts
- Autoregressive models; variational autoencoders; GANs; diffusion models; normalizing flows; energy-based models
- 9
LLM foundations and application design
- Tokenization; vocabulary; embeddings; positional representation; attention; context window; pretraining objective
7 more points
- Tokenization
- Vocabulary
- Embeddings
- Positional representation
- Attention
- Context window
- Pretraining objective
- Pretraining; instruction tuning; supervised fine-tuning; preference optimization; RLHF/DPO concepts; quantization
6 more points
- Pretraining
- Instruction tuning
- Supervised fine-tuning
- Preference optimization
- RLHF/DPO concepts
- Quantization
- Prompt structure; system/developer/user instructions; examples; structured output; tool definitions; context management
6 more points
- Prompt structure
- System/developer/user instructions
- Examples
- Structured output
- Tool definitions
- Context management
- Hallucination; grounding; citation checking; refusal behavior; uncertainty; prompt injection; model limitations
7 more points
- Hallucination
- Grounding
- Citation checking
- Refusal behavior
- Uncertainty
- Prompt injection
- Model limitations
- Tokenization; vocabulary; embeddings; positional representation; attention; context window; pretraining objective
- 10
Retrieval-augmented generation (RAG)
- Document ingestion; parsing; chunking; metadata; embeddings; vector index; lexical index; hybrid retrieval
8 more points
- Document ingestion
- Parsing
- Chunking
- Metadata
- Embeddings
- Vector index
- Lexical index
- Hybrid retrieval
- Query rewriting; filters; reranking; context assembly; citations; answer generation; abstention
7 more points
- Query rewriting
- Filters
- Reranking
- Context assembly
- Citations
- Answer generation
- Abstention
- Evaluation set; retrieval recall; groundedness; context precision; answer relevance; latency/cost trade-off
6 more points
- Evaluation set
- Retrieval recall
- Groundedness
- Context precision
- Answer relevance
- Latency/cost trade-off
- Document ingestion; parsing; chunking; metadata; embeddings; vector index; lexical index; hybrid retrieval
- 11
Agents and tool-using systems
- Model vs agent; goal; plan; tool schema; action; observation; state; memory; orchestration
9 more points
- Model vs agent
- Goal
- Plan
- Tool schema
- Action
- Observation
- State
- Memory
- Orchestration
- Single-agent workflow; multi-agent coordination; task decomposition; handoff; retry; stop conditions; human approval
7 more points
- Single-agent workflow
- Multi-agent coordination
- Task decomposition
- Handoff
- Retry
- Stop conditions
- Human approval
- Sandboxing; least privilege; tool permissions; prompt injection; secret leakage; audit logs; failure recovery
7 more points
- Sandboxing
- Least privilege
- Tool permissions
- Prompt injection
- Secret leakage
- Audit logs
- Failure recovery
- Model vs agent; goal; plan; tool schema; action; observation; state; memory; orchestration
- 12
AI engineering and MLOps
- Data/model pipeline; experiment tracking; model registry; feature store; deployment; inference server; batching
7 more points
- Data/model pipeline
- Experiment tracking
- Model registry
- Feature store
- Deployment
- Inference server
- Batching
- Offline/online inference; model serving; monitoring; drift; retraining; rollback; canary; shadow testing
8 more points
- Offline/online inference
- Model serving
- Monitoring
- Drift
- Retraining
- Rollback
- Canary
- Shadow testing
- GPU/accelerator basics; throughput; latency; tokens per second; caching; quantization; distillation; batching
8 more points
- GPU/accelerator basics
- Throughput
- Latency
- Tokens per second
- Caching
- Quantization
- Distillation
- Batching
- Versioned datasets; reproducible environments; CI/CD/CT; model card; dataset documentation; lineage
6 more points
- Versioned datasets
- Reproducible environments
- CI/CD/CT
- Model card
- Dataset documentation
- Lineage
- Data/model pipeline; experiment tracking; model registry; feature store; deployment; inference server; batching
- 13
AI safety, governance and frontier questions
- Risk assessment; NIST AI RMF; generative-AI risk profile; data rights; privacy; security; provenance
7 more points
- Risk assessment
- NIST AI RMF
- Generative-AI risk profile
- Data rights
- Privacy
- Security
- Provenance
- Evaluation contamination; benchmark overfitting; red-teaming; adversarial robustness; jailbreak resistance
5 more points
- Evaluation contamination
- Benchmark overfitting
- Red-teaming
- Adversarial robustness
- Jailbreak resistance
- Synthetic data; model collapse risk; copyright questions; bias; accessibility; labour and environmental impact
6 more points
- Synthetic data
- Model collapse risk
- Copyright questions
- Bias
- Accessibility
- Labour and environmental impact
- Interpretability; mechanistic interpretability; scalable oversight; agentic-system evaluation; post-training research
5 more points
- Interpretability
- Mechanistic interpretability
- Scalable oversight
- Agentic-system evaluation
- Post-training research
- Risk assessment; NIST AI RMF; generative-AI risk profile; data rights; privacy; security; provenance
