# ML crash course - Production ML systems

> Machine learning crash course 중 "Production ML systems" 챕터.

[Machine learning crash course](https://wiki.g15e.com/pages/Machine%20learning%20crash%20course.txt) 중 "Production ML systems" 챕터.

https://developers.google.com/machine-learning/crash-course/production-ml-systems

## Introduction

## Static versus dynamic training

## Static versus dynamic inference

### Static inference

### Dynamic inference

## When to transform data?

### Transforming data before training

### Transforming data while training

## Deployment testing

### About the unicorn model

### Test model updates with reproducible training

### Test calls to machine learning API

### Write integration tests for pipeline components

### Validate model quality before serving

### Validate model-infrastructure compatibility before serving

## Monitoring pipelines

### Write a data schema to validate raw data

### Write unit tests to validate feature engineering

### Check metrics for important data slices

### Use real-world metrics

### Check for training-serving skew

### Check for label leakage

### Monitor model age throughout pipeline

### Test that model weights and outputs are numerically stable

### Monitor model performance

### Test the quality of live model on served data

### Randomization

## Questions to ask

### Is each feature helpful?

### Is your data source reliable?

### Is your model part of a feedback loop?

## What's next?

- [ML crash course - AutoML](https://wiki.g15e.com/pages/ML%20crash%20course%20-%20AutoML.txt)