A complete implementation of a Vanilla Generative Adversarial Network trained on MNIST handwritten digits, built with TensorFlow/Keras. This project covers the full pipeline: architecture, training loop, visualization, latent space analysis, and results.
flowchart TB
subgraph INPUTS["INPUTS"]
direction LR
Z["Noise Vector z
Shape: batch x 100
Distribution: N(0,1)"]
REAL["Real Images x
Shape: batch x 28 x 28 x 1
Source: MNIST Range: -1 to +1"]
end
subgraph GEN["GENERATOR — G(z)"]
direction TB
G_IN["Input — Shape: 100
Random noise z"]
subgraph G_B1["Block 1"]
direction LR
G_D1["Dense — 256 units"]
G_L1["LeakyReLU — alpha=0.2"]
G_BN1["BatchNorm — momentum=0.8"]
G_D1 --> G_L1 --> G_BN1
end
subgraph G_B2["Block 2"]
direction LR
G_D2["Dense — 512 units"]
G_L2["LeakyReLU — alpha=0.2"]
G_BN2["BatchNorm — momentum=0.8"]
G_D2 --> G_L2 --> G_BN2
end
subgraph G_B3["Block 3"]
direction LR
G_D3["Dense — 1024 units"]
G_L3["LeakyReLU — alpha=0.2"]
G_BN3["BatchNorm — momentum=0.8"]
G_D3 --> G_L3 --> G_BN3
end
subgraph G_B4["Output Block"]
direction LR
G_D4["Dense — 784 units
28 x 28 x 1"]
G_TANH["tanh activation
Range: -1 to +1"]
G_RESHAPE["Reshape
784 to 28x28x1"]
G_D4 --> G_TANH --> G_RESHAPE
end
G_IN --> G_B1 --> G_B2 --> G_B3 --> G_B4
end
FAKE["Fake Image G(z)
Shape: batch x 28 x 28 x 1
Synthesized MNIST digit"]
subgraph DISC["DISCRIMINATOR — D(x)"]
direction TB
D_IN["Input — Shape: 28 x 28 x 1
Receives real OR fake image"]
subgraph D_FLAT["Preprocessing"]
FLAT["Flatten
28x28x1 to 784"]
end
subgraph D_B1["Block 1"]
direction LR
D_D1["Dense — 512 units"]
D_L1["LeakyReLU — alpha=0.2"]
D_DR1["Dropout — rate=0.3"]
D_D1 --> D_L1 --> D_DR1
end
subgraph D_B2["Block 2"]
direction LR
D_D2["Dense — 256 units"]
D_L2["LeakyReLU — alpha=0.2"]
D_DR2["Dropout — rate=0.3"]
D_D2 --> D_L2 --> D_DR2
end
subgraph D_B3["Output Block"]
direction LR
D_D3["Dense — 1 unit"]
D_SIG["sigmoid activation
Range: 0 to 1"]
D_D3 --> D_SIG
end
D_IN --> D_FLAT --> D_B1 --> D_B2 --> D_B3
end
D_OUT["Discriminator Score
0.0 = Fake — 1.0 = Real"]
subgraph LOSSES["LOSS FUNCTIONS — Binary Cross Entropy"]
direction LR
subgraph D_LOSS["Discriminator Loss"]
direction TB
DL1["Real Loss
BCE(D(x), 1)
Target: D(real) = 1"]
DL2["Fake Loss
BCE(D(G(z)), 0)
Target: D(fake) = 0"]
DL3["Total D Loss
= Real Loss + Fake Loss"]
DL1 --> DL3
DL2 --> DL3
end
subgraph G_LOSS["Generator Loss"]
GL1["Adversarial Loss
BCE(D(G(z)), 1)
Target: fool D
Make D(fake) = 1"]
end
end
subgraph OPTIM["OPTIMIZERS — Adam lr=0.0002 beta1=0.5"]
direction LR
OPT_G["Generator Optimizer
Adam lr=0.0002
beta_1=0.5 beta_2=0.999
Updates G weights only"]
OPT_D["Discriminator Optimizer
Adam lr=0.0002
beta_1=0.5 beta_2=0.999
Updates D weights only"]
end
subgraph TRAIN["TRAINING LOOP — Per Batch"]
direction TB
T1["Step 1 — Sample
z ~ N(0,1) shape: 128x100
x ~ MNIST shape: 128x28x28x1"]
T2["Step 2 — Forward Pass
Generate fakes via G(z)
Score real: D(x)
Score fake: D(G(z))"]
T3["Step 3 — Compute Losses
D loss = BCE(D(x),1) + BCE(D(G(z)),0)
G loss = BCE(D(G(z)),1)"]
T4["Step 4 — Backward Pass
Two separate GradientTapes
One for G — One for D"]
T5["Step 5 — Update Weights
D optimizer on D gradients
G optimizer on G gradients
Fully independent updates"]
T1 --> T2 --> T3 --> T4 --> T5
end
subgraph PARAMS["MODEL PARAMETERS"]
direction LR
P1["Generator — 1,493,520 params
Block1: 100x256+bias = 25,856
Block2: 256x512+bias = 131,584
Block3: 512x1024+bias = 525,312
Output: 1024x784+bias = 803,600
BatchNorm = 7,168"]
P2["Discriminator — 533,505 params
Block1: 784x512+bias = 401,920
Block2: 512x256+bias = 131,328
Output: 256x1+bias = 257
Combined Total = 2,027,025"]
end
Z -->|"z shape: batch x 100"| GEN
GEN -->|"G(z) shape: batch x 28x28x1"| FAKE
FAKE -->|"Fake path — D targets 0"| DISC
REAL -->|"Real path — D targets 1"| DISC
DISC --> D_OUT
D_OUT -->|"Scores D(x) and D(G(z))"| LOSSES
LOSSES -->|"Gradients"| OPTIM
OPT_G -->|"Update G — D frozen"| GEN
OPT_D -->|"Update D — G frozen"| DISC
classDef inputNode fill:#1e3a5f,stroke:#3b82f6,stroke-width:2px,color:#bfdbfe
classDef genNode fill:#14532d,stroke:#22c55e,stroke-width:2px,color:#bbf7d0
classDef discNode fill:#4c1d95,stroke:#a855f7,stroke-width:2px,color:#e9d5ff
classDef lossNode fill:#7c2d12,stroke:#f97316,stroke-width:2px,color:#fed7aa
classDef optimNode fill:#0c4a6e,stroke:#38bdf8,stroke-width:2px,color:#bae6fd
classDef trainNode fill:#1c1917,stroke:#a8a29e,stroke-width:2px,color:#e7e5e4
classDef paramNode fill:#1e1b4b,stroke:#6366f1,stroke-width:1px,color:#c7d2fe
classDef outputNode fill:#422006,stroke:#f59e0b,stroke-width:2px,color:#fde68a
class Z,REAL inputNode
class G_IN,G_D1,G_L1,G_BN1,G_D2,G_L2,G_BN2,G_D3,G_L3,G_BN3,G_D4,G_TANH,G_RESHAPE genNode
class D_IN,FLAT,D_D1,D_L1,D_DR1,D_D2,D_L2,D_DR2,D_D3,D_SIG discNode
class DL1,DL2,DL3,GL1 lossNode
class OPT_G,OPT_D optimNode
class T1,T2,T3,T4,T5 trainNode
class P1,P2 paramNode
class FAKE,D_OUT outputNode
Two fully-connected networks compete in a minimax game:
GENERATOR
Input: Noise vector (100,)
Dense(256) -> LeakyReLU(0.2) -> BatchNorm
Dense(512) -> LeakyReLU(0.2) -> BatchNorm
Dense(1024) -> LeakyReLU(0.2) -> BatchNorm
Dense(784) -> tanh -> Reshape(28, 28, 1)
Output: Fake image in [-1, 1]
DISCRIMINATOR
Input: Image (28, 28, 1)
Flatten(784)
Dense(512) -> LeakyReLU(0.2) -> Dropout(0.3)
Dense(256) -> LeakyReLU(0.2) -> Dropout(0.3)
Dense(1) -> sigmoid
Output: P(real) in [0, 1]
Training objective (Minimax):
tensorflow
numpy
matplotlib
Install with:
pip install tensorflow numpy matplotlibOpen Vanilla_GAN_Implementation.ipynb and run all cells. All output files are saved to an outputs/ folder automatically.
OUTPUT_DIR = 'outputs' # All PNGs and model files go here| Parameter | Value |
|---|---|
| Epochs | 50 |
| Batch size | 128 |
| Noise dimension | 100 |
| Learning rate | 0.0002 |
| Optimizer | Adam (beta_1=0.5) |
| Loss function | Binary Cross Entropy |
| Model | Parameters |
|---|---|
| Generator | 1,493,520 |
| Discriminator | 533,505 |
| Total | 2,027,025 |
| Metric | Value |
|---|---|
| Initial D Loss | 0.5738 |
| Final D Loss | 1.3276 |
| Initial G Loss | 2.3641 |
| Final G Loss | 0.7822 |
| Nash Equilibrium target | 0.6931 |
| Total training time | 4.8 minutes |
| Average time per epoch | 5.8 seconds |
The same 16 fixed noise vectors are passed through the Generator at each checkpoint, making it easy to track learning progress.
Epoch 000 — Pre-training (pure noise):
Epoch 001 — First patterns emerge:
Epoch 005 — Digit outlines forming:
Epoch 010 — Legible digits:
Epoch 015:
Epoch 020:
Epoch 025:
Epoch 030:
Epoch 035:
Epoch 040:
Epoch 045:
Epoch 050 — Final output:
Smooth linear interpolation between two random noise vectors z1 and z2, showing the Generator transitions continuously between digit styles.
| Epoch | Observation |
|---|---|
| 0 | Pure Gaussian noise — no structure |
| 1 | Blob shapes appear. D Loss drops sharply to 0.5738 as D immediately learns to separate real from fake |
| 5 | Noisy but recognizable digit outlines emerge |
| 10 | Clear digit structure — most outputs are legible |
| 20 | Digits sharpen. Stroke weight becomes consistent |
| 50 | High-quality handwritten-style digits with clean strokes |
- D Loss opened at 0.5738 (epoch 1) — the Discriminator learned extremely fast. This early dominance temporarily suppressed Generator learning.
- D Loss stabilized at ~1.33, which is 0.63 above the Nash equilibrium (0.693). This gap indicates D can still detect some fakes, which is expected and healthy — it means both networks are still actively competing.
- G Loss opened at 2.3641 (Generator was producing obvious noise) and fell consistently to 0.7822 by epoch 50 — confirming G learned to fool D with high reliability.
- Both losses plateau after epoch 25, indicating diminishing returns from further training with this architecture.
The interpolation plot shows smooth, continuous transitions between digit styles (3 → 8 style morph). This confirms the Generator learned a structured, continuous latent space rather than discretely memorizing training examples.
- Mild digit bias: Digits 0, 1, and 9 appear more frequently in generated outputs than 2, 5, and 8. This is a soft form of mode collapse typical in fully-connected GANs.
- Noise artifacts: Some images show faint noise specks. This is caused by the Dense architecture lacking the spatial inductive bias that convolutional layers provide.
- Loss plateau: Training stabilizes well before epoch 50 — the fully-connected design limits further quality improvement without architectural changes.
D Loss = G Loss = 0.693 <- Nash equilibrium = ln(2)
D Loss too low -> Discriminator too strong, Generator can't learn
G Loss too low -> Mode collapse (Generator produces the same image)
Oscillating losses is normal — GANs are inherently unstable by design
All files are saved to outputs/ during training:
| File | Description |
|---|---|
real_samples.png |
Sample real MNIST images used for training |
epoch_000.png — epoch_050.png |
Generator output at each visualization checkpoint |
final_generated.png |
8x8 grid of final generated images |
real_vs_fake.png |
Side-by-side real vs generated comparison |
training_curves.png |
D loss, G loss, and per-epoch timing charts |
interpolation.png |
Latent space linear interpolation (z1 → z2) |
discriminator_analysis.png |
Score distributions and box plots |
vanilla_gan_generator.keras |
Saved Generator model |
vanilla_gan_discriminator.keras |
Saved Discriminator model |
| Architecture | Improvement |
|---|---|
| DCGAN | Convolutional layers — sharper images, better spatial structure |
| WGAN | Wasserstein loss — more stable training and meaningful loss metric |
| Conditional GAN | Add class labels to control which digit is generated |
| StyleGAN | State-of-the-art for high-fidelity image generation |
- Goodfellow et al. (2014) — Generative Adversarial Nets
- Radford et al. (2015) — DCGAN
- Arjovsky et al. (2017) — Wasserstein GAN