目录
- 前言
- 1. NLP
- 1.1 Transformer架构
- 2. CV
- 2.1 CNN架构
- 2.2 Transformer架构
前言
本篇博客将记录深度学习领域常见模型的大小，具体算法如下
torchinfo.summary(model)
模型可能来自于PyTorch官方，HuggingFace等。
如有错误或者建议欢迎在评论区指出。
| 第三方库 | 版本 |
|---|---|
| transformers | 4.30.2 |
| PyTorch | 2.0.1 |
1. NLP
1.1 Transformer架构
Encoder-Only架构
| 模型 | 来源 | 总参数量 | 总参数量 |
|---|---|---|---|
| BERT-base | HuggingFace | 109,482,240 | 109.5M |
| BERT-large | HuggingFace | 335,141,888 | 335.1M |
| RoBERTa-base | HuggingFace | 124,645,632 | 124.6M |
| RoBERTa-large | HuggingFace | 355,359,744 | 355.3M |
| DeBERTa-base | HuggingFace | 138,601,728 | 138.6M |
| DeBERTa-large | HuggingFace | 405,163,008 | 405.2M |
| DeBERTa-xlarge | HuggingFace | 757,804,032 | 757.8M |
| DistilBERT | HuggingFace | 66,362,880 | 66.4M |
Decoder-Only架构文章来源：https://uudwc.com/A/orn50
| 模型 | 来源 | 总参数量 | 总参数量 |
|---|---|---|---|
| GPT | HuggingFace | 116,534,784 | 116.5M |
| GPT-2 | HuggingFace | 124,439,808 | 124.4M |
| GPT-2-medium | HuggingFace | 354,823,168 | 354.8M |
| GPT-2-large | HuggingFace | 774,030,080 | 774.0M |
| GPT-J | HuggingFace | 5,844,393,984 | 5.8B |
| LLaMA | HuggingFace | 6,607,343,616 | 6.6B |
Encoder-Decoder架构文章来源地址https://uudwc.com/A/orn50
| 模型 | 来源 | 总参数量 | 总参数量 |
|---|---|---|---|
| Transformer | PyTorch | 44,140,544 | 44.1M |
| T5-small | HuggingFace | 93,405,696 | 93.4M |
| T5-base | HuggingFace | 272,252,160 | 272.3M |
| T5-large | HuggingFace | 803,466,240 | 803.5M |
2. CV
2.1 CNN架构
| 模型 | 来源 | 总参数量 | 总参数量 |
|---|---|---|---|
| AlexNet | PyTorch | 61,100,840 | 61.1M |
| GoogleNet | PyTorch | 13,004,888 | 13.0M |
| VGG-11 | PyTorch | 132,863,336 | 132.9M |
| VGG-13 | PyTorch | 133,047,848 | 133.0M |
| VGG-16 | PyTorch | 138,357,544 | 138.4M |
| VGG-19 | PyTorch | 143,667,240 | 143.7M |
| ResNet-18 | PyTorch | 11,689,512 | 11.7M |
| ResNet-34 | PyTorch | 21,797,672 | 21.8M |
| ResNet-50 | PyTorch | 25,557,032 | 25.6M |
| ResNet-101 | PyTorch | 44,549,160 | 44.5M |
| ResNet-152 | PyTorch | 60,192,808 | 60.2M |
2.2 Transformer架构
| 模型 | 来源 | 总参数量 | 总参数量 |
|---|---|---|---|
| SwinTransformer-tiny | PyTorch | 28,288,354 | 28.3M |
| SwinTransformer-small | PyTorch | 49,606,258 | 49.6M |
| SwinTransformer-base | PyTorch | 87,768,224 | 87.8M |
| ViT-base-16 | PyTorch | 86,567,656 | 86.6M |
| ViT-base-32 | PyTorch | 88,224,232 | 88.2M |
| ViT-large-16 | PyTorch | 304,326,632 | 304.3M |
| ViT-large-32 | PyTorch | 306,535,400 | 306.5M |
| ViT-Huge-14 | PyTorch | 632,045,800 | 632.0M |
