重构:修复线性回归梯度更新Bug、优化神经网络初始化与代码效率#3253
Closed
wutong88yuyu wants to merge 17 commits into
Closed
Conversation
Member
|
截图中运行报错:没有安装tqdm |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
修改概述:
修改详细描述
1.修复Bug(错误修复)
src/chap02_linear_regression/linear_regression-tf2.0.py:
修正偏置不更新问题:原代码在tape.gradient和apply_gradients中只计算了权重w的微调,导致偏置b无法训练。现在修改为同时更新[model.w, model.b]。
图例显示,去掉了多余的修改“测试”标签。
src/chap03_SVM/svm.py:
修复维度匹配问题:在计算最小值时,给y增加了一个轴 ( np.newaxis) 以匹配X的维度,解决了广播错误。
2.模型收敛与训练优化(Optimization)
src/chap04_simple_neural_network/tutorial_minst_fnn-numpy-exercise.py:
优化权重初始化:将权重初始化的标准差从默认的1.0修改为scale=0.1。防止因初始权重过大导致的Softmax缓冲与陡消失,使模型训练更容易。
src/chap03_softmax_regression/softmax_regression-exercise.py:
更换优化器:将SGD更换为Adam优化器(LR=0.02),显着加快收敛速度,分类边界更加稳定。
src/chap04_simple_neural_network/tutorial_minst_fnn-tf2.0-exercise.py:
调整学习率:针对全量数据训练,将Adam的学习率调整为0.01,加快收敛。
src/chap02_linear_regression/exercise-linear_regression.py:
利用 NumPy 广播 ( x ** np.arange(...)) 生成挖矿式特征,替代了高效的列表推导方式和concatenate操作,代码机制更简洁。
src/chap03_softmax_regression/logistic_regression-exercise.py:
使用tf.round进行二分类判别,替代了发音的tf.where/ones_like操作。
src/chap03_SVM/svm.py:
使用np.where简化了预测函数的逻辑。
经历过什么样的测试?
操作系统: Windows / Linux / macOS (请根据您的实际情况保留一个)
Python版本:Python 3.x、TensorFlow 2.x、NumPy
测试结果:
运行线性回归 TF2.0 版本,Loss 能够正常下降(偏置项 b 持续变化)。
运行FNN(NumPy版),模型在初始阶段不再出现损失居高不下难以恢复的情况。
各章节练习代码均无报错运行通过。
