LOSS Diagram
Text Only
================================================================================
LOSS & ACCURACY CALCULATION DIAGRAM (MATRIX OPERATIONS)
================================================================================
(INPUTS)
+---------------------------------+ +---------------------+
| softmax_outputs (ŷ) | | class_targets (y) |
| (N x C Prediction Matrix) | | (Ground Truth) |
| [[0.7, 0.2, 0.1], | | [0, 1, 1] | <-- Sparse format
| [0.5, 0.1, 0.4], | | OR |
| [0.02, 0.9, 0.08]] | | [[1, 0, 0], |
+---------------------------------+ | [0, 1, 0], | <-- One-Hot format
| [0, 1, 0]] |
+---------------------+
|
v
+-------------------------------------------------------------------------------+
| FLOW 1: LOSS CALCULATION |
+-------------------------------------------------------------------------------+
|
v
(ACTION: Clip to prevent log(0))
np.clip(y_pred, 1e-7, 1-1e-7)
|
v
+-------------------------------+
| y_pred_clipped |
| [[0.7, 0.2, 0.1], |
| [0.5, 0.1, 0.4], |
| [0.02, 0.9, 0.08]] |
+-------------------------------+
|
+-------------------------+-------------------------+
| |
v (IF TARGETS ARE SPARSE) v (IF TARGETS ARE ONE-HOT)
+------------------------------------+ +--------------------------------------+
| Get values by index | | Element-wise multiplication |
| y_pred[range(N), y_true] | | y_pred_clipped * y_true |
| | | |
| [0.7, 0.2, 0.1] ---<--- [0] | | [[0.7, 0.2, 0.1], * [[1, 0, 0], |
| [0.5, 0.1, 0.4] -------<--- [1] | | [0.5, 0.1, 0.4], [0, 1, 0], |
| [0.02, 0.9, 0.08] -----^---<--- [1] | | [0.02, 0.9, 0.08]] [0, 1, 0]] |
| | | | = |
| v | | [[0.7, 0.0, 0.0], |
| (Vector of correct probabilities)| | [0.0, 0.1, 0.0], |
| [0.7, 0.5, 0.9] | | [0.0, 0.9, 0.0]] |
+------------------------------------+ | | |
| v (np.sum(axis=1)) |
+--------------------------------------+
| |
+----------->-------------+
|
v
+--------------------------+
| correct_confidences |
| (Vector of correct probs)|
| [0.7, 0.5, 0.9] |
+--------------------------+
|
v (ACTION: Take Negative Log)
-np.log()
|
v
+--------------------------+
| sample_losses |
| [0.356, 0.693, 0.105] |
+--------------------------+
|
v (ACTION: Take the mean)
np.mean()
|
v
+-----------------------+
| average_loss |
| (Scalar) |
| 0.385 |
+-----------------------+
+-------------------------------------------------------------------------------+
| FLOW 2: ACCURACY CALCULATION - CORRECTED |
+-------------------------------------------------------------------------------+
|
(Using original `softmax_outputs`) |
+---------------------------------+ | +---------------------------------------+
| softmax_outputs (ŷ) | | | class_targets (y) |
| [[0.7, 0.2, 0.1], | | | |
| [0.5, 0.1, 0.4], | | | If One-Hot: |
| [0.02, 0.9, 0.08]] | | | [[1,0,0], [0,1,0], [0,1,0]] |
+---------------------------------+ | | | |
| | | v np.argmax(axis=1) |
v np.argmax(axis=1) | | |
| | | If Sparse (no action needed): |
+----------------+ | | [0, 1, 1] |
| predictions | | +-----------------|---------------------+
| [0, 0, 1] | | |
+----------------+ +-----------><-------+
| |
+---------------><-------------+
|
v (ACTION: Compare for equality)
predictions == y
|
v
+----------------------------+
| [0, 0, 1] == [0, 1, 1] |
+----------------------------+
|
v
+----------------------------+
| [True, False, True] |
+----------------------------+
|
v (ACTION: Take the mean, True=1, False=0)
np.mean()
|
v
+-----------------------+
| accuracy |
| (Scalar) |
| 0.666666... |
+-----------------------+
================================================================================
FINAL RESULTS
+------------------+ +------------------+
| average_loss | | accuracy |
| 0.385 | | 0.666666... |
+------------------+ +------------------+
================================================================================