DK Forest LiDAR - Random Forest Model Performance
Jakob Assmann
09/08/2022
Models trained using BIOWIDE stratification
For these models the training data was split according to the BIOWIDE stratification.
Variable importance
Variable importance for this random forest model, determined using the “permutation” option in ranger.
| Overall | |
|---|---|
| Sand_utm32_10m | 100.000000 |
| treetype_bjer_dec | 82.402128 |
| dtm_10m | 77.419004 |
| Clay_utm32_10m | 70.345552 |
| dtm_10m_sd_110m | 57.163281 |
| ns_groundwater_summer_utm32_10m | 56.787379 |
| Soc_utm32_10m | 38.041657 |
| amplitude_sd | 37.983167 |
| canopy_height | 36.898430 |
| normalized_z_sd | 35.367769 |
| openness_difference | 30.409109 |
| slope | 20.910563 |
| ns_groundwater_summer_sd_110m | 16.561303 |
| vegetation_density | 15.630589 |
| amplitude_mean | 13.397601 |
| solar_radiation | 12.448666 |
| canopy_height_sd_110m | 4.933190 |
| vegetation_density_sd_110m | 3.406289 |
| foliage_height_diversity | 0.000000 |
Performance in BIOWIDE regions:
Performance map based on the independent validation data:
Performance table based on the independent validation data:
| Measure | Overall | Bornholm | Fune_Lolland | Nordjlland | Oestjylland | Sjaelland | Vestjylland |
|---|---|---|---|---|---|---|---|
| Accuracy | 0.82 | 0.82 | 0.81 | 0.91 | 0.82 | 0.76 | 0.90 |
| Error | 0.18 | 0.18 | 0.19 | 0.09 | 0.18 | 0.24 | 0.10 |
| Sensitivity (True Positive Rate) | 0.87 | 0.90 | 0.92 | 0.95 | 0.89 | 0.83 | 0.71 |
| Specificity (True Negative Rate) | 0.78 | 0.69 | 0.60 | 0.86 | 0.74 | 0.69 | 0.96 |
| Fall-out (False Positive Rate) | 0.22 | 0.31 | 0.40 | 0.14 | 0.26 | 0.31 | 0.04 |
| Positive predictive value (User Accuracy) | 0.80 | 0.83 | 0.83 | 0.89 | 0.80 | 0.74 | 0.88 |
Performance table based on the dependent training data:
| Measure | Overall | Bornholm | Fune_Lolland | Nordjlland | Oestjylland | Sjaelland | Vestjylland |
|---|---|---|---|---|---|---|---|
| Accuracy | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| Error | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Sensitivity (True Positive Rate) | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| Specificity (True Negative Rate) | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| Fall-out (False Positive Rate) | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Positive predictive value (User Accuracy) | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
Performance in Derek’s regions:
Performance map based on the independent validation data:
Performance table based on the independent validation data:
| Measure | Overall | Region 1 | Region 2 | Region 3 |
|---|---|---|---|---|
| Accuracy | 0.82 | 0.80 | 0.81 | 0.88 |
| Error | 0.18 | 0.20 | 0.19 | 0.12 |
| Sensitivity (True Positive Rate) | 0.87 | 0.89 | 0.88 | 0.78 |
| Specificity (True Negative Rate) | 0.78 | 0.68 | 0.71 | 0.92 |
| Fall-out (False Positive Rate) | 0.22 | 0.32 | 0.29 | 0.08 |
| Positive predictive value (User Accuracy) | 0.80 | 0.80 | 0.80 | 0.81 |
Performance table based on the dependent training data:
| Measure | Overall | Region 1 | Region 2 | Region 3 |
|---|---|---|---|---|
| Accuracy | 1 | 1 | 1 | 1 |
| Error | 0 | 0 | 0 | 0 |
| Sensitivity (True Positive Rate) | 1 | 1 | 1 | 1 |
| Specificity (True Negative Rate) | 1 | 1 | 1 | 1 |
| Fall-out (False Positive Rate) | 0 | 0 | 0 | 0 |
| Positive predictive value (User Accuracy) | 1 | 1 | 1 | 1 |
Performance by forest type (boradleaf vs. coniferous)
Performance table based on the independent validation data:
| Measure | Overall | Broadleaf | Coniferous |
|---|---|---|---|
| Accuracy | 0.82 | 0.81 | 0.87 |
| Error | 0.18 | 0.19 | 0.13 |
| Sensitivity (True Positive Rate) | 0.87 | 0.91 | 0.57 |
| Specificity (True Negative Rate) | 0.78 | 0.66 | 0.96 |
| Fall-out (False Positive Rate) | 0.22 | 0.34 | 0.04 |
| Positive predictive value (User Accuracy) | 0.80 | 0.80 | 0.83 |
Performance table based on the dependent training data:
| Measure | Overall | Broadleaf | Coniferous |
|---|---|---|---|
| Accuracy | 1 | 1 | 1 |
| Error | 0 | 0 | 0 |
| Sensitivity (True Positive Rate) | 1 | 1 | 1 |
| Specificity (True Negative Rate) | 1 | 1 | 1 |
| Fall-out (False Positive Rate) | 0 | 0 | 0 |
| Positive predictive value (User Accuracy) | 1 | 1 | 1 |
Models trained using Derek’s stratification
Variable importance
Variable importance for this random forest model, determined using the “permutation” option in ranger.
| Overall | |
|---|---|
| Sand_utm32_10m | 100.000000 |
| treetype_bjer_dec | 81.570818 |
| dtm_10m | 73.198738 |
| Clay_utm32_10m | 69.218368 |
| dtm_10m_sd_110m | 58.622528 |
| ns_groundwater_summer_utm32_10m | 56.875166 |
| amplitude_sd | 39.117492 |
| Soc_utm32_10m | 37.597694 |
| canopy_height | 36.925847 |
| normalized_z_sd | 34.734356 |
| openness_difference | 30.391050 |
| slope | 19.174338 |
| ns_groundwater_summer_sd_110m | 16.546973 |
| amplitude_mean | 14.016066 |
| vegetation_density | 13.985474 |
| solar_radiation | 11.460056 |
| canopy_height_sd_110m | 4.549356 |
| vegetation_density_sd_110m | 3.270389 |
| foliage_height_diversity | 0.000000 |
Performance in BIOWIDE regions:
Performance map based on the independent validation data:
Performance table based on the independent validation data:
| Measure | Overall | Bornholm | Fune_Lolland | Nordjlland | Oestjylland | Sjaelland | Vestjylland |
|---|---|---|---|---|---|---|---|
| Accuracy | 0.82 | 0.83 | 0.83 | 0.91 | 0.81 | 0.76 | 0.90 |
| Error | 0.18 | 0.17 | 0.17 | 0.09 | 0.19 | 0.24 | 0.10 |
| Sensitivity (True Positive Rate) | 0.87 | 0.90 | 0.92 | 0.95 | 0.89 | 0.83 | 0.71 |
| Specificity (True Negative Rate) | 0.77 | 0.70 | 0.64 | 0.85 | 0.71 | 0.70 | 0.96 |
| Fall-out (False Positive Rate) | 0.23 | 0.30 | 0.36 | 0.15 | 0.29 | 0.30 | 0.04 |
| Positive predictive value (User Accuracy) | 0.79 | 0.84 | 0.84 | 0.88 | 0.77 | 0.75 | 0.86 |
Performance table based on the dependent training data:
| Measure | Overall | Bornholm | Fune_Lolland | Nordjlland | Oestjylland | Sjaelland | Vestjylland |
|---|---|---|---|---|---|---|---|
| Accuracy | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| Error | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Sensitivity (True Positive Rate) | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| Specificity (True Negative Rate) | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| Fall-out (False Positive Rate) | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Positive predictive value (User Accuracy) | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
Perfromance in Derek’s regions:
Performance map based on the independent validation data:
Performance table based on the independent validation data:
| Measure | Overall | Region 1 | Region 2 | Region 3 |
|---|---|---|---|---|
| Accuracy | 0.82 | 0.80 | 0.80 | 0.88 |
| Error | 0.18 | 0.20 | 0.20 | 0.12 |
| Sensitivity (True Positive Rate) | 0.87 | 0.87 | 0.90 | 0.79 |
| Specificity (True Negative Rate) | 0.77 | 0.69 | 0.68 | 0.92 |
| Fall-out (False Positive Rate) | 0.23 | 0.31 | 0.32 | 0.08 |
| Positive predictive value (User Accuracy) | 0.79 | 0.80 | 0.78 | 0.78 |
Performance table based on the dependent training data:
| Measure | Overall | Region 1 | Region 2 | Region 3 |
|---|---|---|---|---|
| Accuracy | 1 | 1 | 1 | 1 |
| Error | 0 | 0 | 0 | 0 |
| Sensitivity (True Positive Rate) | 1 | 1 | 1 | 1 |
| Specificity (True Negative Rate) | 1 | 1 | 1 | 1 |
| Fall-out (False Positive Rate) | 0 | 0 | 0 | 0 |
| Positive predictive value (User Accuracy) | 1 | 1 | 1 | 1 |
Performance by forest type (boradleaf vs. coniferous)
Performance table based on the independent validation data:
| Measure | Overall | Broadleaf | Coniferous |
|---|---|---|---|
| Accuracy | 0.82 | 0.80 | 0.88 |
| Error | 0.18 | 0.20 | 0.12 |
| Sensitivity (True Positive Rate) | 0.87 | 0.91 | 0.60 |
| Specificity (True Negative Rate) | 0.77 | 0.65 | 0.96 |
| Fall-out (False Positive Rate) | 0.23 | 0.35 | 0.04 |
| Positive predictive value (User Accuracy) | 0.79 | 0.79 | 0.80 |
Performance table based on the dependent training data:
| Measure | Overall | Broadleaf | Coniferous |
|---|---|---|---|
| Accuracy | 1 | 1 | 1 |
| Error | 0 | 0 | 0 |
| Sensitivity (True Positive Rate) | 1 | 1 | 1 |
| Specificity (True Negative Rate) | 1 | 1 | 1 |
| Fall-out (False Positive Rate) | 0 | 0 | 0 |
| Positive predictive value (User Accuracy) | 1 | 1 | 1 |