Oriented mAP: what to report, and how to read it

AP50 asks one question: do the rotated rectangles overlap by half?

That is the number I quote for a DOTA pretrain, and it is the official Task 1 score. On the Hub the field is eval_task1_map50.

Say which images the number was measured on, then give the percentage. How a rotated box becomes that percentage is below.

Here is how that percentage is built.

A match is a rotated overlap Link to heading

Sort detections of one class by score, high to low. Walk them. A detection is a true positive when its rotated rectangle overlaps a still-unmatched ground-truth rectangle of the same class, on the same image, at IoU at least the threshold. Anything else at the end of the walk is a false positive. One ground truth can pay for one detection.

The IoU is the area of the polygon intersection of the two rotated rectangles, divided by the union. It is not the IoU of the axis-aligned boxes drawn around them. A ship and a dock that share a horizontal extent and miss in heading can look fine in an HBB metric and fail here. That is the whole reason the series reports rotated AP.

The published path uses exact polygon IoU on CPU. Sampled GPU IoU is the matcher inside training, where a CUDA kernel would otherwise be required. It is the wrong backend for a number you put in a table. evaluation.val_exact_rotated_iou defaults to exact. Leave it there for anything you quote.

DOTA difficult objects, when they are still in the evaluated set, follow Pascal VOC: a detection that lands on one is neither a true positive nor a false positive. The default loader strategy is difficult_strategy: "drop", which removes them before train and before eval. "ignore" keeps them as ignore regions. Say which one the run used. A 1-point gap between two logs is often this flag.

From the walk to one percentage Link to heading

Precision and recall are the usual cumulative curves in that score order. With a single IoU threshold — the DOTA setting — AP is the VOC 2007 11-point interpolation: the maximum precision at or above recall 0.0, 0.1, …, 1.0, averaged over those eleven levels. mAP is the unweighted mean of the per-class APs.

That is the local implementation of what the DOTA Task 1 server reports for oriented boxes. It is not COCO AP. COCO averages the area under the curve from IoU 0.50 to 0.95 in steps of 0.05. oriented-det only does that style of average when a recipe sets evaluation.map_iou_thresholds. OSSDD recipes set 0.10, 0.15, …, 0.95, and on that path each threshold uses the area under the precision-recall curve (VOC 2010), then the mean of those areas. iou_threshold stays 0.5 for the other diagnostics. An OSSDD mAP and a DOTA mAP50 are different functions. They can both be “mAP” in a log line.

Two smaller accounting rules:

• The local mean includes classes that have at least one detection or one ground truth in the scored set. A class with neither is absent, not a zero. The DOTA server averages its fifteen classes. A tile subset that never contains a helipad is not the server’s denominator.
• Published local eval_map50 figures were scored with a score floor of 0.05. The precision-recall curve is built from boxes kept at or above that floor. It is not an integral down to score 0.

A deploy threshold is a different object. Oriented R-CNN’s production floor is 0.55. That chooses which polygons a GIS client sees. It does not change AP, because AP already walks the scores. The Docker post uses 0.55 on demo.jpg and counts boxes. That count is an operating point. It is not 76.73%.

NMS does change the set that enters the metric. The zoo numbers use rotated NMS at IoU 0.1. A looser merge keeps duplicates, and those duplicates become false positives once the first box has taken the ground truth.

AP50 is overlap. AP75 is tightness Link to heading

AP50 asks whether the rotated boxes overlap by half. AP75 asks for three quarters. On a long ship, a few degrees of heading error fails 0.5. On a compact vehicle, the same angular error can still pass 0.5 and fail 0.75. The lessons note is why that shows up as a loss problem. The numbers below are why a single AP50 cannot rank a 1× checkpoint against a 3× one.

Official DOTA v1.0 Task 1. The ResNet rows are the v0.3.1 zoo. The RTMDet row is v0.4.0 — one Hub weight, the 3×.

Read the ResNet columns in pairs. For Oriented R-CNN, Faster R-CNN, and FCOS, another 24 epochs does not buy hidden-test AP50. The 3× movement that is real is AP75. RetinaNet is the AP50 exception, and only the OBB head: circum-HBB weights live at rotated_retinanet_dota_le90_{1x,3x}_hbb and are a lower, different metric. RTMDet is not that pair. Its only published schedule is the 3×, and that single weight is both the AP50 peak and the AP75 peak. Which detector to train uses the split on purpose: RTMDet-S for that peak, Oriented R-CNN when a diagonal pack needs oriented RoIAlign, Faster R-CNN or FCOS when recall and a cheaper schedule matter more.

The server number and the log number Link to heading

DOTA zoo recipes train on trainval. Official Task 1 is the hidden test. Local make eval-val scores val tiles that were in that train. On Faster R-CNN 3× that log says 83.46% and the server says 74.48%. The gap is memorized tiles. Quote the server number for DOTA. Keep eval-val as a convergence plot.

That word is DOTA-only. HRSC eval-val is the ImageSets test. SSDD and HRSID eval-val are the official test splits. FAIR1M eval-val is official val, and the 36.70% Faster R-CNN figure is tiled val, not a hidden Gaofen test. Those are held-out. They are also not Task 1, and they are not comparable to 76.73% just because the detector family is the same.

Pairs I will not subtract Link to heading

The HRSID row is the one that looks most like a leaderboard. That post already says the Wei number is COCO horizontal AP. Putting 78.55 next to 84.7 without the box type is how a rotated model gets described as behind on a metric it did not enter.

Links Link to heading

• DOTA evaluation · compute_oriented_map
• Tutorial 07 — config, train, eval
• Lessons learned on DOTA
• le90 geometry
• v0.3.1 zoo table · v0.4.0 RTMDet
• Previous: le90
• Next: Does MMRotate Faster R-CNN also mess up the bus lot? (26 Oct)

Written on October 22, 2026 by Jeff Faudi. Link to heading